Skip to main content

Drama 💃 A/B No premium features in Starter

https://powrtrak.atlassian.net/browse/PM-6355

Drama 💃 A/B No premium features in Starter for Form Builder

Goal

Understand whether moving all Starter premium features to Pro will increase revenue, when compared with the current pricing plan.

Description​

Test moving all existing features and limits from the Starter plan to Pro for the Form Builder app only. Paying for a Starter subscription will only hide the POWr Watermark.

Test should primarily measure for impact to revenue (not upgrades).

Test should exclude existing users with active subscriptions

Hypothesis​

Users upgrade to gain access to premium features and limits. With no premium features included in the Starter plan, fewer users will upgrade to Starter. Users that are upgrading to access features and unlock limits will be more inclined to upgrade to Pro, and will do so.

Stories

All Wix users - no change

As a Wix user with or without an account

When view the price table and encounter limits in the editor

I should be excluded from this A/B

AND I should encounter the same premium limits that previously existed for Starter plan

So that my limits are in-line with the Wix price table

Logged in with existing subscription users Starter - no change

As a Logged in user with an active subscription on a Starter (premium) plan

When I return to the editor to make changes to my plugin

I want my features and limits to be at the same level as when I purchased my subscription

So I feel that I am getting the value I was promised from my purchase

Logged out user or logged in user with no active subscription Starter - pricing page remove watermark only

As a Logged out user or logged in user with no active subscription buying a Starter (premium) plan

When I visit the pricing page

I want to see that the Starter plan only removes the watermark

Because the pricing page should reflect the product and value I will purchase

Logged out user or logged in user with no active subscription Starter - features remove watermark only

As a Logged out user or logged in user with no active subscription that has purchased a Starter (premium) plan

When I am editing my plugin

I want to see that the watermark has been removed

AND to have access to the same features I had for free

Because this is the value I was promised from my purchase

Logged out user or logged in user with no active subscription Pro - pricing plan

As a Logged out user or logged in user with no active subscription buying a Pro plan

When I visit the pricing page

I want to see that the Pro plan includes all features that were previously available on Pro, including features that were previously available on Starter

Because the pricing page should reflect the product and value I will purchase

Logged out user or logged in user with no active subscription Pro - features and limits start at pro

As a Logged out user or logged in user with no active subscription that has purchased a Pro plan

When I am editing my plugin

I want to see that all features available on the Pro and below plans are available

Because this is the value I was promised from my purchase

POWr Product Manager - A/B tracking

As a POWr PM

When I go to POWr AB (https://www.powr.io/admin/powr_ab) to check on the status of this test

I want to see the test in the POWr AB logs

So that I can use the admin controls to examine the results of the AB and set parameters

POWr Product Manager - Upgrade Distribution

As a POWr PM

When I am reviewing the results of the AB

I want to see the distribution for number of upgrades for each level/subscription (e.g. Pro/Yearly) of each group in the test

So that I can compare distributions between the test groups to measure the impact

POWr Product Manager - Upgrade Distribution

As a POWr PM

When I am reviewing the results of the AB

I want to see the distribution for number of upgrades for each level/subscription (e.g. Pro/Yearly) of each group in the test

So that I can compare distributions between the test groups to measure the impact

POWr Product Manager - Revenue Tracking

As a POWr PM

When I am reviewing the results of the AB

I want to see the LTV for upgrades for each level/subscription (e.g. Pro/Yearly) for each group in the test (A vs. B)

So that I can compare overall impact to revenue and measure LTV changes per level/subscription

Comments

Edward Fischer’s comments:

  • Price table exists in Shopify (could edit, but will need to be the same for both groups in the A/B)
  • Probably wouldn’t want to run in Wix
  • All the help center articles will be wrong for this test. Consider following up with a card to remove mentions of specific feature/plan mentions in our Zendesk articles.
  • Will need to make sure everyone in support is aware of the A/B, so as to not confuse users
  • On Shopify, the pricing table should show the most restrictive version (Starter only removes the watermark)

Pilar’s comments:

  • What about eeeeeemails? Some of them mention mention plan-specific features. I’d recommend syncing with Emilie before she leaves for vacation.
  • Specs should clarify that this is an A/B test that will use ab_logs
  • Also make sure one of your engineers or Ivan can write a script to give you better A/B results than what’s on powr_ab (seems like you will want to look at upgrade distribution, revenue, and LTV?)

Images and supporting information

Price Table

Price Table and enabled features should reflect that Starter has the same features listed as Free, AND “Remove POWr Logo”, and that Pro has the same features and limits that it currently has.