Skip to main content

How to create defaults abs test app defaults

New version notes from Praneeta:

  • grab content for default by going to https://www.powr.io/raw/<app_id>.json
  • Go to the defaults/<app_type>.rb
  • Rename existing design version as design<ticket_number>_control and the new version as design<ticketnumber><useful_name>.
  • Name of the ab will always be <app_type>_defaults, so is important to have the variations updated to keep test results clean.
  • Note 1: We should clean up the old split ab logic from app/helpers/app_views_helper.rb. For now I have added an if condition for tabs app.
  • Note 2: You don't have to initialize the ab or mark user as exposed. Because of where the logic exists, it all happens when the user loads standalone and is simple, so leave it as is.

Example: https://gitlab.com/powr/powr/-/merge_requests/5613 and https://gitlab.com/powr/powr/-/merge_requests/5675

Can test by forcing yourself into a group like this: http://powr.io/plugins/tabs/standalone?tabs_defaults=design_ACTIV8_1708_control

Old Version

Step 1: Designer creates new default app

Anyone at POWr who understands design and how our users use our plugins can create a new default. Typically, we let our wonderful UX designers do this part as there are many things to consider - how many elements, what design to use so it looks great on first load but the user won't have to change too much to make it work on their site, etc.

Once the designer has created the default, have them send you the standalone link.

Step 2: Copy app content

Once you have the link to the app, open its view URL. Then, go to the console and copy everything that's in APP_MODEL.attributes. Remember to expand data to grab things from in there in case this part is different in the new default.

Step 3: Paste into default overrides file

Path looks like this: app/views/apps/defaults/[appName].rb

Paste in everything from the app model's attributes into the file and give the new default a name (often we just use design_2).

Important: Clean up the formatting and make sure you have no syntax errors. One thing you'll have to do is to convert the JS into Ruby, so remember to wrap quotes around everything and add commas to the end of each line!

To preview your work, add a param to your standalone URL that looks like this: ?[appName]_defaults=[name of your default) For example, https://www.powr.io/plugins/form-builder/standalone?formBuilder_defaults=design_1

If you're NOT seeing the correct default in your link, you probably messed up the formatting and you're instead seeing the original rather than one of the defaults in your defaults rb file.

Example MR for reference: https://gitlab.com/powr/powr/-/merge_requests/5042/diffs

Step 4: Verify that data is being added to /better-split

You don't need to do anything to activate the A/B test, it gets automatically created as soon as the new default is loaded in an app.

The test will have its name look like this: [appName]_defaults It will appear on here: https://www.powr.io/better-split

If you're not seeing the A/B or there is no data being added to your test group, something's fucked.