Skip to main content

Powr ab tests

We have 2 main ways for creating ab's:

  • First one is using main function for creating ab's powr_ab(ab_test_name, *ab_options). Example of using: powr_ab('ab_name', 'group_a', 'group_b') do |test| @globals['SOME GLOBAL VARIABLE'] = test #test = group_a or group_b end

  • Second and usually more preferable is by adding to initializers.rb your ab test by using add_ab_test(ab_test) function. It will add your ab test to the list of current ab's and excecute all of them once after initialized function call, example: def initialize_growth_ab_tests add_ab_test({ name: 'beks_clean_ab', groups: ['control', 'fake_ab_group'], do_ab_test: true })

    add_ab_test({ name: 'exit_intent_modal_abc', groups: ['control', 'discount_intent', 'wiifm_intent'], do_ab_test: true })

    add_ab_test({ name: 'more_reviews_via_dashboard', groups: ['control', 'progress_bar'], do_ab_test: user_in_ab_test?('more_reviews_via_dashboard') || current_user&.apps.blank? })

    add_ab_test({ name: 'project_casper_ab', groups: ['control', 'modern'], do_ab_test: user_in_ab_test?('project_casper_ab') || current_user&.apps.blank? }) end
    It will create a global window variable with all ab's that you can access in front-end: window.GLOBALS.ACTIVE_AB_TESTS

Using the exposed field.​

We add a user to an AB before they get exposed to the ab fork.

Eg: There is an ab where we want to show/hide themes in design tab. If the user never clicks on design tab, they will never experience the ab, so the ab results are not true. This is a deeper problem for some pricing abs which needed to run in isolation with other abs and were initialized much ahead of time. We now have "exposed" field and that is what is use to display number of participants on the ab dashboard and we can mark user as exposed from both front end and backend

Problem: Adding users in an ab at the right time (# of participant != # of people exposed) Solution: We add “exposed” to the ablog to indicate if user came across the ab variation experience https://gitlab.com/powr/powr/merge_requests/3592/diffs

Serverside: user_exposed_to_ab!(ab_test_name)

Clientside:

isInABTest(testName, groupName, exposed)
getABGroup(testName, exposed)
markUserExposedToAb(testName)

Example: isInABTest('template_minimarket_ab', 'template_flow', true) in template_modal_helper.jsx .

Note! Be careful if you are using isInABTest('ab_name', 'group', true) on view side, because it can call multiple ajax requests from wobblies sites, when wobblies' users see the plugin.