Documentation
Feedback
Guides

AB Tester

The A/B Tester app allows you to list, start and finish A/B tests via the VTEX Admin.

{"base64":"  ","img":{"width":2856,"height":1518,"type":"png","mime":"image/png","wUnits":"px","hUnits":"px","length":230761,"url":"https://cdn.jsdelivr.net/gh/vtexdocs/dev-portal-content@main/images/vtexarg-abtester-0.png"}}

In the following, you'll learn how to install and use the A/B Tester app.


Configuration

Take the following steps to install the AB Tester app on your VTEX Admin.

  1. Open the terminal and use the VTEX IO CLI to log in to the desired VTEX account.

  2. Install the A/B Tester app in the master workspace to enable A/B testing on your store website by running:


    _10
    vtex use master
    _10
    vtex install vtex.ab-tester

  3. Install the A/B Tester Admin app in the master workspace:


    _10
    vtex install vtexarg.abtester

Usage

After installing the AB Tester app, you'll be able to:

  • Create A/B tests.
  • Compare A/B test results.
  • Finish an A/B test.

Creating an A/B test

In the following step by step, learn how to set up a new A/B test. Notice that, before proceeding any further, you must already have a production workspace containing the changes you want to evaluate.

  1. Access the VTEX Admin.
  2. Go to Extensions Hub > Installed Apps > AB Tester.
  3. Click on Create new AB test.
  4. Fill out the presented fields as in the following:
    • Workspace Name - Enter the name of the production workspace containing the changes you want to examine via the A/B test.
    • Traffic Proportion - Enter the desired traffic proportion to be destined for the master workspace during the first hours of the A/B test. This value must be any whole number between 0 and 10000. For example, to set 90% of the traffic to the master workspace, enter 9000.
    • Amount of Time - Enter the amount of time (in hours) during which the specified Traffic Proportion will be maintained. After this period, the traffic proportions of the test will be automatically updated by the A/B testing system. Enter 0 if you prefer the A/B testing system to take care of your website traffic distribution from the start.

      The A/B testing automatically balances traffic every 3 minutes according to the conversion or revenue rates.

    • Test Type - Enter what the A/B test must evaluate: revenue or conversion.
  5. Click on Save.

Once you save your changes, the A/B test will start immediately, splitting your website traffic according to your preferences.

Comparing results

After creating your A/B test, the AB Tester app will provide a table containing some metrics that will help you evaluate and interpret the test status. This table contains the following information:

  • Conversion - Conversion rate percentage that each workspace presented during the test.
  • Conversion (last 24hrs) - Conversion rate percentage that each workspace presented during the test's last 24 hours.
  • N. of Sessions - Total number of sessions of each workspace since the beginning of the test.
  • N. of Sessions (last 24hrs) - Number of sessions for each workspace during the test's last 24 hours.

Finishing an A/B Test

Although automatic, an A/B test will not end by itself. Take the following steps to terminate an A/B test:

  1. Access the VTEX Admin.
  2. Go to Extensions Hub > Installed Apps > AB Tester and scroll the horizontal bar until you see a Kebab menu.
  3. Click on the Kebab menu and click on Finish A/B Test.
Contributors
3
Photo of the contributor
Photo of the contributor
Photo of the contributor
+ 3 contributors
Was this helpful?
Yes
No
Suggest edits (Github)
See also
Running A/B tests
App Development
Contributors
3
Photo of the contributor
Photo of the contributor
Photo of the contributor
+ 3 contributors
On this page