Build a simple A/B test in Mojito

Let's imagine we're working for Bing and we want to test showing no image on the homepage to 50% of users. What's needed is a simple A/B test.

Pre-requisites 1) You've cloned Mojito JS Delivery locally and installed the package dependencies. 2) You've setup tracking through your storageAdapter, or used our example one.

Experiment parameters

Parameter	Details
Hypothesis	Defaulting the homepage to no_image will increase the rate of searches.
Targeting	All homepage users
Traffic / Sample	100%
Variants	Control: 50%, Treatment 50%

Control

Show the Bing homepage with images.

Treatment

Disable the homepage image.

1. Create a new test

Start by creating your test's scaffolding. You can do this on the command line, like so:

npm run new -- --ab ex1

The command will output all your experiment files under lib/waves/ex1 (where ex1 can be replaced by the parameter you pass into the generator command above):

• config.yml: The experiment configuration YAML - where you define your experiment's parameters.
• 1.js: Recipe #1's variant code - the JS responsible for changing the look and feel of the treatment page.
• trigger.js: The activation trigger function - which dictates when your experiment will run.

2. Set your experiment parameters

Let's give the experiment a proper name and set its traffic allocation to 100% of traffic. Note: Your experiment will not accept traffic until you change the state from staging to live (we'll do that later):

• name: Bing Homepage without image
• sampleRate: 1
• Recipe 1: Treatment (no image)

By default, traffic to recipes will be evenly defined amongst them - so we don't need to set the sampleRate parameter for each recipe. Refer to the API reference for more details about the parameters.

Your experiment config should now look like this:

state: staging
sampleRate: 1
id: ex1
name: Bing Homepage without image
recipes:
  '0':
    name: Control
  '1':
    name: Treatment (no image)
    js: 1.js
trigger: trigger.js

3. Write your variant code function

Bing makes it easy to switch images off on its home page. We only need to apply the no_image CSS class to the div.hp_body element:

This is quite easy to do in JavaScript. We just need to change our Treatment recipe's code in 1.js, like so:

function js () {
    var elem = document.querySelector('div.hp_body');
    if (elem) elem.classList.add('no_image');
}

Note: The classList API is not supported well by older browsers, so you may need to create a default exclusion rule in your container shared-code.js.

4. Define your trigger properly

Your experiment will currently activate on all pages when DOMContentLoaded fires.

function trigger(test) {
    Mojito.utils.domReady(test.activate);
}

Recall that we want to run this experiment only on the home page (/). We just need to add an if statement for conditional activation:

// A good way to activate
function trigger(test) {
    if (document.location.pathname === '/') Mojito.utils.domReady(test.activate);
}

Even better, we can activate the experiment as soon as the background element is ready in the DOM. Simple replace Mojito.utils.domReady with Mojito.utils.waitForElement, so we can wait until div.hp_body is available:

// A better way to activate
function trigger(test) {
    if (document.location.pathname === '/') Mojito.utils.waitForElement('div.hp_body', test.activate);
}

Mutation observers are a great way to activate an experiment that treads between flicker (or FOOC) and race conditions (where you might attempt to interact with an element before it's loaded). Also, if there happens to be a version of the home page without the div.hp_body element, your experiment won't activate.

5. Build, test & publish your container

Congratulations! Your experiment is now set up. It's ready to build and publish to staging, so you may QA it:

npm run deploy

Preview your experiment

NB: You must have Mojito installed on the page or use a Chrome Extension like Requestly to inject Mojito into the page for testing.

1. Browse to www.bing.com.

2. Using the Mojito Chrome Inspector, open dev tools, browse to the Mojito tab and force your new experiment to the treatment.

3. Refresh the page, your treatment will be enabled.

6. Launch your experiment

When you've tested it's working, you can launch it by setting its state to live, and buildin/publishing the container:

state: live
sampleRate: 1
id: ex1
name: Bing Homepage without image
recipes:
  '0':
    name: Control
  '1':
    name: Treatment (no image)
    js: 1.js
trigger: trigger.js

Now you can build and publish your experiment again, to send it live for all your visitors:

# Build your container file & publish to AWS S3 (if configured)
npm run deploy