Targets for A/B tests

You can target your A/B tests to users based on specific behavior, attribute, or event. The segment and target options for A/B tests are identical to those in a message or campaign. To select your audience for a test, click the Add segment dropdown.

See Building segments and targets for more details about segments and targets.

Change the distribution

Leanplum randomly assigns your targeted users to one of the test groups (control or variant). You can adjust the Distribution toggle to change the portion of users in each test group.

Sticky test targets

By default, when you select a target for your A/B test, the test will be "sticky." When the magnet button is on (sticky), the background is blue, like so:

A sticky test means that users will remain in a test once they enter it, even if they eventually fall out of the test's target group. The magnet defaults to "on" because most tests require users to stay in the group they are assigned to until the test is finished.

With stickiness off, users are taken out of your test when they no longer fit the test's target audience. This means users may stop seeing their test experience if they don’t fit your test’s target audience anymore.

For example, consider a test that is targeting users in the US. A user launches the app in the US and is placed into the test. Then, they fly to Europe and open the app again. If the magnet is selected, the user would stay in the test, even though they no longer fit the target group. If the magnet is unselected, they will be removed from the test once their location changes to Europe.

📘

Re-entering users are placed back into the same test group

In an unsticky test, it's possible for a user to exit the test, then re-enter it later. If this occurs, the user will be assigned a variant again (Leanplum assigns users to test groups based on User ID), and as a result could see a different variant offering a different experience from the previous test entry.
Sticky tests ensure users don't see more than one test experience.

Targets for tests with messages

If you start a new A/B test directly from the message setup page, the targets defined in the message will be copied over to the test as well. If the message campaign and the A/B test have exactly the same targets, then only users within the A/B test will be served that message.

If the targets do not match identically, users outside of the test may also receive the message (if they meet the target criteria for the message but not the test). See A/B testing a message for more on setting up a test for a message.

Adding specific users to a test

There may be times when you want to add a specific user to one of your A/B tests, even if that user doesn't fall into your target audience.

🚧

Random assignment is important for accurate results. Forcing users into a variant should only be used sparingly, for testing purposes.

In the Users tab, you can force a user into the control or variant of an A/B test. Click the ID of the user you want in your test, then scroll to the bottom of their user info. You should be able to see all active A/B tests with options for each control and variant group.

Clicking the circle under the variant or control group will force a user into that particular test group, even if the user does not belong to the target for the A/B test. A check mark will appear in the circle you select for that user. This feature is mainly used for preview/testing purposes — so you can put yourself in a variant and experience what the A/B test is like in the app, even if you don't match the specified target.

🚧

If a user is force-added into a campaign or test (using the method above), they will not exit automatically when they don't meet the target criteria. The user must be forced out of the campaign (by unchecking the same circle for the variant in the Users tab). Once forced out, the user will exit the campaign on their next session start.