Skip to main content

What is Bandito? How Do I Use Content Testing in Arc XP?

Bandito is Arc XP’s Content Testing tool. Integrated natively with PageBuilder Editor, Bandito makes it easy to try out different variants of content to learn which performs better with your audience.

Bandito terminology

Through Bandito documentation, we’ll be using a few key terms, which are defined here:

  1. Block: The front-end component that will have varying content being tested.

  2. Default Variant: The default value of the block.

  3. Variant: Any non-default value being tested in a block. A Bandito test can have infinite* variants

  4. Test: The period of time during which a feature is being served different content. A test begins when a PageBuilder Editor page is published that contains a block with variants added.

  5. Serves: The number of times a piece of content is presented to the audience.

  6. Clicks: The number of times a piece of content is clicked on by an audience member.

  7. Click-Through Rate (CTR): The percent of serves that result in a click.

* Recommendations on maximum variants are given below.

Why should I use Bandito?

The goal of Bandito is to assist editors in maximizing the performance of content on their site.

Performance in Arc is measured with Click Through Rate, which is a measure of how often an audience member clicks on a link out of how many times it is served to the audience. Bandito works by allowing editors to add variants of their content - often headlines, promotional images, or description text - which then get presented to the audience as part of the experiment.

Over time, Bandito learns which variants are outperforming others and begins to route traffic to the better variants. Eventually, the experiment will converge on a winner that can be selected as the new default for the feature. Bandito implemented a Multi-Armed Bandit test. More about Multi-Armed Bandits can be found in our Content Testing Overview.

Editorial workflow

Prerequisites

Let’s assume you already have a Story created and published in Composer with the headline, “Team A Wins Event!” This Story has been added to the Sports section front page in PageBuilder Editor.

Workflow

  1. An Editor navigates to PageBuilder Editor and selects their Sports Front Page.

  2. An Editor finds the block with the headline, and follows the steps outlined here to add a variant, “Team B Narrowly Defeated!”

  3. Bandito adds the variant, creating a test that begins when the Editor published the Sports Front Page.

  4. Bandito begins to segment traffic to the different variants, learning which is performing better and routing more traffic to the winning variant.

  5. Eventually, the CTR of “Team B Narrowly Defeated!” is significantly higher than that of “Team A Wins Event!”, so Bandito declares it the winner and begins routing most traffic to the variant.

  6. Back in the PageBuilder Editor, the Editor can set the winning variant to the default, thus ending the experiment.

What blocks can I test using Bandito

Bandito is available out of the box on the following Themes Blocks:

  • XL Promo Block

  • L Promo Block

  • Medium Promo Block

  • Small Promo Block

  • XL Manual Promo Block

  • L Manual Promo Block

  • Medium Manual Promo Block

  • Small Manual Promo Block

Bandito can also be used an any custom block given you have set the block up in Engine, as described below:

  • The block has to be dynamic and CANNOT be static. Bandito renders variants client-side which is why static components cannot support content testing because they will not re-render in the reader’s browser.

  • The block MUST use a registerSuccessEvent hook. See the “How to: Using registerSuccessEvent”. The registerSuccessEvent hook MUST be used in order to tell Bandito that the user interacted with the Feature. For more details, check this documentation on RegisterSuccessEvent() Method.

  • Content testing is not yet supported for Linkable Blocks, Chains, and Chained Blocks on parent and/or child page. If you have variants setup on a non-linking Block/Chained Block, the linking card will not be available for the control or the variant. Likewise, if you have Linked Blocks you will not have the option to create variants.

The following are NOT required but make for a much easier developer experience:

  • Ensure your blocks is using editableField instead of editableContent.

  • Your block should be a functional component as opposed to a class component. While this is not a requirement, it does provide a better developer experience as Engine will automatically update content with local edits without any extra development work.

Note

It’s important to consider combinations of the above that may impact this as well. For example, if the registerSuccessEvent has been implemented on a static component, then that feature will not work.

Content testing limitations

Bandito does not limit the number of variants an Editor can add to a single test. However, a content test can only be successful if each variant receives enough clicks that one variant can have a statistically higher click rate. In Bandito, we recommend ensuring every variant is going to receive at least 100 total clicks.

The number of clicks each variant will receive is determined by a combination of three variables:

  1. The number of variants

  2. The overall traffic or total number of clicks a story is expected to generate

  3. The length of time your test is running

An hour long test on a story that will only generate 100 clicks per hour, for example, will only be able to support a total of two variants (the default and one additional). The same content with a 6 hour test could support up to 6 total variants.

Total Click Count

Recommended Maximum Variants

<100

Recommended to not use Bandito

100 - 500

2

500 - 2,000

4

2,000 - 5,000

8

5,000 +

16

Learn more

See PageBuilder Editor: Content Testing with Bandito Guide to

  • learn more about Content Testing in Bandito,

  • walk through a Bandito Test step by step, and

  • learn about integrating Bandito in your front-end feature.

In this section: