Increase Conversions Using Amazon’s Manage Your Experiments

Shoppers’ needs, desires and subjective tastes are never static. 

They evolve or can dramatically change seemingly overnight, influenced by the latest hot trends, new technology, or by spot-on marketing. 

The right marketing efforts can discover what consumers need and why they desire it, and then position products as the answer to those needs and desires. That’s why mega-brands spend thousands of dollars just on market research.

For Amazon sellers, accurately predicting their audience’s ever-changing needs and desires can be a hair-pulling difficult task where the odds are 50-50 that they will make the right call for which products to promote. 

What is the Manage Your Experiments Tool?

Amazon’s Manage Your Experiments tool is designed to help sellers to determine and create conversion driving content that answers the needs and wants of their current audiences, while expanding their reach to new ones.

By utilizing the Manage Your Experiments (MYE) feature, sellers are able to sharpen their sales strategies, refine their product offerings and listings, to appeal to current popular shopping trends. When used properly, Manage Your Experiments enables sellers to more effectively market their products. 

Available in Seller Central, sellers are able to run A/B split tests on titles and main images, bullet points and descriptions, and A+ content. 

MYE allows sellers to compare two versions of their listings content against each other to figure out which one performs better. Once the A/B test has been completed, the data will point to which listing performed best, based upon conversions and sales. Sellers would then publish the winning experiment content.

To publish MYE content, sellers can use the standard listing tools ‘Add a product’, or ‘Add a product via upload’ etc. To publish MYE A+ content, sellers can use the A+ content manager.

How Does Manage Your Experiments Work?

Manage Your Experiments follows a traditional A/B split test format. Randomly, fifty percent of shoppers see Variant A of a main image, product title, or A+ Content, while the other half are shown Variant B for the entire experiment. 

Content does not rotate over time, but both variants are viewed by different experimental groups of shoppers wherever it is available. 

Each variant appears as normal on the search results pages, product pages, etc., replacing the previous content (with good or bad results to be determined). Experiments don’t affect search rankings.

MYE A/B test results are updated once a week until the experiment ends. Sellers can access their results by clicking on the experiment name in the experiments dashboard.

The tool will then display metrics for conversion rates, units sold for unique visitor, projected one-year impact (predicts units sold/sales for one year if a particular version is used), and sample size. Also, results show how well each variant performed with shoppers and the probability of that the winning content is better.

Pro Tip: Sellers should run an experiment for at least 10 weeks to acquire enough data.

Eligibility Requirements for Amazon’s Manage Your Experiments

A seller must own the brand and be responsible for selling the brand in an Amazon Store and have at least one eligible ASIN based on traffic to run experiments. The ASIN is eligible if it’s part of a seller’s brand and has had high traffic in recent weeks so that enough data can be collected to justify experimental results (confidently determine a winner at the end of the experiment).

The Manage Your Experiments tool will show the eligibility status of most candidate ASINs, however, ASINs with low traffic may not be shown. Sellers can run only a single experiment on any ASIN at a time.

A+ experiments must have published A+ content to be eligible.

Pro Tip: Sellers should consider driving more traffic to ineligible ASINs using Amazon advertising or software tools like PixelMe.

Determine Goals Before Experimenting

Sellers should decide what things they wish to test with an eye towards accomplishing a specific goal. Should the title be tested to see if a shorter title converts better than a longer one? MYE is ideal for testing hypotheses like this to gather data on how a target audience will respond to content changes.

Sellers should test only one component of a listing at a time. If multiple changes are made simultaneously, and the new listing out-performs the original one, sellers won’t know for sure what specific change was responsible for superior results (more sales) and they will be unable to apply it to their other listings.

Ideas for Experiments

• Reducing product title length to under 100 characters to entice more shoppers to visit the detail page
• Revising product title to include brand name
• Revising product title to make it more descriptive (include features of interest to target audience)
• Adding a new product image to stand apart from niche competition
• Adding high-resolution images to content, or updating existing images
• Using only lifestyle images or standard product images
• Showcasing one set of features over a different set of features
• Using new A+ content modules on the detail pages
• Using a comparison chart in the A+ content

How to Create an Experiment

Once a seller has decided what content to test, creating an experiment is a straightforward process.

1. From the Seller Central dashboard, “Brands” to access the MYE screen
2. Click the ‘Create a New Experiment’ dropdown
3. Select the type of experiment that s to be run
4. See which products are eligible and those that are not
5. Choose a Reference ASIN (product to be tested)
6. List experiment details (start date, duration, content for each variant)
7. Submit content for review (once Amazon validates it, the experiment will begin on the date selected by the seller)

How to Design a Good Experiment

The best experiments have these attributes in common:

Experimental content is different from existing content. If it’s too similar, there will be less likelihood of it affecting shopper behavior, preventing the seller from clearly determining a winner.

Variant A and Variant B are very different from each other. This helps to ensure that any identified differences in performance are relevant and not a random occurrence. 

The experiment runs its entire course. Manage Your Experiments gives sellers options for how long they wish an experiment to run starting with four weeks. Sellers should keep in mind that a short testing period affects accuracy because of the limited data that is generated when the experiment is ended too soon. Amazon recommends an experimentation timeframe of at least 10 weeks.

Even though there may be promising results shortly after the start of an experiment, it’s always better to complete the entire timeframe for the experiment before making any new changes to minimize the risk of overestimating the results, or of possibly choosing the wrong variant.

Pros and Cons of Manage Your Experiments

Pros:

✔ Manage Your Experiments conducts its experiments with real shoppers in real time within the Amazon marketplace.

✔ The data retrieved from the MYE tool is provided in real time, so there isn’t any possibility for outdated information to skew results.

✔ All data is Amazon relevant. This helps to provide more accurate data which sellers can use to better manage their content on Amazon.

Cons:

• Manage Your Experiments’ A/B split testing is limited to only 3 categories presently (product titles, main images, A+ Content). If a seller wishes to test other parts of their content (e. g. bullet points, videos, other listing images), they can’t do so. 
• Only high traffic listings can be tested. The poorer performing, low traffic listing are ineligible and therefore can’t take advantage of MYE to improve their content.
• Experiments are done in real time. Sellers can present their normal content to only 50%  of their targeted audience. The other half will receive inferior content, making this type of testing risky to the seller’s reputation and brand, and may be costing them sales.

Eligible sellers should consider adding the Manage Your Experiments tool to their marketing arsenal because it gives them data insights that can save them valuable time and reduce their workload when testing new content.

Entourage: Software to Scale Amazon Ads and Results Driven Management.