What is A/B testing?

In a digital environment where everything can be tracked and measured, testing your strategies is a no brainer. Content strategy plays a key role in driving organic traffic and converting sales, so by leveraging and optimizing your content, you can increase the chances of shoppers purchasing your product over your competitors. A/B testing is one of a handful of the best practices you can use to enhance content on the platform.

Also known as split testing, A/B testing on Amazon is a method of determining the best-performing product listing variation. By comparing two versions of the same content, you can identify the exact element that is driving purchase-ready shoppers to act, taking into account metrics such as conversion rate, sessions, and total sales. That being said, you should have a strong understanding of your current metrics, performance and challenges before building an A/B testing plan. This will allow you to have a solid base foundation before making any changes.

Start by running an A/B test experiment on Amazon.

Amazon launched its own A/B testing tool in 2019 called Manage Your Experiments, which allows US brand owners to test two variations of one product listing element. As a brand owner wanting to test different A+ content elements, you must have eligible ASINs, otherwise they will not be displayed in the A/B test experience. To be eligible, ASINs must belong to your brand, and they must have high enough traffic in their respective categories to determine content winners with confidence. 

Keep best practices for A/B testing in mind.

By following a few tips, you can begin your test experiment with confidence. Here are some recommendations:

• Strategically create a hypothesis to learn something from the experiment regardless of the outcomes.
• To get a larger sample size, experiment on high-traffic ASINs.
• Stick to making one change at a time to avoid confusion on which variables influenced the experiment outcomes. 
• Don’t stop early – you can run experiments for four, six, eight or ten weeks to ensure accurate results.

Get to know the elements of A/B testing.

Nearly any element of your A+ content can be A/B tested. However, you should use your own judgment to determine what to test based on your current performance. Consider working with an Amazon Ads Partner to strategically outline the testing method. Some variation ideas for your A/B experiment: 

• Utilize a comparison chart.
• Rearrange and update your modules and images.
• Present the same images and text using a different module layout.
• Include lifestyle images.
• Highlight one set of product features versus a different set.
• Add your brand name to the product title.
• Use different headlines to engage and motivate shoppers to learn more about your product.

Keep in mind that your experimental content should differ from the existing content, otherwise it is less likely to affect consumer behavior and you may not be able to confidently determine a winning variation. The key to a unified branding strategy is consistency, which applies to your color scheme, icons, layout, and even text fonts.

Wait for optimal data.

Patience is key here as most tests need several weeks to gather enough data to determine a winning variation. While Amazon recommends testing for eight to ten weeks, you can adjust the schedule or even turn off the test while it is running. You will start to see data within one or two weeks, although this preliminary data is not representative of the true impact of your experiment. Be sure to let your experiment run its full course before interpreting the results and making a decision. 

Gauge the effectiveness of your experiment.

While launching new marketing initiatives on Amazon can lead to an increase in traffic, you should know how to gauge the effectiveness of each initiative to dial in on what works best. 

Once your A/B test ends, you will get the following from Amazon: 

• Recommendations on which content variation is more effective
• A confidence level of the recommendations
• A confidence interval of likely outcomes from that content
• Estimated 12-month impact on sales

These insights will highlight the most effective content for your product detail pages to boost total sales and conversions for your Amazon products. By learning from these experiments, you can optimize and improve your other product listings. You might find it beneficial to run experiments during different seasonal periods as this will provide you with a better understanding of your consumers and their expectations. 

When analyzing your results, be sure to keep your audience in mind to make the most informed decision. Although it takes time and patience to run A/B tests, the optimization is worth it if you want to dominate your market. Happy testing and selling!

Earlier this year, Ashley Musumeci, Director Go-to-Market, CRM at Media.Monks, spoke to a packed house at Salesforce Connections. This exclusive 15-minute video captures her presentation in which she outlines how brand marketers are using Salesforce to deliver a personalized customer experience at scale when a clear data strategy is in place.

Think of, let’s say, a multinational holding company with about six subsidiaries in retail and banking, operating in seven countries with physical stores and both website and app platforms. Not all subsidiaries are present everywhere, and not all platforms use the same language. Now imagine wanting to unify all of that into a scalable system that provides updated, accurate information about your consumers—processed through reliable metrics that allow you to get a better grasp of their needs and interests. Not exactly child’s play, is it?

Our partners at Falabella, one of the largest retail companies in Latin America, were faced with that exact challenge in their ambition to develop their technological capabilities and strengthen their logistics to support the rapid growth of their online sales. The ultimate goal was simple: to better serve customers through personalized solutions and increase the company’s overall efficiency. The path to get there, though, required considerable expertise in the use and implementation of data. 

Growing Its Technological Muscles

Falabella’s flagship department stores, as well as its shopping centers and supermarkets, have crowned it the largest retail company in Chile, and one of the most important ones in Latin America. But while this long-standing brand was built on the foundations of a traditional physical store, the importance of having a strong online presence has never eluded them. 

Today, Falabella is in pursuit of becoming one of the region’s top ecommerce players. In other words, to take the relevance it’s built in the physical world and replicate it in the digital space. But to become the go-to ecommerce platform for a variety of needs, it had to integrate its four pre-existing platforms and their data into a single marketplace and a measuring system.

Our Head of Data Growth, South Cone, Walter Rebollo, was there to support them on that journey. “We put together a dedicated team that worked side by side with the brand’s, serving as a technological muscle,“ he says.

For a company of this magnitude, any technological miscommunication between properties could hinder the accuracy of their data analysis, which explains the need for a foundational transformation. “Our team of analytics experts, consultants and engineers helped the brand design measurement plans that were aligned throughout the organization. They developed dashboards, documentation, manuals and provided technical support for their implementations,” adds Rebollo.  

It’s More Than Just Statistics

In addition to serving as the technological muscle, the team introduced the brand to a more analytical and data-driven mindset. After all, being cognizant of the potential that lies in one’s first-party data is the first step towards building a sharpened customer experience. And it’s not about playing with statistics to see what sticks. Think of first-party data as a source of truth that illustrates our customers’ personalities and behaviors; a roadmap of sorts to support your audience across the entire customer journey.

Javier Fernández Morales, Falabella’s Regional Head of Performance & Growth, puts it plainly, “As one of the sites with the highest traffic in the region, our first-party data is a key asset in order to provide a better shopping and browsing experience.

In retail, consumers expect an omnichannel experience that’s tailored to them. They want to be able to switch between app, web and devices across the purchase journey from start to finish. We designed the analytics framework to capture that information, creating a setup that eased the understanding of how audiences behave and what they are interested in. 

“The digital transformation of Falabella encompassed a bunch of steps that led to the final goal,” explains Gastón Fossati, our VP of Data Growth SPLA. “For example, the implementation of the web ecommerce funnel for all countries, assessments to define the attribution model to be used, baking in machine learning for audience prediction, the implementation of enhanced conversions in Google Analytics and a monthly consulting service to work on the Firebase project for the app, among other things.” 

An Empowered Team Leads to Fast Decision-Making

Beyond providing a seamless customer experience, having a single measurement system aligned throughout the organization can be as helpful internally as it is for customer-facing interactions. Think about it: refined measurement leads to effective automated models, which can then save the team time and energy. “From a backend standpoint, having a better data management system has made it possible for us to develop a well-executed product scalability model,” says Fernández Morales. By allowing marketing technology tools to be executed with greater agility, we empower specialists to make better decisions, faster.

That said, the development of an integrated measurement system from the ground up is not something that happens overnight. In this year-long project—which stemmed from a three-year partnership—both teams worked in lockstep to create a measurement strategy according to each unit’s goals, integrate Google Analytics and train the brand’s team to act upon the information. “Our approach is one of democratizing knowledge, so we always make sure we’re not just delivering but also teaching our partners how and why we do what we do,” says Rebollo. 

Throughout that process, both teams blended with one another to the point where there was almost no distinction between each. “I find it truly remarkable that we’ve created a single team with not only a common goal but also a shared spirit,” says Lorena Alva Salazar, Head of Growth & Martech at Falabella.

Today, they can bank not only on a solid, unified team, but also on reliable metrics that are immediately available for quick decision-making. The phase-out of third-party cookies poses no threat, and they are in fact better prepared to forge a closer relationship with consumers. First, an integrated framework and multiple data collection points provide invaluable information on their consumers. And second, the brand can now empower users to choose how much information they want to share, thus safeguarding their privacy.