25 Ecommerce A/B Testing Ideas For Your 5 Top Store Pages

Bonus Material: Ecommerce A/B Testing Setup Guide

The biggest question in ecommerce A/B testing is not “how.”

It’s not even “what.”

It’s “why not?”

A survey by ConversionXL of 722 participants shows that 43% run fewer than five A/B tests per month![*]

They know experiments can improve their business, but they don’t act.

Why?

Because they don’t know how!

Regular A/B testing helps you to iterate site changes and give you a competitive advantage!

That’s what this guide is for: to help ecommerce businesses understand how to run an A/B test and give them inspiration for different A/B testing ideas.

All A/B Testing Ideas In One Table

A/B testing is a data-based approach to ensure we understand the impact of the changes we make to our site, ads, or emails.

At a very high level, A/B testing is done in four steps:

  1. Define your hypothesis

  2. Design the experiment (sample size, impacting factors, when is the difference meaningful, run time, participants, goal)

  3. Gather data

  4. Analyze data

When you begin with A/B testing, your list of ideas gets long quickly; after a while, that momentum to test and add ideas gets lost. That’s why I collected the most important ideas in a table, to inspire you and keep your momentum going.

The table is sorted by page type because not every page is equal. The homepage, for example, receives significant but unqualified traffic: it’s not yet clear what the user wants to accomplish. Did she come to the homepage because she heard about the store in an ad, or because she wants to search for a product in your shop?

The deeper we move into the site hierarchy (homepage at the top, product pages at the bottom), the more refined our understanding of the user's intent becomes. On the homepage, it’s very fuzzy, but on a product page, we know the user is considering buying the product. Once she visits there, our questions are “does she understand the value?” and “is this the right product for her?”.

We need to look at different success metrics and segments when testing different pages and channels. The success metrics are simply the numbers you look at to understand whether the control or variation of your experiment performed better. The segments are specific attributes within that experiment that can give you deeper insights about your A/B test.

When you perform A/B tests for specific segments, you're much more likely to find a winner because you have less noise in your target audience.


Metric
Segments
Homepage
  • Clicks (CTA)

  • Bounce rate

  • Dwell time

  • Email signups

  • New vs. returning customer

  • Mobile vs. desktop

  • Referral source

  • Social shares/likes

Category page
  • Clicks (CTA)

  • Bounce rate

  • Dwell time

  • Mobile vs. desktop

Product page
  • Average order size

  • Conversions

  • Revenue

  • Mobile vs. desktop

  • Paid vs. organic search traffic

  • Referral source

Search page
  • Clicks (CTA)

  • Bounce rate

  • Pages per visit

n/a
Email
  • Open rate

  • Clicks

  • Traffic

  • CTA

  • Country

  • New vs. existing reader

SEO
  • Rankings

  • Traffic (organic entrances)

n/a
Ads
  • Impressions

  • Clicks

  • Conversions

  • CTA

  • Demographics

  • Branded vs. non-branded keywords

Testing different segments is important because not all users behave the same. There are differences depending on what channel they came through or what device they use. Your goal is to understand the impact of different channels, devices, and user profiles on your revenue to make the best business decisions possible.

With a better understanding of the numbers and segments to test, we can move on to actual experiments.

Elements/
page types
Homepage
Category page
Product page
Search page
Page layout
One product offer at top of the page
Filter position and number
Product description
Mosaic layout vs. list
Personalization for logged in or repeat customers
Show several products vs. one product at the top
More info above the fold vs. less
More vs. fewer details
Best-sellers vs. promotions

Open vs. closed spec accordion

Full vs. lean



Navigation
Drop down vs. static
Sidebar nav.
Navigation vs. no navigation
Filters to narrow search down
Search box placement & size
Big vs. small breadcrumb nav.


Super lean (fold-out)
Highlight best-sellers


Category order



Show notifications in nav.



Headlines
Description of what store offers vs. no description
Small vs. big
Color set apart from rest of content vs. equal color
Headlines of different sections vs. none
Big vs. small section headlines
Color


Copy
Highlighting product value vs. description
Copy at top vs. bottom of page
Descriptive copy vs. none
n/a
Copy in promotion

Above specs vs. below

CTAs
Color (contrast vs. none)
Color (contrast vs. none)
Urgency (countdown)
Email signup popup
Email signup popup
Email signup popup
Email signup popup


Sales marks
Written out vs. symbol



Above vs. below the fold



Position

Images
Larger + fewer or smaller + more images
Product callouts (focus on 1-5 products at top of category page)
Freestanding product hero vs. crowded
Product images vs. text only

Amount of results
Image gallery


Image sizes
Videos vs. images



360 view of product



Product only vs. product in use



Product hero size

Price
Prices vs. no prices
Prices vs. no prices
Price size, color, placement
Prices vs. no prices
Size and color or price

Price with shipping vs. without



Zeros vs. no zeros

Promotions/
offers
Promotions at the top vs. bottom
Promotions vs. none
Promotions vs. none
Highlighting a price reduction
First-time visitor promotion



Seasonal promotion



Highlighting sales in your navigation



Content
Personalized products
Default type of product sorting (price, relevance, new to old, top-rated)
FAQ
Product recommendations in search box
Specific products vs. categories
Products other users bought in this category
Ask question right on the page
Search suggestions (“other people also searched”)
Social media feed
Descriptions/details/specs (how much info below a product image is enough to make user click?)
Promotional offer vs. no offer
Specs in list versus images
Show blog articles
Brands vs. categories
Number of related products

Hero image vs. video
Subcategories
Instagram/social feed


Related categories
Complementary products


Promoting guides and shopping advice
Specs in list or image


Showing only subcategories vs. subcategories + products


Social proof
Testimonials
n/a
Reviews + ratings (placement, amount, highlighting good reviews, testimonials)
Product recommendations on search pages
Images of people using/wearing product(s)

Person vs. no person in image showing how product is used/ worn

You might have a question or two about the table, so I want to zoom in a bit on each page type and provide a few examples. We’ll start with the homepage, then move on to category pages, product pages, and search pages. I’ll also cover some often-forgotten areas for testing outside of that.

Let’s go!

Homepage A/B Testing Ideas

As I mentioned before, the homepage is one of the most important pages for traffic, but not for revenue. Your primary goal here is to convert first-time visitors into leads, for example by getting them to sign up for your newsletter. For existing customers, the goal is to provide a personalized experience that leads them to product and category pages, or promotions.

Idea #1: Making the homepage leaner

User experience often improves when making a page leaner — not fuller. One mistake that’s often made on the homepage is trying to do 100 things at once. Instead, think about how you can narrow the experience down to one, maybe two goals.

The Shopify store Faucet Face below looks very noisy. There are too many elements above the fold. I would test a version that has just one element above the fold, maybe a product hero image, and push the rest further down.

Screenshot showing different aspects of a page

I would also test taking away the signup button, review button, and the chat window.

An extreme example of such a homepage is the Shopify store Triangl, which only has a few elements on the site but immediately shows what the product is.

Screenshot showing a landing page featuring a bikini model

Idea #2: First-time visitor promotion

For first-time visitors, it might be valuable to make an offer right away — depending on the source of their visit. They might have read an article about your product and clicked on a link, which indicates a certain prequalification. Or they heard about your store, searched for it on Google and want to check it out. Both use cases come with different implications, which is why testing an immediate offer is interesting.

Screenshot showing a subscribe popup

Idea #3: Seasonal promotion

Ecommerce provides many opportunities for seasonal promotions, from Christmas to Valentine’s Day and more. A viable experiment could look at the right place for a seasonal holiday promotion, whether as a popup or a fixed part of the homepage.

Screenshot showing nordstrom

Idea #4: Hamburger menu versus full navigation

In a mobile world, most navigation comes in a hamburger menu format. It’s interesting to test that format for desktop as well since it’s becoming such a standard. I would measure the impact on clicks and pages per visit in such an experiment, and also test using the work “Menu” instead of the hamburger icon.

Screenshot showing a hamburger menu

Idea #5: Highlighting sales in your navigation

Highlighting an important sales offer in your site navigation has the benefit that it’s visible on every page and gets a ton of attention.

Screenshot showing Williams Sonoma

Besides SALE, some other categories you can test highlighting in your navigation are best-sellers, top-rated, or seasonal.

Idea #6: Showing notifications in your navigation

Notifications are everywhere these days. Sometimes they’re annoying; sometimes they’re helpful. On your store, you can only test how users react to them. Making them aware of a current promotion, of an item they saved in their cart or of an important update about the company are legit notifications you can test with clicks and conversions.

Shopify store Taylor Stitch uses a small notification on the top right of their homepage to get more visitors to sign up for their email newsletter.

Screenshot showing an informational popup on a landing page

Idea #7: Shopping on Instagram

Instagram provides a shopping functionality that lets users buy the product right from their feed. Here’s what it looks like on HELM Boots Shopify store:

Screenshot showing an Instagram promotion page on a website

Try adding the feed to your homepage and test if users are more likely to convert from that feed (or at least follow you on Instagram). The idea is that users are more likely to order through an Instagram picture because they see the product from a different angle.

Screenshot showing a product page

Category Page A/B Testing Ideas

A category page is not only an important hub for search engines (in terms of SEO) but also for users. It’s a page for the middle of the funnel, when users consider buying something but don’t know which product yet. Browsing and exploring is a vital part of shopping. A buyer’s decision is more refined, based on whether she browses a category or subcategory page.

For category page experiments, it makes sense to measure:

  • Clicks

  • Revenue

  • Conversions

  • Bounce rate

  • Pages per visit

  • Dwell time

Idea #8: Highlight one product at the top

For category and subcategory pages, it makes sense to test highlighting the best-seller or best-rated product of that category at the top. On the category page for electronics on walmart.com, you can see an iPad highlighted, for example. Walmart has a slider with 4-5 products in the header, probably depending on offers, season, and other factors.

Screenshot showing a landing page

Idea #9: Lean category page

The idea of a lean page, one without many elements and mostly visuals, can be applied to category pages. You need to find the right balance between [x and y], but you can go as far as showing product images only and providing more details when hovering over the image. Here’s how Shopify store Triangl does it:

Screenshot showing an ecommerce page featuring bikinis

Idea #10: Sales marked on the category page

Indicating a sale on a category page helps visitors to make a product decision. The idea is to turn a browsing behavior into a specific buying intent. Here’s how Shopify store Chalkd does it:

Screenshot showing multiple products

Idea #11: Promoting guides and shopping advice

Given that category and subcategory pages are used for researching, it makes sense to experiment with guidance and help specific to that category. Test and measure the impact of adding a buyer’s guide or offer shopping advice from an expert.

Screenshot showing home audio products

Idea #12: Showing only subcategories vs. subcategories and products

Even though the main job of category pages is to narrow down the user’s search, for example from “audio” to “speakers,” it’s interesting to test showing selected products on them as well. A good indicator of success would be clicks and conversions for such an experiment.

Walmart shows only subcategories at the top of its category pages, for example, and hints at more products below with a product feed.

Screenshot showing different categories of electronics

Product Page A/B Testing Ideas

Product pages are often the core of experimentation — the money is made here! Not only are product pages the center of conversion, but it’s also the template with the most pages on an ecommerce site. Hence, optimization of product pages often has the biggest impact.

Based on that position, product page experiments should be measured by the following metrics:

  • Revenue

  • Conversions

  • Average order size

Idea #13: Full versus lean product page

Amazon’s product page is full of information, which leads to the assumption that more information leads to better conversion rates. So, what do we do with assumptions? Test!

Brainstorm what information you can show for your product and experiment with it. Don’t test 100% of your ideas with the current version. Select a few and iterate on the optimal product page instead.

Screenshot showing a product page on amazon

Idea #14: Specs in list versus images

Another interesting experiment is the format you use to communicate product specifications and descriptions.

One approach is images that visualize product attributes like size, quality, material, and more. Here’s how Shopify store Brilliant Bicycle Co. does it:

Screenshot showing information about a product

In some cases, a simple list might be enough, like Shopify store Great George Watches uses:

Screenshot showing product specifications

There is no right or wrong way, so you have to experiment with it.

Idea #15: Highlighting a price reduction

Price reductions are one of the main drivers for buyers, and it’s worth experimenting with different ways to display that.

Screenshot showing a product page

Idea #16: Showing the product versus how it’s used

Of course, you have to show your product. But does it matter whether users see the product in action versus just on a white/black background? An experiment reveals the answer.

What is more attractive: Seeing just the product?

Picture showing a pair of shoes

Or seeing it in action?

Picture showing a man wearing a pair of shoes

Test it!

Search Page A/B Testing Ideas

The bulk of your experimentation efforts should happen on the pages that make you money. But you shouldn’t completely forget the pages and channels that bring traffic to your conversion pages and the pages handling the transaction.

One essential part of every shop is search. Search pages have only one purpose: to bring a user to another page. Hence, it’s smart to test different layouts, filter navigation, and recommendations.

Idea #17: Product recommendations on search pages

Recommendations in search pages can be tricky from a technical standpoint, but if manageable they can have an interesting impact on revenue. Users have a goal when they search, so from this point on it’s about whether you can help them find it.

Jet.com shows best-seller recommendations on top of its search results for “non-fiction books.”

Screenshot showing products on an online book store

Idea #18: Show recommendations while users search

Sometimes users aren’t sure how to formulate a search. Try helping them as they are entering their query. Providing product recommendations, suitable categories and even blog articles in the search field is something you should test.

Here’s how Shopify store Harris Farm Markets does it to help visitors quickly find groceries they want to buy:

Screenshot showing search results on an ecommerce store

Idea #19: Mosaic versus list view

Making it as easy as possible for users to find the right product is also a matter of user experience. In some cases, a list view is better, in other cases a grid view. It depends on your product and how broad the search is. The narrower, the more sense it makes to use a list view to provide more details about the product. One way to test is to compare user engagement for a mosaic view versus a list view.

Here is what the mosaic view looks like:

Screenshot showing results on a bookstore

Here is what the list view looks like:

Screenshot showing results on a bookstore

You should provide the option to use both, then once you have enough A/B test data, you can show the winning version.

Checkout Page A/B Testing Ideas

Of course, you also have to test downstream from the conversion page: the checkout process. There is nothing worse for an ecommerce store than losing customers in the checkout process.

Idea #20: Offer at checkout

When users check out, they are already very invested in the decision to buy this product from you. That’s a good opportunity to have them sign up for a membership or subscription.

It’s important to find the right balance here: how useful is it for first-time customers to sign up versus repeat customers? What is the impact of order size on the likelihood to sign up?

Screenshot showing the checkout page on Barnes & Noble

Idea #21: Recommendations in shopping cart

When users add an item to their shopping cart, it can make sense to show them items other users bought as well. Often, shoppers have similar interests, and this is a way to capitalize on that.

Screenshot showing "people also bought" popup

Idea #22: One-page versus multi-page checkout

Providing checkout on a single page versus multiple pages can have a big impact on shopping cart abandonments. You have to find out what makes it as easy as possible for your customers while not being confusing at the same time. On the screenshot below, you can see how Walmart creates a convenient checkout flow on one page.

Screenshot showing the checkout page on Walmart

Of course, you can test more than just pages. You can also test marketing channels, such as SEO, email, and ads!

Marketing Channel A/B Testing Ideas

Idea #23: Ad testing

Instead of testing content or layout when the user gets to your site, why not test how to get her to the site in the first place? That’s what ad A/B testing is for, and luckily most ad platforms make it very easy to design such experiments.

Screenshot showing questions about AB testing

Facebook’s ad manager comes with built-in A/B testing functionality. To set it up, all you need to do is add the Facebook pixel to your site. You should have at least 100 conversions per test to get statistically valid results. So if you are just starting out and have five conversions per week, it might be a bit too early to experiment.

Knowing how much a conversion costs you, for example from previous campaigns, can also help you estimate the budget you need for ad testing. Say you have to pay $5 on average for a conversion and want to run an experiment with two variations. With 100 conversions for statistically relevant results, you now know you’ll need to spend roughly $5 x 100 x 2 = $1,000.

You can test ad copy, images, calls-to-action, or showing the same ad to different audiences. Facebook will tell you not only how many clicks each ad got, but also how many conversions!

There is much to be learned from testing ad variations:

  • What message (copy) your audience resonates more with

  • What images catch their attention

  • What audience your product is most appealing to

  • etc.

Idea #24: Email testing

Email testing brings similar insights as ad testing: understanding your audience better and what offers make them convert. But email goes a step further: what subject line makes a user open an email? You can test subject lines to see what messages get more of your subscribers to open them.

Screenshot showing an AB test setup page

You should test different email formats:

  • Welcome Email

  • Promotion

  • Reminder/reactivation

  • Cart abandonment

  • Etc.

Each of these has a different purpose and context in which it is relevant. Hence, email testing should be a staple in your experimentation culture.

Idea #25: SEO testing

SEO follows a slightly different framework, as it’s more of a quasi-experiment with bots than users. However, that doesn’t make it less useful to us. Companies like Thumbtack, Pinterest, Etsy, and others proved that when you have enough traffic and lots of templatized pages — which ecommerce stores have — you can do SEO testing. The image below shows an SEO experiment from Thumbtack, where three different title tag variations were tested.

Thumbtack put a few thousand pages into each title tag bucket, then launched the test on October 7. On October 14 they could see that the alternate variations had underperformed the baseline title tag by 20-30%. So they reverted the titles back to the original title tag, and the traffic returned to a normal level.

Graph showing data about ab testing

This process is very similar to other forms of testing: you take a group of pages with the same template, change one variable and see how their traffic, CTR, or rankings diverge. Strictly speaking, SEO testing is a quasi-experiment because it lacks the randomization factor. You know exactly which pages get which treatment (changes made), as opposed to actual A/B testing, in which a user randomly sees the control or variation page. The randomization is important because it doesn’t favor a specific user or group. It’s okay for SEO, though, because the user is a machine and therefore shouldn’t be susceptible to bias.

A/B Testing Mistakes

Many costly mistakes in A/B testing can be avoided when you know what to pay attention to. Starting out isn’t easy but it’s better than not starting with experimentation at all. Some mistakes have to be made, but I want to give you a list of tips that should prevent you from making the ones costing the most money:

  • Start experimenting with the page that brings the most expensive traffic (highest CPA/CPC): “start where the money is.”

  • Experiments must be falsifiable, meaning you need to be able to prove that doing the opposite leads to worse results.

  • Somebody in the company needs to be responsible for testing, otherwise it tends to fall off the table. Buying the software and spending time to develop tests costs you money with no return if you don't actually do the testing.

  • Test one change at a time. The more variables you change, the lower your confidence in the results.

  • If it takes too long to build an experiment, e.g., because of the time it takes to design or code a variant, it’s probably not a good idea. It usually means that the changes are too complicated to understand and roll out.

  • Perform the same experiment on mobile and desktop. The results can be completely different.

  • Hold regular brainstorming meetings to fill your testing idea backlog.

  • Don’t focus too much on button colors, test hero shots and big page elements instead.

Three Key Takeaways

Here are the three key takeaways:

  • Test designs and success metrics depend on what page type you test, like homepage versus product page.

  • Test pages and marketing channels: Email, ad campaigns, and SEO.

  • Get started, keep your momentum, and create a routine of testing.

To get a short step-by-step guide on how to get started A/B testing for your ecommerce business, grab my ecommerce A/B testing guide below.

Ecommerce A/B Testing Setup Guide

PROVEN STRATEGIES TO GROW YOUR ECOMMERCE BUSINESS

Sumo Shortcuts are based on proven strategies. Try them free for 14 days. If they don't work, don't pay a cent.

Trusted by some of your favorite companies