25 July 2023

CRO | Launching results to new heights with A/B testing

0 min read
CRO | Launching results to new heights with A/B testing
Mark LillicrappWritten ByMark Lillicrapp

As Propeller's Technical Director, Mark is responsible for developing digital policy and strategy, implementing infrastructure and leveraging technology to help Propeller and its amazing clients achieve their goals.

Conversion rate optimisation campaigns increase the flow of visitors to a website. They encourage visitors to take desired actions such as mailing list subscriptions, registering for an event, or more.

CRO campaigns require testing to find the most impactful assets in converting website visitors into customers. With such a wealth of options available- copy, colours and CTAs- it can be challenging to settle on a final design. 

Knowing what customers will respond to with the desired action is not a straightforward formula. This is where A/B testing takes centre stage. Split-testing campaigns is a key way to optimise performance; ensuring conversion rate campaigns are running at their best.

Conversion rate optimisation: a brief overview

Why is it important?

Conversion Rate Optimization is vital for businesses seeking to thrive in the digital landscape. By optimizing the conversion process, businesses can achieve higher ROI, better user experiences, and a competitive edge, while also aligning their strategies with customer needs and staying adaptive to market changes. Ultimately, CRO empowers businesses to make informed decisions, improve customer satisfaction, and achieve sustainable growth.

Where A/B testing comes in

A/B testing is a fantastic way to compare the effectiveness of two variations of the same campaign. It allows testers to understand what will achieve more conversions. 

What is A/B?

Ideally, every element of a campaign will work as hard as possible to convert more visitors. A/B testing involves sampling different copy, designs, colours and offers to understand which receives the highest click-through-rate, or conversions.

This is achieved through sending out two variants of the same campaign to different shoppers at the same time. In turn, this allows marketers to strengthen their understanding of what creates better results. 

Typically ‘A’ refers to ‘control’ or the original testing variable. Meanwhile, ‘B’ refers to ‘variation’ or a new version of the original testing variable. 

Testing different elements of a campaign against one another while keeping every other variable consistent is key to successful A/B testing. If too much is changed from one sample to another, it would be increasingly difficult to understand which factors are making a difference.

What can you test/ change?


If customers cannot understand the CTA, they won’t know what to do next. A strong CTA is the crux of any campaign. Marketers must trial positions, colours and copy to see which combination earns the most clicks.


Lengthy sentences with an unclear direction will prevent the delivery of messages. Sampling different phrases, and sentence lengths will boost understanding of what resonates with audiences.


Colours can truly make or break a campaign. Finding the balance between attracting a customer’s attention but not overwhelming them can be a difficult task.  


High-quality images are a must for an engaging campaign. However, not all customers will respond in the same way to certain images. Sampling a selection will enable marketers to determine what their customers respond well to.

Why is it effective? / benefits

Data-driven results

Data from these tests validates changes to strategy, as it demonstrates tried and tested samples.

Reduce bounce rates

If website visitors are leaving pages quickly after arriving on a website, it is worthwhile trialling A/B testing. Tweaking the copy, titles, imagery and designs will establish what best catches visitors’ attention. 

Run conversion rate campaigns to their full potential

With A/B testing, it is possible to fgigure out the type of content that converts more website visitors into customers. This marketing tactic enables a deeper understanding of what works for a brand. 

Best practices for A/B testing

Run your tests for a reasonable amount of time

Time is essential in testing. A common mistake in A/B testing is ending an experiment too early. In order to obtain statistically viable results, tests must run until a sufficient number of conversions have been recorded. Shorter stints might lead to inconclusive or misleading results, wasting time and resources.

Make only minor adjustments each time you conduct a test

By testing components individually, marketers clearly identify which changes positively impact the outcome. In turn, this makes future optimisations more effective.

Incorporating A/B testing into your broader CRO strategy

A/B testing is just one piece of the larger CRO puzzle. Utilising A/B testing comes hand-in-hand with other essential CRO tools.

Other CRO tools/ strategies to incorporate

User Surveys and Feedback:

Gathering direct feedback from customers via surveys or interviews provides strong insight into their preferences and pain points. This qualitative data complements A/B testing results, boosting understanding of why certain changes led to specific outcomes.

Heatmaps and User Behaviour Analytics:

Tools like heatmaps and user behaviour analytics offer visual representations of how visitors interact with a website. These insights highlight areas that require attention and help analysts identify usability issues or bottlenecks in the conversion process.

Continuous Iteration and Optimisation:

CRO is an ongoing process, and no website will ever be beyond further optimisation. Continuously monitoring A/B test results and user data to identify new opportunities for improvement. Websites must be regularly updated and optimised to adapt to changing customer behaviour.

In summary

A/B testing ensures that campaigns are performing at their optimum level. Understanding how to tweak campaigns and what subtle changes can make a dynamic impact will rocket eCommerce conversions. 

Exceptional digital experiences built on insight and strategy

Utilising data-driven decision making, the teams at Propeller are empowered to build smart strategies that engage your customers and build relationships that last. It’s about more than a beautiful website – as a strong growth marketing agency, we ensure every move, every click, every feature is defined by a clear goal.

With a unique expertise in developing digital platforms and growth marketing, our team works on integrating the best technology stack tailored to our clients.

Our teams’ expertise in growth marketing strategy is why we’ve consistently worked with a number of FTSE listed businesses for over a decade, developing and refining the brands’ digital growth as the landscape has changed.