Setup A/B Testing in Sitecore - Part 1

 

Setup A/B Testing in Sitecore - Part 1


In this article we will understand What is A/B Testing, Types of Testing in Sitecore, Actual Use Case and later we will see how we can setup A/B Testing in Sitecore XP solution with Rendering having different data source items. 

Before we go further, lets understand WHAT IS A/B Testing?

A/B testing, also known as split testing, is a method used in marketing and product development to compare two versions of something to determine which one performs better. In an A/B test, two versions, A and B, are created, with one varying element between them. This element could be anything from the wording of an email subject line, the design of a web page, the placement of a button, or even the pricing of a product. 

The versions, A and B, are then randomly shown to similar audiences or users, and their responses are measured and compared. The version that performs better in terms of predefined metrics (such as click-through rate, conversion rate, or revenue) is considered the winner and is often implemented or further optimized. 

The original version is called Control and another version is called as Variant. The Idea is to determine which content is better suitable for audience. This is being used to eliminate the guess work, instead coming to a decision based on statistics data.

 


 

As visitors are served either the control or variation, their engagement with each experience is measured and collected in a dashboard and analyzed through a statistical engine. You can then determine whether changing the experience (variation or B) had a positive, negative or neutral effect against the baseline (control or A). 

Types of Testing in Sitecore

Below are the available tests in Sitecore: 

1. Basic Content Tests: A basic content test compares the old version of a page to the new version of a page. 

2. Component tests: Component tests let you test alternative combinations of your website's content to find out which components or combinations of components are the most effective with visitors. This we might discuss in our later blog. 

3. Page tests: A page test tests a complete page against another page. The pages in the test can be different versions of the same page, or they can be completely different pages. Page tests are different from content tests. I will be discussing this type of test in this article.

Actual Use Case

Recently, we got an requirement to understand  Does a user changes his account number upon showing the alert box after validating it thru some API? So we created a A/B Test (Page Test) on home page in which two different version of forms were shown thru MVC Form component. The original version i.e Control was not showing any alert box but the second variant shows the alert box upon entering the account number. 

After completing the tests, the stats showed that Variant B, which shows the alert box, helped user to identify the incorrect account number and they changed it before final submission of form. So in that case we finalized the Version B and made it live. 

Setup A/B Test in Sitecore

A/B Tests are the part of Experience Optimization feature under Marketing Applications section on Launchpad:

 



For our A/B test, I have a created a component named Article which accepts a data source item having the content in it. I will be creating 2 version of a page item having the Article component applied on it having different data source item. 

Lets understand the practical implementation in my upcoming blog.

Comments

Post a Comment

Popular posts from this blog

Working with Device Detection Module in Sitecore

Image
  What is Device Detection in Sitecore? Sitecore has an in-built feature called Device Detection which helps to detect at which device the content is shown into. When I say in-built that means its included as a standard feature in Sitecore XP 9.0 and later. This feature is also present in Sitecore XM Cloud , hence you do not need to buy a separate add-on or module to utilize device detection features.   The information gather from Device Detection Module can help content makers, allowing them to tailor the website experience based on the device type they are accessing it with, such as by displaying different layouts or content depending on screen size and capabilities. For Example, An iPhone and Android users might be looking at different content on same page.   Moreover, developers uses the same mechanism to personalize the components based on the device specific audience. They can setup Device Layout based on specific device parameters or can use the in-built rules...
Read more

Working with Webhooks in Sitecore - Part 1

Image
  Working with Webhooks in Sitecore - Part 1 In this blog we will understand the concept of Webhooks and how we can use them in Sitecore. So before jumping into Webhooks in Sitecore, first lets understand what webhooks are?   What is a Webhook? They are typically created to connect/link different applications. They are much like API but different from them. Webhooks are one way that apps can send automated messages or information to other apps.   Consider an example below:     Suppose, there is a user facing front application which showcase the product catalogs to its customers. This application is responsible for booking an order from its users. BUT the logistics is handle by different application.   The second application (logistic app) has setup a webhook which is exposed over http and waiting to receive any confirmed orders. Once someone place any order which can be called as an event for the first application, then along with its operation it is passin...
Read more