A/B testing
A/B Testing
A/B testing, also known as split testing, is a method used in marketing and development teams to compare two versions of a web page and see which one performs better. If you have a User Journey checking a website that uses A/B testing, this essentially means you are testing two different websites.
This can cause User Journeys to fail because the two websites might have completely different selectors, or follow a different set of interactions. The easiest way to identify A/B testing in User Journeys is by comparing screenshots of a passing and failing test. Additionally, you will find the site fails intermittently about half the time depending on which version of the site is loaded on each test.
A User Journey with A/B testing. The journey seems to fail intermittently and about half of the time.
Fixing A/B Testing Failures
You will need to add logic into the script that checks which version of the site the User Journey loads. This is called an IfElement. By using an IfElement you can tell the User Journey to perform different actions or different selectors based on the website version. For example:
- Website A:
- Wait for the selector
.smallButtonto load - Click the button
- Wait for a form to appear
- Type the details into the form
- Wait for the selector
- Website B:
- Wait for the selector
.largeButtonto load - Click the button
- Wait for a form to appear
- Type the details into the form
- Select an option from a dropdown menu
- Wait for the selector
You might have similar websites where just one element is different. For example
- The User Journey
- Waits for the selector
.buttonto load - Clicks the button
- Waits for a form to appear
- Waits for the selector
- Form for Website A:
- Type the details into the form
- Form for Website B:
- Type the details into the form
- Select an option from a dropdown menu
Using an IfElement, you tell the User Journey that if it sees a selector from site A or site B then it should proceed with a specific set of actions. You can include up to 7 if conditions.
Enter the selector for each condition and set the time the journey waits for it. You will then have an empty if element.
You should now add actions to each condition. Lets turn the first example into an IfElement:
As you can see, the IfElement is divided into Website A and Website B, with different selectors actions for each. If the User Journey encounters Website B, rather than failing, it will complete only the actions that are specified within the Website B IfElement.
Keeping it Simple
Sometimes, you might have similar websites where just one element is different. Let’s imagine a different example than the one above. In this case, only the form would need to be included within the IfElement:
- The User Journey
- Waits for the selector
.buttonto loaUnchanged: - Clicks the button
- Waits for a form to appear
- Waits for the selector
- Form for Website A:
- Type the details into the form.
- Form for Website B
- Type the details into the form.
- Select an option from a dropdown menu
You would only put the forms within the IfElement, not the whole journey.
Only the form is inside the IfElement as that is the only difference in site A and B.
It is always worth keeping IfElements as simple as possible because it helps debug your script later. You should consider how different the two versions of the site are. If only one thing is different, you don’t want to script everything twice.
You should also still make sure to use Steps and Actions as intended, with one step recommended for one page on your site. Don’t be tempted to put your whole user journey into one IfElement! This not only is hard to read and debug but will limit your ability to view test results by step. Instead, use multiple IfElements for each step where there are differences between your sites.
Read more