Why A/B Testing? Why does it make sense to introduce and use this new method? In order to better understand and analyze customers, a new testing method needs to be established. A/B Testing helps to answer questions that cannot be measured by qualitative methods. Analytics tools often reach their limits with before/after comparisons – the fluctuations of visitors on a store can never be eliminated as cleanly as with a classic A/B test. However, the purchase of a tool alone does not solve all problems. It is often forgotten that there must also be a process behind it within the company. But what does an A/B testing tool offer?
A first challenge is the purchase of the tool. In my use case, there were quite a few privacy issues that needed to be resolved first. Further a first A/A test of the tool followed to make sure that the traffic split works correctly and shows clean results. It is recommended to implement the tool with a first test case. It is best to take a simple hypothesis (see practical example).
At the same time, the implementation of the testing process in the company is also a challenge that must be faced. In addition to purchasing a tool, a process for this must also be created and established in the company.
One challenge is of course to design a process, but the bigger challenge is to establish it in the company. How can all employees tip in ideas? How is it evaluated and prioritized? There is probably no one-size-fits-all solution. It’s a constant learning, adapting & responding flexibly. I believe processes have to grow with the company. What helps a lot is a standardization for scoring and evaluating the tests. Also, a communication concept for the presentation of the results has proven to be useful in the concrete use case.
During customer interviews as well as mouse tracking, it was found that customers do not scroll much on the PDP. Most see – if at all – only 50% of the page. Web analytics data also confirmed this.
To conduct a test, a test concept is created and evaluated – with the following points:
The implementation of a new tool was one of my first big projects I did independently. From contacting the service provider, supporting the implementation, coordinating all deadlines & processes as well as establishing the prioritization process. In doing so, I learned that data protection is an extremely important topic and that even a new process that is introduced in a company usually needs some changes and feedback loops until it works. Of course, a process is never perfect and must be continuously developed.
In addition to all the process-related topics, I am now naturally good at designing, coordinating, executing, evaluating and presenting A/B tests in practice.