769 | Add payment method icons to the shopping cart | Desktop
In this experiment, we tested the addition of payment method icons to the shopping cart on a desktop website.
This hypothesis is grounded in the following psychological principles:
-
Reduce Uncertainty: Displaying all available payment methods can alleviate doubts or questions about payment options, which is especially important for new visitors who have not yet established trust with the website. By clarifying the payment process before the final checkout step, we aim to minimize the cognitive load and decision-making stress for the user.
-
Attract Attention: Strategically placed icons can draw the user's eye and emphasize the variety of payment options, making the information more salient and easily processed.The visual representation of payment methods can serve as a heuristic cue that the site is trustworthy and professional.
Based on this background, we believe that adding payment method icons to the shopping cart for desktop visitors will cause an increase in the number of visitors proceeding to checkout and completing transactions.
The experiment did not achieve the expected positive outcome. Across all tests, the addition of payment method icons consistently showed a negative impact on transaction rates.
Possible explanations why the impact was not as hypothesized:
- Information overload: Adding icons might have distracted users or created unnecessary cognitive load.
- Misalignment with expectations: Users might expect to see payment methods only at checkout and found it out of place in the cart.
- Perceived complexity: The icons could have unintentionally suggested that payment was required immediately, leading to hesitation.
All A/B tests in Evidoo have been analysed using Bayesian Statistics. The most important advantage of Bayesian statistics is that it is easy to understand. If there is a difference between the control and the variant, we determine the probability that there is a difference. The probability that the variation differs from the control, is indicated in a percentage.
An A/B test labeled as ">80%" (winner), indicates that the hypothesis has a high probability (>80%) of being true.
An A/B test labeled as "21 - 79%" (inconclusive), suggests the hypothesis has an intermediate chance of being true. This probability range indicates that there is still uncertainty regarding the hypothesis. Therefore it can not be clearly categorized as true or false.
An A/B test labeled as "< 20%" (loser), likely represents a hypothesis that has less than a 20% chance of being true. This suggests that the hypothesis is likely false.
If the primairy KPI is ‘transactions’, there is no impact on average order value, unless mentioned in the learnings.
All A/B tests in Evidoo are also analysed on secondary KPIs, for example 'add to carts'. If we found remarkable results on other KPI’s, check the tab 'learnings'. We also analysed different segments, for example 'new visitors' or 'returning visitors'. If we found remarkable results on specific segments, check also the tab 'learnings'.
All conducted A/B tests in Evidoo comply with:
- conducting sample size and power checks.
- performing the experiment only after an A/A test has been completed.
- implementing Sample Ratio Mismatch (SRM) checks.
- maintaining a minimum runtime of 2 weeks.
- measuring A/B tests only when they are visible to the website visitor, such as counting an A/B test in the middle of the product page only if the visitor has scrolled to that point.