Automatically Discovering Major Usability Issues in e-Commerce Sites via Machine Learning

Researchers
Dr Richard Jensen
Professor Qiang Shen
Dr Neil Mac Parthaláin

The Overview

Research undertaken by the Advanced Reasoning Group (ARG) at Aberystwyth University (AU) has led to the development of robust techniques for data mining that can handle uncertainty, incompleteness, and imprecision in data. This provided a foundation for work with UserReplay, a leading software company specialising in e-Commerce, who were looking to automate the discovery of usability issues in e-Commerce systems as part of their analytics solutions. UserReplay and their clients have benefitted commercially from this research due to reduced losses, increased automation, and competitiveness. Competitors subsequently had to make similar improvements leading to further economic impact. Ultimately, customers using these sites benefitted from an improved experience. 

The Research

New data mining algorithms have been developed to effectively and efficiently discover usability issues in e-Commerce sites employing UserReplay’s analytics solutions, saving significant amounts of money for businesses globally. Cases have shown estimated annual savings of over $86M.

The technology is developed from data mining research at AU. The ARG has a strong track record in the development of robust and effective methods for various steps in the data mining process, such as feature selection, instance selection, missing data imputation, and rule induction. This has been achieved through the hybridisation of fuzzy sets (that model vagueness and noise) and rough sets (that model indiscernibility).

Having recognised this, UserReplay approached AU and, through successful collaboration, subsequently secured an Innovate UK grant in March 2016. The ARG provided scientific input to and technical support for this project.

The total estimated annualised revenue opportunity is $86.2M. The machine learning technology is used to record, analyse, and segment over 2.5 billion user sessions a month, which would be impossible to achieve manually.

The Impact

Reduced Losses and Better Customer Experience

The approach discovered many usability issues potentially costing e-Commerce sites significant revenue. In one instance an anomaly was found on a customer’s website where multiple presses of the “Buy” button were occurring, affecting up to 40% of possible transactions. This bug was undiscovered by prior system testing or human analysts and would have cost millions if it had remained active during the Black Friday trading season. Another example saw stock being reported incorrectly on the website, preventing customers from completing their purchases, resulting in an estimated annualised revenue opportunity of nearly $1.5M. In another example, customers were unable to pay when one of the items in their baskets had sold out, which led to cart abandonment. Fixing this problem was estimated to save nearly £1M per annum.

From these cases alone, the total estimated annualised revenue opportunity is $86.2M. The machine learning technology is used to record, analyse, and segment over 2.5 billion user sessions a month, which would be impossible to achieve manually. The problems flagged by the analytics include usability issues, technical issues, and fraudulent activity, demonstrating the range and complexity of issues that can be detected.

Adoption of Machine Learning

Machine learning is at the centre of UserReplay’s latest product, which performs analytics and session replay for mobile shopping apps. This uses the data mining developments described above and their associated additional tools. The technology today is deployed across at least 22 companies globally, including some of the Fortune 500 in the USA. Further to the substantial commercial benefits that UserReplay’s clients have been afforded by the research, end customers also have an improved experience when using the e-Commerce sites.

Patent

A patent was filed by UserReplay detailing the above approach and an extension using neural networks to aid prediction. Following on from the successful R&D conducted with Aberystwyth University, the company has focused more on data mining and machine learning, recruiting full-time machine learning experts.

Get in touch

As a University, we’re always keen to share our knowledge and expertise more widely for the benefit of society. If you’d like to find out more or explore how you can collaborate with our researchers, get in touch with our dedicated team of staff in the Department of Research, Business and Innovation. We’d love to hear from you. Just drop an e-mail to:

research@aber.ac.uk

Research Impact Case Studies | Research Theme: Technology