logo de lucie figuet

LDLC.com UX funnel

Setting up of usability tests

LDLC usability tests

LDLC Group is a french online business group, created in 1996, ranked 5th in France by FEVAD in 2016. Its major brand, LDLC.com is a major player in e-commerce computing and high-tech in France. In addition to the common characteristics of most e-commerce websites (top of the category sales, online product reviews, etc.), the site quickly set up a faceted search to facilitate the search for products such as motherboards, RAMs or monitors whose offer is sometimes in hundreds of models. The site also has the particularity of offering computers delivered without a pre-installed operating system.

About the mission

During my time at LDLC, I was asked to set up usability tests to evaluate our solution and be able to identify the real problems of use that our customers might encounter. At the time, no test of this kind had ever been set up. Honestly, I didn't really know how to start and I was given complete freedom to choose how to proceed. I remember telling myself that there was mostly 2 ways to do it. The first one was to search for particular issues using analytics or expert evaluation for example and then, target the most critical ones to fix them. The second approach was much broader and was based on the main user goals such as buying a product or check for the availability of a product in a store. This is the second option I chose because it would allow me to observe the participants in more realistic conditions. Furthermore, I could also, at the end of the user tests, ask everyone to describe me the most common things he does on our website.

LDLC usability tests
Sketching the experience

Project process

I first decided to use the 4 personas created earlier and for each of them, I defined two relevant scenarios to be used during the user test.

LDLC usability tests
The main persona, Tom
LDLC usability tests
Matthieu, the nerd
LDLC usability tests
Maud, Mrs Everybody
LDLC usability tests
Stéphane, the professional

Each scenario described a particular goal to accomplish (like "buying a BlueTooth speaker"). Except one or two constraints maximum (like "for 65€ or less" or "pay by credit card"), the participant could choose how he intended to perform these tasks.

LDLC usability tests
Example of a scenario document

    To set up these tests, I had to write some technical or legal documents :


  • The scenario sheet given to the participant included the constraints he had to respect, but also a little intro text to thank him for participating, as well as a sentence reminding him that it was not him who was tested, but the website.
  • A usertest guide completed by the test observer gathering the collected information (participant personal informations, description of each action taken during the test, duration, encountered errors or blockages, quotes). The usertest guide also includes a DEEP questionnaire (Design-oriented Evaluation of Perceived Usability) and some final questions to have the participant's opinion about the test itself.
  • A consent form explaining the conditions of the test, the anonymous, confidential, and provisional nature of the collected data, and the right for the participant to withdraw from the study at any time and without justification.

Next step was to organize pre-tests to check that anything was missing. It was therefore necessary to make requests for computer equipment to the general services of the Group, reserve a meeting room, find volunteers for the pre-tests, and then test the material, softwares, duration and content itself of the test. Meanwhile, I asked the communication service to help me recruiting users inside the company (while we were over 700 employees) according to some criterias defined by the 4 personas, so the participants could perfectly match with them.

    These criterias were :


  • The age of the participants
  • Their socio-professionnal category
  • Their internet use frequency
  • Their interest in high-tech and computing
  • The frequency of online shopping on our website
  • Are they already a customer or not
  • Their most used devices

For each of the 4 personas, we recruited 5 participants. We decided to meet 2 of these 20 participants a day, to organize a user test in the morning, and one in the afternoon, in order not to exceed 2 weeks of consecutive user tests.

    Then came the analysis of the test results which included :


  • The number of errors made for each scenario
  • The number of actions relative to the minimum number required to perform the task
  • The average time taken for each scenario
  • The results of the interviews and participants relevant quotes
  • The DEEP questionnaire results
LDLC usability tests
The DEEP questionnaire

Results

After having analyzed the results, we finally prioritized the different corrective actions with the marketing and IT teams. In an iterative and continuous improvement approach, these user tests are now reused to improve the customer experience. They are also completed by other kind of user tests like "in situ" tests.

You have a project ?
Lucky you, I've got an email address !