Usability testing: myths vs reality

on 17th December 2018

(Last updated 6th March 2024)

Myth#1 - It costs loads of money

The truth is it can, but it doesn’t have to. The most important thing to learn about usability testing is that some testing is better than none. Even just testing a handful of users at the start of the project can provide a profound improvement on your products user experience, and to do this, you only really need to test a handful of users over half a day.

Myth#2 - You need to be a usability testing expert to conduct tests

Usability testing has a low barrier of entry. In the most popular testing method, the “Think aloud protocol”, all that is needed is to ask users to complete basic tasks on your product, discovering flaws you have been blind to.

Yes, it would help to be an expert. With more experience you will be able to extract more information from your participants, ensure you have removed bias and would further understand the nuances that make a test go smoothly, but these skills will develop quickly. Everyone is surprised how much useful information they gain from their first test, even if there are a handful of errors.

Myth#3 - You need to recruit lots of participants

Many people believe fairly, that to obtain useful test results you need to test loads of participants. In quantitative studies where the objective is to obtain as much data as possible for analysis, this is true, however, in quantitative studies such as the previously mentioned “Think aloud protocol”, the best return on investment is to only test 5 users. To find this magic number, usability stalwarts Nielsen / Norman Group conducted a study analysing how successful 85 of their usability tests were with varying numbers of participants and discovered most findings can be found with just 5 participants. You can see the results below.

Myth#4 - You need a usability lab

Ditch your idea of high-tech labs with interrogation room style mirrored windows. This kind of setup may be necessary for serious HCI studies and large-scale evaluations, but for most teams just trying to improve the experience of their product, it just isn’t necessary.  All you need is a meeting room, a test device and a dictaphone.


Myth#5 - It’s most important to test at the end of the project

Because you wish to iron out your designs biggest flaws before going live this may seem natural, but it’s much easier and far more effective to discover design issues earlier in the project. In “Don’t make me think” Steve Krug writes that “Testing one user early in the project is better than testing 50 near the end”. The truth is that while it may seem easy to go in and make changes to your website, most of the time it can cause havoc, especially if the issue is major. Test early and iron out the early flaws for best results! 

Myth#6 - There’s no point testing the existing website

There’s no better prototype of your new site than your existing one! While it may be tempting to wipe the slate clean, it can be extremely valuable in discovering the positives and negatives of the existing design that you may not currently be aware of. While you may think you know what users find most annoying about your product, the only way to truly discover what works and what doesn’t is to conduct tests. What if you were to remove a feature that kept users coming back or missed an opportunity to include a feature that will make your users lives ten times easier? Consider testing your existing product. As mentioned previously, it doesn’t have to take long and we guarantee you’ll be surprised by what you discover.

Do you know any other Usability myths that might put people of getting involved? If so we’d love to hear them. 

This article was posted in Design, User Experience