Alexa, our Senior Design Researcher, responds to Harry Brignull’s tweet about the pitfalls of being ill-prepared for usability testing. Alexa shares with us, therefore, how to bring multi-project teams together and how to get the best out of validation programmes.

usability testing


“When this Twitter conversation popped up on my feed I had to laugh: “Usability testing is not a washing machine”.”


It’s true though if you put something in that is not thought through, or coherent, it will not make sense in testing.

You will simply not get miraculous answers, a clear direction or great insight as an output.

We regularly run testing programmes here at the Nile office in Edinburgh and there have been several occasions when we have received a prototype the day before testing (or dare I say it the morning of the testing!) and it’s clear it’s has been thrown together in a bit of a panic.

This usually has less to do with the skills of the designers and more to do with the team who are creating the prototype being detached from the process of scheduling the testing or organising the project. In other words, more effective communication would help a lot.

What does more effective communication look like?

It is essential that all the different teams/agencies involved in producing this service or, proof of concept, are aware of the project plan, timescales, critical paths and sprint cycles. That way you can all align on what is expected and when. Particularly in the build-up to the testing, when working out hypothesis, key questions for the research, reviewing journeys, and discussion guide creation.

Other key stages for communication and collaboration:

  • During the testing, it is really valuable opportunity to understand people’s behaviour and create empathy – both required for good design!
  • During the results stage, collectively analysing the results is beneficial to any design project because the output should not be a shopping list of things wrong with the design

The prototype is not always to blame

You don’t need an elaborate high-fidelity prototype to get the insight you need. You just need one that clearly reflects the questions you want to test.

It’s more than possible to take a low-fidelity prototype into testing and gain great insight from it, as long as it has some sort of clear journey or context for the participant to comprehend.

Without this, it is just too confusing and effortful for the participant to follow what is going on, or understand what is an error in the prototype, or what is a planned route in a journey versus simply an erroneous page because someone wasn’t sure if it should be there or not.

Using testing to make tricky political decisions

Unlike Harry Brignull (referenced above – Twitter) however I do think user testing can be used when there are tricky political decisions to make.

Gathering user feedback can help you build a case which can be used to persuade stakeholders from what you might have originally thought was an unmovable position, to reconsider.

Don’t be tempted to test all the arguments though. It can be useful to take two versions of something into testing or occasionally three. However they have to be significantly different enough for a participant to perceive. The team needs to understand why it is being tested, have a shared knowledge of its purpose.

And remember, no matter how many prototypes or versions you test, you’ll still have to make a decision at the end of the day


So, having read this, you’re committed to not treating your testing like a washing machine. How can you make sure that you are getting the most value from your validation programme?

We’ve come up with some good questions to consider prior to building your prototype and embarking upon testing:

  1. Have we agreed on timings and communication channels?
  2. What is the overall objective of this prototype, what is its purpose what is the purpose of wider testing?
  3. What’s the user story / job that needs to be done?
  4. What are our hypotheses?
  5. What will we do with the insight/recommendations?

Of course, we are always on hand for our clients when they have concerns about their testing programme, and we are open to new clients who wish to run testing for their build work.

Read more about usability testing and customer experience design or if you have any questions about user testing at Nile, please do get in touch.