3. Evaluation

WHEN

Use iteratively throughout the development process, but particularly as a design progresses towards the final idea.

ISSUES

• Choose users carefully: Make sure you have a wide range of lead users within your target market
• Choose whether to test ideas with people that you have already conducted research with in order to make use of their familiarity with the project, or whether to use new people for a fresh perspective
• Do you test for a couple of hours or ask people to test for longer periods?

STRENGTHS

• Feedback from a user perspective on a full range of design characteristics including aesthetics, function, ergonomics, user-friendliness etc.
• Can test ideas at varying levels of completeness -
• Brings fresh opinions to a design that you are by now very familiar with
• Can catch obvious design flaws early on, giving you time to address them
• Ideas tested in “real-world” scenarios. Lead users will test the idea in unexpected ways, adding a robustness to your approach
• Enables comparison of your potential solutions against existing ideas from competitors to benchmark your idea before it goes to market

WEAKNESSES

• Users can be influenced by the appearance and sophistication of the prototype. This may bias the feedback on other aspects of the design 
• Feedback from users may influence the design in favour of their own specific needs
• Be careful when basing design decisions on the opinions of a small number of users. You must retain the role of editor. Not everything that every user says is correct
• Users may lack the experience or training to give useful feedback. Many may just say “It’s OK” and not articulate any further
• Testing out of context will provide limited insight, while in-depth testing in context can be timeconsuming to conduct

HOW

1. Start testing as early as possible before too many decisions have been made. This will save you spending a lot of time on solutions that do not work.
2. Plan what you want to test, and tailor this to suit the current stage of the design work. An early test may include an 10-minute assessment of simple functionality. Later tests may include in-depth testing over several weeks.
3. Increase the scope of your evaluation as the design progresses. Start by testing the first impressions, and build up to use over a longer period of time.
4. The number of users will vary depending on the scope and nature of the design. Aim to find 8-10 different users.
5. Test with one user at a time whenever possible to get more focused feedback.
6. Try to test alternative solutions. It is often easier to give comparative feedback than to assess a single design.
7. Try to test the idea in context and in a variety of circumstances.
8. Test one time too many rather than one time too few. Your users’ knowledge and growing experience of the product is a valuable resource.
9. Document the testing through video and photographs. This is important for analysing and sharing the information with the rest of your team.

GOES WELL WITH

• Interview: ask users to give feedback on prototype features in relation to current products and experiences
• Controlled observation: when introducing a prototype or setting tasks to test functionality, use controlled observation and ask users to verbalise their reactions and thoughts
• Research kits: suitable for lengthier tests carried out in context such as an individual’s home. A research kit could be developed to capture feedback whilst the prototype is being used

BEST SUITED TO

• Checking the design throughout the development process across the full range of characteristics from function and ergonomics to materials and finish
• Comparative testing of alternatives to focus on the best solution to take forward
• In-depth testing of an idea before going to market