Testing and evaluation

PHASE FIVE

Test and evaluation

will it work?

Start!

how?

Intro

why?

Tools

index

Introduction

Looking at your prototype, what a sense of accomplishment! It has been a long way and now you have developed an actual soon-to-be-product that will disrupt the world! But, will it? You will never know until you test it. Design thinking is a methodolgy and an approach to create innovative, user-centric products. It is a living process that is constantly evolving. This final phase "test and evaluation" is a stage where you purposfully design a testing secnario to reveal areas of improvment and futher research.

Testing leads to failure, and failure leads to understanding.

- Burt Rutan

+ info

Why do we test?

Save time and money

Improve the prototype

Learn more about users

Refine the problem statement

How to test?

Define Goals and Scope

Iterate and re-test

Who to test with

Conduct the test

Consider the logistics

Built it - Break it - Fix it

Feedback Grid

tools

Wizard of Oz

Usability test

Build it- Break it - Fix it

Often used in software development, "Build it - Break it - fix it" aims to stretch and test a prototype under extreme situation and make it fail-proof. Build it - Turn your concept into a prototype FAST. Don't drag on or wait for more market analysis. Build it, even if it is not perfect Break it - Once you have your prototype, find someone (users, colleagues) and ask them to break it. Essentially, you want identify the weak points and things that are not working well. Fix it - Analyse your "broken prototype". Look into things that broke and also things that bent. Use the "5 Why" to find the root cause and re-bulid it.

+ info

Wizard of Oz

No one wants to spend all the money and time to build a full prototype that no one likes. WELL WELL WELL, call on the Wizard of Oz to do the trick for you! Mostly used to test digital systems, once you decide what function you want test and find a way to fake it (by using tablets, email systems, online chat etc.) Make the user thinks the response is computer-driven, when in fact it’s human-controlled. Voilà! Now you can observe and collect users responses without a fully functional prototype!

video

+ info

Usability Test

A usability test or user test is to let selected users test your prototype, observe their behaviour and reaction, and identify weaknesses and potentials from the user's point of view. Usability test is an essential step of product development. The sooner the prototype is put in front of a real user, the earlier and easier you can improve. There are many types of usability tests, like

Moderated and Unmoderated Test

Each usability test is differently. The key is to pre-define what questions you want answers for.

+ info

video

sample questions to get you started

Moderated

Unmoderated

vs

+ info

+ info

Feedback Grid

A 2x2 Feedback Grid is simple yet effective way to capture the responses and insight from users. Ask participants four questions about their user experiences with your prototype: 1. What do you like about it? (Likes) 2. What would you change? (Wishes) 3. What questions came to mind during the test? (Questions) 4. Any new ideas? (Ideas)Once you have collect all the feedback. Analyise them, look for patterns and theme, then take actions and adjust!

video

+ info