Test a prototype with real people

You get a link you can send to participants. They work through the tasks you set on your prototype, and you see how many finished, how long it took and where they clicked wrongly.

What a usability test is

A usability test asks people to do something with your screens, such as "Find where to change your plan", and records whether they could. It tells you where the design gets in their way. Research Mate runs it without a call: the participant opens the link, with no account, and works alone. You can also watch a participant use the same link on a call.

A test belongs to one prototype board and covers the screens on one page of it. Screens on other pages of the prototype are not part of the test, and the set-up lists only the screens on the page you opened. To test a flow that crosses pages, put its screens on one page.

Make a test

Open a prototype board.
Click Test, at the top right.
Click New test.

The test opens, named "Untitled test", on the Set up tab. The same menu lists the board's tests, each with its status (Draft, Live or Closed) and its number of sessions, and clicking one opens it.

Set it up

A draft saves as you go: a field when you leave it, a task when you pick or finish typing. If a save is refused, the field goes back to what was saved and a message says why.

Title: shown to participants on the first page.
Objective: for you and your team. It is never shown to participants.
Introduction: shown before the consent text.
Consent text: what a participant reads before agreeing. It starts with a plain default. Read it and change it to suit your study, see What is recorded below. Participants must agree before anything is recorded.
Thank-you message: shown at the end. If you leave it empty the participant sees "That was the last task. You can close this page now."
Participant limit: how many participants may take the test, 1 to 200. It starts at 30. When it is reached a new participant sees "This test is closed". A session you delete (see Read the results below) stops counting towards it.

Tasks

Under Tasks, click Add task. A test has up to 10. Each task has:

What the participant is asked to do: the prompt, up to 500 characters.
Starts on: the screen the participant begins on. It starts as the first screen of the page, or the screen a flow starts on if you have set one.
Success: reaching any of these screens: tick one or more screens. The task ends as soon as the participant reaches any of them. The start screen is not in the list, because it cannot also be a success screen.
Time limit in seconds (optional): 10 to 1,800. When time runs out the task ends as timed out. Leave it empty for no limit.

A task that is not complete stays on screen with a note saying what it needs, such as "Pick at least one success screen to save this task.", and is not saved until it is. Remove deletes a task. Participants get the tasks in the order you list them.

Preview it

Click Preview, beside the title. The test plays full screen as a participant would see it, and "Preview: nothing you do here is recorded." is shown above it. Click Close preview, at the top, to leave at any time. Preview is off until the test has a task.

A draft previews the board as it is now. A published test previews the copy it was published with.

Publish it

Under Publish, choose Link works for: 7 days, 30 days or 90 days. 30 is the default. Then click Publish test. It is off until every task is complete and there is at least one.

Publishing freezes the tasks, the wording and the screens as they are now, and makes a link anyone can open. Later changes to the board do not reach the test, and a published test cannot be edited. To test a changed design, make a new test.

The link appears under Participant link, with Copy. It is shown once: "Shown once. Copy it now: it cannot be shown again." Leave the page and it is gone. If you lose it, you cannot get it back, and a new test makes a new link.

A test link is not a share link. A share link shows a presentation. A test link gives a participant the tasks and records what they do. See Share a presentation with a link.

What the participant sees

The title, your introduction and the consent text. Nothing is recorded, and no session exists, until the participant presses Agree. Decline ends the page with "You chose not to take part" and "None of your answers were recorded."
Each task in turn. A page shows "Task 1 of 3" and the prompt. The participant presses Start and the prototype appears under a bar with the task, a Give up button and, if you set a limit, the time left.
The prototype. They click through it as in Present. There is no project or board name, no Why or Process layer and no keys for stepping through the screens.
The end of each task. Reaching a success screen ends the task at once, and so does Give up or running out of time. The next task's page says whether the last one was done.
Thank you. Your thank-you message.

Each screen is fitted whole to the participant's window, so a desktop-sized screen is too small to read on a phone: a 1,233 px wide screen at 390 px shows 12 px text at about 3 px.

A participant who closes the page stops. What was recorded before that stays.

If the test is full, the participant sees "This test is closed". If the test is closed, the link has expired, or the test or its board is deleted, they see a not-found page.

Read the results

A published test opens on the Results tab. A line says how many sessions there are. Click Refresh to read new ones, because results do not update on their own.

Each task has a table:

Completion: how many attempts ended on a success screen, out of all who started the task, such as "6 of 8 (75%)".
Median time: the middle time to success. Only successes count.
Middle half of times: the range between the quarter and three-quarter marks of those times.
Misclicks: clicks on something that did nothing, in total and per attempt.
Gave up, Timed out: attempts that ended that way.
Abandoned: attempts with no end, because the participant closed the page or went quiet.

A participant who has not finished and started within the last four hours is in progress: a line shows "2 in progress", and they are left out of the numbers and the list until they finish or the four hours pass.

Under Sessions, each participant is P1, P2 and so on, in the order they started. A label box ("Add a label") lets you note who it was, such as "Sam, recruited 12 Nov". Labels are seen only by your team. Each row shows when it started, whether it is finished, and how many tasks were done.

To remove a participant, for example because they withdrew, click the bin button on their row ("Delete P1"). They move to Deleted sessions and drop out of the numbers. Click Restore within 30 days to bring one back. After 30 days it is removed for good, with its clicks.

Close a test

On a live test's Set up tab, under Close this test, click Close test and then Close test again to confirm. The link stops taking new participants. A participant partway through keeps their answers. Closing cannot be undone, and the results stay.

Deleted tests

To delete a test, open it and click Delete test, then Delete test again to confirm. You go back to the board. A deleted test stops answering its link at once. It is kept for 30 days, with its sessions. To bring it back, click Test on the board, find it under Deleted tests and click Restore. After 30 days it is removed for good, with its sessions and clicks.

A test follows its board. If the board is deleted the link stops working, and it works again when you restore the board. See Delete and restore a board.

What is recorded

Once a participant agrees, a session records:

The clicks they make: which screen, where on it, which element, and whether it did anything.
Which screens they see, and when they start and end each task, as time since they agreed.
The width and height of their window, and whether they use a touch screen or a mouse.
When they agreed.

It does not record a name, an email address, an account, their voice, their camera, their screen or their IP address with their answers. The participant is a number such as P1, and a label only you add. Search engines are told not to list the page.

The default consent text says in plain words that clicks, the screens seen and the time each task takes are recorded, and that a name, voice, camera, screen and IP address are not. It does not mention the window size or touch or mouse, so add a sentence about those if you want participants told. If you change the text, keep it true: if your text does not carry the sentence "we do not store your IP address with your answers", the participant page adds it. You are responsible for getting consent from the people you test. See Limits and how long things are kept for the numbers.