User testing: How to recruit the right participants, as it won’t happen by chance!

One of the most common comments we hear from clients straight after a usability testing session is about the quality of the respondent they have just seen. Clients are happy if the respondent is chatty and vivacious but there is much more than that to being the right participant for the project. Openness is just one of the key ingredients to being a good participant. Here are some of our top 5 recruitment tips, gleaned from years and years of research, which will help ensure the highest possible calibre participant.

 

Your ideal respondent is out there but you must be mindful that they need to found on time and on budget. In order to do this you have to make the pool of potential participants as big as possible.  Unless it is absolutely critical to the project the one situation you want to avoid is trying to find a needle in a haystack. In recruiter speak, you want the ‘strike rate’ to be high as possible.

Tip 1: Establish the right behaviour parameters

It is important to think carefully about the correct screening questions. Do the participants have to exhibit certain online behaviour, do they need to be making a decision about your brand imminently or do certain people need to be excluded if they are brand rejectors? Recently we tested a car website and it was important that we spoke to non brand rejectors. If we had not explicitly excluded brand rejectors, we could not be certain their views on the brand website were not clouded by their unfavourable disposition to the brand (which would have been hidden).

The strategic objectives of the project will dictate the necessary parameters but each and every one needs to be justified. Irrelevant or unnecessary parameters will hinder the recruitment process, it may encourage potential participants to stretch the truth (i.e. lie) just to qualify or the right candidates may get bored and disinterested. When looking at the parameters set down ask yourself, “Do these type of people actually exist in the real world?” Any skilled researcher will challenge and promote the best parameters to use in the test screener.

Tip 2: Match the panel type to your audience

There is much debate about whether to keep the recruiting in house, use a client list or a recruiter panel and it usually boils down to a ‘horse for courses’ type decision. Clients’ lists can be useful to fast track to people with particular behaviours i.e. have a certain bank account, inquired about trialling one of your products or made a donation to your organisation recently, however, the downside can be that they may not want to participate in research. You will also need to be certain that they have given permission to be contacted. In order to be useful the list must contain current phone numbers. Typically strike rates on clients’ lists can be on the low side, which will mean that more time will need to be allowed in the recruitment process and therefore the recruiter may charge you more!  When recruiting from a list the organisation’s reputation is on the line, so everything must be above board or you may risk losing more just the right participant.

With the development of Research 2.0 there is growing interest in creating online panels for recruitment. This area has much potential and organisations need to be aware that resources need to be dedicated to it to ensure a high quality of membership. Time must be allocated to ensure member details and preferences are up to date. There will also be a need to constantly grow this database, as participants should really only attend 2 testing sessions per year (AMSRS guidelines: http://www.mrsa.com.au/files/Qualitative_Recruiters_Guidelines.pdf).

The online panel audience will also be somewhat skewed to those that enjoy being online and contributing in cyberspace and this might very well exclude your ideal target market. With the advent of Research 2.0 the key sticking point is the handover of power, or more the increasing equality of power between the researcher/recruiter/client and the audience. If you are comfortable with sharing of power then this may well appeal to your business operation.  Some organisations have recruited via posts on Facebook and this is fine if there is a high penetration of Facebook usage amongst your target audience, or a high level of online savviness!

If you do not have the resources or inclination to create your own panel, existing recruiter panels remain a credible source of test participants. Typically they have large (and willing) databases that they can call on in a timely fashion. They also have the greatest breadth of participant types covering a large variety of socio economic variables and locations. This channel might not appear to be cutting edge however it does provide that critical big pool of passionate participants who will participate. Often government projects require the inclusion of a broad range of participant types (especially without a bias towards technology) so recruiters’ panels are ideal here.        

Tip 3: Make it convenient

So you have carefully crafted the right audience the next challenge is to get them to the test. People are busy and although you may spend your working day pouring over your project, participants have other priorities. Never underestimate the amount of notice that you need to provide, at the very least it should be 7 days. The venue should also be easy to get to. Ideally it should be close to public transport or where it is easy to park and if participants are travelling at night they need to feel safe in the testing location, females and older participants especially. Locations maybe convenient for the client team but if it’s not for participants you will invariably lose them. Consideration must also be given to the audience’s lifestyle routines – household responsibilities like school drop offs and pick-ups, holidays and even important sporting events will have higher priority!  When recruiting for a children’s entertainment website our testing happened after 4pm in the afternoon to accommodate school hours in a central location where it was easy for parent to park their vehicle. It made all the difference. 

Tip 4: Show respect

Everyone’s time is precious so demonstrate that you understand this. Compensating participants for their time and effort is crucial. The old adage, ‘you pay peanuts, you get monkeys’ holds a lot of currency here. Tap into recruiter’s expertise here in determining what is an appropriate amount. They deal with a vast array of audiences across a broad range of topics. They will be able to provide guidance on what to pay when. Scrimping on incentives may provide a short-term gain but it may have an adverse effect on the quality of your participant and in turn your insights. If you are really having trouble getting people to agree to participate you are probably not paying them enough! Typically for a one-hour testing session we recommend participants be offered between $80 and $100.

Tip 5: Don’t assume anything

When inviting participants along to a testing session, especially a website usability test, it is imperative that the recruiters are primed to check specifics. For example, that the participant is comfortable using the Internet, i.e. that they can use it comfortably without direct and constant assistance, or that they can comfortably use a mouse. The difference between what they think they can do and reality can sometimes be quite vast. In o
ne example a participant correctly answered that they used the internet at home however they neglected to share that they did it only with their teenager sitting beside them guiding through the process. Whatever requirements are mandatory ensure that they are confirmed long before the testing begins.

 

Great recruitment is an art form, which requires much thought and attention than simply getting ‘bums on seats’. We take great pride in ensuring we gather the best people for our user testing, if you would like to more about how we conduct our recruiting process or access to our recruitment templates please give us a call on 1300 858 015 or email us at lphillips@objectivedigital.com.

 

 

 

This entry was posted in Uncategorized by James Breeze. Bookmark the permalink.

About James Breeze

Objective Digital Holdings Pty Ltd has offices in Sydney (Objective Experience Sydney) and Singapore (Objective Experience Singapore). De-coding behavioural insights so your Customer Experiences are fantastic! Eye Tracking, CX, UX, Usability Testing, Shopper Research & Design Thinking across Australasia and South East Asia.

2 thoughts on “User testing: How to recruit the right participants, as it won’t happen by chance!

  1. Hello there just happened upon your blog from Bing after I typed in, “User testing: How to recruit the right participants, as it won’t happen by chance! | Understanding the User Experience” or something similar
    (can’t quite remember exactly). Anyhow, I’m relieved I
    found it because your content is exactly what I’m searching for (writing a college paper) and I hope you don’t mind if I collect some information from here and I will of course credit you as
    the reference. Thank you.

  2. Wow that was strange. I just wrote an really long comment but after I clicked submit my comment didn’t appear. Grrrr… well I’m not writing all that over again.

    Regardless, just wanted to say fantastic blog!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s