Understanding Usability

Previously, we conducted a user needs analysis project with a client in a particularly political environment.  This meant that every decision made during the redesign had to be well reasoned. The client needed to see exactly how each of the various features, functions and content items (FF&C) were understood during the user research process.  In particular they wanted to know;

”How do you choose the right cards to do a card sort with?”

This made me realise that, in many user centred design projects, the user research is simply used to educate the Information Architect (a person). Often the client doesn’t see an overt relationship between the research findings and the final design choices. They simply trust the Informationa Architecture (IA).

User centred research

To show the relationship between each research exercise and each FF&C I created a simple Excel spread sheet like this (click to enlarge).

Across the top I used the following headings:

User Research methods

• User focus groups suggestions
• Online Survey support
• Online user forum support
• Competitor analysis support
• Stakeholder suggestions from Face to Face Research
• External stakeholder suggestions
• Recommended content & features (cards for sort shaded)

Strategic decisions

• Priority (1, 2, 3)
• Justification
• Additional info
• Phase
• Responsible

Features, functions & content

Then I listed all the possible FF&C down the left, including:

• everything on the existing site
• all the stakeholders’ business requirements (preferences)
• competitor ideas
• requirements uncovered and tested, and
• new ideas.

Next I simply went through each FF&C and checked whether it ‘passed’ each user research ‘checkpoint’.

This can be done very quickly with a client in a workshop.  That way the client has full visibility of what is in or out in the design, and most importantly, why?

The last thing to be done is putting a priority on each FFC.

Just last week I used it for another client. We did less research therefore there were less columns. Here’s a partially completed example (click to enlarge):

Table of content, features and functions in IA

This method was incredibly successful!

It allowed us to generate valuable and insightful discussion with the client and their senior colleagues. In this case, the colours on the left were used to show the priority that people gave in the cards sorted in the face-to-face workshops.

By looking at the spreadsheet you can very easily see if each of the things that stakeholders thought they needed was also a requirement of users. And also what new ideas users had come up with, and whether they are in or out.  The list provides the information architect with a checklist, a heuristic framework, to ensure nothing is missed.  It also lets the client quickly see that everything is justified.

How do you choose cards for a card sort? Don’t just guess, make use of all of the user research that you have completed.

We recommend recruiting people with a professional market research recruitment agency. We use Farron Research*. Farron has been doing recruitment for us from the beginning and also recruit for many other usability and market research companies. It costs about $100 - $130 to find people from specific demographics for your tests.

It is generally not a good idea to do recruitment yourself. As it is very time consuming and can be very frustrating for the uninitiated.  There is nothing worse that being set up for testing and the participant not showing up. If you have limited funds, you can read our popular tips about recruitment blog post. 

Sometimes clients give us lists of their customers and this is totally fine. However, recruitment may cost a bit more with a list. This is because recruiters generally use a panel of known people. These people have signed up to be participants and expect irregular calls from Farron.  She manages the lists so that people do not do too many of the same types of tests. Those on Farraons list are profiled, so that Farron can select the right demographic for your requirenents. List recruitment usually costs a little bit more than using Farron's panel and people from lists are prone to not turning up to testing :)

Another reason to use a professional recruiter is that they are very strict when it comes to meeting the criteria. Their screeners are carefully put together and only include people based on an ordered priority of criteria.

Something we have become cautious about is a request to test with people with low-level web skills. Someone too lowly skilled can be a waste of a test opportunity. It can be frustrating for everyone involved, and  unfair on the test participant who feels like a failure. When it comes to web skills, we ask about:

• frequency of use
• tasks completed and importantly,
• unassisted use.

We like to give Farron about a week to find 6 - 12 people and we brief Farron with the following usability recruitment guideline and briefing document.

http://www.scribd.com/doc/33003448/Recruitment-Brief

To incentivise participants to take time out of their day, we provide a cash payment. The higher $ amount it is, the easier Farron's job. Currently, the standard rate for a one hour usability session with general public is between $80 to $100.

If you need testing done, please be ready to complete the recruitment brief. Of course, we will help you with it!

You'll find a list of other important usability testing requently asked questions on our website.

*Of course we are happy to use your chosen recruitment method.