6 Things You Didn’t Know About Heuristic Evaluations
A Heuristic evaluation is a process where someone trained in usability principles reviews an application (a website or software). She compares the website against a set of guidelines or principles (“Heuristics”) that tend to make for more usable applications.
For example, if while completing a task a user gets a message that says “Error 1000xz Contact System Administrator” this would violate a Heuristic: “Error messages should be expressed in plain language (no codes).”
Maybe you already have heard-of and use Heuristics Evaluations. Here are six things you might NOT know about this popular usability method:
- An effective Heuristic Evaluation needs to have multiple evaluators (3 to 5) not just one because different evaluators tend to find different problems.
- Heuristics Evaluations should be done prior to and in addition to user-testing, not instead of user-testing. If you have easy access to a few experts then you can identify a lot of low-hanging problems prior to subjecting them to users. What’s more, you can then check your expert effectiveness by seeing how many HE problems are encountered by users during the user-test.
- Rolf Molich, one of the co-creators of the method said “Heuristic Evaluations are 99% bad” at UPA 2009 on a panel with Jakob Nielsen and Chauncey Wilson on Heuristic Evaluations. His reaction was in large part because HE is being used instead of user-testing and only loosely based on any heuristics.
- Double experts are better than single experts: Being an expert in both usability principles and the domain make for better, more focused heuristic evaluations. So if you’re testing a financial application you’d have better results by having a usability expert either with an accounting background or well versed in the lexicon and flow of accounting and finance software.
- There is not a single set of Heuristics. Nielsen’s 10 famous heuristics are the ones most used but there are also others:
- There is controversy over just how effective Heuristic Evaluations are in finding problems as compared to user-testing. There is even controversy over how you determine the effectiveness of a usability method. This was one of the key criticisms in Gray and Salzman’s 1998 influential paper Damaged Merchandise: A simple count of problems isn’t a reliable or valid measure of effectiveness.