Heuristic evaluation

Last updated

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles (the "heuristics"). These evaluation methods are now widely taught and practiced in the new media sector, where user interfaces are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing.

Contents

Introduction

The main goal of heuristic evaluations is to identify any problems associated with the design of user interfaces. Usability consultants Rolf Molich and Jakob Nielsen developed this method on the basis of several years of experience in teaching and consulting about usability engineering. Heuristic evaluations are one of the most informal methods [1] of usability inspection in the field of human–computer interaction. There are many sets of usability design heuristics; they are not mutually exclusive and cover many of the same aspects of user interface design. Quite often, usability problems that are discovered are categorized—often on a numeric scale—according to their estimated impact on user performance or acceptance. Often the heuristic evaluation is conducted in the context of use cases (typical user tasks), to provide feedback to the developers on the extent to which the interface is likely to be compatible with the intended users' needs and preferences.

The simplicity of heuristic evaluation is beneficial at the early stages of design and prior to user-based testing. This usability inspection method does not rely on users which can be burdensome due to the need for recruiting, scheduling issues, a place to perform the evaluation, and a payment for participant time. In the original report published, Nielsen stated that four experiments showed that individual evaluators were "mostly quite bad" at doing heuristic evaluations and suggested multiple evaluators were needed, with the results aggregated, to produce and to complete an acceptable review. Most heuristic evaluations can be accomplished in a matter of days. The time required varies with the size of the artifact, its complexity, the purpose of the review, the nature of the usability issues that arise in the review, and the competence of the reviewers. Using heuristic evaluation prior to user testing is often conducted to identify areas to be included in the evaluation or to eliminate perceived design issues prior to user-based evaluation.

Although heuristic evaluation can uncover many major usability issues in a short period of time, a criticism that is often leveled is that results are highly influenced by the knowledge of the expert reviewer(s). This "one-sided" review repeatedly has different results than software performance testing, each type of testing uncovering a different set of problems.

Methodology

Heuristic evaluation are conducted in variety of ways depending on the scope and type of project. As a general rule of thumb, there are researched frameworks involved to reduce bias and maximize findings within an evaluation. There are various pros and cons to heuristic evaluation. A lot of it depends on the amount of resources and the time available to the user.

Pros: Because there’s a very detailed list of criteria the evaluator goes through, it is a very detailed process and provides good feedback on areas that could be improved on. In addition, since it is done by several people the designer can get feedback from multiple perspectives. As it is a relatively straightforward process, there are less ethical and logistical concerns related to organizing the evaluation and executing it.

Cons: Since there is a specific set of criteria, the quality of the evaluation is limited by the skill and knowledge of the people who evaluate it. This leads to another issue: finding experts and people qualified enough to conduct this evaluation. However, if you have close resources of experts and qualified evaluators, this wouldn’t pose an issue. In addition, because the evaluations are more just personal observations, there’s no hard data in the results — the designer just has to take all the information and evaluations with these considerations in mind.

Number of Evaluators

According to Nielsen, three to five evaluators are recommended within a study. [2] Having more than five evaluators does not necessarily increase the amount of insights, and this may add more cost than benefit to the overall evaluation.

Individual and Group Process

Heuristic evaluation must start individually before aggregating results in order to reduce group confirmation bias. [2] The evaluator should examine the prototype independently before entering group discussions to accumulate insights.

Observer Trade-offs

There are costs and benefits associated when adding an observer to an evaluation session. [2]

In a session without an observer, evaluators would need to formalize their individual observations within a written report as they interact with the product/prototype. This option would require more time and effort from the evaluators, and this would also require further time for the conductors of the study to interpret individual reports. However, this option is less costly because it reduces the overhead costs associated with hiring observers.

With an observer, evaluators can provide their analysis verbally while observers transcribe and interpret the evaluators' findings. This option reduces the amount of workload from the evaluators and the amount of time needed to interpret findings from multiple evaluators.

Nielsen's heuristics

Jakob Nielsen's heuristics are probably the most-used usability heuristics for user interface design. An early version of the heuristics appeared in two papers by Nielsen and Rolf Molich published in 1989-1990. [3] [4] Nielsen published an updated set in 1994, [5] and the final set still in use today was published in 2005: [6]

  1. Visibility of system status:
    The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
  2. Match between system and the real world:
    The system should speak the user's language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
  3. User control and freedom:
    Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
  4. Consistency and standards:
    Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
  5. Error prevention:
    Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
  6. Recognition rather than recall:
    Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
  7. Flexibility and efficiency of use:
    Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
  8. Aesthetic and minimalist design :
    Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
  9. Help users recognize, diagnose, and recover from errors:
    Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
  10. Help and documentation:
    Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

Gerhardt-Powals' cognitive engineering principles

Although Nielsen is considered the expert and field leader in heuristic evaluation, Jill Gerhardt-Powals developed a set of cognitive engineering principles for enhancing human-computer performance. [7] These heuristics, or principles, are similar to Nielsen's heuristics but take a more holistic approach to evaluation. The Gerhardt Powals' principles [8] are listed below.

  1. Automate unwanted workload:
    Eliminate mental calculations, estimations, comparisons, and any unnecessary thinking, to free cognitive resources for high-level tasks.
  2. Reduce uncertainty:
    Display data in a manner that is clear and obvious to reduce decision time and error.
  3. Fuse data:
    Bring together lower level data into a higher level summation to reduce cognitive load.
  4. Present new information with meaningful aids to interpretation:
    New information should be presented within familiar frameworks (e.g., schemas, metaphors, everyday terms) so that information is easier to absorb.
  5. Use names that are conceptually related to function:
    Display names and labels should be context-dependent, which will improve recall and recognition.
  6. Group data in consistently meaningful ways:
    Within a screen, data should be logically grouped; across screens, it should be consistently grouped. This will decrease information search time.
  7. Limit data-driven tasks:
    Use color and graphics, for example, to reduce the time spent assimilating raw data.
  8. Include in the displays only that information needed by the user at a given time:
    Exclude extraneous information that is not relevant to current tasks so that the user can focus attention on critical data.
  9. Provide multiple coding of data when appropriate:
    The system should provide data in varying formats and/or levels of detail in order to promote cognitive flexibility and satisfy user preferences.
  10. Practice judicious redundancy:
    Principle 10 was devised by the first two authors to resolve the possible conflict between Principles 6 and 8, that is, in order to be consistent, it is sometimes necessary to include more information than may be needed at a given time.

Shneiderman's Eight Golden Rules of Interface Design

Ben Shneiderman's book was published a few years prior to Nielsen, Designing the User Interface: Strategies for Effective Human-Computer Interaction (1986) covered his popular list of the, "Eight Golden Rules". [9] [10]

  1. Strive for consistency:
    Consistent sequences of actions should be required in similar situations ...
  2. Enable frequent users to use shortcuts:
    As the frequency of use increases, so do the user's desires to reduce the number of interactions ...
  3. Offer informative feedback:
    For every operator action, there should be some system feedback ...
  4. Design dialog to yield closure:
    Sequences of actions should be organized into groups with a beginning, middle, and end ...
  5. Offer simple error handling:
    As much as possible, design the system so the user cannot make a serious error ...
  6. Permit easy reversal of actions:
    This feature relieves anxiety, since the user knows that errors can be undone ...
  7. Support internal locus of control:
    Experienced operators strongly desire the sense that they are in charge of the system and that the system responds to their actions. Design the system to make users the initiators of actions rather than the responders.
  8. Reduce short-term memory load:
    The limitation of human information processing in short-term memory requires that displays be kept simple, multiple page displays be consolidated, window-motion frequency be reduced, and sufficient training time be allotted for codes, mnemonics, and sequences of actions.

Weinschenk and Barker classification

In 2000, Susan Weinschenk and Dean Barker [11] created a categorization of heuristics and guidelines used by several major providers into the following twenty types: [12]

  1. User Control:
    The interface will allow the user to perceive that they are in control and will allow appropriate control.
  2. Human Limitations:
    The interface will not overload the user’s cognitive, visual, auditory, tactile, or motor limits.
  3. Modal Integrity:
    The interface will fit individual tasks within whatever modality is being used: auditory, visual, or motor/kinesthetic.
  4. Accommodation:
    The interface will fit the way each user group works and thinks.
  5. Linguistic Clarity:
    The interface will communicate as efficiently as possible.
  6. Aesthetic Integrity:
    The interface will have an attractive and appropriate design.
  7. Simplicity:
    The interface will present elements simply.
  8. Predictability:
    The interface will behave in a manner such that users can accurately predict what will happen next.
  9. Interpretation:
    The interface will make reasonable guesses about what the user is trying to do.
  10. Accuracy:
    The interface will be free from errors.
  11. Technical Clarity:
    The interface will have the highest possible fidelity.
  12. Flexibility:
    The interface will allow the user to adjust the design for custom use.
  13. Fulfillment:
    The interface will provide a satisfying user experience.
  14. Cultural Propriety:
    The interface will match the user’s social customs and expectations.
  15. Suitable Tempo:
    The interface will operate at a tempo suitable to the user.
  16. Consistency:
    The interface will be consistent.
  17. User Support:
    The interface will provide additional assistance as needed or requested.
  18. Precision:
    The interface will allow the users to perform a task exactly.
  19. Forgiveness:
    The interface will make actions recoverable.
  20. Responsiveness:
    The interface will inform users about the results of their actions and the interface’s status.

Domain or culture-specific heuristic evaluation

For an application with a specific domain and culture, the heuristics mentioned above do not identify the potential usability problems. [13] These limitations of heuristics occur because these heuristics are incapable of considering the domain and culture-specific features of an application. This results in the introduction of domain-specific or culture-specific heuristic evaluation. [14]

See also

Related Research Articles

Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.

Usability engineering is a professional discipline that focuses on improving the usability of interactive systems. It draws on theories from computer science and psychology to define problems that occur during the use of such a system. Usability Engineering involves the testing of designs at various stages of the development process, with users or with usability experts. The history of usability engineering in this context dates back to the 1980s. In 1988, authors John Whiteside and John Bennett—of Digital Equipment Corporation and IBM, respectively—published material on the subject, isolating the early setting of goals, iterative evaluation, and prototyping as key activities. The usability expert Jakob Nielsen is a leader in the field of usability engineering. In his 1993 book Usability Engineering, Nielsen describes methods to use throughout a product development process—so designers can ensure they take into account the most important barriers to learnability, efficiency, memorability, error-free use, and subjective satisfaction before implementing the product. Nielsen’s work describes how to perform usability tests and how to use usability heuristics in the usability engineering lifecycle. Ensuring good usability via this process prevents problems in product adoption after release. Rather than focusing on finding solutions for usability problems—which is the focus of a UX or interaction designer—a usability engineer mainly concentrates on the research phase. In this sense, it is not strictly a design role, and many usability engineers have a background in computer science because of this. Despite this point, its connection to the design trade is absolutely crucial, not least as it delivers the framework by which designers can work so as to be sure that their products will connect properly with their target usership.

<span class="mw-page-title-main">Don Norman</span> American researcher, professor, and writer (born 1935)

Donald Arthur Norman is an American researcher, professor, and author. Norman is the director of The Design Lab at University of California, San Diego. He is best known for his books on design, especially The Design of Everyday Things. He is widely regarded for his expertise in the fields of design, usability engineering, and cognitive science, and has shaped the development of the field of cognitive systems engineering. He is a co-founder of the Nielsen Norman Group, along with Jakob Nielsen. He is also an IDEO fellow and a member of the Board of Trustees of IIT Institute of Design in Chicago. He also holds the title of Professor Emeritus of Cognitive Science at the University of California, San Diego. Norman is an active Distinguished Visiting Professor at the Korea Advanced Institute of Science and Technology (KAIST), where he spends two months a year teaching.

<span class="mw-page-title-main">Usability</span> Capacity of a system for its users to perform tasks

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

<span class="mw-page-title-main">Jakob Nielsen (usability consultant)</span> American computer scientist and usability professional (born 1957)

Jakob Nielsen is a Danish web usability consultant, human–computer interaction researcher, and co-founder of Nielsen Norman Group. He was named the “guru of Web page usability” in 1998 by The New York Times and the “king of usability” by Internet Magazine.

Interaction design, often abbreviated as IxD, is "the practice of designing interactive digital products, environments, systems, and services." While interaction design has an interest in form, its main area of focus rests on behavior. Rather than analyzing how things are, interaction design synthesizes and imagines things as they could be. This element of interaction design is what characterizes IxD as a design field, as opposed to a science or engineering field.

The following outline is provided as an overview of and topical guide to human–computer interaction:

The cognitive walkthrough method is a usability inspection method used to identify usability issues in interactive systems, focusing on how easy it is for new users to accomplish tasks with the system. A cognitive walkthrough is task-specific, whereas heuristic evaluation takes a holistic view to catch problems not caught by this and other usability inspection methods. The method is rooted in the notion that users typically prefer to learn a system by using it to accomplish tasks, rather than, for example, studying a manual. The method is prized for its ability to generate results quickly with low cost, especially when compared to usability testing, as well as the ability to apply the method early in the design phases before coding even begins.

Web usability of a website consists of broad goals of usability, presentation of information, choices made in a clear and concise way, a lack of ambiguity and the placement of important items in appropriate areas as well as ensuring that the content works on various devices and browsers.

<span class="mw-page-title-main">Ben Shneiderman</span> American computer scientist

Ben Shneiderman is an American computer scientist, a Distinguished University Professor in the University of Maryland Department of Computer Science, which is part of the University of Maryland College of Computer, Mathematical, and Natural Sciences at the University of Maryland, College Park, and the founding director (1983-2000) of the University of Maryland Human-Computer Interaction Lab. He conducted fundamental research in the field of human–computer interaction, developing new ideas, methods, and tools such as the direct manipulation interface, and his eight rules of design.

<span class="mw-page-title-main">User interface design</span> Planned operator–machine interaction

User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and the user experience. In computer or software design, user interface (UI) design primarily focuses on information architecture. It is the process of building interfaces that clearly communicate to the user what's important. UI design refers to graphical user interfaces and other forms of interface design. The goal of user interface design is to make the user's interaction as simple and efficient as possible, in terms of accomplishing user goals. User-centered design is typically accomplished through the execution of modern design thinking which involves empathizing with the target audience, defining a problem statement, ideating potential solutions, prototyping wireframes, and testing prototypes in order to refine final interface mockups.

In user interface design, a mode is a distinct setting within a computer program or any physical machine interface, in which the same user input will produce perceived results different from those that it would in other settings. Modal interface components include the Caps lock and Insert keys on the standard computer keyboard, both of which typically put the user's typing into a different mode after being pressed, then return it to the regular mode after being re-pressed.

Cognitive ergonomics is a scientific discipline that studies, evaluates, and designs tasks, jobs, products, environments and systems and how they interact with humans and their cognitive abilities. It is defined by the International Ergonomics Association as "concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Cognitive ergonomics is responsible for how work is done in the mind, meaning, the quality of work is dependent on the persons understanding of situations. Situations could include the goals, means, and constraints of work. The relevant topics include mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress and training as these may relate to human-system design." Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance. It is a subset of the larger field of human factors and ergonomics.

Cognitive dimensions or cognitive dimensions of notations are design principles for notations, user interfaces and programming languages, described by researcher Thomas R.G. Green and further researched with Marian Petre. The dimensions can be used to evaluate the usability of an existing information artifact, or as heuristics to guide the design of a new one, and are useful in Human-Computer Interaction design.

User experience design, upon which is the centralized requirements for "User Experience Design Research", defines the experience a user would go through when interacting with a company, its services, and its products. User experience design is a user centered design approach because it considers the user's experience when using a product or platform. Research, data analysis, and test results drive design decisions in UX design rather than aesthetic preferences and opinions, for which is known as UX Design Research. Unlike user interface design, which focuses solely on the design of a computer interface, UX design encompasses all aspects of a user's perceived experience with a product or website, such as its usability, usefulness, desirability, brand perception, and overall performance. UX design is also an element of the customer experience (CX), and encompasses all design aspects and design stages that are around a customer's experience.

<i>The Design of Everyday Things</i> 1988 book by Donald Norman

The Design of Everyday Things is a best-selling book by cognitive scientist and usability engineer Donald Norman. Originally published in 1988 with the title The Psychology of Everyday Things, it is often referred to by the initialisms POET and DOET. A new preface was added in 2002 and a revised and expanded edition was published in 2013.

The pluralistic walkthrough is a usability inspection method used to identify usability issues in a piece of software or website in an effort to create a maximally usable human-computer interface. The method centers on recruiting a group of users, developers and usability professionals to step through a task scenario, discussing usability issues associated with dialog elements involved in the scenario steps. The group of experts used is asked to assume the role of typical users in the testing.

Usability inspection is the name for a set of methods where an evaluator inspects a user interface. This is in contrast to usability testing where the usability of the interface is evaluated by testing it on real users. Usability inspections can generally be used early in the development process by evaluating prototypes or specifications for the system that can't be tested on users. Usability inspection methods are generally considered to be less costly to implement than testing on users.

<span class="mw-page-title-main">Human–computer interaction</span> Academic discipline studying the relationship between computer systems and their users

Human–computer interaction (HCI) is research in the design and the use of computer technology, which focuses on the interfaces between people (users) and computers. HCI researchers observe the ways humans interact with computers and design technologies that allow humans to interact with computers in novel ways. A device that allows interaction between human being and a computer is known as a "Human-computer Interface (HCI)".

Soft ergonomics is the study of designing virtual interfaces that cater towards the wellness of the human body, its emotional and cognitive abilities.

References

  1. ↑ Nielsen, J., and Molich, R. (1990). Heuristic evaluation of user interfaces, Proc. ACM CHI'90 Conf. (Seattle, WA, 1–5 April), 249–256
  2. 1 2 3 Experience, World Leaders in Research-Based User. "Heuristic Evaluation: How-To: Article by Jakob Nielsen". Nielsen Norman Group. Retrieved 3 December 2021.{{cite web}}: |first= has generic name (help)
  3. ↑ Nielsen, J.; Molich, R. (1989). "Teaching user interface design based on usability engineering". ACM SIGCHI Bulletin. 21 (1): 45–48. doi:10.1145/67880.67885. ISSN   0736-6906. S2CID   41663689 . Retrieved 25 May 2022.
  4. ↑ Molich, Rolf; Nielsen, Jakob (1990). "Improving a human-computer dialogue". Communications of the ACM. 33 (3): 338–348. doi:10.1145/77481.77486. ISSN   0001-0782. S2CID   11462820 . Retrieved 4 February 2022.
  5. ↑ Nielsen, Jakob (1994). Enhancing the explanatory power of usability heuristics. the SIGCHI conference. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Celebrating Interdependence - CHI '94. Boston, Massachusetts, United States: ACM Press. pp. 152–158. doi:10.1145/191666.191729. ISBN   978-0-89791-650-9 . Retrieved 25 May 2022.
  6. ↑ Nielsen, Jakob (2005), Ten usability heuristics, S2CID   59788005
  7. ↑ Gerhardt-Powals, Jill (1996). "Cognitive engineering principles for enhancing human – computer performance". International Journal of Human-Computer Interaction. 8 (2): 189–211. doi:10.1080/10447319609526147.
  8. ↑ Heuristic Evaluation – Usability Methods – What is a heuristic evaluation? Archived 28 June 2013 at the Wayback Machine Usability.gov
  9. ↑ Shneiderman (1998, p. 75); as cited in: "Eight Golden Rules of Interface Design". at www.cs.umd.edu.
  10. ↑ Malviya, Kartik (20 November 2020). "8 Golden Rules of Interface Design". Medium. UX Planet. Retrieved 2 March 2021.
  11. ↑ Weinschenk, S and Barker,D. (2000) Designing Effective Speech Interfaces. Wiley.
  12. ↑ Jeff Sauro. "What's the difference between a Heuristic Evaluation and a Cognitive Walkthrough?". MeasuringUsability.com. Archived from the original on 24 September 2014. Retrieved 28 November 2011.
  13. ↑ Nizamani, Sehrish; Khoumbati, Khalil; Nizamani, Sarwat; Memon, Shahzad; Nizamani, Saad; Laghari, Gulsher (20 March 2021). "A methodology for domain and culture-oriented heuristics creation and validation". Behaviour & Information Technology. 41 (8): 1769–1795. doi:10.1080/0144929X.2021.1903080. ISSN   0144-929X. S2CID   233682515.
  14. ↑ Nizamani, Sehrish; Nizamani, Saad; Basir, Nazish; Memon, Muhammad; Nizamani, Sarwat; Memon, Shahzad (5 April 2021). "Domain and culture-specific heuristic evaluation of the websites of universities of Pakistan". University of Sindh Journal of Information and Communication Technology. 5 (1): 45–51. ISSN   2523-1235.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.