Comparison of usability evaluation methods

Last updated

Usability testing methods aim to evaluate the ease of use of a software product by its users. As existing methods are subjective and open to interpretation, scholars have been studying the efficacy of each method [1] [2] [3] and their adequacy to different subjects, comparing which one may be the most appropriate in fields like e-learning, [4] e-commerce, [5] or mobile applications. [6]

Contents

Evaluation MethodEvaluation Method TypeApplicable StagesDescriptionAdvantagesDisadvantages
Think-aloud protocol Testing Design, coding, testing and release of applicationParticipants in testing express their thoughts on the application while executing set tasks
  • Less expensive
  • Results are close to what is experienced by users
  • The Environment is not natural to the user
Remote Usability testing Testing Design, coding, testing and release of applicationThe experimenter does not directly observe the users while they use the application though activity may be recorded for subsequent viewing
  • Efficiency, effectiveness and satisfaction, the three usability issues, are covered
  • Additional Software is necessary to observe the participants from a distance
Focus groups InquiryTesting and release of applicationA moderator guides a discussion with a group of users of the application
  • If done before prototypes are developed, can save money
  • Produces a lot of useful ideas from the users themselves
  • Can improve customer relations
  • The environment is not natural to the user and may provide inaccurate results.
  • The data collected tends to have low validity due to the unstructured nature of the discussion
Interviews InquiryDesign, coding, testing and release of applicationThe users are interviewed to find out about their experience and expectations
  • Good at obtaining detailed information
  • Few participants are needed
  • Can improve customer relations
  • Can not be conducted remotely
  • Does not address the usability issue of efficiency
Cognitive walkthrough Inspection Design, coding, testing and release of applicationA team of evaluators walk through the application discussing usability issues through the use of a paper prototype or a working prototype
  • Good at refining requirements
  • does not require a fully functional prototype
  • Does not address user satisfaction or efficiency
  • The designer may not behave as the average user when using the application
Pluralistic walkthrough Inspection DesignA team of users, usability engineers and product developers review the usability of the paper prototype of the application
  • Usability issues are resolved faster
  • Greater number of usability problems can be found at one time
  • Does not address the usability issue of efficiency

See also

Related Research Articles

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Psychometrics</span> Theory and technique of psychological measurement

Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally refers to specialized fields within psychology and education devoted to testing, measurement, assessment, and related activities. Psychometrics is concerned with the objective measurement of latent constructs that cannot be directly observed. Examples of latent constructs include intelligence, introversion, mental disorders, and educational achievement. The levels of individuals on nonobservable latent variables are inferred through mathematical modeling based on what is observed from individuals' responses to items on tests and scales.

Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.

Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware products. A text-to-speech (TTS) system converts normal language text into speech; other systems render symbolic linguistic representations like phonetic transcriptions into speech. The reverse process is speech recognition.

<span class="mw-page-title-main">Machine learning</span> Study of algorithms that improve automatically through experience

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. When there was a vast amount of potential answers, the correct ones needed to be labeled as valid by human labelers initially and human supervision was needed.

A think-aloudprotocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences.

<span class="mw-page-title-main">Usability</span> Capacity of a system for its users to perform tasks

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

<span class="mw-page-title-main">Mathematics education</span> Mathematics teaching, learning and scholarly research

In contemporary education, mathematics education—known in Europe as the didactics or pedagogy of mathematics—is the practice of teaching, learning, and carrying out scholarly research into the transfer of mathematical knowledge.

<span class="mw-page-title-main">Jakob Nielsen (usability consultant)</span> American computer scientist (born 1957)

Jakob Nielsen is a Danish web usability consultant, human–computer interaction researcher, and co-founder of Nielsen Norman Group. He was named the “guru of Web page usability” in 1998 by The New York Times and the “king of usability” by Internet Magazine.

<span class="mw-page-title-main">Bioremediation</span> Process used to treat contaminated media such as water and soil

Bioremediation broadly refers to any process wherein a biological system, living or dead, is employed for removing environmental pollutants from air, water, soil, flue gasses, industrial effluents etc., in natural or artificial settings. The natural ability of organisms to adsorb, accumulate, and degrade common and emerging pollutants has attracted the use of biological resources in treatment of contaminated environment. In comparison to conventional physicochemical treatment methods bioremediation may offer considerable advantages as it aims to be sustainable, eco-friendly, cheap, and scalable. Most bioremediation is inadvertent, involving native organisms. Research on bioremediation is heavily focused on stimulating the process by inoculation of a polluted site with organisms or supplying nutrients to promote the growth. In principle, bioremediation could be used to reduce the impact of byproducts created from anthropogenic activities, such as industrialization and agricultural processes. Bioremediation could prove less expensive and more sustainable than other remediation alternatives.

A recommender system, or a recommendation system, is a subclass of information filtering system that provide suggestions for items that are most pertinent to a particular user. Typically, the suggestions refer to various decision-making processes, such as what product to purchase, what music to listen to, or what online news to read. Recommender systems are particularly useful when an individual needs to choose an item from a potentially overwhelming number of items that a service may offer.

Computational Economics is an interdisciplinary research discipline that involves computer science, economics, and management science. This subject encompasses computational modeling of economic systems. Some of these areas are unique, while others established areas of economics by allowing robust data analytics and solutions of problems that would be arduous to research without computers and associated numerical methods.

<span class="mw-page-title-main">Interactive media</span> Digital media which make use of moving images, animations, videos and audio

Interactive media normally refers to products and services on digital computer-based systems which respond to the user's actions by presenting content such as text, moving image, animation, video and audio. Since its early conception, various forms of interactive media have emerged with impacts on educational and commercial markets. With the rise of decision-driven media, concerns surround the impacts of cybersecurity and societal distraction.

Structural health monitoring (SHM) involves the observation and analysis of a system over time using periodically sampled response measurements to monitor changes to the material and geometric properties of engineering structures such as bridges and buildings.

<span class="mw-page-title-main">Learning to rank</span> Use of machine learning to rank items

Learning to rank or machine-learned ranking (MLR) is the application of machine learning, typically supervised, semi-supervised or reinforcement learning, in the construction of ranking models for information retrieval systems. Training data consists of lists of items with some partial order specified between items in each list. This order is typically induced by giving a numerical or ordinal score or a binary judgment for each item. The goal of constructing the ranking model is to rank new, unseen lists in a similar way to rankings in the training data.

Partial concurrent thinking aloud is a method used to gather data in usability testing with screen reader users. It is a particular kind of think aloud protocol created by Stefano Federici and Simone Borsci at the Interuniversity Center for Research on Cognitive Processing in Natural and Artificial Systems of University of Rome "La Sapienza". The partial concurrent thinking aloud is built up in order to create a specific usability assessment technique for blind users, eligible to maintain the advantages of concurrent and retrospective thinking aloud while overcoming their limits. Using PCTA blind users' verbalizations of problems could be more pertinent and comparable to those given by sighted people who use a concurrent protocol. In the usability evaluation with blind people, the retrospective thinking aloud is often adopted as a functional solution to overcome the structural interference due to thinking aloud and hearing the screen reader imposed by the classic thinking aloud technique; such a solution has yet a relapse in the evaluation method, because the concurrent and the retrospective protocols measure usability from different points of view, one mediated by navigation experience (retrospective) one more direct and pertinent (concurrent). The use of PCTA could be widened to both summative and formative usability evaluations with mixed panels of users, thus extending the number of problems' verbalizations according to disabled users' divergent navigation processes and problem solving strategies.

Gamification is the strategic attempt to enhance systems, services, organizations, and activities by creating similar experiences to those experienced when playing games in order to motivate and engage users. This is generally accomplished through the application of game-design elements and game principles in non-game contexts.

<span class="mw-page-title-main">Explainable artificial intelligence</span> AI in which the results of the solution can be understood by humans

Explainable AI (XAI), also known as Interpretable AI, or Explainable Machine Learning (XML), is artificial intelligence (AI) in which humans can understand the reasoning behind decisions or predictions made by the AI. It contrasts with the "black box" concept in machine learning, where even the AI's designers cannot explain why it arrived at a specific decision.

In software engineering, containerization is operating system-level virtualization or application-level virtualization over multiple network resources so that software applications can run in isolated user spaces called containers in any cloud or non-cloud environment, regardless of type or vendor.

References

  1. Genise, Pauline (August 28, 2002.). "Usability Evaluation: Methods and Techniques". University of Texas
  2. Dhouib, A.; Trabelsi, Abdelwaheb; Kolski, C.; Neji, M. (2016). "A classification and comparison of usability evaluation methods for interactive adaptive systems". 2016 9th International Conference on Human System Interactions (HSI). pp. 246–251. doi:10.1109/HSI.2016.7529639. ISBN   978-1-5090-1729-4. S2CID   19110009. Archived from the original on 2021-02-14. Retrieved 2021-02-07.
  3. Hocko, Jennifer M. (2002). "Reliability of Usability Evaluation Methods".{{cite journal}}: Cite journal requires |journal= (help)
  4. Vukovac, Dijana Plantak; Kirinic, V.; Klicek, B. (2010). "A Comparison of Usability Evaluation Methods for e-Learning Systems" (PDF). A Comparison of Usability Evaluation Methods for e- Learning Systems. doi:10.2507/daaam.scibook.2010.27. ISBN   9783901509742. Archived (PDF) from the original on 2018-06-03. Retrieved 2021-02-07.
  5. Hasan, L.; Morris, Anne; Probets, S. (2012). "A comparison of usability evaluation methods for evaluating e-commerce websites". Behav. Inf. Technol. 31 (7): 707–737. doi:10.1080/0144929X.2011.596996. S2CID   9998763. Archived from the original on 2021-02-18. Retrieved 2021-02-07.
  6. Mathur, P.; Chande, Swati V. (2020). "Empirical Investigation of Usability Evaluation Methods for Mobile Applications Using Evidence-Based Approach". Microservices in Big Data Analytics. pp. 95–110. doi:10.1007/978-981-15-0128-9_9. ISBN   978-981-15-0127-2. S2CID   214128768. Archived from the original on 2021-02-18. Retrieved 2021-02-07.