Component-based usability testing

Last updated November 15, 2023

Component-based usability testing (CBUT) is a testing approach which aims at empirically testing the usability of an interaction component. The latter is defined as an elementary unit of an interactive system, on which behavior-based evaluation is possible. For this, a component needs to have an independent, and by the user perceivable and controllable state, such as a radio button, a slider or a whole word processor application. The CBUT approach can be regarded as part of component-based software engineering branch of software engineering.

Theory

CBUT is based on both software architectural views such as model–view–controller (MVC), presentation–abstraction–control (PAC), ICON and CNUCE agent models that split up the software in parts, and cognitive psychology views where a person's mental process is split up in smaller mental processes. Both software architecture and cognitive architecture use the principle of hierarchical layering, in which low level processes are more elementary and for humans often more physical in nature, such as the coordination movement of muscle groups. Processes that operate on higher level layers are more abstract and focus on a person's main goal, such as writing an application letter to get a job.

The layered protocol theory (LPT),^[1] which is a special version of perceptual control theory (PCT), brings these views together by suggesting that users interact with a system across several layers by sending messages. Users interact with components on high layers by sending messages, such as pressing keys, to components operating on lower layers, which on their turn relay a series of these messages into a single high-level message, such as DELETE, to a component on a higher layer. Components operating on higher layers, communicate back to the user by sending messages to components operating on lower-level layers. Whereas this layered-interaction model explains how the interaction is established, control loops explain the purpose of the interaction. LPT sees the purpose of the users' behavior as the users' attempt to control their perception, in this case the state of the component they perceive. This means that users will only act if they perceive the component to be in an undesirable state. For example, if a person has an empty glass but want a full glass of water, he or she will act (e.g. walk to the tap, turning the tap on to fill the glass). The action of filling the glass will continue until the person perceives the glass as full. As interaction with components takes places on several layers, interacting with a single device can include several control loops. The amount of effort put into operating a control loop is seen as an indicator for the usability of an interaction component.

Testing

CBUT can be categorized according to two testing paradigms, the single-version testing paradigm (SVTP) and the multiple-versions testing paradigm (MVTP). In SVTP only one version of each interaction component in a system is tested. The focus is to identify interaction components that might reduce the overall usability of the system. SVTP is therefore suitable as part of a software-integration test. In MVTP on the other hand, multiple versions of a single component are tested while the remaining components in the system remain unchanged. The focus is on identifying the version with the highest usability of specific interaction component. MVTP therefore is suitable for component development and selection. Different CBUT methods have been proposed for SVTP and MVTP, which include measures based on recorded user interaction and questionnaires. Whereas in MVTP the recorded data can directly be interpreted by making a comparison between two versions of the interaction component, in SVTP log file analysis is more extensive as interaction with both higher and lower components must be considered.^[2] Meta-analysis on the data from several lab experiments that used CBUT measures suggests that these measures can be statistically more powerful than overall (holistic) usability measures.^[3]

Usability questionnaire

While holistic oriented usability questionnaires such as the system usability scale (SUS) examine the usability of a system on several dimensions such as defined in ISO 9241 Part 11 standard effectiveness, efficiency and satisfaction, a component-based usability questionnaire (CBUQ)^[4] is a questionnaire which can be used to evaluate the usability of individual interaction components, such as the volume control or the play control of a MP3 player. To evaluate an interaction component, the six perceived ease-of-use (PEOU) statements from the technology acceptance model are taken with a reference to the interaction component, instead of to the entire system.

Users are asked to rate these statements on a seven-point Likert scale. The average rating on these six statements is regarded as the user's usability rating of the interaction component. Based on lab studies with difficult to use interaction components and easy to use interaction components, a break-even point of 5.29 on seven-point Likert scale has been determined.^[4] Using a one-sample student's t-test, it is possible to examine whether users' rating of an interaction component deviates from this break-even point. Interaction components that receive rating below this break-even point can be regarded as more comparable to the set of difficult to use interaction components, whereas ratings above this break-even point would be more comparable to the set of easy-to-use interaction components.

If engineers like to evaluate multiple interaction components simultaneously, the CBUQ questionnaire exists of separate sections, one for each interaction component, each with their own 6 PEOU statements.

Related Research Articles

Social software, also known as social apps or social platform includes communications and interactive tools that are often based on the Internet. Communication tools typically handle capturing, storing and presenting communication, usually written but increasingly including audio and video as well. Interactive tools handle mediated interactions between a pair or group of users. They focus on establishing and maintaining a connection among users, facilitating the mechanics of conversation and talk. Social software generally refers to software that makes collaborative behaviour, the organisation and moulding of communities, self-expression, social interaction and feedback possible for individuals. Another element of the existing definition of social software is that it allows for the structured mediation of opinion between people, in a centralized or self-regulating manner. The most improved area for social software is that Web 2.0 applications can all promote co-operation between people and the creation of online communities more than ever before. The opportunities offered by social software are instant connections and opportunities to learn.An additional defining feature of social software is that apart from interaction and collaboration, it aggregates the collective behaviour of its users, allowing not only crowds to learn from an individual but individuals to learn from the crowds as well. Hence, the interactions enabled by social software can be one-to-one, one-to-many, or many-to-many.

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

In computer science, a software agent is a computer program that acts for a user or another program in a relationship of agency.

A Likert scale is a psychometric scale named after its inventor, American social psychologist Rensis Likert, which is commonly used in research questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, although there are other types of rating scales.

ISO 9241 is a multi-part standard from the International Organization for Standardization (ISO) covering ergonomics of human-computer interaction. It is managed by the ISO Technical Committee 159. It was originally titled Ergonomic requirements for office work with visual display terminals (VDTs). From 2006 onwards, the standards were retitled to the more generic Ergonomics of Human System Interaction.

Across the many fields concerned with interactivity, including information science, computer science, human-computer interaction, communication, and industrial design, there is little agreement over the meaning of the term "interactivity", but most definitions are related to interaction between users and computers and other machines through a user interface. Interactivity can however also refer to interaction between people. It nevertheless usually refers to interaction between people and computers – and sometimes to interaction between computers – through software, hardware, and networks.

The following outline is provided as an overview of and topical guide to human–computer interaction:

Software prototyping is the activity of creating prototypes of software applications, i.e., incomplete versions of the software program being developed. It is an activity that can occur in software development and is comparable to prototyping as known from other fields, such as mechanical engineering or manufacturing.

The user experience (UX) is how a user interacts with and experiences a product, system or service. It includes a person's perceptions of utility, ease of use, and efficiency. Improving user experience is important to most companies, designers, and creators when creating and refining products because negative user experience can diminish the use of the product and, therefore, any desired positive impacts; conversely, designing toward profitability often conflicts with ethical user experience objectives and even causes harm. User experience is subjective. However, the attributes that make up the user experience are objective.

User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and the user experience. In computer or software design, user interface (UI) design primarily focuses on information architecture. It is the process of building interfaces that clearly communicate to the user what's important. UI design refers to graphical user interfaces and other forms of interface design. The goal of user interface design is to make the user's interaction as simple and efficient as possible, in terms of accomplishing user goals.

The systems architect is an information and communications technology professional. Systems architects define the architecture of a computerized system in order to fulfill certain requirements. Such definitions include: a breakdown of the system into components, the component interactions and interfaces, and the technologies and resources to be used in its design and implementation.

LoadRunner is a software testing tool from OpenText. It is used to test applications, measuring system behavior and performance under load.

(In the automation and engineering environments, the hardware engineer or architect encompasses the electronics engineering and electrical engineering fields, with subspecialities in analog, digital, or electromechanical systems.)

A rating scale is a set of categories designed to elicit information about a quantitative or a qualitative attribute. In the social sciences, particularly psychology, common examples are the Likert response scale and 1-10 rating scales in which a person selects the number that is considered to reflect the perceived quality of a product.

User experience design is the process of defining the experience a user would go through when interacting with a company, its services, and its products. Design decisions in UX design are often driven by research, data analysis, and test results rather than aesthetic preferences and opinions. Unlike user interface design, which focuses solely on the design of a computer interface, UX design encompasses all aspects of a user's perceived experience with a product or website, such as its usability, usefulness, desirability, brand perception, and overall performance. UX design is also an element of the customer experience (CX), which encompasses all aspects and stages of a customer's experience and interaction with a company.

An authoring system is a program that has pre-programmed elements for the development of interactive multimedia software titles. Authoring systems can be defined as software that allows its user to create multimedia applications for manipulating multimedia objects.

Sexuality can be inscribed in a multidimensional model comprising different aspects of human life: biology, reproduction, culture, entertainment, relationships and love.

A software widget is a relatively simple and easy-to-use software application or component made for one or more different software platforms.

IDEF4, or Integrated DEFinition for Object-Oriented Design, is an object-oriented design modeling language for the design of component-based client/server systems. It has been designed to support smooth transition from the application domain and requirements analysis models to the design and to actual source code generation. It specifies design objects with sufficient detail to enable source code generation.

The Questionnaire For User Interaction Satisfaction (QUIS) is a tool developed to assess users' subjective satisfaction with specific aspects of the human-computer interface. It was developed in 1987 by a multi-disciplinary team of researchers at the University of Maryland Human–Computer Interaction Lab. The QUIS is currently at Version 7.0 with demographic questionnaire, a measure of overall system satisfaction along 6 scales, and measures of 9 specific interface factors. These 9 factors are: screen factors, terminology and system feedback, learning factors, system capabilities, technical manuals, on-line tutorials, multimedia, teleconferencing, and software installation. Currently available in: German, Italian, Portuguese, and Spanish.

References

↑ Farrell, P.S.E., Hollands, J.G., Taylor, M.M., Gamble, H.D., (1999). Perceptual control and layered protocols in interface design: I. Fundamental concepts. International Journal of Human-Computer Studies 50 (6), 489–520. doi : 10.1006/ijhc.1998.0259
↑ Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2007), Towards an empirical method of efficiency testing of system parts: a methodological study, Interacting with Computers, vol. 19, no. 3, pp. 342–356. preliminary version doi : 10.1016/j.intcom.2007.01.002
↑ Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2008). Component-Specific Usability Testing, IEEE Transactions on Systems, Man, and Cybernetics – Part A, vol. 38, no. 5, pp. 1143–1155, September 2008. preliminary version doi : 10.1109/TSMCA.2008.2001056
1 2 Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2009), Theoretical foundation and validity of a component-based usability questionnaire, Behaviour and Information Technology, 2, no. 28, pp. 121 – 137. preliminary version MP3 example study doi : 10.1080/01449290701306510

External links

Example study that uses component-based usability questionnaire including instructions, questionnaires, data analysis and additional instructions .

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[Farrel1999-1] Farrell, P.S.E., Hollands, J.G., Taylor, M.M., Gamble, H.D., (1999). Perceptual control and layered protocols in interface design: I. Fundamental concepts. International Journal of Human-Computer Studies 50 (6), 489–520. doi : 10.1006/ijhc.1998.0259

[Brinkman2007-2] Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2007), Towards an empirical method of efficiency testing of system parts: a methodological study, Interacting with Computers, vol. 19, no. 3, pp. 342–356. preliminary version doi : 10.1016/j.intcom.2007.01.002

[Brinkman2008-3] Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2008). Component-Specific Usability Testing, IEEE Transactions on Systems, Man, and Cybernetics – Part A, vol. 38, no. 5, pp. 1143–1155, September 2008. preliminary version doi : 10.1109/TSMCA.2008.2001056

[Brinkman2009-4] 1 2 Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2009), Theoretical foundation and validity of a component-based usability questionnaire, Behaviour and Information Technology, 2, no. 28, pp. 121 – 137. preliminary version MP3 example study doi : 10.1080/01449290701306510

[1]

[2]

[3]

[4]