Out-of-the-loop performance problem

Last updated

The out-of-the-loop performance problem (OOL or OOTL [1] ) arises when an operator suffers from performance decrement as a consequence of automation. [2] [3] The potential loss of skills and of situation awareness caused by vigilance and complacency problems might make operators of automated systems unable to operate manually in case of system failure. Highly automated systems reduce the operator to monitoring role, which diminishes the chances for the operator to understand the system. [4] It is related to mind wandering. [4]

Contents

Etymology

OOL is also known as out-of-the-loop syndrome [5] and out-of-the-loop effect. [6] One of the first mentions of the term "out of the loop" is found in a patent by Willard Meilander from Goodyear Aerospace Corporation for automated aircraft control in 1972. [7] [8] More early mentions of OOL came up in the context of flight automation in 1980s. [9]

Consequences

Three Mile Island accident in 1979, USAir Flight 5050 crash in 1989, Air France Flight 447 in 2009 and the loss of $400 million by Knight Capital Group in 2012 are attributed to OOL. [3] [10]

Automatic train operation

Automatic train operation is meant to reduce manual operation. This results in OOL performance problem for train drivers. [11]

See also

Related Research Articles

Automated theorem proving is a subfield of automated reasoning and mathematical logic dealing with proving mathematical theorems by computer programs. Automated reasoning over mathematical proof was a major impetus for the development of computer science.

<span class="mw-page-title-main">Call centre</span> Office dealing with a large volume of enquiries by telephone

A call centre or call center is a managed capability that can be centralised or remote that is used for receiving or transmitting a large volume of enquiries by telephone. An inbound call centre is operated by a company to administer incoming product or service support or information inquiries from consumers. Outbound call centres are usually operated for sales purposes such as telemarketing, for solicitation of charitable or political donations, debt collection, market research, emergency notifications, and urgent/critical needs blood banks. A contact centre is a further extension of call centres telephony based capabilities, administers centralised handling of individual communications, including letters, faxes, live support software, social media, instant message, and email.

<span class="mw-page-title-main">Automation</span> Use of various control systems for operating equipment

Automation describes a wide range of technologies that reduce human intervention in processes, mainly by predetermining decision criteria, subprocess relationships, and related actions, as well as embodying those predeterminations in machines. Automation has been achieved by various means including mechanical, hydraulic, pneumatic, electrical, electronic devices, and computers, usually in combination. Complicated systems, such as modern factories, airplanes, and ships typically use combinations of all of these techniques. The benefit of automation includes labor savings, reducing waste, savings in electricity costs, savings in material costs, and improvements to quality, accuracy, and precision.

<span class="mw-page-title-main">Self-driving car</span> Vehicle operated with reduced human input

A self-driving car, also known as an autonomous car (AC), driverless car, or robotic car (robo-car), is a car that is capable of operating with reduced or no human input. Self-driving cars are responsible for all driving activities including perceiving the environment, monitoring important systems, and controlling the vehicle, including navigating from origin to destination.

<span class="mw-page-title-main">Laboratory robotics</span> Using robots in biology or chemistry labs

Laboratory robotics is the act of using robots in biology, chemistry or engineering labs. For example, pharmaceutical companies employ robots to move biological or chemical samples around to synthesize novel chemical entities or to test pharmaceutical value of existing chemical matter. Advanced laboratory robotics can be used to completely automate the process of science, as in the Robot Scientist project.

Mathematical psychology is an approach to psychological research that is based on mathematical modeling of perceptual, thought, cognitive and motor processes, and on the establishment of law-like rules that relate quantifiable stimulus characteristics with quantifiable behavior. The mathematical approach is used with the goal of deriving hypotheses that are more exact and thus yield stricter empirical validations. There are five major research areas in mathematical psychology: learning and memory, perception and psychophysics, choice and decision-making, language and thinking, and measurement and scaling.

Situational awareness or situation awareness (SA) is the understanding of an environment, its elements, and how it changes with respect to time or other factors. Situational awareness is important for effective decision making in many environments. It is formally defined as:

“the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”.

Cognitive ergonomics is a scientific discipline that studies, evaluates, and designs tasks, jobs, products, environments and systems and how they interact with humans and their cognitive abilities. It is defined by the International Ergonomics Association as "concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Cognitive ergonomics is responsible for how work is done in the mind, meaning, the quality of work is dependent on the persons understanding of situations. Situations could include the goals, means, and constraints of work. The relevant topics include mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress and training as these may relate to human-system design." Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance. It is a subset of the larger field of human factors and ergonomics.

Neuroergonomics is the application of neuroscience to ergonomics. Traditional ergonomic studies rely predominantly on psychological explanations to address human factors issues such as: work performance, operational safety, and workplace-related risks. Neuroergonomics, in contrast, addresses the biological substrates of ergonomic concerns, with an emphasis on the role of the human nervous system.

<span class="mw-page-title-main">Vehicular automation</span> Automation for various purposes of vehicles

Vehicular automation involves the use of mechatronics, artificial intelligence, and multi-agent systems to assist the operator of a vehicle such as a car, lorries, aircraft, or watercraft. A vehicle using automation for tasks such as navigation to ease but not replace human control, qualify as semi-autonomous, whereas a fully self-operated vehicle is termed autonomous.

In psychology and neuroscience, memory span is the longest list of items that a person can repeat back in correct order immediately after presentation on 50% of all trials. Items may include words, numbers, or letters. The task is known as digit span when numbers are used. Memory span is a common measure of working memory and short-term memory. It is also a component of cognitive ability tests such as the WAIS. Backward memory span is a more challenging variation which involves recalling items in reverse order.

Augmented cognition is an interdisciplinary area of psychology and engineering, attracting researchers from the more traditional fields of human-computer interaction, psychology, ergonomics and neuroscience. Augmented cognition research generally focuses on tasks and environments where human–computer interaction and interfaces already exist. Developers, leveraging the tools and findings of neuroscience, aim to develop applications which capture the human user's cognitive state in order to drive real-time computer systems. In doing so, these systems are able to provide operational data specifically targeted for the user in a given context. Three major areas of research in the field are: Cognitive State Assessment (CSA), Mitigation Strategies (MS), and Robust Controllers (RC). A subfield of the science, Augmented Social Cognition, endeavours to enhance the "ability of a group of people to remember, think, and reason."

Human-in-the-loop or HITL is used in multiple contexts. It can be defined as a model requiring human interaction. HITL is associated with modeling and simulation (M&S) in the live, virtual, and constructive taxonomy. HITL along with the related human-on-the-loop are also used in relation to lethal autonomous weapons. Further, HITL is used in the context of machine learning.

Design Automation usually refers to electronic design automation, or Design Automation which is a Product Configurator. Extending Computer-Aided Design (CAD), automated design and Computer-Automated Design (CAutoD) are more concerned with a broader range of applications, such as automotive engineering, civil engineering, composite material design, control engineering, dynamic system identification and optimization, financial systems, industrial equipment, mechatronic systems, steel construction, structural optimisation, and the invention of novel systems.

<span class="mw-page-title-main">Mica Endsley</span> Former Chief Scientist of the U.S. Air Force

Mica Endsley is an American engineer and a former Chief Scientist of the United States Air Force.

Neuroenhancement or cognitive enhancement is the experimental use of pharmacological or non-pharmacological methods intended to improve cognitive and affective abilities in healthy people who do not have a mental illness. Agents or methods of neuroenhancement are intended to affect cognitive, social, psychological, mood, or motor benefits beyond normal functioning.

Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct. Automation bias stems from the social psychology literature that found a bias in human-human interaction that showed that people assign more positive evaluations to decisions made by humans than to a neutral object. The same type of positivity bias has been found for human-automation interaction, where the automated decisions are rated more positively than neutral. This has become a growing problem for decision making as intensive care units, nuclear power plants, and aircraft cockpits have increasingly integrated computerized system monitors and decision aids to mostly factor out possible human error. Errors of automation bias tend to occur when decision-making is dependent on computers or other automated aids and the human is in an observatory role but able to make decisions. Examples of automation bias range from urgent matters like flying a plane on automatic pilot to such mundane matters as the use of spell-checking programs.

Human performance modeling (HPM) is a method of quantifying human behavior, cognition, and processes. It is a tool used by human factors researchers and practitioners for both the analysis of human function and for the development of systems designed for optimal user experience and interaction. It is a complementary approach to other usability testing methods for evaluating the impact of interface features on operator performance.

Ironies of Automation is a research paper written by Lisanne Bainbridge and published in Automatica in 1983, and has been widely recognized as a pioneering statement of the problems inherent in automation.

Cognitive systems engineering (CSE) is a field of study that examines the intersection of people, work, and technology, with a focus on safety-critical systems. The central tenet of cognitive systems engineering is that it views a collection of people and technology as a single unit that is capable of cognitive work, which is called a joint cognitive system.

References

  1. Merat, Natasha; Seppelt, Bobbie; Louw, Tyron; Engström, Johan; Lee, John D.; Johansson, Emma; Green, Charles A.; Katazaki, Satoshi; Monk, Chris; Itoh, Makoto; McGehee, Daniel; Sunda, Takashi; Unoura, Kiyozumi; Victor, Trent; Schieben, Anna; Keinath, Andreas (1 February 2019). "The "Out-of-the-Loop" concept in automated driving: proposed definition, measures and implications". Cognition, Technology & Work. 21 (1): 87–98. doi: 10.1007/s10111-018-0525-8 . hdl: 10919/93316 . ISSN   1435-5566. S2CID   52279007.
  2. Endsley, Mica R.; Kiris, Esin O. (June 1995). "The Out-of-the-Loop Performance Problem and Level of Control in Automation". Human Factors: The Journal of the Human Factors and Ergonomics Society. 37 (2): 381–394. doi:10.1518/001872095779064555. ISSN   0018-7208. S2CID   2147200.
  3. 1 2 Kaber, David B.; Endsley, Mica R. (1997). "Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety". Process Safety Progress. 16 (3): 126–131. doi:10.1002/prs.680160304. ISSN   1066-8527. S2CID   14070085.
  4. Nadj, Mario; Maedche, Alexander; Schieder, Christian (1 August 2020). "The effect of interactive analytical dashboard features on situation awareness and task performance". Decision Support Systems. 135: 113322. doi:10.1016/j.dss.2020.113322. ISSN   0167-9236. PMC   7234950 . PMID   32834262.
  5. Endsley, Mica R. (February 2017). "From Here to Autonomy: Lessons Learned From Human–Automation Research". Human Factors: The Journal of the Human Factors and Ergonomics Society. 59 (1): 5–27. doi: 10.1177/0018720816681350 . ISSN   0018-7208. PMID   28146676. S2CID   3771328.
  6. Merat, Natasha; Seppelt, Bobbie; Louw, Tyron; Engström, Johan; Lee, John D.; Johansson, Emma; Green, Charles A.; Katazaki, Satoshi; Monk, Chris; Itoh, Makoto; McGehee, Daniel; Sunda, Takashi; Unoura, Kiyozumi; Victor, Trent; Schieben, Anna; Keinath, Andreas (1 February 2019). "The "Out-of-the-Loop" concept in automated driving: proposed definition, measures and implications". Cognition, Technology & Work. 21 (1): 87–98. doi:10.1007/s10111-018-0525-8. hdl: 1721.1/118125 . ISSN   1435-5566. S2CID   254136498.
  7. Meilander, Willard C. (6 June 1972). "Method and apparatus for vehicle traffic control" . Retrieved 26 March 2023.
  8. Wiener, Earl L.; Nagel, David C. (1988). Human Factors in Aviation. Gulf Professional Publishing. ISBN   978-0-12-750031-7.
  9. Berberian, Bruno; Gouraud, Jonas; Somon, Bertille; Sahai, Aisha; Le Goff, Kevin (2017). "My Brain is Out of the Loop: A Neuroergonomic Approach of OOTL Phenomenon". Augmented Cognition. Neurocognition and Machine Learning. Lecture Notes in Computer Science. Vol. 10284. Springer International Publishing. pp. 3–18. doi:10.1007/978-3-319-58628-1_1. ISBN   978-3-319-58627-4.
  10. Wang, Aobo; Guo, Beiyuan; Du, Hao; Bao, Haifeng (2022). "Impact of Automation at Different Cognitive Stages on High-Speed Train Driving Performance". IEEE Transactions on Intelligent Transportation Systems. 23 (12): 24599–24608. doi:10.1109/TITS.2022.3211709. ISSN   1558-0016. S2CID   253325209.