FORR

Last updated

FORR (FOr the Right Reasons) is a cognitive architecture for learning and problem solving inspired by Herbert A. Simon's ideas of bounded rationality and satisficing. It was first developed in the early 1990s at the City University of New York. It has been used in game playing, robot pathfinding, recreational park design, spoken dialog systems, and solving NP-hard constraint satisfaction problems, and is general enough for many problem solving applications.

A cognitive architecture can refer to a theory about the structure of the human mind. One of the main goals of a cognitive architecture is to summarize the various results of cognitive psychology in a comprehensive computer model. However, the results need to be formalized so far as they can be the basis of a computer program. The formalized models can be used to further refine a comprehensive theory of cognition, and more immediately, as a commercially usable model. Successful cognitive architectures include ACT-R and SOAR.

Machine learning branch of statistics and computer science, which studies algorithms and architectures that learn from observed facts

Machine learning (ML) is the scientific study of algorithms and statistical models that computer systems use to effectively perform a specific task without using explicit instructions, relying on patterns and inference instead. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model of sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to perform the task. Machine learning algorithms are used in a wide variety of applications, such as email filtering, and computer vision, where it is infeasible to develop an algorithm of specific instructions for performing the task. Machine learning is closely related to computational statistics, which focuses on making predictions using computers. The study of mathematical optimization delivers methods, theory and application domains to the field of machine learning. Data mining is a field of study within machine learning, and focuses on exploratory data analysis through unsupervised learning. In its application across business problems, machine learning is also referred to as predictive analytics.

Problem solving consists of using generic or ad hoc methods in an orderly manner to find solutions to problems. Some of the problem-solving techniques developed and used in philosophy, artificial intelligence, computer science, engineering, mathematics, or medicine are related to mental problem-solving techniques studied in psychology.

Contents

Background

Bounded rationality

FORR does not have perfect knowledge of how to solve a problem, but instead learns from experience. Intelligent agents are not optimal, but make decisions based on only a subset of all possible good reasons and informative data. These agents can still be considered rational. This idea of bounded rationality was introduced by Herbert A. Simon, [1] who along with Allen Newell developed the early foundations of the study of cognitive architectures and also inspired early architectures such as Soar and ACT-R.

Bounded rationality is the idea that when individuals make decisions, their rationality is limited by the tractability of the decision problem, the cognitive limitations of their minds, and the time available to make the decision. Decision-makers in this view act as satisficers, seeking a satisfactory solution rather than an optimal one. Herbert A. Simon proposed bounded rationality as an alternative basis for the mathematical modeling of decision-making, as used in economics, political science and related disciplines. It complements "rationality as optimization", which views decision-making as a fully rational process of finding an optimal choice given the information available. Simon used the analogy of a pair of scissors, where one blade represents "cognitive limitations" of actual humans and the other the "structures of the environment", illustrating how minds compensate for limited resources by exploiting known structural regularity in the environment.

Herbert A. Simon American political scientist, economist, sociologist, and psychologist

Herbert Alexander Simon was an American economist, political scientist and cognitive psychologist, whose primary research interest was decision-making within organizations and is best known for the theories of "bounded rationality" and "satisficing". He received the Nobel Prize in Economics in 1978 and the Turing Award in 1975. His research was noted for its interdisciplinary nature and spanned across the fields of cognitive science, computer science, public administration, management, and political science. He was at Carnegie Mellon University for most of his career, from 1949 to 2001.

Allen Newell was a researcher in computer science and cognitive psychology at the RAND Corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology. He contributed to the Information Processing Language (1956) and two of the earliest AI programs, the Logic Theory Machine (1956) and the General Problem Solver (1957). He was awarded the ACM's A.M. Turing Award along with Herbert A. Simon in 1975 for their basic contributions to artificial intelligence and the psychology of human cognition.

Multiple good reasons

FORR depends upon the idea that there are multiple reasons or rationales for performing actions while solving a problem. These reasons can be always right (it's always right to make a move in chess that will put the opponent in checkmate) or just sometimes right. The always-right reasons are the minority. The sometimes-right reasons can complete with each other: for example, in game playing, one good reason might be to capture pieces, while another might be to control some area of the board. In FORR, these competing reasons are called Advisors.

The tiered Advisor system is general enough that any potential good reason, such as probabilistic, deductive, or perceptual can be implemented, so long as it gives advice on its preference of one action over another.

Perception organization, identification, and interpretation of sensory information in order to represent and understand the environment

Perception is the organization, identification, and interpretation of sensory information in order to represent and understand the presented information, or the environment.

Because of its reliance on a set of independent agents (the Advisors), FORR can be considered a connectionist architecture.

Connectionism

Connectionism is an approach in the fields of cognitive science, that hopes to explain mental phenomena using artificial neural networks (ANN).

The Architecture

A FORR architecture has three components: a set of descriptives that describe the state of the problem, a tiered set of Advisors that are consulted in order to decide what action to perform, and a behavioral script that queries the Advisors and performs the action that they suggest. [2]


Advisors

The Advisors are the set of rationales or heuristics for making a decision. They can be considered the procedural memory component of the architecture. Upon each new decision, Advisors are queried in order to decide which action to perform. Advisors never communicate with each other or learn on their own: they simply ask for information about the state of the problem stored in the form of descriptives, and make a suggestion based on that information. The Advisors are divided into three tiers, which are queried in the following order:

Procedural memory is a type of implicit memory and long-term memory which aids the performance of particular types of tasks without conscious awareness of these previous experiences.

Descriptives

The declarative memory component of the architecture, the descriptives represent the state of the problem and are available to any Advisor.

Behavioral script

The behavioral script queries each tier of Advisors sequentially. If a tier 1 Advisor suggests an action, the script performs the action. Otherwise, if a tier 2 Advisor is triggered, it means that a sub-problem has been encountered. A tier 1 Advisor guarantees that only one tier 2 Advisor is active at any time. If no tier 1 Advisor comments and no tier 2 Advisor is triggered, the behavioral script asks for suggestions or comments from all tier 3 Advisors and lets them vote. The script performs the action with the highest vote among all tier 3 advisors.

Implementing a FORR architecture

A problem domain is a set of similar problems, called the problem classes. If the problem domain is playing simple board games, then tic-tac-toe is a problem class, and one particular game of tic-tac-toe is a problem instance. If navigating a maze is the problem domain, then a particular maze is the class and one attempt at its navigation is an instance. Once the problem domain is identified, the implementation of a FORR architecture for that domain has two basic stages: finding possible right reasons (the Advisors) and learning their weights for a particular class.

How to build a FORR architecture

  1. Decide on a problem domain.
  2. Use domain knowledge, surveys of the literature, intuition and good sense to enumerate a list of possible rationales for making a decision, which can be good or bad for different classes within the domain. These rationales are the Advisors.
  3. Divide the Advisors into tiers:
    1. The Advisors that are always right are in Tier 1. For example, it's always right to make a winning move in a board game.
    2. The Advisors which identify a sub-problem go into Tier 2. For example, going around a wall in a maze.
    3. Every other Advisor is Tier 3.
  4. Code the Advisors. Each Advisor returns a set of suggested actions along with weights for each suggested action. The weights are initially set to a uniform value, such as 0.05.
  5. Identify all information about the state of the problem needed by all Advisors. These are the descriptives. Code these.
  6. Code the behavioral script which queries the Advisors and performs the action they suggest.
  7. Learn the weights for the Advisors on a set of particular problem instances in the Learning Phase using a Reinforcement learning algorithm.
  8. Test the architecture on a set of previously unencountered problem instances.

Learning Advisor weights

The Advisors are the same for all problem classes in a domain, but the weights can be different for each class within the domain. Important heuristics for tic-tac-toe might not be important for a different board game. FORR learns the weights for its tier 3 Advisors by experience. Advisors that suggest an action resulting in failure have their weights penalized, and Advisors whose suggestions result in success have their weights increased. Learning algorithms vary between implementations.

Applications

FORR has been used for game playing, robot pathfinding, constraint satisfaction problems, park design, and spoken dialog systems. [3] [4]

See also

Related Research Articles

Reason is the capacity of consciously making sense of things, establishing and verifying facts, applying logic, and changing or justifying practices, institutions, and beliefs based on new or existing information. It is closely associated with such characteristically human activities as philosophy, science, language, mathematics and art, and is normally considered to be a distinguishing ability possessed by humans. Reason, or an aspect of it, is sometimes referred to as rationality.

Rationality is the quality or state of being rational – that is, being based on or agreeable to reason. Rationality implies the conformity of one's beliefs with one's reasons to believe, and of one's actions with one's reasons for action. "Rationality" has different specialized meanings in philosophy, economics, sociology, psychology, evolutionary biology, game theory and political science.

Constraint satisfaction problems (CSPs) are mathematical questions defined as a set of objects whose state must satisfy a number of constraints or limitations. CSPs represent the entities in a problem as a homogeneous collection of finite constraints over variables, which is solved by constraint satisfaction methods. CSPs are the subject of intense research in both artificial intelligence and operations research, since the regularity in their formulation provides a common basis to analyze and solve problems of many seemingly unrelated families. CSPs often exhibit high complexity, requiring a combination of heuristics and combinatorial search methods to be solved in a reasonable time. The Boolean satisfiability problem (SAT), the satisfiability modulo theories (SMT) and answer set programming (ASP) can be roughly thought of as certain forms of the constraint satisfaction problem.

Procedural knowledge, also known as imperative knowledge, is the knowledge exercised in the performance of some task. See below for the specific meaning of this term in cognitive psychology and intellectual property law.

Modularity of mind is the notion that a mind may, at least in part, be composed of innate neural structures or modules which have distinct established evolutionarily developed functions. Somewhat different definitions of "module" have been proposed by different authors.

Soar is a cognitive architecture, originally created by John Laird, Allen Newell, and Paul Rosenbloom at Carnegie Mellon University. It is now maintained and developed by John Laird's research group at the University of Michigan.

A cognitive tutor is a particular kind of intelligent tutoring system that utilizes a cognitive model to provide feedback to students as they are working through problems. This feedback will immediately inform students of the correctness, or incorrectness, of their actions in the tutor interface; however, cognitive tutors also have the ability to provide context-sensitive hints and instruction to guide students towards reasonable next steps.

Learning classifier system

Learning classifier systems, or LCS, are a paradigm of rule-based machine learning methods that combine a discovery component with a learning component. Learning classifier systems seek to identify a set of context-dependent rules that collectively store and apply knowledge in a piecewise manner in order to make predictions. This approach allows complex solution spaces to be broken up into smaller, simpler parts.

Spatial memory

In cognitive psychology and neuroscience, spatial memory is that part of the memory responsible for the recording of information about one's environment and spatial orientation. For example, a person's spatial memory is required in order to navigate around a familiar city, just as a rat's spatial memory is needed to learn the location of food at the end of a maze. It is often argued that in both humans and animals, spatial memories are summarized as a cognitive map. Spatial memory has representations within working, short-term memory and long-term memory. Research indicates that there are specific areas of the brain associated with spatial memory. Many methods are used for measuring spatial memory in children, adults, and animals.

Piagets theory of cognitive development piaget psycology theory

Piaget's theory of cognitive development is a comprehensive theory about the nature and development of human intelligence. It was first created by the Swiss developmental psychologist Jean Piaget (1896–1980). The theory deals with the nature of knowledge itself and how humans gradually come to acquire, construct, and use it. Piaget's theory is mainly known as a developmental stage theory. Piaget "was intrigued by the fact that children of different ages made different kinds of mistakes while solving problems". He also believed that children are not like "little adults" who may know less; children just think and say words in a different way. By Piaget thinking that children have great cognitive abilities, he came up with four different cognitive development stages, which he put out into testing. Within those four stages he managed to group them with different ages. Each stage he realized how children managed to develop their cognitive skills. For example, he believed that children experience the world through actions, representing things with words, thinking logically, and using reasoning.

Unified Theories of Cognition is a 1990 book by Allen Newell. Newell argues for the need of a set of general assumptions for cognitive models that account for all of cognition: a unified theory of cognition, or cognitive architecture.

Intelligent agent

In artificial intelligence, an intelligent agent (IA) is an autonomous entity which acts, directing its activity towards achieving goals, upon an environment using observation through sensors and consequent actuators. Intelligent agents may also learn or use knowledge to achieve their goals. They may be very simple or very complex. A reflex machine, such as a thermostat, is considered an example of an intelligent agent.

John Robert Anderson is a Canadian-born American psychologist. He is currently professor of Psychology and Computer Science at Carnegie Mellon University.

Outline of thought Overview of and topical guide to thought

The following outline is provided as an overview of and topical guide to thought (thinking):

The worked-example effect is a learning effect predicted by cognitive load theory. Specifically, it refers to the learning effect observed when worked-examples are used as part of instruction, compared to other instructional techniques such as problem-solving and discovery learning. According to Sweller: "The worked example effect is the best known and most widely studied of the cognitive load effects".

Psi-theory, developed by Dietrich Dörner at the University of Bamberg, is a systemic psychological theory covering human action regulation, intention selection and emotion. It models the human mind as an information processing agent, controlled by a set of basic physiological, social and cognitive drives. Perceptual and cognitive processing are directed and modulated by these drives, which allow the autonomous establishment and pursuit of goals in an open environment.

Neo-Piagetian theories of cognitive development criticize and build upon Jean Piaget's theory of cognitive development.

Most of the terms listed in Wikipedia glossaries are already defined and explained within Wikipedia itself. However, glossaries like this one are useful for looking up, comparing and reviewing large numbers of terms together. You can help enhance this page by adding new terms or writing definitions for existing ones.

References

  1. Simon, H.A. (1981) The Sciences of the Artificial
  2. Epstein (1992) Capitalizing on Conflict: The FORR Architecture
  3. For example applications see the FORR homepage
  4. For the use of FORR in spoken dialog systems, see FORRSooth at The Loqui project
Notes