Inference engine

In the field of artificial intelligence, an inference engine is a component of an intelligent system that applies logical rules to the knowledge base to deduce new information. The first inference engines were components of expert systems. The typical expert system consisted of a knowledge base and an inference engine. The knowledge base stored facts about the world. The inference engine applied logical rules to the knowledge base and deduced new knowledge. This process would iterate as each new fact in the knowledge base could trigger additional rules in the inference engine. Inference engines work primarily in one of two modes either special rule or facts: forward chaining and backward chaining. Forward chaining starts with the known facts and asserts new facts. Backward chaining starts with goals, and works backward to determine what facts must be asserted so that the goals can be achieved. ^[1]

Additionally, the concept of 'inference' has expanded to include the process through which trained neural networks generate predictions or decisions. In this context, an 'inference engine' could refer to the specific part of the system, or even the hardware, that executes these operations. This type of inference plays a crucial role in various applications, including (but not limited to) image recognition, natural language processing, and autonomous vehicles. The inference phase in these applications is typically characterized by a high volume of data inputs and real-time processing requirements.

Architecture

The logic that an inference engine uses is typically represented as IF-THEN rules. The general format of such rules is IF <logical expression> THEN <logical expression>. Prior to the development of expert systems and inference engines, artificial intelligence researchers focused on more powerful theorem prover environments that offered much fuller implementations of first-order logic. For example, general statements that included universal quantification (for all X some statement is true) and existential quantification (there exists some X such that some statement is true). What researchers discovered is that the power of these theorem-proving environments was also their drawback. Back in 1965, it was far too easy to create logical expressions that could take an indeterminate or even infinite time to terminate. For example, it is common in universal quantification to make statements over an infinite set such as the set of all natural numbers. Such statements are perfectly reasonable and even required in mathematical proofs but when included in an automated theorem prover executing on a computer may cause the computer to fall into an infinite loop. Focusing on IF-THEN statements (what logicians call modus ponens ) still gave developers a very powerful general mechanism to represent logic, but one that could be used efficiently with computational resources. What is more, there is some psychological research that indicates humans also tend to favor IF-THEN representations when storing complex knowledge. ^[2]

A simple example of modus ponens often used in introductory logic books is "If you are human then you are mortal". This can be represented in pseudocode as:

Rule1: Human(x) => Mortal(x)

A trivial example of how this rule would be used in an inference engine is as follows. In forward chaining, the inference engine would find any facts in the knowledge base that matched Human(x) and for each fact it found would add the new information Mortal(x) to the knowledge base. So if it found an object called Socrates that was human it would deduce that Socrates was mortal. In backward chaining, the system would be given a goal, e.g. answer the question is Socrates mortal? It would search through the knowledge base and determine if Socrates was human and, if so, would assert he is also mortal. However, in backward chaining a common technique was to integrate the inference engine with a user interface. In that way, rather than simply being automated the system could now be interactive. In this trivial example, if the system was given the goal to answer the question if Socrates was mortal and it didn't yet know if he was human, it would generate a window to ask the user the question "Is Socrates human?" and would then use that information accordingly.

This innovation of integrating the inference engine with a user interface led to the second early advancement of expert systems: explanation capabilities. The explicit representation of knowledge as rules rather than code made it possible to generate explanations to users: both explanations in real time and after the fact. So if the system asked the user "Is Socrates human?", the user may wonder why she was being asked that question and the system would use the chain of rules to explain why it was currently trying to ascertain that bit of knowledge: that is, it needs to determine if Socrates is mortal and to do that needs to determine if he is human. At first these explanations were not much different than the standard debugging information that developers deal with when debugging any system. However, an active area of research was utilizing natural language technology to ask, understand, and generate questions and explanations using natural languages rather than computer formalisms. ^[3]

An inference engine cycles through three sequential steps: match rules, select rules, and execute rules. The execution of the rules will often result in new facts or goals being added to the knowledge base, which will trigger the cycle to repeat. This cycle continues until no new rules can be matched.

In the first step, match rules, the inference engine finds all of the rules that are triggered by the current contents of the knowledge base. In forward chaining, the engine looks for rules where the antecedent (left hand side) matches some fact in the knowledge base. In backward chaining, the engine looks for antecedents that can satisfy one of the current goals.

In the second step, select rules, the inference engine prioritizes the various rules that were matched to determine the order to execute them. In the final step, execute rules, the engine executes each matched rule in the order determined in step two and then iterates back to step one again. The cycle continues until no new rules are matched. ^[4]

Implementations

Early inference engines focused primarily on forward chaining. These systems were usually implemented in the Lisp programming language. Lisp was a frequent platform for early AI research due to its strong capability to do symbolic manipulation. Also, as an interpreted language it offered productive development environments appropriate to debugging complex programs. A necessary consequence of these benefits was that Lisp programs tended to be slower and less robust than compiled languages of the time such as C. A common approach in these early days was to take an expert system application and repackage the inference engine used for that system as a re-usable tool other researchers could use for the development of other expert systems. For example, MYCIN was an early expert system for medical diagnosis and EMYCIN was an inference engine extrapolated from MYCIN and made available for other researchers. ^[1]

As expert systems moved from research prototypes to deployed systems there was more focus on issues such as speed and robustness. One of the first and most popular forward chaining engines was OPS5, which used the Rete algorithm to optimize the efficiency of rule firing. Another very popular technology that was developed was the Prolog logic programming language. Prolog focused primarily on backward chaining and also featured various commercial versions and optimizations for efficiency and robustness. ^[5]

As expert systems prompted significant interest from the business world, various companies, many of them started or guided by prominent AI researchers created productized versions of inference engines. For example, Intellicorp was initially guided by Edward Feigenbaum. These inference engine products were also often developed in Lisp at first. However, demands for more affordable and commercially viable platforms eventually made personal computer platforms very popular.

Open source implementations

ClipsRules and RefPerSys (inspired by CAIA ^[6] and the work of Jacques Pitrat). The Frama-C static source code analyzer also uses some inference engine techniques.

See also

• Geometric and Topological Inference
• Action selection
• Backward chaining
• Expert system
• Forward chaining
• Inductive inference

References

1. Hayes-Roth, Frederick; Donald Waterman; Douglas Lenat (1983). Building Expert Systems. Addison-Wesley. ISBN   0-201-10686-8.
2. Feigenbaum, Edward; Avron Barr (September 1, 1986). The Handbook of Artificial Intelligence, Volume I. Addison-Wesley. p. 195. ISBN   0201118114.
3. Barzilayt, Regina; Daryl McCullough; Owen Rambow; Jonathan DeCristofaro; Tanya Korelsky; Benoit Lavoie. "A New Approach to Expert System Explanations" (PDF). USAF Rome Laboratory Report. Archived from the original on July 5, 2016.
4. Griffin, N.L., A Rule-Based Inference Engine which is Optimal and VLSI Implementable, University of Kentucky.
5. Sterling, Leon; Ehud Shapiro (1986). The Art of Prolog. Cambridge, MA: MIT. ISBN   0-262-19250-0.
6. Pitrat, Jacques (2009). Artificial Beings, the conscience of a conscious machine. Wiley. ISBN   978-1848211018.