Hindsight optimization

Last updated

Hindsight optimisation (HOP) is a computer science technique used in artificial intelligence for analysis of actions which have stochastic results. HOP is used in combination with a deterministic planner. By creating sample results for each of the possible actions from the given state (i.e. determinising the actions), and using the deterministic planner to analyse those sample results, HOP allows an estimate of the actual action.

Related Research Articles

<span class="mw-page-title-main">Finite-state machine</span> Mathematical model of computation

A finite-state machine (FSM) or finite-state automaton, finite automaton, or simply a state machine, is a mathematical model of computation. It is an abstract machine that can be in exactly one of a finite number of states at any given time. The FSM can change from one state to another in response to some inputs; the change from one state to another is called a transition. An FSM is defined by a list of its states, its initial state, and the inputs that trigger each transition. Finite-state machines are of two types—deterministic finite-state machines and non-deterministic finite-state machines. A deterministic finite-state machine can be constructed equivalent to any non-deterministic one.

In theoretical computer science, a nondeterministic Turing machine (NTM) is a theoretical model of computation whose governing rules specify more than one possible action when in some given situations. That is, an NTM's next state is not completely determined by its action and the current symbol it sees, unlike a deterministic Turing machine.

A pseudorandom sequence of numbers is one that appears to be statistically random, despite having been produced by a completely deterministic and repeatable process. Simply put, the problem is that many of the sources of randomness available to humans rely on physical processes not readily available to computer programs.

In electronics and telecommunications, jitter is the deviation from true periodicity of a presumably periodic signal, often in relation to a reference clock signal. In clock recovery applications it is called timing jitter. Jitter is a significant, and usually undesired, factor in the design of almost all communications links.

In cryptography, pseudorandom noise (PRN) is a signal similar to noise which satisfies one or more of the standard tests for statistical randomness. Although it seems to lack any definite pattern, pseudorandom noise consists of a deterministic sequence of pulses that will repeat itself after its period.

<span class="mw-page-title-main">Determinism</span> Philosophical view that events are pre-determined

Determinism is the philosophical view that events are completely determined by previously existing causes. Deterministic theories throughout the history of philosophy have developed from diverse and sometimes overlapping motives and considerations. Like eternalism, determinism focuses on particular events rather than the future as a concept. The opposite of determinism is indeterminism, or the view that events are not deterministically caused but rather occur due to chance. Determinism is often contrasted with free will, although some philosophers claim that the two are compatible. 

Monte Carlo methods, or Monte Carlo experiments, are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use randomness to solve problems that might be deterministic in principle. They are often used in physical and mathematical problems and are most useful when it is difficult or impossible to use other approaches. Monte Carlo methods are mainly used in three problem classes: optimization, numerical integration, and generating draws from a probability distribution.

<span class="mw-page-title-main">Reinforcement learning</span> Field of machine learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.

Indeterminism is the idea that events are not caused, or are not caused deterministically.

<span class="mw-page-title-main">Bandlimiting</span> Limiting a signal to contain only low-frequency components

Bandlimiting refers to a process which reduces the energy of a signal to an acceptably low level outside of a desired frequency range.

Global optimization is a branch of applied mathematics and numerical analysis that attempts to find the global minima or maxima of a function or a set of functions on a given set. It is usually described as a minimization problem because the maximization of the real-valued function is equivalent to the minimization of the function .

<span class="mw-page-title-main">Monte Carlo integration</span> Numerical technique

In mathematics, Monte Carlo integration is a technique for numerical integration using random numbers. It is a particular Monte Carlo method that numerically computes a definite integral. While other algorithms usually evaluate the integrand at a regular grid, Monte Carlo randomly chooses points at which the integrand is evaluated. This method is particularly useful for higher-dimensional integrals.

<span class="mw-page-title-main">Automated planning and scheduling</span> Branch of artificial intelligence

Automated planning and scheduling, sometimes denoted as simply AI planning, is a branch of artificial intelligence that concerns the realization of strategies or action sequences, typically for execution by intelligent agents, autonomous robots and unmanned vehicles. Unlike classical control and classification problems, the solutions are complex and must be discovered and optimized in multidimensional space. Planning is also related to decision theory.

The Planning Domain Definition Language (PDDL) is an attempt to standardize Artificial Intelligence (AI) planning languages. It was first developed by Drew McDermott and his colleagues in 1998 mainly to make the 1998/2000 International Planning Competition (IPC) possible, and then evolved with each competition. The standardization provided by PDDL has the benefit of making research more reusable and easily comparable, though at the cost of some expressive power, compared to domain-specific systems.

<span class="mw-page-title-main">Hip hop production</span> Creation of hip hop music in a recording studio

Hip hop production is the creation of hip hop music in a recording studio. While the term encompasses all aspects of hip hop music creation, including recording the rapping of an MC, a turntablist or DJ providing a beat, playing samples and "scratching" using record players and the creation of a rhythmic backing track, using a drum machine or sequencer, it is most commonly used to refer to recording the instrumental, non-lyrical and non-vocal aspects of hip hop.

Motion planning, also path planning is a computational problem to find a sequence of valid configurations that moves the object from the source to destination. The term is used in computational geometry, computer animation, robotics and computer games.

<span class="mw-page-title-main">Think (About It)</span> 1972 single by Lyn Collins

"Think (About It)" is a funk song recorded by Lyn Collins and released as a single on James Brown's People Records in 1972. The recording was produced by Brown (who also wrote the song) and features instrumental backing from his band The J.B.'s. It was the title track of Collins' 1972 debut album. The song is very popular for its raw drumbeat dressed with tambourine and multiple background vocals, which suggest the song was recorded altogether in one take, with Jabo Starks playing drums. It peaked at No. 9 on the Billboard Best Selling Soul Singles chart and No. 66 on the Hot 100. Owing to the composition, it became a fan favourite and has been featured on various compilation albums posthumously. In the closing lyrics, Collins sings lines from "Think", which shows that this song was one of the few adaptations of the 5 Royales song that Brown loved to do. Think (About It) is among the most sampled songs of all-time.

<span class="mw-page-title-main">Rapidly exploring random tree</span>

A rapidly exploring random tree (RRT) is an algorithm designed to efficiently search nonconvex, high-dimensional spaces by randomly building a space-filling tree. The tree is constructed incrementally from samples drawn randomly from the search space and is inherently biased to grow towards large unsearched areas of the problem. RRTs were developed by Steven M. LaValle and James J. Kuffner Jr. They easily handle problems with obstacles and differential constraints and have been widely used in autonomous robotic motion planning.

<span class="mw-page-title-main">Sampling (music)</span> Reuse of sound recording in another recording

In sound and music, sampling is the reuse of a portion of a sound recording in another recording. Samples may comprise elements such as rhythm, melody, speech, sound effects or longer portions of music, and may be layered, equalized, sped up or slowed down, repitched, looped, or otherwise manipulated. They are usually integrated using electronic music instruments (samplers) or software such as digital audio workstations.

In probability theory, a piecewise-deterministic Markov process (PDMP) is a process whose behaviour is governed by random jumps at points in time, but whose evolution is deterministically governed by an ordinary differential equation between those times. The class of models is "wide enough to include as special cases virtually all the non-diffusion models of applied probability." The process is defined by three quantities: the flow, the jump rate, and the transition measure.

References