Reflective agent architecture with self-reflection, evaluation, short/long-term memory, and environment interaction.
Reflection in artificial intelligence is ability of large language models (LLMs) to examine, evaluate, and refine their own outputs. By incorporating self-assessment and internal deliberation, reflective AI aims to improve reasoning accuracy, reduce errors (such as hallucinations), and enhance interpretability. It is a form of "test-time compute".
This internal process of "thinking" about the steps leading to an answer is analogous to human metacognition or "thinking about thinking." It helps AI systems approach tasks that require multi-step reasoning, planning, and logical thought. Reflection can occur either after completing a full processing cycle (intermediate outputs may be hidden) and generating output, or continuously throughout the process.[1][2] In LLMs, special tokens can mark the beginning and end of reflection before producing a final response (e.g., <thinking>).
Background
Traditional neural networks process inputs in a feedforward manner, generating outputs in a single pass. However, their limitations in handling complex reasoning tasks have led to the development of methods that simulate internal deliberation. Techniques such as chain-of-thought prompting encourage models to generate intermediate reasoning steps, thereby providing a form of self-reflection that can improve performance on tasks including arithmetic, commonsense reasoning, and more.
Techniques
Increasing the length of the Chain-of-Thought reasoning process, by passing the output of the model back to its input and doing multiple network passes, has been used to increases inference-time scaling[3]. Reinforcement learning frameworks such as Group Relative Policy Optimization [4] have been used to steer the Chain-of-Though to produce desired final outputs[5]
Sequential scaling
Refines reasoning by allowing later computations to build upon earlier ones, improving accuracy through iterative refinement.
Parallel scaling
Processes multiple reasoning paths independently and selects the most reliable outcome based on evaluation metrics, enhancing robustness and diversity of solutions.
In the 2025 paper "s1: Simple test-time scaling,"[6] demonstrated the effectiveness of budget forcing and scaling techniques using the s1-32B model, a fine-tuned version of Qwen2.5-32B-Instruct. By training on a carefully curated dataset of 1,000 examples (s1K) and implementing budget forcing, s1-32B matched or outperformed larger proprietary models like OpenAI's o1-preview. Notably, it exceeded o1-preview by up to 27% on competition math benchmarks (MATH and AIME24). Furthermore, applying scaling strategies improved its AIME24 score from 50% to 57% without additional test-time intervention.
Types of reflection
Post-hoc reflection
Analyzes and critiques an initial output separately. This often involves prompting the model to identify errors or suggest improvements after generating a response. The Reflexion framework follows this approach.[7][8]
Iterative reflection
Revises earlier parts of a response dynamically during generation. Self-monitoring mechanisms allow the model to adjust reasoning as it progresses. Methods like Tree-of-Thoughts exemplify this, enabling backtracking and alternative exploration.
Intrinsic reflection
Integrates self-monitoring directly into the model architecture rather than relying solely on external prompts. This approach could enable models with inherent awareness of their reasoning limitations and uncertainties. This has been used by Google DeepMind in a technique called Self-Correction via Reinforcement Learning (SCoRe) which rewards the model for improving its responses.[9]
Model architectures
Architectures incorporating reflection mechanisms
Certain model architectures integrate explicit reflection mechanisms to enhance self-monitoring and reasoning awareness. These architectures may include additional components designed to analyze intermediate steps and refine outputs based on iterative feedback.
Self-monitoring mechanisms
Some architectures embed modules that continuously assess confidence levels and logical consistency. These mechanisms help improve response quality by identifying potential reasoning flaws in real-time.
Modular approaches for dynamic reasoning feedback
Incorporates specialized reasoning modules that can be selectively activated depending on the complexity of the task. These modular systems enable dynamic adjustments to reasoning depth and strategy during inference.
Training strategies
Reasoning templates
Includes structured reasoning examples in training data, helping the model generalize logical steps more effectively and improving coherence.
Specialized loss functions
Encourages intermediate reasoning steps and penalizes inconsistencies between generated outputs and self-reflections. This approach reinforces logical consistency and reduces hallucinated responses.
Benchmarks
Reflective models generally outperform non-reflective models in most benchmarks, especially on tasks requiring multi-step reasoning.
However, some benchmarks exclude reflective models due to longer response times.
Humanity's Last Exam
The HLE, a rigorous benchmark designed to assess expert-level reasoning across mathematics, humanities, and the natural sciences, reveals substantial performance gaps among models. State-of-the-art reasoning models have demonstrated low accuracy on HLE, highlighting significant room for improvement. In particular, the full reasoning model o3 achieved an accuracy of 26.6%,[10] while its lighter counterpart, o3‑mini (high) (evaluated on text‑only questions), reached 13%.[11]
AIME
The American Invitational Mathematics Examination (AIME) benchmark, a challenging mathematics competition, demonstrates significant performance differences between model types. Non-reasoning models typically solve less than 30% of AIME. In contrast, models employing reasoning techniques score between 50% and 80%.[12] While OpenAI's o1 maintained or slightly improved its accuracy from reported 2024[citation needed] metrics to 2025 AIME results, o3-mini (high) achieved a higher accuracy (80%) at a significantly lower cost (approximately 12 times cheaper).
o3-mini performance
According to OpenAI's January 2025 report on o3-mini, adjustable "reasoning effort" significantly affects performance, particularly in STEM. Increasing reasoning effort from low to high boosts accuracy on benchmarks like AIME 2024, GPQA Diamond, and Codeforces, providing performance gains typically in the range of 10-30%. With high reasoning effort, o3-mini (high) achieved 87.3% in AIME (different from the MathArena AIME benchmark results), 79.7% in GPQA Diamond, 2130 Elo in Codeforces, and 49.3 in SWE-bench Verified.[13]
Integration with search capabilities
In December 2024, Google introduced Deep Research in Gemini,[14] a feature in Gemini that conducts multi-step research tasks.
On January 25, 2025, DeepSeek launched a feature in their DeepSeek R1 model, enabling the simultaneous use of search and reasoning capabilities, which allows for more efficient integration of data retrieval with reflective reasoning processes.
Subsequently, OpenAI's o3-mini model gained the ability to combine search and reasoning in a unified process.
On February 2, 2025, OpenAI released deep research,[15] a tool that integrates reasoning and web search in a unified workflow, allowing users to perform complex research tasks that require multi-step reasoning and data synthesis from multiple sources. It is based on o3 and can take from 5 to 30 minutes to generate comprehensive reports.[16]
History
2024
o1-preview, a LLM with enhanced reasoning released in September 2024, showed significant improvements on benchmarks.[17] In December 2024, the full version o1 was released, incorporating lessons learned from the preview stage. OpenAI also shared preliminary results on its successor, o3, featuring new records on benchmarks for coding, science and mathematics.[18]
Alibaba also released reasoning versions of its Qwen family of LLMs, such as QwQ-32B-Preview and QvQ-72B-Preview in late 2024.
2025
In January 2025, the Chinese company DeepSeek gained significant attention by releasing DeepSeek R1, a reasoning model competitive with o1 but at a much lower cost.[19] In the following weeks, OpenAI released o3-mini and a variant using more "reasoning effort" called o3-mini-high. It also released deep research, that uses o3.[16]
Applications
Mathematical and logical reasoning
Reflection enables LLMs to solve multi-step problems, demonstrated on benchmarks like FrontierMath,[20] GSM8K (mathematical word problems), GPQA Diamond (PhD-level Science Questions) and Big-Bench Hard (challenging reasoning tasks). A model might initially produce an incorrect solution but, through self-reflection, identify the flawed step and generate a corrected answer.
Vision-Language tasks
Frameworks like R3V allow vision-language models to iteratively refine reasoning on complex multimodal tasks. In visual question answering, the model might first generate a plausible but incorrect answer based on a superficial understanding. Through reflection, it could identify inconsistencies between its answer and image details, leading to a revised, more accurate response.[21]
General Problem Solving
Enhanced reflection leads to improved coherence, long-term planning, and reduced hallucinations. This is valuable in tasks requiring planning, sequential decision-making, or creative problem-solving, like writing code, composing stories, or designing experiments.
QvQ-72B-Preview – an experimental visual reasoning model launched on December 24, 2024, which integrates image understanding with verbal chain-of-thought reasoning.
QwQ-32B-Preview – an experimental text-based reasoning model released in late November 2024 that emphasizes complex, step-by-step analysis.
On December 16, 2024, an experiment using a Llama 3B model demonstrated that by scaling test-time compute, a relatively small model could outperform a much larger Llama 70B model on challenging reasoning tasks. This result highlighted that improved inference strategies can unlock latent reasoning capabilities even in compact models.[22]
Reflective models require significantly more test-time compute than non-reasoning models. On the AIME benchmark, reasoning models were 10 to 74 times more expensive[12] than non-reasoning counterparts. The cheapest model, Gemini 2.0-Flash, cost just $0.06 per benchmark.
Latency Issues
Reflective reasoning significantly increases response times, with current models taking anywhere from three seconds to several minutes to generate an answer. As reasoning depth improves, future models may require even longer processing times.
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
