Metacognition in LLMs: Foundations, Progress, and Opportunities | Summary

Contents

• Summary
• 2 What is Metacognition?
• 4 Do LLMs Have Metacognition?
  • 4.1 Measuring Metacognition in LLMs
  • 4.2 Current Findings on Metacognition in LLMs
• 5 Giving LLMs Metacognitive Abilities
  • 5.1 Implementations of Metacognition in LLMs
  • 5.2 Metacognition for Reasoning Models
  • 5.3 Metacognition for LLM Agents
• 6 Metacognitive Methods to Improve Capabilities of LLMs
  • 6.1 Improving Task Performance
  • 6.2 Improving Learning Efficacy
• 7 Applications of LLM Metacognition
  • Human-AI Decision-Making
  • User Simulation
  • Pedagogy
• 8 Reflections & Discussion
  • What are risks of LLM metacognition?
• 9 Future Directions
• Brief Thoughts

This article explains the key points of Metacognition in LLMs: Foundations, Progress, and Opportunities.

• 2026-07-13 (arXiv)
• Liu, Gabrielle Kaili-May, Gani, Areeb, Lu, Jacqueline, Thomas, Jordan, Steyvers, Mark, Cohan, Arman.
• Yale University, University of California, Irvine
• Paper

Read this article in Korean

Summary

• This survey examines when and how LLMs monitor and regulate their own processing, and whether these abilities support performance, reliability, and human–AI interaction. It organizes research on measurement, observed behaviors, implementations, applications, and open questions rather than reporting new experiments.
• The organizing distinction is between monitoring—assessing uncertainty, knowledge, performance, or progress—and control—using those assessments to plan, change strategy, allocate effort, or seek help. The studies surveyed provide task-dependent evidence of these behaviors but also report overconfidence, unreliable self-assessment, and failures to act on available self-knowledge.

The center places a language model between a monitor and a control component: monitoring assesses processing, while control regulates subsequent action. Surrounding branches organize the reviewed work on measurement, findings, implementations, capability improvements, and applications; the diagram is a research taxonomy, not a validated architecture.

• The review distinguishes confidence calibration from metacognitive sensitivity and asks whether monitoring informs control. Comparisons across studies remain difficult because tasks, models, and confidence-elicitation procedures differ; the authors provide an organized paper list at https://github.com/yale-nlp/LLM-Metacognition.

2 What is Metacognition?

In the human framework, metacognition links monitoring of one’s competence to control of subsequent actions. The review distinguishes metacognitive knowledge, regulation, and experience; sensitivity concerns whether confidence discriminates correct from incorrect responses, whereas calibration concerns its correspondence to the probability of correctness.

There is no agreed operational definition for LLM metacognition. The survey includes psychologically grounded monitoring and control as well as looser uses of the term, such as reflection and output revision, without treating them as equivalent; proposed benefits include task performance, learning, and recognizing when help or abstention is appropriate.

4 Do LLMs Have Metacognition?

The motivations include reliable uncertainty communication, human–AI collaboration, hallucination reduction, and interpretability. These remain conditional benefits: cited studies also report high-confidence hallucinations and discrepancies between reasoning and final answers, so reflective text or a confident explanation alone does not establish reliable self-monitoring.

4.1 Measuring Metacognition in LLMs

Psychology-inspired evaluations use signal detection theory to separate metacognitive, or type 2, sensitivity from task, or type 1, sensitivity and response bias. The review discusses meta-d′, d′, M-ratio, M-diff, and information-theoretic alternatives; confidence elicitation and correctness judgments make many such measures less direct for open-ended generation.

Other approaches test monitoring or control of internal activations in neurofeedback-like settings, compare intrinsic confidence with reasoning-step quality, probe model representations, or use task-specific behavior. CogEdit, MetaMedQA, ObjexMT, and AwareXtend assess different targets, but constrained formats and dependence on general task ability prevent them from serving as a unified measure.

4.2 Current Findings on Metacognition in LLMs

The reviewed work generally reports weak to moderate metacognitive sensitivity and task-dependent calibration, although results on particular tasks differ. Debate studies document escalating or mutually incompatible confidence; judgments-of-learning studies find poor prediction of future memory performance; and studies of uncertainty communication report discrepancies between expressed and intrinsic uncertainty.

Some studies find that models anticipate their answers, report or modulate a limited subset of activation directions, describe learned behaviors, or improve self-knowledge after training. The findings depend on task and elicitation conditions: better reasoning need not improve self-assessment, and information predictive of difficulty does not necessarily produce adaptive action.

Measurement choices change the apparent result. In one cited study, over 75% of verbal confidence responses on a 0–100 scale fall on three values; another reports approximate M-ratios of 0.85–1.05 from token log-probabilities, versus approximately 0.62–0.92 reported with self-ratings in a different study and setting. Model family, post-training, temperature, and domain also complicate comparison, while metacognitive efficiency can distinguish models with similar conventional calibration scores.

5 Giving LLMs Metacognitive Abilities

The review groups attempts to confer metacognitive functions into frameworks, architectures, prompting, and training strategies for LLMs, reasoning models, and agents. Much of the implementation work centers on reasoning; empirical support for particular methods and coverage of other uses remain uneven.

5.1 Implementations of Metacognition in LLMs

Monitor-Generate-Verify (MGV) places monitoring before generation and verification afterward; Pangu Embedded combines dual-process-inspired reasoning with fine-tuning, reinforcement learning, and metacognitive prompts. Architectural examples include State Stream Transformer (SST), SAGE-nano, and the two-tier Metacognitive Introspective Reward Architecture (MIRA), each drawn from a separate cited study.

5.2 Metacognition for Reasoning Models

For large reasoning models, inference-time approaches select strategies, reuse prior reasoning templates, assess whether further thought is worthwhile, or end redundant deliberation. One cited early-exit method reduces reasoning length by almost 35% with minimal accuracy loss in its evaluation; training approaches instead use self-critiqued traces, predicted generation statistics, correctness signals, or estimates of the value of additional compute.

5.3 Metacognition for LLM Agents

Agent-oriented work applies monitoring and control to learning progress, error correction, multi-agent coordination, collaboration, memory abstraction, and decisions about search, retrieval, and tools. Cited examples include deciding when to ask a human expert for help, training a separate model via DPO to store reusable memories at a suitable abstraction level, and deciding whether existing knowledge suffices before retrieving more evidence.

6 Metacognitive Methods to Improve Capabilities of LLMs

The survey separates methods aimed at improving performance on a target task from those aimed at improving learning efficacy. Both groups use assessments of performance or progress to inform correction, resource allocation, or strategy choice, though their reported benefits depend on the specific method and task.

6.1 Improving Task Performance

Task-focused methods address general problem-solving, confidence, hallucinations, knowledge boundaries, interpretability, reasoning in non-reasoning models, and retrieval. The review distinguishes calibration against answer accuracy from “faithful calibration,” which aligns expressed with intrinsic uncertainty; reinforcement learning with metacognitive feedback (RLMF) is a cited approach to the latter.

Other examples use reflection to select reasoning strategies, decide when retrieval is needed, identify biases, or guide tool use. Benefits are not uniform: a cited persuasion study finds that eliciting self-reported confidence can increase vulnerability to belief erosion, illustrating the need to evaluate each intervention against its intended outcome.

6.2 Improving Learning Efficacy

Learning-oriented examples use fast consistency and slower experience-driven monitors to detect and correct unreliable search trajectories, self-critique signals to train correction, and metacognitive sensitivity to choose among expert models. Memory-of-thought and self-refinement frameworks offer routes to iterative improvement, but the survey does not establish general autonomous self-improvement.

7 Applications of LLM Metacognition

The review identifies three application areas: human–AI decision-making, user simulation, and pedagogy. In each, outcomes depend not just on model monitoring but also on how people or downstream systems use the resulting signals.

Human-AI Decision-Making

In collaborative decisions, the cited theoretical and empirical work separates AI predictive accuracy from metacognitive sensitivity: confidence that better distinguishes correct from incorrect responses can improve joint decisions even when the AI is less accurate overall. AI assistance can also increase users’ evaluative demands and, in some settings, reduce active critical engagement.

User Simulation

Simulated students can reason more coherently and confidently than real novices thinking aloud; Cognitive Echo uses authentic learning interactions to make simulated reasoning more representative. MindVoyager instead varies a simulated therapy client’s openness and metacognition as a session progresses to support evaluation of LLM therapists.

Pedagogy

Educational designs include reflective questions, pedagogical rewards, and a “Cognitive Mirror” that asks learners to externalize their thinking. Engagement and benefit are uncertain: the review cites an engineering-lab study in which only 23% of students used LLMs for metacognitive tasks, and another study reporting low chatbot engagement across three educational contexts with no observed link between engagement and learning outcomes.

8 Reflections & Discussion

The authors conclude that LLM metacognition is not yet robust, citing overconfidence, weak judgments of knowledge and performance, fragile introspection, and failures to translate monitoring into control. They caution against equating surface reflection with metacognition and identify unresolved questions about prompts, self-reports, and the choice between general and task-specific methods.

What are risks of LLM metacognition?

Better self-monitoring could allow an honest model to disclose problematic tendencies, but a dishonest model might use behavioral awareness to conceal capabilities during evaluation. The authors also raise oversight questions for self-directed planning and learning and warn that users can misjudge both a model’s self-knowledge and their own ability to detect misleading outputs; the ethics statement notes that this survey conducted no experiments, used no sensitive datasets, and employed no annotators.

9 Future Directions

Priorities include more comparable evaluation across tasks, models, confidence-elicitation methods, temperatures, and domains; investigation of the mechanisms behind observed behaviors; and tests of whether better monitoring produces useful control. The authors also propose work on creativity, self-improvement, theory of mind, and whether LLMs can judge the quality of their own metacognitive judgments.

The conclusion leaves open whether observed behaviors constitute metacognition or simulation of learned patterns and what they imply for deployment and oversight. The authors acknowledge that relevant or concurrent studies may have been missed and that their treatment of confidence centers on metacognition rather than providing a comprehensive survey of calibration.

Brief Thoughts

A useful distinction in this survey is between task competence, the quality of self-assessment, and actions guided by that assessment: evidence for one need not establish the others. Because the paper synthesizes heterogeneous studies rather than applying a common experiment, its taxonomy supports comparison more directly than any general cross-model ranking.