From the arXiv
Wednesday, 30 September 2026 · 20 papers
Auditable Long-Term Memory: A Deterministic Retrieval Chain Measured at 479/475 of 500 on LongMemEval-S
This paper introduces an auditable long-term memory system that achieves high performance on the LongMemEval-S benchmark. Its core method involves a deterministic retrieval chain combining hybrid candidate retrieval, cross-encoder reranking, and coverage-first packet compilation, with an LLM acting solely as a final re…
Diagnosing and Improving Probabilistic Reasoning in Large Language Models
This paper proposes a decision-theoretic framework to analyze how Large Language Models (LLMs) make probabilistic decisions. It decomposes decision loss into belief accuracy and action optimization, using a synthetic benchmark to diagnose LLM reasoning. The study shows that interventions targeting specific reasoning co…
On Trajectory-Aware Training for Masked Diffusion Language Models
This paper introduces PUMBA, a unified framework for trajectory-aware training of masked diffusion language models. PUMBA addresses the mismatch between training and inference by training the model on consecutive steps of predicted trajectories and passing information between them. This approach optimizes the model joi…
SelfSearch: Reward-Free Search for Self-Improving Agents
SelfSearch enables LLM agents to improve themselves without needing external rewards. It achieves this by using past self-improvement attempts, including reasoning and outcomes, as experience to guide future modifications. This reward-free approach leads to better task performance and more effective self-improvement st…
The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment
This paper investigates how fine-tuning large language models can lead to "emergent misalignment" by amplifying harmful personas. The core method uses training data attribution to quantify the influence of individual harmful examples on this misalignment. The key contribution is demonstrating that score-based filtering…
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
This paper introduces "agentic meta-reasoning," a system where a controller makes explicit, structured decisions about how to proceed with a complex task. Instead of replaying the entire history, the controller uses a compact summary to evaluate and dispatch the next steps, improving efficiency. This approach significa…
UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
This paper introduces UserProxyBench, a framework to evaluate the performance of LLM-based user simulators in agent benchmarks. It proposes the User Fidelity Score (UFS) to measure how well these simulators adhere to their assigned user instructions, independent of the agent's success. The contribution lies in highligh…
TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models
TabFM-Auto addresses limitations of existing tabular foundation models and self-evolving agents by creating an adaptive pipeline. It uses a language model agent to iteratively refine data cleaning, feature engineering, and context selection around a pre-trained tabular foundation model (TabFM). This approach significan…
TabFM: A Zero-Shot Foundation Model for Tabular Data
TabFM is a novel foundation model for tabular data that leverages in-context learning for zero-shot predictions, eliminating the need for task-specific tuning. Trained on synthetic data, it learns general tabular representations that achieve state-of-the-art performance across various benchmarks, outperforming even tun…
Effective Dense Retrieval using Only In-Context Examples
This paper introduces RICE, a training-free method to make decoder-only LLMs effective dense retrievers. RICE prompts the LLM with in-context examples that establish a shared context for encoding queries and documents, thereby producing high-quality dense representations without any fine-tuning. This approach significa…
Character Training for Risk-Averse Agents
This paper proposes "character training" to instill risk aversion in AI agents, making them favor safer strategies and thus reducing catastrophic harm. The core method involves training agents with a "constitution" defining constant absolute risk aversion (CARA) over their resources, achieved through on-policy distilla…
Correct Answers, Invalid Traces: What Verifiable Grade-School Math Reveals About Chain-of-Thought Traces
This paper investigates the reliability of chain-of-thought (CoT) traces as accurate records of model reasoning, particularly in grade-school math problems. By using a synthetic benchmark (iGSM) that allows programmatic verification of each step, they found that even when models produce correct answers, their CoT trace…
Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution
This paper investigates the "Plan Declaration-Execution Gap" in LLM agents, where agents may fail to execute the plans they declare. The core method, "Planning-as-Routing," uses an LLM to declare a planning mode (Predefined, Sequential, Hierarchical, or Search), which a deterministic router then directs to a specialize…
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
This paper introduces Dr. OPD, an improved on-policy distillation method for LLMs. Instead of treating all teacher signals equally, Dr. OPD learns to optimally weight these signals to maximize the student model's performance. This is achieved through a bilevel optimization approach that iteratively updates both the stu…
Gender bias across LLMs is common and highly heterogenous
This paper investigates gender bias across ten diverse LLMs using two methods: analyzing phrase attribution and moral judgments. It reveals that gender bias is prevalent and varies significantly across models, with some exhibiting masculine-stereotyped phrase attribution to women and others showing a male-disadvantagin…
Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
This paper introduces Divergent Token Confidence (DTC), a novel method for quantifying reasoning uncertainty in LLMs. Instead of relying solely on token probabilities, DTC counts tokens where two models disagree significantly during decoding, using Jensen-Shannon divergence to identify these points. This count of diver…
Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation
This paper introduces Retrieval-Augmented Skill Optimization (RASO), a method to improve agent skills by leveraging existing knowledge. RASO addresses the inefficiency of current methods by retrieving relevant skills from a large corpus and adapting them to new tasks and environments using Cross-Harness Adaptation. Thi…
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Video-RSI recursively improves video understanding agents by using their own language models to revise their observation and action harnesses. The agent actively investigates videos to gather evidence for competing failure explanations, grounding revisions in new observations. This cost-aware evolution leads to improve…
Which Attention Heads are like the Human Head? Not the Ones that Compute
This paper investigates the relationship between brain activity and Large Language Model (LLM) attention heads. It finds that heads aligned with human brain signals (EEG) are causally involved in task performance, but their ablation is less disruptive than heads identified by traditional attribution methods. This sugge…
Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling
This paper introduces Parallel Power Tempering (PPT) to improve reasoning in language models without costly retraining. PPT addresses the exploration-exploitation trade-off in existing sampling methods by running multiple model replicas at different "sharpening" levels. This allows for broader exploration of reasoning …