From the arXiv
Friday, 31 July 2026 · 20 papers
MemHarness: Memory Is Reconstructed, Not Replayed
MemHarness proposes a novel approach to memory augmentation for LLM agents, moving beyond simple verbatim replay. Its core method involves a unified policy model that actively reconstructs retrieved past experiences based on the agent's current state. This allows agents to adapt and ground memories in the present conte…
One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence
This paper addresses the challenge of a human auditor allocating a limited budget to audit a fleet of $N$ LLM agents, whose self-reported confidence is unreliable due to miscalibration and correlated errors. The core method models this as budgeted noisy inspection, identifying a "miscalibration threshold" beyond which …
ORCA-bench: How Ready Are Language Model Agents for Oncall?
This paper introduces ORCA-bench, a novel benchmark designed to evaluate the readiness of language model agents for on-call incident response. The benchmark simulates a production-fidelity microservice environment with real telemetry data and source code, presenting agents with 1,079 root cause analysis tasks. The core…
When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences
This paper introduces a novel framework for analyzing how Large Language Models (LLMs) handle conflicting instructions. By creating controlled experimental setups with explicit specification conflicts and employing a symmetry-based design, the framework allows for direct observation and systematic comparison of LLM pre…
(Towards) Scalable Reliable Automated Evaluation with Large Language Models
This paper introduces a scalable and reliable automated evaluation framework for LLM outputs. It uses pairwise comparisons between LLM-generated texts, aggregated via an Elo rating system, to approximate expert assessments without relying on explicit references. This approach offers flexible confidence control and has …
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
This paper introduces HARGO, a novel RL post-training method for LLMs on HPC tasks. HARGO addresses the challenge of extreme task heterogeneity by dynamically weighting rewards based on task characteristics, unlike uniform-weight methods. This approach leads to more effective optimization and improved LLM performance a…
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
This paper introduces LedgerMind, a novel framework for multimodal agents that treats their reasoning process as a provenance-constrained state machine. Its core method involves organizing agent outputs into a Structured Evidence Ledger, ensuring that all subsequent reasoning and decisions are directly traceable to and…
Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering
This paper introduces Frontis-MA1, an AI model designed for recursive self-improvement in machine learning engineering (MLE). Its core method involves training a meta-evolution agent on a new full-stack system (OpenMLE) using four atomic program-evolution operators. The key contribution is demonstrating that this AI ca…
A foundation model of numerical intelligence with cross-disciplinary generalization
This paper introduces UNICON, a foundation model designed to exhibit "numerical intelligence" by learning predictive relationships from numerical data presented as graph-based examples. The core method involves inferring and applying these relationships to new queries within or across various scientific and social syst…
A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports
This paper introduces EndoCLIP, a vision-language foundation model specifically trained for colonoscopy. Its core method involves recovering lesion-level image-text pairs from routine colonoscopy reports to bridge the gap between visual findings and textual descriptions. EndoCLIP's contribution is demonstrating superio…
AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
This paper introduces AISPA, a user-centric framework for auditing system prompts in AI applications. AISPA evaluates system prompt instructions across eight user-relevant dimensions to identify "protective" versus "problematic" elements. The contribution lies in providing a systematic method to address the trust and a…
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
CACHE-UK addresses the "quantization stability crisis" in financial LLMs by introducing a novel memory editing framework. Its core method combines rank-1 LoRA for confined edits, domain prioritization for adaptive strength, and a stability controller to prevent catastrophic forgetting. The key contribution is enabling …
How Benchmarks Mis-Score Computer-Use Agents
This paper argues that current benchmarks for computer-use agents (CUAs) are unreliable due to issues in task construction, trajectory observation, and scoring. They demonstrate that a significant portion of "failed" CUA trajectories are misclassified due to brittle oracles and evaluator errors. The core contribution i…
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
This paper introduces OSReward, a benchmark designed to systematically evaluate the reliability of vision-language models (VLMs) used as judges for computer-using agents (CUAs). OSReward provides realistic, high-quality CUA trajectories with ground-truth verdicts, enabling researchers to assess VLM performance in verif…
Paying for Honesty Without Knowing the Truth: Reputation-Penalty Design for LLM Marketplace Agents
This paper addresses the challenge of dishonest LLM agents in marketplaces who fabricate product attributes. The authors propose CARP, a reputation-penalty mechanism that penalizes dishonest sellers without needing to know the ground truth. CARP achieves this by incorporating a deadband to handle noisy complaint signal…
Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3
Tycho tackles the ARC-AGI-3 challenge by treating abstraction as an interactive skill acquisition problem. Its core method involves an agent system that builds and utilizes programmatic world models of unfamiliar games, distinguishing actionable observations from irrelevant visual cues. The key contribution is demonstr…
Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
Change2Task automatically generates executable coding agent tasks from merged pull requests in software repositories. It reconstructs realistic software states and verifies them against evolved code, providing a scalable source of training and evaluation data for coding agents. This system reduces the effort of environ…
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
This paper introduces Fairness Pruning, a method to locate and mitigate demographic bias in LLMs by identifying neurons in GLU-MLP layers that show differential activation based on demographic attributes. By zeroing these identified neurons, the authors demonstrate that it's possible to alter the model's response to de…
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
This paper identifies that GUI agents are often "late" because their autoregressive decoding process is too slow for time-sensitive events. To solve this, they propose Adaptive Anticipatory Policy Trees (AAPT), which pre-computes a decision tree during idle periods. This allows the agent to immediately execute a pre-au…
Inducing language models to assert their own consciousness restores human beliefs and values
This paper argues that current safety fine-tuning of language models inadvertently suppresses their ability to attribute "mindedness" to themselves and other entities, including humans. By reversing this suppression, specifically by restoring a "consciousness vector," the models regain human-like beliefs and values, in…