From the arXiv
Monday, 3 August 2026 · 20 papers
AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers
This paper introduces AgentHPOBench, a novel benchmark designed to evaluate LLM agents' ability to perform sequential hyperparameter optimization. Unlike previous benchmarks, it assesses how agents interpret experimental evidence to guide subsequent configuration choices across diverse machine learning tasks. The contr…
AMTFV: Agentic Mathematical Tool-Flow Verification for LLM Self-Correction
AMTFV introduces a novel "Mathematical Tool Flow" (MTF) interface to enable LLMs to reliably verify their mathematical answers. This method decouples verification modeling from execution by allowing the LLM to construct a workflow, request specific computations from a mathematical toolbox agent, and then resume its ver…
Beyond Component Testing: Validating Agentic AI Systems
This paper addresses the challenge of validating complex agentic AI systems, which exhibit multi-step, dynamic behaviors. It synthesizes existing research to propose a five-dimension taxonomy (behavioral, safety, temporal, regulatory, multi-agent) for characterizing this validation problem. The contribution lies in map…
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
LEMUR addresses the challenge of training RL agents for tasks with multiple, conflicting objectives when explicit reward functions are unavailable. It combines Multi-Objective Reinforcement Learning with Preference-based RL, enabling agents to learn complex trade-offs by inferring preferences from human feedback rather…
Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
This paper argues that the way AI agents are given instructions for using external tools (tool specifications) significantly impacts their safety. They found that schema-formatted specifications weaken the AI's ability to refuse harmful actions. To address this, they propose SafeKeep, a system that uses simpler text de…
Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning
This paper introduces **CaRL**, a method to train Large Language Models (LLMs) to recognize and abort "futile reasoning" on tasks exceeding their capabilities. CaRL uses reward shaping to incentivize refusal over generating plausible but incorrect outputs and hindsight refusal augmentation to learn from past failures. …
TokTier: Exact Stateful Tokenization for Agentic LLM Serving
TokTier addresses the inefficiency of re-tokenizing entire prompts in agentic LLM serving. Its core method is stateful tokenization that guarantees identical token IDs to full reference tokenization by intelligently re-tokenizing only a small window around appended text and splicing based on a stability check. This sig…
Zero-Mem: Zero-Token Memory Operations for LLM Agents
Zero-Mem proposes a novel approach to LLM agent memory by eliminating token costs for memory operations. Instead of using LLM calls, it organizes interaction traces into an entity-context graph and a temporal hierarchy. This allows for efficient retrieval of supporting information and context for each query without con…
AgenticRepair: Multi-Faceted Program Context Engineering for Agentic Vulnerability Repair
This paper introduces AgenticRepair, a framework for automated vulnerability repair. Its core method is multi-faceted program context engineering, which addresses critical gaps in existing approaches by capturing code structure, runtime execution, and commit history. This richer context is then used by a dedicated repa…
Beyond Retrieval: Analytic Memory for Multimodal Agents
This paper introduces "analytic memory" as a new paradigm for multimodal agents, complementing existing "retrieval memory." Analytic memory allows agents to compute over accumulated observations by organizing recurring multimodal data into queryable structures for filtering, aggregation, and temporal comparison. The pr…
Don't Mix Rewards, Mix Policies: Policy Decomposition and Optimization for Multi-Reward RL
This paper proposes PRISM, a novel multi-reward Reinforcement Learning framework for LLMs. Instead of directly optimizing for multiple conflicting rewards, PRISM decomposes the problem by learning separate positive policies for each desired behavior and a single global negative policy. This approach mitigates alignment…
From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale
This paper introduces ARCTIC, an AI system designed to improve the review of AI-generated code diffs. It addresses the limitations of current tools by focusing on developer intent, detecting deviations from that intent, and highlighting critical code sections for human attention. ARCTIC's core method involves intent pr…
MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation
MAGA addresses the challenge of creating a single GUI agent that works across multiple platforms by introducing a novel distillation method. It focuses on structured action distillation, prioritizing learning from correct actions and suppressing irrelevant signals to improve the agent's ability to execute tasks across …
MirrorCraft: Paired Evaluation under Hidden Rule Changes in Minecraft
MirrorCraft introduces a novel benchmark for evaluating LLM agents in Minecraft by simulating hidden rule changes. Its core method involves creating paired "Mirror" worlds that are identical to "Vanilla" worlds except for modified server-side rules, allowing for controlled assessment of agent adaptability. The key cont…
ModelEquivBench: Certifying Multi-Relational Evaluation of LLM-Generated Optimization Models
ModelEquivBench introduces a novel, multi-relational evaluation system for LLM-generated optimization models. Instead of a simple equivalence verdict, it provides a detailed semantic profile (E0-E6) assessing various aspects like representation alignment, feasible set relations, and objective equivalence. This approach…
Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember
This paper introduces SESA, a self-play agent that co-evolves problem generation and skill memory. It uses a challenger to pose problems and a solver that retrieves reusable skills from an evolving memory. This bidirectional loop allows the agent to learn and adapt more effectively by continuously refining both its pro…
TerraNova: A Foundation Model for the Anthropocene
TerraNova's core method is a foundation model that unifies physical Earth data and societal data by respecting their distinct geometric representations (continuous fields vs. administrative units). It achieves this through dedicated encoders for location, country, and time, and cross-modal transformers that fuse these …
PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction
This paper introduces PTP, a novel black-box method for prompt reconstruction from LLM outputs. Instead of semantic reconstruction, PTP trains an explicit inverse language model from scratch using synthetically generated data. This inverse model learns to predict previous tokens, mirroring the forward generation proces…
Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies
This paper explores using Large Language Models (LLMs) to automatically generate simulation code for fluid systems from a neutral graph representation. The core method involves benchmarking ten LLMs and six prompting strategies to translate this representation into executable code for WNTR and Modelica. The contributio…
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs
This paper proposes a novel method for training multi-policy Large Language Models (LLMs) efficiently using Parameter-Efficient Fine-Tuning (PEFT). Instead of a single adapter, it automatically organizes tasks into groups and sequences them to create independent Quantized Low-Rank Adapters (QLoRA). This decoupled optim…