The Morning
From the arXiv
LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs
his paper introduces PropMe, a framework that evaluates LLM memorization not just when prompted adversarially, but also under normal usage. Their core method uses a new propensity-aware metric and a tracing pipeline called SimpleTrace to quantify how often models leak training data during typical generation. The key contribution is demonstrating a significant gap between LLMs' *ability* to memorize (when forced) and their *propensity* to do so naturally, suggesting current evaluations may overestimate real-world data leakage.
ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents
This paper introduces Causal Minimal Tool Filtering (CMTF) to improve the reliability of LLM agents using external tools. CMTF addresses the problem of tool selection by focusing on causal sufficiency rather than just semantic relevance, ensuring only the mini…
Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
Vortex is a system that accelerates the design and deployment of sparse attention algorithms for large language models. It combines a user-friendly Python frontend with an efficient backend, allowing researchers and AI agents to rapidly prototype, evaluate, an…
Will the Agent Recuse Itself? Measuring LLM-Agent Compliance with In-Band Access-Deny Signals
This paper introduces the "Recuse Signal," a novel in-band mechanism for servers to request autonomous LLM agents to voluntarily withdraw access to a resource, acting as a cooperative governance control rather than a security boundary. The core contribution is…
CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
CollabSim introduces a novel methodology for evaluating the collaborative competence of LLM agents in multi-agent systems. It grounds this evaluation in decades of Computer-Supported Cooperative Work (CSCW) research, focusing on how agents establish common gro…
Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
This paper characterizes the system implications of agent memory for long-horizon LLM tasks. It introduces a taxonomy for memory systems, a profiling harness to measure costs, and …
Benchmark Everything Everywhere All at Once
This paper introduces Benchmark Agent, an autonomous system that automates the entire benchmark creation process for LLMs and MLLMs. Its core method involves orchestrating user que…
Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration
This paper introduces ALMANAC, a novel dataset designed to improve agent collaboration. It addresses the lack of data on human collaborative processes by providing action-level ann…
LLM Self-Recognition: Steering and Retrieving Activation Signatures
This paper introduces a method for LLMs to "self-recognize" their own outputs by embedding a detectable fingerprint within their internal activations. By steering the residual stre…
TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management
TokenMizer addresses the LLM context window limitation by modeling session history as a typed knowledge graph, preserving crucial structured information. Its core method involves a…
The Town Square
AI is making progress towards recursive self-improvement, where AI systems can autonomously enhance their own capabilities, leading to potentially rapid advancements.
Workshops
Hermes-Agent is a Python-based AI agent designed for flexible and evolving task execution, featuring a modular architecture and a focus on adaptability to user needs.
This repository provides an agent harness for performance optimization, focusing on skills, instincts, memory, security, and research-first development for various AI code assistants like Claude Code and Codex.
