The Morning
From the arXiv
ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
ctKV addresses the memory burden of LLM agents by prioritizing KV cache entries crucial for action generation. Its core method employs action-oriented eviction and confidence-driven budget allocation to retain relevant information, enabling more efficient and higher-throughput agentic inference. This contributes the first KV cache compression framework specifically designed to optimize agent performance by focusing on action quality.
DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration
DIAL addresses the limitations of LLM judges by developing a framework that combines LLM and human comparisons. It separates position biases in LLM judgments, learns their underlying preferences, and then adaptively calibrates these to align with human prefere…
Game Arena: Strategic LLM Evaluation in Competitive Environments
The paper introduces Kaggle Game Arena, a platform for evaluating LLMs through competitive gameplay. Its core method involves head-to-head matchups in structured game environments, allowing for dynamic performance assessment as models improve. The contribution…
PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
This paper introduces PriceBench, a benchmark designed to diagnose the price, quality, and brand preferences of Large Language Models (LLMs) acting as booking agents. By analyzing LLM booking choices using a logit model, the benchmark reveals that more capable…
Nonparametric In-Context Learning under Growing Geometric Complexity: Minimax Optimality and Local Geometry-Adaptivity of Transformers
This paper analyzes how Transformers perform in-context learning on data with complex, varying geometric structures. The core method involves developing a novel local-polynomial estimator that adapts to these heterogeneous manifolds. The key contribution is es…
AgentXploit: Autonomous Repository-to-Runtime Red-Teaming for AI Agents
AgentXploit is an autonomous system for pre-deployment red-teaming of AI agents. It uses a two-agent approach: an Analyzer Agent discovers potential attack paths within the agent's…
Beyond Approved Actions: Runtime Validation of Persistent Outcomes in Agent Workflows
This paper introduces EffectMatch, a runtime validation system for LLM agents that interact with software. Its core method is to collect and compare persistent changes made by an a…
Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer
This paper introduces a benchmark for evaluating code documentation by testing if regenerated code passes original tests, finding completeness, not length, is key. They developed a…
DeepEdu-v1: Efficient and Scalable Agentic LLMs for Vietnamese Education
This paper introduces DeepEdu-v1, an AI tutoring system for Vietnamese education designed to address data sovereignty and curriculum alignment issues. Its core method, the SCALE fr…
Evaluating Cultural Awareness of LLMs for Haitian Creole
This paper introduces the first systematic evaluation of cultural awareness in LLMs for Haitian Creole. It uses a benchmark of culturally relevant prompts to assess specificity, bi…
The Town Square
Fireworks.ai has launched Ember-1, a powerful, open-source, 7-billion parameter LLM optimized for efficient inference and fine-tuning, aiming to democratize AI development.
Workshops
Hindsight is an AI agent memory system that learns and adapts over time, enabling agents to recall and utilize past experiences to improve future decision-making and performance.
VoiceStudio is an open-source, local ElevenLabs alternative enabling voice cloning, design, video dubbing, dictation, transcription, and audiobook creation across 646 languages.
