AgentRx: Root Cause Attribution for Multi-Agent LLM Failures

Microsoft's AgentRx benchmark, updated in February 2026, provides under 1,000 annotated records of failed multi-agent LLM trajectories. It features step-level failure categories and designated root cause attributions across domains such as retail.

Use Cases

• Training models to identify root cause failures in agentic workflows
• Evaluating LLM debugging capabilities using step-level failure categories
• Developing constraint-based supervision for multi-agent systems

Strengths

• Step-level failure annotations for granular debugging
• Root cause attribution labels for diagnostic tasks
• CC BY 4.0 open license for research use

Limitations

• Small sample size of fewer than 1,000 records
• Focus is restricted to failed trajectories rather than a balanced success/failure set

Provenance

Source

Microsoft (Arxiv 2602.02475)

Collection Method

Annotated failed trajectories from multi-agent systems

Freshness

Last updated February 2026.

Geography

United States