Sotopia-ToM: Multi-Agent Interaction Scenarios with Partitioned Knowledge

Sotopia-ToM is a multi-dimensional benchmarking dataset for evaluating LLM agents' ability to navigate information-asymmetric and privacy-sensitive multi-party interactions. Each scenario involves 3–5 agents with partitioned private knowledge, channel-dependent sharing policies, and realistic coordination goals across professional domains. The dataset was authored by Yashwanth and last updated on 2026-04-28.

Use Cases

• Benchmarking LLM agents' theory of mind capabilities based on scenarios with partitioned private knowledge.
• Evaluating strategic information sharing in multi-agent systems based on channel-dependent sharing policies.
• Training agents for coordination in privacy-sensitive environments based on realistic multi-party interaction goals.

Strengths

• Scenarios involve 3–5 agents, providing a structured multi-agent environment.
• Scenarios are designed with information-asymmetry and privacy-sensitive goals, reflecting complex real-world interactions.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

yashwanthys on Hugging Face

Freshness

Last updated 2026-04-28 12:29:18; freshness should be verified.