RealUserSim: 7,273 Behavioral User Profiles for Grounded LLM Agent Benchmarking

7,273 behavioral user profiles extracted from real conversations, each containing demographics and executable linguistic style commands. The dataset also includes 600 evaluation test cases across 6 splits for measuring user simulation fidelity. It was created by Salesforce and last updated on May 6, 2026.

Use Cases

• Benchmarking the fidelity of conversational AI agents based on the provided 600 evaluation test cases.
• Training user simulation models based on the 7,273 profiles containing demographics and linguistic style commands.
• Studying the relationship between user demographics and conversational style in real-world chat data.

Strengths

• Contains 7,273 user profiles derived from real conversations, providing a grounded basis for simulation.
• Includes 600 structured evaluation test cases across 6 splits for systematic benchmarking.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count for the primary profile data is known, but other dataset dimensions like file formats are unknown.

Provenance

Source

Derived from the WildChat dataset.

Collection Method

User profiles and test cases were extracted from real conversations.

Freshness

Last updated 2026-05-06 19:46:51; freshness should be verified.