OrgAccess: Synthetic Benchmark for Role-Based Access Control in LLMs

OrgAccess is a synthetic benchmark dataset created by respai-lab to evaluate the ability of Large Language Models to operate within organizational hierarchies and role-based access control policies. The dataset was last updated on June 12, 2025. It addresses a critical challenge in ensuring LLMs can reliably function as unified agents within structured organizations.

Use Cases

• Benchmarking LLM performance on RBAC tasks based on the synthetic organizational scenarios
• Training LLMs to understand hierarchical constraints based on the described policy structures
• Evaluating LLM agent reliability in organizational environments based on the described unified agent challenge

Strengths

• Dataset is specifically designed for a novel and underexplored challenge in LLM evaluation.
• The dataset was last updated on June 12, 2025, indicating recent maintenance.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset is synthetic, which may not fully reflect real-world organizational complexity.

Provenance

Source

respai-lab

Collection Method

Synthetic generation for benchmark purposes.

Freshness

Last updated 2025-06-12 05:36:25