SimLingo: 3.3 Million CARLA 2.0 Samples with Driving Commentary

SimLingo-Data consists of 3,308,315 samples of autonomous driving data generated in the CARLA 2.0 simulator by RenzKa. It integrates sensor readings and action labels with natural language annotations for driving commentary, instruction following, and visual question answering, collected using the PDM-Lite rule-based expert.

Use Cases

• Training Vision-Language-Action (VLA) models using sensor data and action labels
• Developing driving commentary systems using the natural language labels to describe simulator states
• Fine-tuning Visual Question Answering (VQA) models on autonomous driving scenarios

Strengths

• 3,308,315 total samples
• Expert trajectories generated by the PDM-Lite rule-based system
• Multi-modal labels covering VQA, commentary, and instructions

Limitations

• Synthetic data from CARLA 2.0 simulator may lack real-world sensor noise and complexity
• Samples are not from unique routes, potentially leading to spatial redundancy across the 3.3 million records

Provenance

Source

RenzKa via Hugging Face

Collection Method

Synthetic generation via CARLA 2.0 simulator using PDM-Lite expert agent

Freshness

Last updated August 2025.

Geography

Synthetic (CARLA simulator)