LiDAR-LLM-Nu-Caption: QA Dataset for Autonomous Driving Caption Tasks

LiDAR-LLM-Nu-Caption is a question-answering dataset designed for training multimodal large language models on caption tasks in autonomous driving scenarios. It is built upon the NuScenes dataset, with answers generated by vision-language models and filtered for LiDAR-specific information. The dataset was created by Senqiao and last updated on December 24, 2024.

Use Cases

• Training LiDAR-only captioning models based on the 'answer_lidar' field designed for the LiDAR modality.
• Fine-tuning vision-language models for autonomous driving scene understanding based on the image-derived 'answer' field.
• Benchmarking model performance on caption generation for driving scenarios based on the QA structure.
• Studying the information gap between LiDAR and vision sensors based on the filtered answer fields.

Strengths

• Built upon the established NuScenes dataset, which suggests a foundation of real-world driving data.
• Includes dual answer fields ('answer' and 'answer_lidar') designed for different sensor modalities.
• Answers were processed using GPT-4o-mini to filter LiDAR-specific information.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Dataset size, file formats, and license information are not provided in the input.

Provenance

Source

Built upon the NuScenes dataset.

Collection Method

Answers generated by vision-language models using image data, with a separate field filtered by GPT-4o-mini for LiDAR-specific information.

Freshness

Last updated 2024-12-24 17:48:20; freshness should be verified.