Clotho: Audio Captioning Dataset with 6,974 Samples and 34,870 Captions

6,974 audio samples of 15 to 30 seconds duration, each paired with five captions of 8 to 20 words, resulting in 34,870 total captions. The dataset was created by Konstantinos Drossos of Tampere University and is described in a 2020 ICASSP paper. It is structured into development, validation, and evaluation splits.

Use Cases

• Training audio captioning models based on the paired audio and text data.
• Evaluating multimodal representation learning based on the structured development, validation, and evaluation splits.
• Benchmarking natural language generation for audio based on the five captions per audio sample.
• Studying audio-text alignment based on the 15-30 second audio clips and their corresponding descriptive captions.

Strengths

• 6,974 audio samples provide a substantial corpus for model training.
• Each audio sample has five independent captions, offering multiple descriptive perspectives.
• Captions are controlled for consistency, with no named entities, speech transcription, or hapax legomena between splits as described in the paper.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Last update date is unknown; freshness unverified.
• The geographic and temporal origins of the audio samples are not specified, which may indicate source bias.

Provenance

Source

Tampere University

Collection Method

Dataset creation described in the associated ICASSP 2020 paper.