Tool Retrieval Benchmark and Training Dataset for Large Language Models

Offering a benchmark and training data for evaluating and optimizing information retrieval models on tool retrieval tasks for large language models. It is described as a large-scale training dataset designed to improve model expertise in this specific domain. The dataset was created by author 'mangopy' and was last updated in March 2025.

Use Cases

• Benchmark existing information retrieval models on the tool retrieval task defined by this project.
• Train or fine-tune information retrieval models using the large-scale training dataset to specialize in tool retrieval.
• Evaluate the performance of retrieval systems on the specific benchmark queries and tasks provided.

Strengths

• Described as a large-scale training dataset, indicating substantial volume for model optimization.
• Positioned as the first comprehensive benchmark for systematically evaluating tool retrieval models.
• Created specifically for the tool learning domain of large language models, providing focused relevance.

Limitations

• The specific scale (row count, file size) and structure (columns, sample data) are not provided in the input.
• The dataset's composition, potential biases, or label quality are not detailed in the available description.
• Relies on external documentation (GitHub) for full details, which may not be immediately accessible.

Provenance

Source

huggingface, author mangopy

Collection Method

Created as part of the ToolRet project for benchmarking and training information retrieval models.

Freshness

Last updated on 2025-03-23.