Sign in to view source links and access this dataset
Description
A blind evaluation dataset of high-quality, diverse, human-written instructions with demonstrations. The dataset was created by HuggingFaceH4 and last updated on February 28, 2023. It is intended for use in step 3 evaluations within a Reinforcement Learning from Human Feedback pipeline.
Use Cases
Evaluating instruction-following capabilities of language models based on human-written instructions.
Benchmarking model performance in a blind evaluation setting as part of an RLHF pipeline.
Training or fine-tuning models to better understand and execute diverse human instructions.
Strengths
Dataset is described as containing high-quality, human-written instructions.
The instructions are noted for their diversity.
Dataset is specifically designed for a critical evaluation step in an RLHF pipeline.
Limitations
Column-level documentation is absent; field semantics must be inferred after download.
Row count, file formats, and overall data size are unknown, which may limit suitability assessment.
Last updated 2023-02-28 22:30:11; freshness should be verified.
Provenance
Source
HuggingFaceH4
Collection Method
Likely collected and curated by human annotators, as described as 'human-written'.
Time Range
null
Freshness
Last updated 2023-02-28 22:30:11.
Geography
null
License information is unknown; users should verify licensing terms before use.