Task-Like Versus Not-Task-Like Text Examples

Prompted text examples labeled as 'task-like' from the P3 collection and 'not-task-like' from the C4 corpus. Examples were sampled from C4 to match the length distribution of P3 examples, and certain P3 datasets were held out for validation.

Use Cases

• Train a binary classifier to distinguish 'task-like' from 'not-task-like' text examples.
• Analyze the distribution of example lengths between the P3 and C4 sources.
• Validate language model performance on held-out 'task-like' datasets from P3.

Strengths

• Examples are labeled with a clear binary classification ('task-like' or 'not-task-like').
• Sampling from C4 was performed to match the length distribution of P3 examples, controlling for a potential confounder.
• Includes a validation set with data from held-out P3 datasets and C4.

Limitations

• The specific datasets from P3 that were ignored or held out are not detailed, limiting reproducibility.
• The size of the dataset and the exact number of rows are unknown.
• The 'not-task-like' validation data from C4 does not intentionally match the length distribution of the training data.

Provenance

Source

P3 and C4 text collections.

Collection Method

Examples were sampled from C4 to match the length distribution of P3 examples. Some P3 datasets were ignored or held out for validation.

Freshness

Last updated in April 2022.