LogicIFEval: A Benchmark for Evaluating Logical Instruction Following in AI Models

LogicIFEval is a benchmark dataset containing 3,050 instructions designed to test the logical instruction-following capabilities of AI models. The dataset, created by author billmianz, includes a full version and a mini version with 749 instructions for quick evaluation. It was last updated on Hugging Face in September 2025.

Use Cases

• Benchmarking model performance on logical instruction-following tasks based on the described evaluation structure.
• Developing and testing evaluation scripts for AI models based on the referenced GitHub repository.
• Analyzing failure modes in AI reasoning based on the structured test cases mentioned in the description.

Strengths

• Contains 3,050 evaluation examples in the full benchmark split.
• Offers a mini version with 749 instructions for rapid prototyping and testing.
• Provides a structured JSONL format with unique task and test case identifiers.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count for the full dataset is provided, but the specific features and data types within each example are unknown.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Hugging Face dataset uploaded by author billmianz.

Collection Method

Likely constructed for the purpose of AI evaluation; methodology details are in the referenced GitHub repository.

Freshness

Last updated 2025-09-25 19:41:51; freshness should be verified.