Nightmare Extraction Test: 148 Adversarial Insurance Documents

148 adversarial insurance documents generated by a closed-source system, released on 2026-05-19. The corpus includes ACORD forms, loss runs, and financial statements, organized into five difficulty packets from easy to nightmare. It was created by TimAginor to stress-test frontier-model document extraction capabilities.

Use Cases

• Stress-testing document extraction models based on adversarial insurance forms.
• Benchmarking model performance across difficulty levels based on the five defined packets.
• Evaluating the robustness of NLP pipelines on synthetic, hand-validated insurance intake artifacts.

Strengths

• 148 documents with hand-validated ground truth.
• Five difficulty packets (N1_easy through N5_nightmare) provide structured benchmarking.
• Corpus is fully public for broad use.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect bias inherent to the closed-source generator used for creation.

Provenance

Source

huggingface

Collection Method

Generated by a closed-source generator; ground truth hand-validated.

Freshness

Last updated 2026-05-19 12:41:27.