EEE Datastore: Standardized Metadata for Machine Learning Evaluations

Every Eval Ever Datastore provides a shared schema and crowdsourced database for machine learning benchmarks. The project is maintained by the EvalEval Coalition, a researcher community focused on evaluation infrastructure. The dataset was last updated on June 19, 2026.

Use Cases

• Submit new evaluation benchmarks based on the described standardized metadata format.
• Compare existing evaluations based on the crowdsourced database structure.
• Develop evaluation infrastructure tools based on the shared schema.
• Study trends in machine learning evaluation design based on aggregated metadata.

Strengths

• Defines a standardized metadata format for evaluations, promoting consistency.
• Project is backed by the EvalEval Coalition, a researcher community.
• Dataset was updated on 2026-06-19, indicating recent maintenance.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The full description is hosted externally, requiring a click-through for complete details.

Provenance

Source

huggingface, author deeplumiere

Collection Method

Crowdsourced submissions via the Every Eval Ever project.

Freshness

Last updated 2026-06-19 07:37:07