SecureVibeBench: A Benchmark for Secure Vibe Coding

SecureVibeBench is a dataset for evaluating secure coding and vulnerability detection, introduced in a paper accepted at the ACL 2026 Main Conference. It is hosted on Hugging Face by the author iCSawyer and was last updated on April 15, 2026. The dataset includes fields such as task IDs, repository URLs, and vulnerability-inducing commit hashes.

Use Cases

• Benchmarking AI models for vulnerability detection based on commit hashes.
• Analyzing patterns in vulnerability-inducing code changes based on repository URLs.
• Training models to identify security flaws in code based on task IDs from the ARVO system.

Strengths

• Associated with a peer-reviewed paper (arXiv:2509.22097) accepted at a major conference (ACL 2026).
• Includes specific fields like vulnerability-inducing commit hashes for targeted analysis.
• Last update timestamp (2026-04-15 08:14:09) is provided.

Limitations

• Row count, file formats, and total dataset size are unknown, limiting suitability assessment.
• Column-level documentation beyond a few fields is absent; field semantics must be inferred after download.
• License information is unknown, which may restrict usage.

Provenance

Source

Hugging Face dataset repository by author iCSawyer.

Collection Method

Likely compiled from software repositories, referencing the ARVO system for task IDs.

Freshness

Last updated 2026-04-15 08:14:09; freshness should be verified.