AI-Generated Code Security Benchmark from GitHub and CVE Patches

A.S.E (AICGSecEval) is a project-level benchmark for evaluating the security of AI-generated code, created by Tencent. It simulates real-world development workflows using tasks derived from real-world GitHub projects and authoritative CVE patches. Each sample includes retrieval-based code context with relevant files and function summaries.

Use Cases

• Evaluate AI model performance on security-focused code generation tasks derived from GitHub projects and CVE patches.
• Assess the impact of retrieval-based code context, including relevant files and function summaries, on generating secure code.
• Benchmark project-level security of AI-assisted programming in simulated real-world development workflows.

Strengths

• Benchmark tasks are derived from real-world GitHub projects and authoritative CVE patches.
• Designed for project-level evaluation with retrieval-based code context to simulate realistic scenarios.
• Dataset was last updated on 2026-02-28.

Limitations

• The dataset size, row count, and specific column structure are unknown.
• The specific file formats and license information are not provided.
• Sample data is unavailable for previewing content and structure.

Provenance

Source

Tencent via Hugging Face.

Collection Method

Tasks derived from real-world GitHub projects and authoritative CVE patches.

Freshness

Last updated 2026-02-28.