Decompile-Bench-Eval includes manually crafted binaries from HumanEval and MBPP, alongside compiled GitHub repositories released after 2025 to mitigate data leakage. The dataset supports both C and C++ source code, whereas earlier models and the HumanEval-Decompile dataset were limited to C only. It was created by LLM4Binary and last updated on September 30, 2025.
Use Cases
- Benchmarking decompilation models based on manually crafted binaries from HumanEval and MBPP.
- Evaluating model performance on compiled GitHub repositories released after 2025.
- Training decompilers for C++ source code, expanding beyond earlier C-only datasets.
- Testing for data leakage issues using post-2025 compiled code.
Strengths
- Includes binaries from well-established HumanEval and MBPP benchmarks.
- Expands language support to C++ source code, beyond earlier C-only datasets.
- Incorporates compiled GitHub repositories released after 2025 to address data leakage.
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download.
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
Provenance
- Source
- LLM4Binary
- Collection Method
- Manually crafted binaries from HumanEval and MBPP, plus compiled GitHub repositories.
- Time Range
- GitHub repositories released after 2025.
- Freshness
- Last updated 2025-09-30 11:12:59; freshness should be verified.