VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

• ID: 95cbe472
• 原文链接: https://arxiv.org/abs/2608.12246
• PDF: https://arxiv.org/pdf/2608.12246v1
• 作者: Jin Lu, Xuening Han, Yang Zhong, Lin Tan, Kevin Luo, Andrew Gacek, Neha Rungta
• 日期: 2026-08-12
• 更新: 2026-08-12
• 分类: coding
• 来源类型: paper
• 标签: benchmark, security, vulnerability-detection, multi-language, vic, cs.cr, cs.cr-cs.ai-cs.cl-cs.se
• 质量评分: 4/5
• 抓取时间: 2026-08-14T04:19:48Z

中文导读

论文发布 VICBench：一个跨多语言以首次引入漏洞的 commit (VIC)为锡点的代码安全检测基准人工 + agentic 双递注释保证质量，覆盖现有数据集在语言 patch 复杂度项目范围上的局限

为什么值得关注

VICBench 以首次引入漏洞的 commit 为锡点重新打造跨语言代码安全基准

关键信息

• 论文标题：VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
• 作者：Jin Lu, Xuening Han, Yang Zhong, Lin Tan, Kevin Luo, Andrew Gacek, Neha Rungta
• arXiv：https://arxiv.org/abs/2608.12246
• 发布时间：2026-08-12
• arXiv 分类：cs.CR, cs.AI, cs.CL, cs.SE
• 关联标签：benchmark, security, vulnerability-detection, multi-language, vic, cs.cr, cs.cr-cs.ai-cs.cl-cs.se

English Abstract

Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range of vulnerable software versions. Existing vulnerability datasets suffer from limited programming language coverage, restricted patch complexity, and narrow project scope. Through our dual annotation by human experts and an agentic workflow, we create a benchmark - VICBench - of 100 verified VICs for 100 CVEs across 88 projects in Python, Java, and C++, covering 48 CWE types. VICBench features complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines - significantly larger than prior work. Our evaluation shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort. VICBench enables robust evaluation of vulnerability detection approaches.

English Summary

VICBench is a multi-language code vulnerability detection benchmark built around vulnerability-inducing commits (VICs) the commits that first introduce a vulnerability. The dataset is dual-annotated by human experts and an agentic workflow, broadening language coverage, patch complexity, and project scope versus existing vulnerability corpora.

Obsidian Notes

• 内容由 opencli arxiv paper 拉取 arXiv 元数据与摘要生成。
• 中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上；未补充论文摘要之外的实验细节。