DIP: Dead code Insertion based Black-box Attack for Programming Language Model

CheolWon Na, YunSeok Choi, Jee-Hyong Lee


Abstract
Automatic processing of source code, such as code clone detection and software vulnerability detection, is very helpful to software engineers. Large pre-trained Programming Language (PL) models (such as CodeBERT, GraphCodeBERT, CodeT5, etc.), show very powerful performance on these tasks. However, these PL models are vulnerable to adversarial examples that are generated with slight perturbation. Unlike natural language, an adversarial example of code must be semantic-preserving and compilable. Due to the requirements, it is hard to directly apply the existing attack methods for natural language models. In this paper, we propose DIP (Dead code Insertion based Black-box Attack for Programming Language Model), a high-performance and effective black-box attack method to generate adversarial examples using dead code insertion. We evaluate our proposed method on 9 victim downstream-task large code models. Our method outperforms the state-of-the-art black-box attack in both attack efficiency and attack quality, while generated adversarial examples are compiled preserving semantic functionality.
Anthology ID:
2023.acl-long.430
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7777–7791
Language:
URL:
https://aclanthology.org/2023.acl-long.430
DOI:
10.18653/v1/2023.acl-long.430
Bibkey:
Cite (ACL):
CheolWon Na, YunSeok Choi, and Jee-Hyong Lee. 2023. DIP: Dead code Insertion based Black-box Attack for Programming Language Model. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7777–7791, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
DIP: Dead code Insertion based Black-box Attack for Programming Language Model (Na et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-long.430.pdf
Video:
 https://aclanthology.org/2023.acl-long.430.mp4