Our paper entitled "Generating Prompt-based Adversarial Text Examples via Variable Neighborhood Search" has been accepted by CAC2023.

Generating Prompt-based Adversarial Text Examples via Variable Neighborhood Search

Yupeng Qi, Xinghao Yang, Baodi Liu, Kai Zhang, Weifeng Liu

Natural Language Processing (NLP) models are immensely vulnerable to adversarial text examples. Various word-level attacks have been proposed to modify input texts by carefully-picked substitute words via static or dynamic optimization algorithms. However, existing word-level attack methods usually ignore text fluency and semantic consistency for seeking a high attack success ratio, often resulting in unnatural adversarial text examples. In this paper, we propose to generate Prompt-based adversarial texts via Variable Neighborhood Search (P-VNS), which achieves a high attack success ratio while simultaneously keeping text fluency and semantic similarity. Specifically, the well-designed Prompt texts are constructed for input texts and the substitute words are obtained by mask-and-filling procedure under the effect of Prompt texts, so the text fluency and semantic similarity can be enhanced. Additionally, the word modification priority is adaptively determined by employing the variable neighborhood search algorithm, yielding an improvement in the attack success ratio. Extensive experiments demonstrate that the P-VNS accomplishes the highest attack success ratio meanwhile preserving text fluency and semantic similarity. Besides, the proposed P-VNS also manifests effectiveness in adversarial training and transfer attack.