Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models

arXiv:2608.196802026-08-21

A smart-contract vulnerability detector that keeps learning new bug types without forgetting old ones, then folds everything into one model

New categories of smart contract vulnerabilities keep emerging, but retraining a large language model from scratch every time is too costly. The researchers built a three-stage pipeline that adapts cheaply, rehearses at-risk knowledge to avoid forgetting, and merges all learned versions into a single deployable model. On the DIVE benchmark of real smart contracts, the merged model came within 2.7% of the accuracy of training separate models per task, while merging took only 156 milliseconds.

Original abstract (English)

Smart contract vulnerability detection with Large Language Models (LLMs) faces three causally linked challenges. First, new vulnerability categories demand parameter-efficient adaptation, since full retraining is prohibitive for sequentially arriving tasks. Second, training per-task adapters on a shared backbone causes catastrophic forgetting of previously learned vulnerabilities. Third, the resulting multiplicity of adapters must be consolidated into a single model, since task identity is unknown at inference time. Each challenge arises directly from the solution to its predecessor, making an integrated framework essential. We propose a three-stage pipeline in which each stage addresses one challenge and feeds into the next. The adaptation stage uses Frequency-Aware Low-Rank Adaptation (FA-LoRA), which performs adaptation in the Fourier domain with per-frequency importance gates, requiring only 0.4% trainable parameters while outperforming standard LoRA and QLoRA. The continual learning stage applies Forget-Aware Replay (FAR), which uses these frequency gates to estimate per-sample forgetting risk via loss dynamics and prioritizes vulnerable knowledge for rehearsal, achieving an average Micro-F1 of 0.8022 across sequential tasks. The deployment stage employs Anchor-Protected Progressive Merging (APPM), which exploits the asymmetric generalization produced by FAR training to identify the strongest-generalizing adapter as an anchor and consolidates all adapters into a single model via anchor-protected weighted merging with frequency-domain gate competition. APPM achieves a Micro-F1 of 0.8085, within 2.7% of the independent per-task upper bound, at a merge cost of 156 ms and no additional runtime memory. Experiments on DIVE confirm the framework effectively addresses all three challenges for evolving blockchain ecosystems.

Authors · Tenghui Huang, Jiawen Kang, Dongning Liu, Changyan Yi, Chengjun Cai, Anjia Yang, Li Li, Dong In Kim

Figures: Tenghui Huang et al., arXiv:2608.19680, arxiv-nonexclusive