Evasion Attacks in Continual Learning

Continual learning (CL) enables machine learning models to adapt to evolving tasks while addressing challenges such as catastrophic forgetting. However, it inherits vulnerabilities from conventional settings, notably evasion attacks where adversarial perturbations degrade model performance. This study investigates the impact of evasion attacks, specifically Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD), in a class incremental continual learning scenario using the CIFAR-10 dataset. Results show that adversarial examples generated during training largely retain their effectiveness across CL steps, demonstrating transferability over time. Their success varies depending on the similarity between newly introduced and previously learned classes, sometimes increasing or decreasing accordingly. Adversarial training, adapted for the CL setting, is also evaluated. While it improves robustness against specific attacks (mean gain ~30%), it introduces trade-offs such as reduced accuracy on benign inputs and potential overfitting to adversarial examples. These findings highlight the challenge of balancing robustness, generalization, and efficiency, and emphasize the importance of understanding how adversarial examples transfer across tasks in continual learning.