Evaluating the Robustness of Deep-Learning Algorithm-Selection Models by Evolving Adversarial Instances

Hart, Emma; Renau, Quentin; Sim, Kevin; Alissa, Mohamad

Abstract

Deep neural networks (DNN) are increasingly being used to perform algorithm-selection in combinatorial optimisation domains, particularly as they accommodate input representations which avoid designing and calculating features. Mounting evidence from domains that use images as input shows that deep convolutional networks are vulnerable to adversarial samples, in which a small perturbation of an instance can cause the DNN to misclassify. However, it remains unknown as to whether deep recurrent networks (DRN) which have recently been shown promise as algorithm-selectors in the bin-packing domain are equally vulnerable. We use an evolutionary algorithm (EA) to find perturbations of instances from two existing benchmarks for online bin packing that cause trained DRNs to misclassify: adversarial samples are successfully generated from up to 56% of the original instances depending on the dataset. Analysis of the new misclassified instances sheds light on the 'fragility' of some training instances, i.e. instances where it is trivial to find a small perturbation that results in a misclassification and the factors that influence this. Finally, the method generates a large number of new instances misclassified with a wide variation in confidence, providing a rich new source of training data to create more robust models.

Citation

Hart, E., Renau, Q., Sim, K., & Alissa, M. (2024, September). Evaluating the Robustness of Deep-Learning Algorithm-Selection Models by Evolving Adversarial Instances. Presented at 18th International Conference on Parallel Problem Solving From Nature PPSN 2024, Hagenburg, Austria

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 18th International Conference on Parallel Problem Solving From Nature PPSN 2024
• Start Date: Sep 14, 2024
• End Date: Sep 18, 2024
• Acceptance Date: May 31, 2024
• Online Publication Date: Sep 7, 2024
• Publication Date: 2024
• Deposit Date: Jun 4, 2024
• Publicly Available Date: Sep 8, 2025
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Pages: 121-136
• Series Title: Lecture Notes in Computer Science
• Series Number: 15149
• Series ISSN: 0302-9743
• Book Title: Parallel Problem Solving from Nature – PPSN XVIII
• ISBN: 9783031700675
• DOI: https://doi.org/10.1007/978-3-031-70068-2_8
• External URL: https://ppsn2024.fh-ooe.at/