Variable Importance Estimation for High-Dimensional Optimisation

Hunter, Kelly; Thomson, Sarah L.; Hart, Emma

Abstract

Machine learning models trained on the solution spaces of optimisation problems can potentially shed light on variable importance. In prior work the recently established combinatorial benchmark, Polynomial Unconstrained Binary Optimisation with variable importance (PUBOi) was used in this way. Small search spaces were considered so that it was possible to fully enumerate as well as sample. The study confirmed that explainable artificial intelligence (XAI) feature attribution methods can detect these ground-truth importances in this combinatorial optimisation problem. In the present work, we consider larger problem dimensions with the aim of establishing whether the results and XAI methods scale. We compare the distributions of important and unimportant variables across PUBOi instances for prevalent XAI methods to uncover how well important variables are captured. We found that in high-dimensional instances the important variables were captured but to a lesser extent than in low-dimensional instances. The analysis will help to inform future work in adapting search operators during optimisation.

Citation

Hunter, K., Thomson, S. L., & Hart, E. (2025, September). Variable Importance Estimation for High-Dimensional Optimisation. Paper presented at 24th UK Workshop in Computational Intelligence (UKCI 2025), Edinburgh, United Kingdom

• Presentation Conference Type: Conference Paper (unpublished)
• Conference Name: 24th UK Workshop in Computational Intelligence (UKCI 2025)
• Start Date: Sep 3, 2025
• End Date: Sep 5, 2025
• Acceptance Date: Jul 1, 2025
• Deposit Date: Aug 7, 2025
• Publicly Available Date: Aug 8, 2025
• Peer Reviewed: Peer Reviewed
• Keywords: explainable artificial intelligence; variable importance; pseudo- boolean optimisation