Interactive Effect of Learning Rate and Batch Size to Implement Transfer Learning for Brain Tumor Classification

Usmani, Irfan Ahmed; Qadri, Muhammad Tahir; Zia, Razia; Alrayes, Fatma S.; Saidani, Oumaima; Dashtipour, Kia

Abstract

For classifying brain tumors with small datasets, the knowledge-based transfer learning (KBTL) approach has performed very well in attaining an optimized classification model. However, its successful implementation is typically affected by different hyperparameters, specifically the learning rate (LR), batch size (BS), and their joint influence. In general, most of the existing research could not achieve the desired performance because the work addressed only one hyperparameter tuning. This study adopted a Cartesian product matrix-based approach, to interpret the effect of both hyperparameters and their interaction on the performance of models. To evaluate their impact, 56 two-tuple hyperparameters from the Cartesian product matrix were used as inputs to perform an extensive exercise, comprising 504 simulations for three cutting-edge architecture-based pre-trained Deep Learning (DL) models, ResNet18, ResNet50, and ResNet101. Additionally, the impact was also assessed by using three well-known optimizers (solvers): SGDM, Adam, and RMSProp. The performance assessment showed that the framework is an efficient framework to attain optimal values of two important hyperparameters (LR and BS) and consequently an optimized model with an accuracy of 99.56%. Further, our results showed that both hyperparameters have a significant impact individually as well as interactively, with a trade-off in between. Further, the evaluation space was extended by using the statistical ANOVA analysis to validate the main findings. F-test returned with p < 0.05, confirming that both hyperparameters not only have a significant impact on the model performance independently, but that there exists an interaction between the hyperparameters for a combination of their levels.

Citation

Usmani, I. A., Qadri, M. T., Zia, R., Alrayes, F. S., Saidani, O., & Dashtipour, K. (2023). Interactive Effect of Learning Rate and Batch Size to Implement Transfer Learning for Brain Tumor Classification. Electronics, 12(4), Article 964. https://doi.org/10.3390/electronics12040964

• Journal Article Type: Article
• Acceptance Date: Feb 10, 2023
• Online Publication Date: Feb 15, 2023
• Publication Date: 2023
• Deposit Date: Mar 16, 2023
• Publicly Available Date: Mar 16, 2023
• Journal: Electronics
• Electronic ISSN: 2079-9292
• Publisher: MDPI
• Peer Reviewed: Peer Reviewed
• Volume: 12
• Issue: 4
• Article Number: 964
• DOI: https://doi.org/10.3390/electronics12040964
• Keywords: brain tumor classification, transfer learning, learning rate, batch size, ANOVA analysis, hyperparameter