Time-Series in Hyper-parameter Initialization of Machine Learning Techniques

Horváth, Tomáš; Mantovani, Rafael G.; de Carvalho, André C. P. L. F.

Abstract

Initializing the hyper-parameters (HPs) of machine learning (ML) techniques became an important step in the area of automated ML (AutoML). The main premise in HP initialization is that a HP setting that performs well for a certain dataset(s) will also be suitable for a similar dataset. Thus, evaluation of similarities of datasets based on their characteristics, named meta-features (MFs), is one of the basic tasks in meta-learning (MtL), a subfield of AutoML. Several types of MFs were developed from which those based on principal component analysis (PCA) are, despite their good descriptive characteristics and relatively easy computation, utilized only marginally. A novel approach to HP initialization combining dynamic time warping (DTW), a well-known similarity measure for time series, with PCA MFs is proposed in this paper which does not need any further settings. Exhaustive experiments, conducted for the use-cases of HP initialization of decision trees and support vector machines show the potential of the proposed approach and encourage further investigation in this direction.

Citation

Horváth, T., Mantovani, R. G., & de Carvalho, A. C. P. L. F. (2021, November). Time-Series in Hyper-parameter Initialization of Machine Learning Techniques. Presented at 22nd International Conference on Intelligent Data Engineering and Automated Learning (IDEAL2021), Manchester

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 22nd International Conference on Intelligent Data Engineering and Automated Learning (IDEAL2021)
• Start Date: Nov 25, 2021
• End Date: Nov 27, 2021
• Online Publication Date: Nov 23, 2021
• Publication Date: 2021
• Deposit Date: Apr 8, 2024
• Publisher: Springer
• Pages: 246-258
• Series Title: Lecture Notes in Computer Science
• Series Number: 13113
• Series ISSN: 0302-9743
• Book Title: Intelligent Data Engineering and Automated Learning – IDEAL 2021: 22nd International Conference, IDEAL 2021, Manchester, UK, November 25–27, 2021, Proceedings
• ISBN: 9783030916077
• DOI: https://doi.org/10.1007/978-3-030-91608-4_25
• Public URL: http://researchrepository.napier.ac.uk/Output/3587420
• Related Public URLs: https://ideal-conf.com/home21