Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Meta-hyperband: Hyperparameter Optimization with Meta-learning and Coarse-to-Fine

: Payrosangari, S.; Sadeghi, A.; Graux, D.; Lehmann, J.


Analide, C.:
Intelligent Data Engineering and Automated Learning - IDEAL 2020. 21st International Conference. Proceedings. Pt.II : Guimaraes, Portugal, November 4-6, 2020
Cham: Springer Nature, 2020 (Lecture Notes in Computer Science 12490)
ISBN: 978-3-030-62364-7 (Print)
ISBN: 978-3-030-62365-4 (Online)
ISBN: 978-3-030-62366-1
International Conference on Intelligent Data Engineering and Automated Learning (IDEAL) <21, 2020, Online>
Fraunhofer IAIS ()

Hyperparameter optimization is one of the main pillars of machine learning algorithms. In this paper, we introduce Meta-Hyperband: a Hyperband based algorithm that improves the hyperparameter optimization by adding levels of exploitation. Unlike Hyperband method, which is a pure exploration bandit-based approach for hyperparameter optimization, our meta approach generates a trade-off between exploration and exploitation by combining the Hyperband method with meta-learning and Coarse-to-Fine modules. We analyze the performance of Meta-Hyperband on various datasets to tune the hyperparameters of CNN and SVM. The experiments indicate that in many cases Meta-Hyperband can discover hyperparameter configurations with higher quality than Hyperband, using similar amounts of resources. In particular, we discovered a CNN configuration for classifying CIFAR10 dataset which has a 3% higher performance than the configuration founded by Hyperband, and is also 0.3% more accurate than the best-reported configuration of the Bayesian optimization approach. Additionally, we release a publicly available pool of historically well-performed configurations on several datasets for CNN and SVM to ease the adoption of Meta-Hyperband.