Publica
Hier finden Sie wissenschaftliche Publikationen aus den FraunhoferInstituten. Scalable hyperparameter optimization with lazy gaussian processes
 Institute of Electrical and Electronics Engineers IEEE; IEEE Computer Society; Association for Computing Machinery ACM: IEEE/ACM 5th Workshop on Machine Learning in High Performance Computing Environments, MLHPC 2019. Proceedings : November 1722, 2019, Denver, Colorado, USA, held in conjunction with SC 2019, International Conference for High Performance Computing, Networking, Storage and Analysis Piscataway, NJ: IEEE, 2019 ISBN: 9781728159850 ISBN: 9781728159867 pp.5665 
 Workshop on Machine Learning in High Performance Computing Environments (MLHPC) <5, 2019, Denver/Colo.> International Conference for High Performance Computing, Networking, Storage and Analysis (SC) <2019, Denver/Colo.> 

 English 
 Conference Paper 
 Fraunhofer ITWM () 
Abstract
Most machine learning methods require careful selection of hyperparameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample efficiency, Bayesian Optimization over a Gaussian Processes modeling of the parameter space has become the method of choice. Unfortunately, this approach suffers from a cubic compute complexity due to underlying Cholesky factorization, which makes it very hard to be scaled beyond a small number of sampling steps. In this paper, we present a novel, highly accurate approximation of the underlying Gaussian Process. Reducing its computational complexity from cubic to quadratic allows an efficient strong scaling of Bayesian Optimization while outperforming the previous approach regarding optimization accuracy. First experiments show speedups of a factor of 162 in single node and further speed up by a factor of 5 in a parallel environment.