Hyper-parameter Selection with Bayesian Optimization

Matthew B. Blaschko
1.10.2020 14:30 to 2.10.2020 16:00
Course will be organized 100% remotely

Schedule (Finnish time):

Lecture 1: 01.10.2020 at 14:30-16:00

Lecture 2: 02.10.2020 at 14:30-16:00

Registration and more information: To receive a zoom link and register, please send an email to Dr. Aleksei Tiulpin (firstname.lastname(at)oulu.fi) with the course title in the subject.

Credits: 1 ECTS


In this module, we will cover the theory and practice of hyperparameter selection using Bayesian optimization.  Bayesian optimization is closely related to optimal experimental design, and iteratively refines a proxy model by selecting a new point to evaluate.  In the application of hyperparameter selection in machine learning, the evaluation can be performed by training and testing a model with hyperparameters determined by the Bayesian optimization procedure.  The resulting procedure is more efficient than grid search, and more principled than stochastic search algorithms such as evolutionary computing.  The theory section will cover aspects of Gaussian process modeling (the most common model underlying Bayesian optimization), acquisition functions, and model selection in machine learning.  In the practical assignment, you will get hands on experience setting up and applying state-of-the-art Bayesian optimization software packages to hyperparameter search.  The practical assignment will use the Python programming language, and require that you be able to set up a Python environment on your computer.

Requirements: Basic programming knowledge in Python.

Short Bio

Matthew B. Blaschko received the B.S. degree from Columbia University and the M.S. degree from the University of Massachusetts Amherst. He also received the Doctorate in electrical engineering and computer science (summa cum laude) from the Technische Universität Berlin for work done at the Max Planck Institute for Biological Cybernetics, Tübingen, Germany. Subsequently, he was a Newton International Fellow in the Department of Engineering Science, University of Oxford and received the Habilitation from the Ecole Normale Suprieure de Cachan, France.  Prior to joining KU Leuven, he was a Permanent Research Scientist in the INRIA Saclay Research Center and a Faculty Member at Ecole Centrale Paris.  From 2015 he is a Professor in the department of Electrical Engineering at KU Leuven, Belgium.

His research interests include machine learning techniques applied to visual data.  Prof. Blaschko has been guest editor of a special issue in the International Journal of Computer Vision, co-oranizer of the First and Second workshops on Learning with Limited Labeled Data held at NeurIPS and ICLR, respectively, and an area chair for top venues in computer vision and machine learning including Neural Information Processing Systems (NeurIPS), the International Conference on Learning Representations (ICLR), the International Conference on Machine Learning (ICML), the International Converence on Computer Vision (ICCV), the European Conference on Computer Vision (ECCV), Computer Vision and Pattern Recognition (CVPR),  Artificial Intelligence and Statistics (AISTATS), and the British Machine Vision Conference (BMVC).  He received the Main Prize of the German Association for Pattern Recognition and Best Paper Awards at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) and European Conference on Computer Vision (ECCV).


Last updated: 11.9.2020