Doctoral course - Self-Supervised Learning for Multimodal Data: From Models to Loss Functions

External teacher(s): Zheng-Hua Tan
External teacher(s) organization: Aalborg University
ECTS -credits: 2
Grade: Pass or Fail
Assessment: oral exam after all lectures and research report within four weeks after lectures.

Lecture hall TS127
(Estimated) Date of the course *May 15-17, 2023
May 15, at 9-11am, Monday morning: Lecture 1
May 16, at 9-11am, Tuesday morning: Lecture 2
May 16, at 13-15pm, Tuesday afternoon: Lecture 3
May 17, at 9-11am, Wed. morning: Lecture 4

May 17, at 13-16pm, Wed. afternoon: Oral exam

Learning objectives and contents:
The course gives an introduction to self-supervised learning methods for learning representations of single- and multiple-modality data, covering deep architectures, training target and loss functions used in state-of-the-art methods, and selected downstream applications. A focus will be given to loss functions including both contrastive and predictive losses. The course will be taught through a combination of lectures and mini-projects.

Tentative topics:
Self-supervised learning concept and history
Deep learning architectures
Key methods: training target and loss functions
Applications

Amount of contact teaching hours *8 hours lecture

Bio of the lecturer:

Zheng-Hua Tan is a Professor in the Department of Electronic Systems and a Co-Head of the Centre for Acoustic Signal Processing Research at Aalborg University, Aalborg, Denmark. He is also a Co-Lead of the Pioneer Centre for AI, Denmark. He received the Ph.D. degree from Shanghai Jiao Tong University (SJTU), Shanghai, China, in 1999. He was a Visiting Scientist at the Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, USA, an Associate Professor at the Department of Electronic Engineering, SJTU, Shanghai, China, and a postdoctoral fellow at the AI Laboratory, KAIST, Daejeon, Korea. His research interests include machine learning, deep learning, pattern recognition, speech and speaker recognition, noise-robust speech processing, multimodal signal processing, and social robotics. He has (co)-authored over 200 refereed publications. His work has been recognized by the prestigious IEEE Signal Processing Society 2022 Best Paper Award. He was the Chair of the IEEE Signal Processing Society Machine Learning for Signal Processing Technical Committee (MLSP TC) and is a Member of MLSP TC from 2018-2023. He has served as an Editorial Board Member/Associate Editor for IEEE/ACM Transactions on Audio, Speech and Language Processing, Computer Speech and Language, Digital Signal Processing, and Computers and Electrical Engineering. He was a Lead Guest Editor of the IEEE Journal of Selected Topics in Signal Processing and a Guest Editor of Neurocomputing. He is a TPC Vice-Chair for ICASSP 2024 and was the General Chair for IEEE MLSP 2018 and a TPC Co-Chair for IEEE SLT 2016.