Why Heuristics Work? Three Paradigms in Machine Learning

Speaker:     Prof. Gang Wang 
Time:          Mar.17th 15:00-16:30 
Location:    SIST 1A 200 
Host:            Prof. Ziping Zhao 
  
Abstract 
Heuristics are widely used in machine learning and data science, from high-resolution imaging, to deep and reinforcement learning (RL). Despite the challenges such as the highly nonconvex landscape in training deep neural networks, simple heuristics are often surprisingly effective in finding high-quality solutions. To gain a deeper understanding of why and how heuristics work well, this talk will discuss three concrete problems. The first is a century-old problem known as phase retrieval that emerges in diverse scientific and engineering applications such as X-ray crystallography, where we are given magnitude-only measurements about an image with its phase information completely missing, and we wish to recover the image. The second is the problem of training a two-layer nonlinear (ReLU) neural network over separable data, in which both the  trainability as well as the generalization issues will be investigated. The third is about temporal-difference (TD) learning, one of the most fundamental ideas in reinforcement learning, whose non-asymptotic analysis has proved challenging. We describe three simple solutions, and present some theory explaining how and why they work well, as well as some numerical examples and applications.  Joint work with Drs. Jie Chen (Tongji U.), Georgios B. Giannakis (UMN), Yonina Eldar (Weizmann), and Yousef Saad (UMN).

  
Bio 
Gang Wang received a B.Eng. degree in Automatic Control in 2011, and a Ph.D. degree in Control Science and Engineering in 2018, both from the Beijing Institute of Technology, Beijing, China. He also received a Ph.D. degree in Electrical Engineering from the University of Minnesota, Minneapolis, USA, in 2018, where he stayed as a postdoctoral researcher until 2020. Since August 2020, he has been a professor with the School of Automation, Beijing Institute of Technology. His research interests focus on the areas of signal processing, control, and reinforcement learning with applications to autonomous intelligent systems. He was the recipient of the Best Paper Award from the Frontiers of Information Technology & Electronic Engineering (FITEE, journal of the Chinese Academy of Engineering) in 2021, the Excellent Doctoral Dissertation Award from the Chinese Association of Automation in 2019, the Best Student Paper Award from the 2017 European Signal Processing Conference, and the Best Conference Paper at the 2019 IEEE Power & Energy Society General Meeting. He is currently on the editorial board of Signal Processing.

SIST Seminar 202103