Random Matrix Theory for Machine Learning and Signal Processing: From Neural Networks to Gaussian Universality

Speaker:       Zhenyu Liao

Time:            10:00 am, Dec. 22nd.

Location:      SIST 1A-200

Host:             Prof. Ziping Zhao

Abstract:

Deep neural networks have become the cornerstone of modern machine learning, yet their multi-layer structure, nonlinearities, and intricate optimization processes pose considerable theoretical challenges. In the first part of the talk, I will review recent advances in random matrix analysis that shed new light on these complex ML models. Starting with the foundational case of linear regression, I will demonstrate how the proposed analysis extends naturally to shallow nonlinear and ultimately deep nonlinear network models. I will also discuss practical implications (e.g., compressing and/or designing equivalent neural network models) that arise from these theoretical insights. This part is based on a recent review paper https://arxiv.org/abs/2506.13139 joint with Michael W. Mahoney (University of California, Berkeley).

Gaussian universality is a pervasive concept in statistics, information/data science, and machine learning (ML). It has been both empirically observed and mathematically proven that in the high-dimensional settings, many ML methods are only able to exploit the first and second-order moments of data distribution, behaving as if the data were Gaussian or Gaussian mixtures. In the second part of the talk, we will discuss examples and counterexamples of Gaussian universality in the classification of high-dimensional Gaussian mixture and linear factor mixture models, the latter potentially including non-Gaussian components. With a flexible leave-one-out analysis approach, we derive precise expressions for the generalization performance of ERM classifiers on data drawn from these two models. We also specify the conditions under which Gaussian universality is upheld or fails, as a function of the model's nonlinearity. This part is based on joint work with Xiaoyi Mai (IMT, France).

Bio:

Zhenyu Liao is an Associate Professor in the School of Electronic Information and Communications at the Huazhong University of Science and Technology (HUST), China. He received his Ph.D. from CentraleSupélec, University Paris-Saclay, France, in 2019 and subsequently served as a Postdoctoral Fellow in the Department of Statistics and at the International Computer Science Institute (ICSI) at the University of California, Berkeley, USA. His research primarily focuses on the statistical and computational aspects of machine learning, signal processing, and data science. He has published more than thirty papers in leading machine learning and data science venues and journals, including ICML, NeurIPS, ICLR, COLT, IEEE Transactions, and AAP, and is a coauthor of the monograph Random Matrix Methods for Machine Learning. He has served as an Area Chair for major conferences such as ICML, NeurIPS, ICLR, AISTATS, and IJCNN, and as an editorial board member of Statistics and Computing. He has been invited as a CRM–Simons Visiting Professor in Canada and an ANR–CIMI Visiting Professor in France.