Visit ShanghaiTech University | 中文 | How to find us
HOME > News and Events > Events
Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions
Date: 2018/7/26             Browse: 121

Speaker:     Junru Wu. TAMU

Time:          14:00—15:00, July 26 

Location:    Room 1A-200, SIST Building

Host:          Prof. Shenghua Gao


The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e.g., GoogLeNet, ResNet and Wide ResNet). Further, the high energy consumption of convolutions limits its deployment on mobile devices. To this end, we proposed a simple yet effective scheme for compressing convolutions though applying k-means clustering on the weights, compression is achieved through weight-sharing, by only recording K cluster centers and weight assignment indexes. We then introduced a novel spectrally relaxed k-means regularization, which tends to make hard assignments of convolutional layer weights to K learned cluster centers during re-training. We additionally propose an improved set of metrics to estimate energy consumption of CNN hardware implementations, whose estimation results are verified to be consistent with previously proposed energy estimation tool extrapolated from actual hardware measurements. We finally evaluated Deep k-Means across several CNN models in terms of both compression ratio and energy consumption reduction, observing promising results without incurring accuracy loss.


Junru Wu received the BE degree in Electrical Engineering from Tongji University in 2016. From 2016 to 2017, he was a research assistant at ShanghaiTech University, under the supervision of Prof. Shenghua Gao. He is currently a Ph.D. student at the Visual Informatics Group at Texas A&M University (VITA), supervised by Prof. Zhangyang Wang. He is also working closely with Dr. Xiang Yu and Dr. Manmohan Chandraker at NEC Labs America. His research interests lie in the broad areas of deep learning and computer vision, specifically including image restoration, saliency detection, and deep network compression. His recent works have been published in IEEE TPAMI, ICML, CVPR and IJCAI. He is part of the winning team in CVPR 2018 UG2 Prize Competition.

SIST-Seminar 18068