The Knowledge Within: Methods for Data-Free Model Compression
Published in CVPR 2020, 2020
Recommended citation: Matan Haroush, Itay Hubara, Elad Hoffer, Daniel Soudry. (2020). "The Knowledge Within: Methods for Data-Free Model Compression." CVPR 2020. https://openaccess.thecvf.com/content_CVPR_2020/papers/Haroush_The_Knowledge_Within_Methods_for_Data-Free_Model_Compression_CVPR_2020_paper.pdf
We propose methods for compressing models without access to the original training data. We generate synthetic data that matches the statistics of the original dataset, and use it to distill the teacher model into a smaller student model.