Deep learning has achieved impressive performance across a wide variety of domains in the past decade due to its ability to learn very complex functions by adding more layers and more neurons within a layer. Despite of their great success in practice, a clear understanding of the fundamentals of deep learning is still lacking. Prior work has shown that deep neural networks have enough capacity to memorize training data with random labels. But why deep neural networks generalize well to new data, even when the number of parameters is significantly larger than the amount of training data, still remains unclear. To explain the generalization in deep learning, many complexity measures have been proposed to capture the capacity of neural networks such as VC-dimension, norm and sharpness. However these measures usually depend on the networks size and would become vacuous for very large networks. For a better understanding of the latest advancements in AI, it is critical to understand why deep learning is capable and has the capacity to raise the third wave of AI. In this talk, we will present our investigations, initiatives and insights to the interpretation of the successful deep learning.
Prof Dacheng Tao is the President of the JD Explore Academy and a Senior Vice President of JD.com. He is also an advisor and chief scientist of the digital science institute in the University of Sydney. He mainly applies statistics and mathematics to artificial intelligence and data science, and his research is detailed in one monograph and over 200 publications in prestigious journals and proceedings at leading conferences. He received the 2015 Australian Scopus-Eureka Prize, the 2018 IEEE ICDM Research Contributions Award, and the 2021 IEEE Computer Society McCluskey Technical Achievement Award. He is a fellow of the Australian Academy of Science, AAAS, ACM and IEEE.