Continuous Image Recognition under Micromemory

Tenure Track Assistant Professor Shanghang Zhang
Peking University, China

Although machine vision has brought great success to various fields, embodied perception is often trained in closed environments, with limitations such as closed-set hypothesis and large-sample hypothesis. However, embodied agents in the real world often face the open environment, and there are the following key challenges: 1) There are a lot of data domain offset in the open environment, and it is difficult for existing schemes to adapt to the new data domain and accurately understand the new scenario; 2) New categories appear dynamically in the open environment, and annotations cannot be obtained in time, and it is difficult for existing schemes to accurately identify new things under a small amount of annotations. In response to these challenges, this talk will introduce a series of research efforts to enhance the generalization ability of open world embodied perception, so that it can automatically adapt to new environments and recognize new things. In particular, a new continual generalization learning paradigm and multi-modal large model solution are proposed for Corner Case and other problems.

Biosketch

Shanghang Zhang is a Tenure Track Assistant Professor at the School of Computer Science, Peking University. She has been the postdoc research fellow at Berkeley AI Research Lab (BAIR), UC Berkeley. Her research is about OOD Generalization that enables the machine learning systems to generalize to new domains using limited labels, as reflected in over 80 papers on top-tier journals and conference proceedings. She has been the author and editor of the book “Deep Reinforcement Learning” published by Springer Nature. Its Electronic Edition has been downloaded 200,000 times worldwide. She has received the AAAI 2021 Best Paper Award, several Champions on international competitions, and has been selected to “2018 Rising Stars in EECS, USA”. Dr. Zhang has been the chief organizer of several workshops on ICML/NeurIPS, the special issue on ICMR, and been the Senior PC of AAAI 2023/2024. Dr. Zhang received her Ph.D. from Carnegie Mellon University in 2018, and her Master from Peking University.