PowerPoint 演示文稿

Chengming Xu cmxu18@fudan.edu.cn chmxu.github.io.

Personal Experience. Research Interests Few-shot learning, action understanding, foundation model adaptation.

Few-shot Learning. Normal deep learning training Large-scale dataset Sufficient data for each category.

Few-shot Learning. Normal deep learning training Large-scale dataset Sufficient data for each category.

Few-shot Learning. Base data. Novel data.

FSL Applications. Lee X Y, Vidyaratne L, Alam M, et al. XDNet: A Few-Shot Meta-Learning Approach for Cross-Domain Visual Inspection[C]//CVPRW2023.

FSL Applications. svpdouov øyo uy sofieull Indul.

FSL & meta-learning. Stn.

Developing FSL algorithms. Deep learning based FSL.

Developing FSL algorithms. Deep learning based FSL.

General Augmentation. image Mixup Cutout CutMix.

Augment training data: PatchMix. FSL methods generally suffer from problem of distribution shift and spurious correlation.

Augment training data: PatchMix. Xu C, Liu C, Sun X, et al. PatchMix Augmentation to Identify Causal Features in Few-shot Learning[J]. TPAMI2022..

Augment training data: PatchMix. Xu C, Liu C, Sun X, et al. PatchMix Augmentation to Identify Causal Features in Few-shot Learning[J]. TPAMI2022..

Augment training data: PatchMix. Papyan V, Han X Y, Donoho D L. Prevalence of neural collapse during the terminal phase of deep learning training[J]. PNAS2020..

Augment training data: PatchMix. Xu C, Liu C, Sun X, et al. PatchMix Augmentation to Identify Causal Features in Few-shot Learning[J]. TPAMI2022..

Augment training data: PatchMix. Xu C, Liu C, Sun X, et al. PatchMix Augmentation to Identify Causal Features in Few-shot Learning[J]. TPAMI2022..

Augment training data: PatchMix. Xu C, Liu C, Sun X, et al. PatchMix Augmentation to Identify Causal Features in Few-shot Learning[J]. TPAMI2022..

Cross-domain settings on CUB, Cars, Places, Plantae Unsupervised setting on miniImageNet.

Developing FSL algorithms. Deep learning based FSL.

cÄc:Ä Soft Nearest Neighbor Classification Scores (a) Instance Embedding Embedding Adaptation Set-to-Set Function Soft Nearest Neighbor Test Instance Task Agnostic Embedding Task Specific Em bedding I Classification Scores (b) Embedding Adaptation.

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. A novel dynamic meta-filter for feature alignment in FSL. Dynamic sampling and grouping strategy to improved flexibility and efficiency. Neural ODE for flexible feature alignment.

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Not flexible: The alignment can be written as where ? is alignment function. By gradually decreasing the time step ?, such a recursive residual equation can be transformed into an ODE whose solution is the final alignment feature..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Strengthen model: DMF. Xu C, Fu Y, Liu C, et al. Learning dynamic alignment via meta-filter for few-shot learning[C]//CVPR2021..

Developing FSL algorithms. Deep learning based FSL.

Test-time finetune. External data Pre-trained backbone Class 1 Domain A Class 4 Domain B Class 2 Class 5 Class 3 Class 6 Meta-trained backbone Support set Augmented support set Task-specifically fine-tuned backbone.

Test-time finetune. ViTs have been less touched in FSL. PMF: naïve pretrain  meta-train  finetune pipeline Heavy computation: updating whole network parameters.

Test-time finetune. ViTs have been less touched in FSL. PMF: naïve pretrain  meta-train  finetune pipeline Potential overfitting: cannot handle extremely small number of labeled images.

Efficient Transformer Tuning. We propose a novel efficient Transformer Tuning (eTT) that is tailored for few-shot learning. Fewer learnable parameters, while being flexible and effective enough.

Efficient Transformer Tuning. Xu C, Yang S, Wang Y, et al. Exploring Efficient Few-shot Adaptation for Vision Transformers[J]//TMLR 2022..

Efficient Transformer Tuning. Domain residual adapter Adapter but light structure with only offset Represent gap between source and target domains, and transfer the original manifold to a more appropriate one.

Efficient Transformer Tuning. Xu C, Yang S, Wang Y, et al. Exploring Efficient Few-shot Adaptation for Vision Transformers[J]//TMLR 2022..

eTT. Xu C, Yang S, Wang Y, et al. Exploring Efficient Few-shot Adaptation for Vision Transformers[J]//TMLR 2022..

Developing FSL algorithms. Deep learning based FSL.

Worst-case risk minimization. Lei J, Li D, Xu C, et al. Worst-case Feature Risk Minimization for Data-Efficient Learning[J]//TMLR 2023..

Worst-case risk minimization. Lei J, Li D, Xu C, et al. Worst-case Feature Risk Minimization for Data-Efficient Learning[J]//TMLR 2023..

Worst-case risk minimization. Lei J, Li D, Xu C, et al. Worst-case Feature Risk Minimization for Data-Efficient Learning[J]//TMLR 2023..

FSL in current research. Deep learning based FSL.

Meta-learning in current research. Bar A, Gandelsman Y, Darrell T, et al. Visual prompting via image inpainting[J]. NIPS2022.

FSL in current research. Deep learning based FSL.

FSL in current research. Jia M, Tang L, Chen B C, et al. Visual prompt tuning[C]//ECCV2022. Zhang R, Fang R, Zhang W, et al. Tip-adapter: Training-free clip-adapter for better vision-language modeling[J]. arXiv preprint arXiv:2111.03930, 2021..