| Sign up

PDF (6.5 MB)

Cite

Collect

Submit Manuscript

Research Article | Open Access

Noise-robust few-shot classification via variational adversarial data augmentation

Renjie Xu^¹, Baodi Liu^², Kai Zhang^³, Honglong Chen^², Dapeng Tao^⁴, Weifeng Liu^²()

College of Oceanography and Space Informatics, China University of Petroleum (East China), Qingdao 266580, China

College of Control Science and Engineering, China University of Petroleum (East China), Qingdao 266580, China

School of Petroleum Engineering, China University of Petroleum (East China), Qingdao 266580, China

School of Information Science and Engineering, Yunnan University, Yunnan 650504, China

Show Author Information

Graphical Abstract

View original image Download original image

Abstract

Few-shot classification models trained with clean samples poorly classify samples from the real world with various scales of noise. To enhance the model for recognizing noisy samples, researchers usually utilize data augmentation or use noisy samples generated by adversarial training for model training. However, existing methods still have problems: (ⅰ) The effects of data augmentation on the robustness of the model are limited. (ⅱ) The noise generated by adversarial training usually causes overfitting and reduces the generalization ability of the model, which is very significant for few-shot classification. (ⅲ) Most existing methods cannot adaptively generate appropriate noise. Given the above three points, this paper proposes a noise-robust few-shot classification algorithm, VADA—Variational Adversarial Data Augmentation. Unlike existing methods, VADA utilizes a variational noise generator to generate an adaptive noise distribution according to different samples based on adversarial learning, and optimizes the generator by minimizing the expectation of the empirical risk. Applying VADA during training can make few-shot classification more robust against noisy data, while retaining generalization ability. In this paper, we utilize FEAT and ProtoNet as baseline models, and accuracy is verified on several common few-shot classification datasets, including MiniImageNet, TieredImageNet, and CUB. After training with VADA, the classification accuracy of the models increases for samples with various scales of noise.

Keywords

few-shot learning adversarial learning robustness variational method

References

[1]

Wang, Y.; Yao, Q.; Kwok, J. T.; Ni, L. M. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys Vol. 53, No. 3, Article No. 63, 2020.

Crossref Google Scholar

[2]

Gaikwad, M.; Doke, A. Survey on meta learning algorithms for few shot learning. In: Proceedings of the 6th International Conference on Intelligent Computing and Control Systems, 1876–1879, 2022.

Crossref

[3]

Shorten, C.; Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. Journal of Big Data Vol. 6, No. 1, Article No. 60, 2019.

Crossref Google Scholar

[4]

Zhong, Z.; Zheng, L.; Kang, G.; Li, S.; Yang, Y. Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 34, No. 7, 13001–13008, 2020.

Crossref Google Scholar

[5]

Rebuffi, S. A.; Gowal, S.; Calian, D. A.; Stimberg, F.; Wiles, O.; Mann, T. A. Data augmentation can improve robustness. In: Proceedings of the 35th International Conference on Neural Information Processing Systems, Article No. 2291, 29935–29948, 2024.

[6]

Silva, S. H.; Najafirad, P. Opportunities and challenges in deep learning adversarial robustness: A survey. arXiv preprint arXiv: 2007.00753, 2020.

[7]

Dong, Y.; Fu, Q. A.; Yang, X.; Pang, T.; Su, H.; Xiao, Z.; Zhu, J. Benchmarking adversarial robustness on image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 318–328, 2020.

Crossref

[8]

Cho, W.; Kim, E. Improving augmentation efficiency for few-shot learning. IEEE Access Vol. 10, 17697–17706, 2022.

Crossref Google Scholar

[9]

Fawzi, A.; Moosavi-Dezfooli, S. M.; Frossard, P. Robustness of classifiers: From adversarial to random noise. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 1632–1640, 2016.

[10]

Zhang, X.; Wang, Q.; Zhang, J.; Zhong, Z. Adversarial autoaugment. In: Proceedings of the 8th International Conference on Learning Representations, 2020.

[11]

Goodfellow, I. J.; Shlens, J.; Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1412.6572, 2014.

[12]

Schmidt, L.; Santurkar, S.; Tsipras, D.; Talwar, K.; Madry, A. Adversarially robust generalization requires more data. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 5019–5031, 2018.

[13]

Tsipras, D.; Santurkar, S.; Engstrom, L.; Turner, A.; Madry, A. Robustness may be at odds with accuracy. In: Proceedings of the 7th International Conference on Learning Representations, 2019.

[14]

Rice, L.; Wong, E.; Kolter, J. Z. Overfitting in adversarially robust deep learning. In: Proceedings of the 37th International Conference on Machine Learning, Article No. 749, 8093–8104, 2020.

[15]

You, Z.; Ye, J.; Li, K.; Xu, Z.; Wang, P. Adversarial noise layer: Regularize neural network by adding noise. In: Proceedings of the IEEE International Conference on Image Processing, 909–913, 2019.

Crossref

[16]

Goldblum, M.; Fowl, L.; Goldstein, T. Adversarially robust few-shot learning: A meta-learning approach. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, Article No. 1501, 17886–17895, 2020.

Crossref

[17]

Zhang, H.; Wang, J. Defense against adversarial attacks using feature scattering-based adversarial training. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 164, 1831–1841, 2019.

[18]

Dong, J.; Wang, Y.; Lai, J.; Xie, X. Improving adversarially robust few-shot image classification with generalizable representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9015–9024, 2022.

Crossref

[19]

Wang, H.; Deng, Z. H. Cross-domain few-shot classification via adversarial task augmentation. arXiv preprint arXiv: 2104.14385, 2021.

Crossref

[20]

Hu, Y.; Ma, A. J. Adversarial feature augmentation for cross-domain few-shot classification. In: Computer Vision – ECCV 2022. Lecture Notes in Computer Science, Vol. 13680. Avidan, S.; Brostow, G.; Cissé, M.; Farinella, G. M.; Hassner, T. Eds. Springer Cham, 20–37, 2022.

Crossref

[21]

Kingma, D. P.; Welling, M. Auto-encoding variational Bayes. arXiv preprint arXiv: 1312.6114, 2013.

[22]

Akrami, H.; Joshi, A. A.; Li, J.; Aydöre, S.; Leahy, R. M. A robust variational autoencoder using beta divergence. Knowledge-Based Systems Vol. 238, Article No. 107886, 2022.

Crossref Google Scholar

[23]

Ghojogh, B.; Ghodsi, A.; Karray, F.; Crowley, M. Factor analysis, probabilistic principal component analysis, variational inference, and variational autoencoder: Tutorial and survey. arXiv preprint arXiv: 2101.00734, 2021.

[24]

Bando, Y.; Sekiguchi, K.; Yoshii, K. Adaptive neural speech enhancement with a denoising variational autoencoder. In: Proceedings of the Interspeech, 2437–2441, 2020.

Crossref

[25]

Harford, S.; Karim, F.; Darabi, H. Generating adversarial samples on multivariate time series using variational autoencoders. IEEE/CAA Journal of Automatica Sinica Vol. 8, No. 9, 1523–1538, 2021.

Crossref Google Scholar

[26]

Luo, Y.; Pfister, H. Adversarial defense of image classification using a variational auto-encoder. arXiv preprint arXiv: 1812.02891, 2018.

[27]

Dhamija, L.; Garg, U. An adaptive randomized and secured approach against adversarial attacks. Information Security Journal: A Global Perspective Vol. 32, No. 6, 401–416, 2023.

Crossref Google Scholar

[28]

Ravi, S.; Larochelle, H. Optimization as a model for few-shot learning. In: Proceedings of the 5th International Conference on Learning Representations, 2017.

[29]

Ren, M.; Triantafillou, E.; Ravi, S.; Snell, J.; Swersky, K.; Tenenbaum, J. B.; Larochelle, H.; Zemel, R. S. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv: 1803.00676, 2018.

[30]

Ye, H. J.; Hu, H.; Zhan, D. C.; Sha, F. Few-shot learning via embedding adaptation with set-to-set functions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8805–8814, 2020.

Crossref

[31]

Welinder, P.; Branson, S.; Mita, T.; Wah, C.; Schroff, F.; Belongie, S.; Perona, P. Caltech-UCSD birds 200. Computation & Neural Systems Technical Report 2010-001, 2010.

[32]

Triantafillou, E.; Zemel, R.; Urtasun, R. Few-shot learning through an information retrieval lens. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2252–2262, 2017.

[33]

Chen, W. Y.; Liu, Y. C.; Kira, Z.; Wang, Y. C. F.; Huang, J. B. A closer look at few-shot classification. In: Proceedings of the 7th International Conference on Learning Representations, 2019.

[34]

Bertinetto, L.; Henriques, J. F.; Torr, P. H. S.; Vedaldi, A. Meta-learning with differentiable closed-form solvers. In: Proceedings of the 7th International Conference on Learning Representations, 2019.

[35]

Snell, J.; Swersky, K.; Zemel, R. Prototypical networks for fewshot learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 4080–4090, 2017.

[36]

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; Polosukhin, I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000–6010, 2017.

[37]

Boudiaf, M.; Ziko, I.; Rony, J.; Dolz, J.; Piantanida, P.; Ben Ayed, I. Information maximization for few-shot learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, Article No. 206, 2445–2457, 2020.

[38]

Le, D.; Nguyen, K. D.; Nguyen, K.; Tran, Q. H.; Nguyen, R.; Hua, B. S. POODLE: Improving few-shot learning via penalizing out-of-distribution samples. In: Proceedings of the 35th International Conference on Neural Information Processing Systems, Article No. 1833, 23942–23955, 2024.

[39]

He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778, 2016.

Crossref

[40]

Vinyals, O.; Blundell, C.; Lillicrap, T.; Kavukcuoglu, K.; Wierstra, D. Matching networks for one shot learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 3637–3645, 2016.

[41]

Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Torr, P. H. S.; Hospedales, T. M. Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1199–1208, 2018.

Crossref

[42]

Finn, C.; Abbeel, P.; Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, 1126–1135, 2017.

[43]

Ravi, S.; Larochelle, H. Optimization as a model for few-shot learning. In: Proceedings of the 5th International Conference on Learning Representations, 2017.

[44]

Rusu, A. A.; Rao, D.; Sygnowski, J.; Vinyals, O.; Pascanu, R.; Osindero, S.; Hadsell, R. Meta-learning with latent embedding optimization. In: Proceedings of the 6th International Conference on Learning Representations, 2018.

Computational Visual Media

Volume 11 Issue 1,
February 2025

Pages 227-239

DOI: 10.26599/CVM.2025.9450403

Cite this article:

Xu R, Liu B, Zhang K, et al. Noise-robust few-shot classification via variational adversarial data augmentation. Computational Visual Media, 2025, 11(1): 227-239. https://doi.org/10.26599/CVM.2025.9450403