Attribute transfer and prototype-based transductive methods for few-shot learning in visual object classification

Abstract

Humans are capable of learning a specific task from few observations and examples because we have the ability of learning to learn. Modern artificial intelligence (AI) systems have achieved great performance in many settings and benchmarks, but this performance is achieved through large quantities of annotated samples such as labeled images. While it might be feasible to acquire such large quantities of labeled images in some application, it remains difficult to build such image corpora in others. This is due to the fact that labeled images may be unavailable and expensive for domain experts to annotate. Few shot learning (FSL) has been proposed to tackle the challenge of training an AI system with such a human-like capability. FSL aims to learn a classifier from a set of base classes with many available labeled data, and generalize to a set of novel classes with few available labeled data. Meta-learning frameworks, which follow the key idea of learning to learn, have been proposed for FSL. In such frameworks, FSL classification tasks are sampled from base categories, and a model is optimized to perform well in those tasks which then is used to classify tasks sampled from novel categories. This dissertation introduces two contributions in the field of few-shot learning. One of these is a transductive framework that enables state-of-the-art FSL models to use intra-class attributes from within the model itself. The other contribution is a transductive metric-based iterative approach that achieves state-of-the-art accuracy in the field of few-shot learning. The first contribution of this work addresses the problem of transfer learning between few-shot source and target domains, using synthetic attributes in addition to convolutional neural networks that are pre-trained on larger image corpora where no labeled instances of the target domains are present, though they may contain instances of their superclasses. Using a probabilistic transfer learning approach from predicted classes and inferred attributes, I develop a meta-learning ensemble method. I apply this method to examine the inference approach: specifically, how it can extend and improve upon existing deep learning models for FSL, and how related probabilistic learning architectures can be adapted to use state-of-the-field deep learning components in this framework. The second contribution of this work falls under the metric-based category of FSL. Metric-based approaches use a distance measure over embeddings extracted from convolutional neural networks. While there are many such proposed approaches, a simple one is to calculate similarities between embeddings of the query set and the mean embeddings of the support set (prototypes) using cosine similarity. My second contribution leverages the query set in a supervised learning setting to enhance those prototypes using convolutional neural networks that are pre-trained on base classes and fine-tuned using my iterative prototype-enhancement approach. Beyond the supervised learning setting, this second aspect of the work also studies the effect of leveraging an extra unlabeled set in a semi-supervised learning setting. Through extensive experiments, I show the positive effect of my iterative prototype-enhancement approach in terms of accuracy.