Official code for the paper Attentional Meta-learners for Few-shot Polythetic Classification, ICML 2022 (Ben Day*, Ramon Viñas*, Nikola Simidjievski, Pietro Liò)

Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners, such as Prototypical Networks, require an embedding dimension that is exponential in the number of task-relevant features to emulate these functions. In contrast, attentional classifiers, such as Matching Networks, are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.

Python version: 3.8.5

To install the required libraries, please run the following command:

pip install -r requirements.txt

The main files are:

• generators/*.py: Generator objects that produce tasks for episodic training.
• train*.py: Training scripts. The behaviour of each model is given by forward_fn, which defines how class probabilities are computed as a function of the query/support embeddings and support labels.
• results/*.ipynb: Notebooks to reproduce the paper results.

To use the tieredImagenet dataset, follow these steps:

1. Download dataset from here.
2. Decompress it as follows:

# Download and place "tiered-imagenet.tar" in "$DATA_ROOT/tiered-imagenet".
mkdir -p $DATA_ROOT/tiered-imagenet
cd $DATA_ROOT/tiered-imagenet
mv $DOWNLOAD_DIR/tiered-imagenet.tar .
tar -xvf tiered-imagenet.tar
rm -f tiered-imagenet.tar