This repository contains implementations of 5 classical zero-shot algorithms (SAE, ALE, SJE, ESZSL, and DeViSE) in the usual as well as the Generalized zero-shot learning (GZSL) settings using the
Proposed Split and evaluation protocols (eg. Class-Averaged Top-1 Accuracy) outlined in
Zero-Shot Learning - A Comprehensive Evaluation of the Good, the Bad and the Ugly (ZSLGBU) by Yongqin Xian, Christoph H. Lampert, Bernt Schiele, Zeynep Akata (TPAMI 2018).
This is the first public implementation of SAE, ALE, SJE and DeViSE under the ZSLGBU protocol. An existing implementation of ESZSL can be found here (thanks to @sbharadwajj). To this, I have added the GZSL functionality.
The original papers corresponding to the 5 algorithms are:
[1] SAE (Semantic Autoencoder) - Semantic Autoencoder for Zero-Shot Learning. Elyor Kodirov, Tao Xiang, Shaogang Gong. CVPR, 2017.
[2] ALE (Attribute Label Embedding) - Label-Embedding for Image Classification. Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid. TPAMI, 2016.
[3] SJE (Structured Joint Embedding) - Evaluation of Output Embeddings for Fine-Grained Image Classification. Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, Bernt Schiele. CVPR, 2015.
[4] ESZSL - An embarrassingly simple approach to zero-shot learning. Bernardino Romera-Paredes, Philip H. S. Torr. ICML, 2015.
[5] DeViSE - DeViSE: A Deep Visual-Semantic Embedding Model. Andrea Frome*, Greg S. Corrado*, Jonathon Shlens*, Samy Bengio, Jeffrey Dean, Marc’Aurelio Ranzato, Tomas Mikolov. NIPS, 2013.
| Dataset | Total Images | Attributes | Class Split (Tr+Val+Ts) | ZSL | GZSL | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| tr | val | ts | tr | val | tr+val | ts seen | ts unseen | ||||
| SUN | 14340 | 102 | 580+65+72 | 11600 | 1300 | 1440 | 9280 | 1040 | 10320 | 2580 | 1440 |
| CUB | 11788 | 312 | 100+50+50 | 5875 | 2946 | 2967 | 4702 | 2355 | 7057 | 1764 | 2967 |
| AWA1 | 30475 | 85 | 27+13+10 | 16864 | 7926 | 5685 | 13460 | 6372 | 19832 | 4958 | 5685 |
| AWA2 | 37322 | 85 | 27+13+10 | 20218 | 9191 | 7913 | 16187 | 7340 | 23527 | 5882 | 7913 |
| aPY | 15339 | 64 | 15+5+12 | 6086 | 1329 | 7924 | 4906 | 1026 | 5932 | 1483 | 7924 |
Each folder above has its own README with running instructions, results and their comparisons with those reported in ZSLGBU. I have also put existing code references wherever relevant.
git clone https://github.com/mvp18/Popular-ZSL-Algorithms.git
cd Popular-ZSL-Algorithms
bash setup.sh
This downloads data (splits, Res101 features and class embeddings) corresponding to the Proposed Split for AWA1, AWA2, CUB, SUN and aPY. To know more about the individual files, refer to the README.txt file available inside xlsa17 folder.
- GZSL expts for ALE
- GZSL expts for DeViSE
- GZSL expts for SJE
If you find any errors, kindly raise an issue and I will get back to you ASAP.