Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

A word aligner based on multilingual encoders

Tools for formatting WMT hypothesis and test sets in XML

bin files

Beyond the Imitation Game collaborative benchmark for enormous language models

BERT score for text generation

🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.

XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 typologically diverse languages and includes nine tasks.

BLEURT is a metric for Natural Language Generation based on transfer learning.

The Unicode Cookbook for Linguists

Named Entity Recognition as Dependency Parsing

Multilingual abstractive summarization dataset extracted from WikiHow.

Named Entity Biases in Pre-trained Language Models

Espresso: A Fast End-to-End Neural Speech Recognition Toolkit

PyTorch implementation of MMT on TVCaption dataset

pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level D…