The Sparsely Gated Mixture of Experts Layer for PyTorch

This repository contains the PyTorch implementation of the MoE layer described in the paper Outrageously Large Neural Networks for PyTorch.

Requirements

This example was tested using torch v1.0.0 and Python v3.6.1 on CPU.

To install the requirements run:

pip install -r requirements.py

The file example.py contains an example illustrating how to train and evaluate the MoE layer with dummy inputs and targets. To run the example:

python example.py

The code is based on the TensorFlow implementation that can be found here.

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
README.md		README.md
example.py		example.py
mlp.py		mlp.py
moe.py		moe.py
requirements.py		requirements.py