TeFF: Learning 3D-Aware GANs from Unposed Images with Template Feature Field (ECCV 2024 Oral)

Xinya Chen¹, Hanlei Guo¹, Yanrui Bin², Shangzhan Zhang¹, Yuanbo Yang¹, Yue Wang¹, Yujun Shen³, Yiyi Liao^1*

¹Zhejiang University  ²The Hong Kong Polytechnic University  ³Ant Group  ^*corresponding author

[Project Page] [paper]

Figure: Overview of TeFF.

Requirements

We test our code on PyTorch 1.12.1 and CUDA toolkit 11.3. Please follow the instructions on https://pytorch.org for installation. Other dependencies can be installed with the following command:

    conda env create -f environment.yml
    conda activate TeFF

Training

We provide the scripts for training in ./shell_scripts. Please update the path to the dataset in the scripts.

Inference

The pretrained models are available here.

To perform the inference, you can use the command:

    python gen_videos.py --outdir=${OUTDIR}
            --trunc=0.7 
            --seeds=0-8 
            --grid=3x3 
            --network=${CHECKPOINT_PATH} 
            --reload_modules=True

where CHECKPOINT_PATH should be replaced with the path to the checkpoint. OUTDIR denotes the folder to save the outputs.

BibTeX

@article{Chen2024TeFF,
    author    = {Chen, Xinya and Guo, Hanlei and Bin, Yanrui and Zhang, Shangzhan and Yang, Yuanbo and Wang, Yue and Shen, Yujun and Liao, Yiyi},
    title     = {Learning 3D-Aware GANs from Unposed Images with Template Feature Field},
    journal = {arXiv preprint arXiv:2404.05705},
    year = {2024}}