Project Page | Video | Paper
Painting 3D Nature in 2D: View Synthesis of Natural Scenes From a Single Semantic Mask
Shangzhan Zhang, Sida Peng, Tianrun Chen, Linzhan Mou, Haotong Lin, Kaicheng Yu, Yiyi Liao, Xiaowei Zhou
CVPR 2023
conda create -n paintnature
conda activate paintnature
pip install -r requirements.txt
Install PyTorch3D.
import sys
import torch
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
version_str="".join([
f"py3{sys.version_info.minor}_cu",
torch.version.cuda.replace(".",""),
f"_pyt{pyt_version_str}"
])
!pip install fvcore iopath
!pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
Install Imaginaire.
export CUDA_VERSION=$(nvcc --version| grep -Po "(\d+\.)+\d+" | head -1)
CURRENT=$(pwd)
for p in correlation channelnorm resample2d bias_act upfirdn2d; do
cd imaginaire/third_party/${p};
rm -rf build dist *info;
python setup.py install;
cd ${CURRENT};
done
Download the pretrained model from this link.
The pretrained model should be placed in ./ckpts folder.
Download the processed data from this link.
The processed data should be placed in ./datasets folder.
Download the SPADE model from this link.
Rename the model to lhq-1NIh3_UZ6uqvzS4mJ4JVhfyYQuG9ZMmvA.pt and place it in ./checkpoints folder.
sh configs/render.sh 182 18 0.1
TODO
If you find this code useful for your research, please use the following BibTeX entry.
@inproceedings{Zhang2023Painting3N,
title={Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask},
author={Shangzhan Zhang and Sida Peng and Tianrun Chen and Linzhan Mou and Haotong Lin and Kaicheng Yu and Yiyi Liao and Xiaowei Zhou},
booktitle={CVPR},
year={2023}
}
Some of the code in this repo is borrowed from SinNeRF, AdaMPI, GANcraft, 3DP. We thank the authors for their great work!