ReLER, CCAI, Zhejiang University
✉Corresponding Author
Figure 1. With just a single image, SIFU is capable of reconstructing a high-quality 3D clothed human model, making it well-suited for practical applications such as 3D printing and scene creation. At the heart of SIFU is a novel Side-view Conditioned Implicit Function, which is key to enhancing feature extraction and geometric precision. Furthermore, SIFU introduces a 3D Consistent Texture Refinement process, greatly improving texture quality and facilitating texture editing with the help of text-to-image diffusion models. Notably proficient in dealing with complex poses and loose clothing, SIFU stands out as an ideal solution for real-world applications.
📖 For more visual results, go checkout our project page
This repository will contain the official implementation of SIFU.
- [2024/4/5] Our paper has been accepted as Highlight (Top 11.9% of accepted papers)!
- [2024/2/28] We release the code of geometry reconstruction, including test and inference.
- [2024/2/27] SIFU has been accepted by CVPR 2024! See you in Seattle!
- [2023/12/13] We release the paper on arXiv.
- [2023/12/10] We build the Project Page.
- Ubuntu 20 / 18
- CUDA=11.6 or 11.7 or 11.8, GPU Memory > 16GB
- Python = 3.8
- PyTorch = 1.13.0 (official Get Started)
- PyTorch3D (official INSTALL.md, recommend install-from-local-clone)
git clone https://github.com/River-Zhang/SIFU.git
sudo apt-get install libeigen3-dev ffmpeg
cd SIFU
conda env create -f environment.yaml
conda activate sifu
pip install -r requirements.txt
Please download the checkpoint (google drive) and place them in ./data/ckpt
Please follow ICON to download the extra data, such as HPS and SMPL. There may be missing files about SMPL, and you can download from here and put them in /data/smpl_related/smpl_data/.
python -m apps.infer -cfg ./configs/sifu.yaml -gpu 0 -in_dir ./examples -out_dir ./results -loop_smpl 100 -loop_cloth 200 -hps_type pixie
# 1. Register at http://icon.is.tue.mpg.de/ or https://cape.is.tue.mpg.de/
# 2. Download CAPE testset
bash fetch_cape.sh
# evaluation
python -m apps.train -cfg ./configs/train/sifu.yaml -test
# TIP: the default "mcube_res" is 256 in apps/train.
