cuda 10.0
python 3.6
Windows

chin-editing Dataset

Download the following pretrained models, put each of them to PATH:

install

conda create -n Coarse2Fine python=3.6
activate Coarse2Fine
pip install -r requirements.txt

We provides boundaries in ./interface/boundaries:

Notice that psi-0.5 dataset and psi-0.8 dataset is images and latent codes we generated by stylegan2 with psi=0.5(faces are more stable ) and psi=0.8(faces are more diverse)

We provide scripts to directly remove the double chin.

data prepare :

For real images, we recommend you use the projector of official stylegan2 to obtain the latent codes (in WP(W+) latent space) of real images.

First, align the image.

python align_images.py\
	--raw_dir $DATA_PATH$/raw\
    --aligned_dir $DATA_PATH$/origin

Please check compiler_bindir_search_path list in ./styleGAN2_Projector/stylegan2\dnnlib\tflib\custom_ops.py and revise it according to MSVC/GCC/CLANG installation in your computer.

Then run Projector(tensorflow version):

cd styleGAN2_Projector
python real_img_project.py\
	project-real-images\
	--network=./stylegan2/model/stylegan2-ffhq-config-f.pkl\
    --dataset=ffhq\
    --data-dir=$DATA_PATH$/origin\
	--save-dir=$DATA_PATH$/code\

or run Projector(pytorch version):

cd styleGAN2_model/stylegan2_pytorch
python run_projector.py 
    project_real_images\ 
    --data_dir=$DATA_PATH$/origin\ 
    --num_images=THE_NUMBER_OF_IMAGE_TO_PROJECT \
    --network=../pretrain/Gs.pth\
    --output=$DATA_PATH$/code

For original images in $DATA_PATH$/origin, named {name}.jpg, the corresponding wp latent code should be put in $PATH/code, named {name}_wp.npy.

run:

Finally , for diffuse method (both coarse boundaries and fine boundaries work):

python main_diffuse.py\
--data_dir $DATA_PATH$\
--boundary_path ./interface/boundaries/fine/all\
--boundary_init_ratio -4.0\
--boundary_additional_ratio -1.0\
--latent_space_type WP\
--learning_rate 0.01\
--num_iterations 100\
--loss_weight_feat 1e-4\

for warp method (need fine boundaries):

python main_warp.py\
--data_dir $DATA_PATH$\
--boundary_path ./interface/boundaries/fine/all\
--boundary_init_ratio -4.0\
--latent_space_type WP

You can use the pre-trained boundaries in ./interface/boundaries, or train your own boundary:

1. Data generate:

   python generate_data_and_score.py\
   --output_dir $PATH_TO_DATASET$\
   --num 50000\
   --truncation_psi 0.8\

If you want to generate data from your own latent codes, please set --latent_codes_path asPATH_TO_LATENT_CODE, else the latent codes will be randomly generated.

Notice that if num is more than the number of latent codes(NUM) in your input latent code file, we will randomly generate num-NUM latent codes for you.

If you only want to generate images that have double chin ,set --double_chin_only

​ 2.Coarse boundary training:

python train_coarse_boundary.py\
--output_dir PATH_TO_SAVE_BOUNDARY\
--latent_codes_path $PATH_TO_DATASET$/w.npy \
--scores_path $PATH_TO_DATASET$/double_chin_scores.npy\
--chosen_num_or_ratio 0.1\
--split_ratio 0.9 \

the boundary will be saved in PATH_TO_SAVE_BOUNDARY

We assume you put your data to path 'DATA' in this section

We provide the whole diffusion process step by step :

First, prepare the data for diffusion

1.using coarse boundary to prepare data

python remove_double_chin_step1.py\
    --output_dir $TRAINING_DIR$\
    --boundary_path PATH_TO_LOAD_BOUNDARY\   
    --input_data_dir $PATH_TO_DATASET$

then diffuse the prepared data:

python remove_double_chin_step2.py\
	--data_dir $TRAINING_DIR$

the data_dir should be the same as output_dir that you input in remove_double_chin_step1.py

the results of diffusion will be saved in data_dir

3.After diffuse, you can use the results of diffuse to train a fine boundary:

python train_fine_boundary.py\
--output_dir PATH_TO_SAVE_BOUNDARY\
--latent_codes_path $TRAINING_DIR$/codes \
--chosen_num_or_ratio 0.1\
--split_ratio 0.9 \

the fine boundary has better performance in preserving face identity.

the comparison between fine boundary(right) and coarse boundary(middle):

[email protected]

If you use this code for your research, please cite our paper:

@article{DBLP:journals/tog/WuYX021,
  author    = {Yiqian Wu and
               Yong{-}Liang Yang and
               Qinjie Xiao and
               Xiaogang Jin},
  title     = {Coarse-to-fine: facial structure editing of portrait images via latent
               space classifications},
  journal   = {{ACM} Trans. Graph.},
  volume    = {40},
  number    = {4},
  pages     = {46:1--46:13},
  year      = {2021}
}

We thanks the following works:

StyleGAN2

InterFaceGAN

StyleGAN2(pytorch-version)

face-alignment

idinvert

MaskGAN

@inproceedings{zhu2020indomain,
  title     = {In-domain GAN Inversion for Real Image Editing},
  author    = {Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year      = {2020}
}
@inproceedings{bulat2017far,
  title={How far are we from solving the 2D \& 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)},
  author={Bulat, Adrian and Tzimiropoulos, Georgios},
  booktitle={International Conference on Computer Vision},
  year={2017}
}
@inproceedings{shen2020interpreting,
  title     = {Interpreting the Latent Space of GANs for Semantic Face Editing},
  author    = {Shen, Yujun and Gu, Jinjin and Tang, Xiaoou and Zhou, Bolei},
  booktitle = {CVPR},
  year      = {2020}
}
@inproceedings{Karras2019stylegan2,
  title     = {Analyzing and Improving the Image Quality of {StyleGAN}},
  author    = {Tero Karras and Samuli Laine and Miika Aittala and Janne Hellsten and Jaakko Lehtinen and Timo Aila},
  booktitle = {Proc. CVPR},
  year      = {2020}
}
@inproceedings{CelebAMask-HQ,
  title={MaskGAN: Towards Diverse and Interactive Facial Image Manipulation},
  author={Lee, Cheng-Han and Liu, Ziwei and Wu, Lingyun and Luo, Ping},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020}
}