REMOVE DOUBLE CHIN

Support W+ Space and styleSpace, based on styleGAN2 and styleGAN2-ada

cuda 10.0
python 3.6
torch==1.2.0
torchvision==0.4.0
dlib
face-alignment==1.1.0
numpy==1.16.0
opencv-contrib-python==4.3.0.36
opencv-python==4.3.0.36
Pillow==5.4.1
tensorflow-gpu==1.14.0

for stylegan-ada-pytorch

torch==1.7.0+cu110 
torchvision==0.8.0+cu110 
torchaudio==0.7.0

Double_Chin Dataset

https://drive.google.com/drive/folders/10e6WB4YLb3Mn6Us4mPAgBksGr7kBx8q0?usp=sharing

The dataset is build by our diffusion method(See the Section3.4 in our paper)

Download the following pretrained models from https://drive.google.com/drive/folders/1SQlYvN12JWPsPqXb-QKP7TgOzeGUax6R?usp=sharing, put each of them to $PATH:

Install face-alignment for face detection.

pip install face-alignment

We provides boundaries in ./interface/boundaries:

Notice that psi-0.5 dataset and psi-0.8 dataset is images and latent codes we generated by stylegan2 with psi=0.5(faces are more stable ) and psi=0.8(faces are more diverse)

You can use the pre-trained boundaries in ./interface/boundaries, or train your own boundary:

1. Data generate:

   python generate_data_and_score.py\
   --output_dir PATH_TO_DATASET\
   --num 50000\
   --truncation_psi 0.8\

If you want to generate data from your own latent codes, please set --latent_codes_path asPATH_TO_LATENT_CODE, else the latent codes will be randomly generated.

Notice that if num is more than the number of latent codes(NUM) in your input latent code file, we will randomly generate num-NUM latent codes for you.

If you only want to generate images that have double chin ,set --double_chin_only

​ 2.Coarse boundary training:

python train_coarse_boundary.py\
--output_dir PATH_TO_SAVE_BOUNDARY\
--latent_codes_path PATH_TO_LOAD_LATENT_CODE \
--scores_path PATH_TO_LOAD_SCORES\
--chosen_num_or_ratio 0.1\
--split_ratio 0.9 \

the boundary will be saved in PATH_TO_SAVE_BOUNDARY

We assume you put your data to path 'DATA' in this section

We provide the whole diffusion process step by step :

First, prepare the data for diffusion

1.using coarse boundary to prepare data

python remove_double_chin_step1.py\
--output_dir DATA\
--boundary_path PATH_TO_LOAD_BOUNDARY\
--ratio -4\
--latent_space_type WP\
--num 10000\
--truncation_psi 0.8\

For generate data from given latent code, please set

--input_latent_codes_path PATH_TO_LATENT_CODE.

Notice that if num is more than the number of latent codes(NUM) in your input latent code file, we will randomly generate num-NUM latent codes for you.

then diffuse the prepared data:

python remove_double_chin_step2.py\
--data_dir DATA\
--latent_space_type WP\
--learning_rate 0.01\
--num_iterations 100\
--loss_weight_feat 5e-5\
--truncation_psi 0.8\

the data_dir should be the same as output_dir that you input in remove_double_chin_step1.py

the results of diffusion will be saved in data_dir

3.After diffuse, you can use the results of diffuse to train a fine boundary:

python train_fine_boundary.py\
--output_dir PATH_TO_SAVE_BOUNDARY\
--latent_codes_path $DATA/codes \
--scores_path PATH_TO_LOAD_SCORES\
--chosen_num_or_ratio 0.1\
--split_ratio 0.9 \

the fine boundary have better performance in preserving face identity.

the comparison between fine boundary(right) and coarse boundary(middle):

We provide scripts to directly remove double chin.

data prepare :

For real images, we recommend you to use the projector of official stylegan2 to obtain the latent codes of real images. (Notice that we can only get the WP latent code of real images, therefore you need to use boundaries in WP(W+) latent space.)

Download the official checkpoint from https://nvlabs-fi-cdn.nvidia.com/stylegan2/networks/ , put the stylegan2-ffhq-config-f.pkl to ./styleGAN2_Projector/stylegan2/model , download the vgg model from https://nvlabs-fi-cdn.nvidia.com/stylegan/networks/metrics/vgg16_zhang_perceptual.pkl, put the vgg16_zhang_perceptual.pkl to ./styleGAN2_Projector/stylegan2/model, then check compiler_bindir_search_path list in ./styleGAN2_Projector/stylegan2\dnnlib\tflib\custom_ops.py

then run:

cd styleGAN2_Projector
python real_img_project.py\
	project-real-images\
	--network=./stylegan2/model/stylegan2-ffhq-config-f.pkl\
    --dataset=ffhq\
    --data-dir=PATH_TO_REAL_IMGS\
	--save-dir=PATH_TO_SAVE_LATENT_CODE\

then put original images in $PATH/origin, named {name}.jpg, the corresponding wp latent code should be put in $PATH/code, named {name}_wp.npy.

run:

Finally , for diffuse method (both coarse boundaries and fine boundaries work):

python main_diffuse.py\
--data_dir $PATH\
--boundary_path PATH_TO_LOAD_BOUNDARY\
--boundary_init_ratio -4.0\
--boundary_additional_ratio -1.0\
--latent_space_type WP\
--learning_rate 0.01\
--num_iterations 100\
--loss_weight_feat 1e-4\

for warp method (need fine boundaries):

python main_warp.py\
--data_dir $PATH\
--boundary_path PATH_TO_LOAD_FINE_BOUNDARY\
--boundary_init_ratio -4.0\
--latent_space_type WP

stylegan2

InterFaceGAN

stylegan2(pytorch-version)

face-alignment

idinvert