Wataru Shimoda¹, Daichi Haraguchi², Seiichi Uchida², Kota Yamaguchi^1
1CyberAgent.Inc, ² Kyushu University
Accepted to ICCV2021. [Publication] [Arxiv] [project-page]

This repository contains the codes for "De-rendering stylized texts".

We propose to parse rendering parameters of stylized texts utilizing a neural net.

The proposed model parses rendering parameters based on famous 2d graphic engine[Skia.org|python implementation], which has compatibility with CSS in the Web. We can export the estimated rendering parameters and edit texts by an off-the-shelf rendering engine.

• Python >= 3.7
• Pytorch >= 1.8.1
• torchvision >= 0.9.1

pip install -r requirements.txt

• The proposed model is trained with google fonts.
• Download google fonts and locate in data/fonts/ as gfonts.
  • Note: the organization of font files in the google fonts is updated from our environment.
• Download font files from this link(ofl) and locate in data/fonts/gfonts/.

- cd　data/fonts
- git clone https://github.com/google/fonts.git gfonts
+ mkdir data/fonts/gfonts; cd data/fonts/gfonts
+ tar xvzf ofl.tar.gz

• The proposed model parses rendering parameters and refines them through the differentiable rendering model, which uses pre-rendered alpha maps.
• Generate pre-rendered alpha maps.

python -m util_lib.gen_pams

Pre-rendered alpha maps would be generated in data/fonts/prerendered_alpha.

• Download the pre-trained weight from this link (weight).
• Locate the weight file in weights/font100_unified.pth.

Example usage.

python test.py --imgfile=example/sample.jpg

Note

• imgfile option: path of an input image
• results would be generated in res/

The proposed model generates a reconstructed image and a pickle file for the parsed rendering parameters.
Here, we prepare a notebook file:text_edit.ipynb for the guide of the processings to edit text images using the parsed rendering parameters.

Background editing

Text editing

Border effect editing

Shadow effect editing

Text offsets editing

Font editing

Quick start.

python gen.py --bgtype=load --bg_dir=src/modules/generator/example/bg --mask_dir=src/modules/generator/example/mask

The generated text images would be located in gen_data/.

For the detail, see generator.

Quick start. Generate training data using simple background dataset.

python gen.py --bgtype=color

Train text parser model with the generated simple background data.

python train.py

For the detail, see trainer.

• Testing codes
• Codes for the text image generator
• Notebook for text editing
• Training codes for text paraser model
• Training codes for inpainting model
• Demo app

@InProceedings{Shimoda_2021_ICCV,
    author    = {Shimoda, Wataru and Haraguchi, Daichi and Uchida, Seiichi and Yamaguchi, Kota},
    title     = {De-Rendering Stylized Texts},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {1076-1085}
}

This repository is maintained by Wataru shimoda(wataru_shimoda[at]cyberagent.co.jp).