# face3d: python tools for processing 3d face

## Introduction

 I organize this repo just for fun & my personal research. 

Hope this helps and you'll enjoy it. : )

## Structure

```python
# Since triangle mesh is the most popular representation of 3D face, 
# the main part is mesh processing.
mesh/             # written in python(numpy)
|  io.py                 # read & write obj
|  vis.py                # plot mesh
|  transform.py          # transform mesh & estimate matrix
|  light.py              # add light & estimate light(to do)
|  render.py             # obj to image using rasterization render

# When can not use vercorization to optimize, loops in python are too slow to use. 
# So write core parts(loop) in c++, then use cython to compile them.
mesh_cython/      # written in c++(cython)
|  render.py             # the same API as in mesh/render.py, with faster speed.
|  light.py              # the same API as in mesh/light.py, with faster speed.
|  mesh_core*            # c++ codes of core parts.

# 3DMM is one of the most popular methods to generate & reconstruct 3d face.
morphabel_model/  # witten in python(numpy)
|  morphable_model.py    # morphable model class: generate & fit
|  fit.py                # estimate shape&expression parameters. 3dmm fitting.
|  load.py               # load 3dmm data
```


## Examples:

* **3dmm**.  `run examples/2_3dmm.py`

  left:     random example generated by 3dmm

  right:  3d face generated by fitted 3dmm&pose  parameters using 68 key points

  ![](examples/results/3dmm/generated.jpg) ![](examples/results/3dmm/fitted.jpg)

* **transform.**  `python examples/3_transform.py`
  left:  

  fix camera position & use orthographic projection.  (often used in reconstruction)

  transform face object:

  scale, change pitch angle, change yaw angle, change roll angle

  right: 

  fix obj position & use perspective projection(fovy=30).  (simulating real views)

  move camera position and rotate camera:

  from far to near,  down & up, left & right, rotate camera

  ![](examples/results/transform/obj.gif) ![](examples/results/transform/camera.gif)

* **light**.   `python examples/4_light.py`

  single point light:

  from left to right, from up to down, from near to far

  ![](examples/results/light/position.gif)

* **image map** `python examples/6_image_map.py`

  render different attributes in image pixels.

  : depth, pncc, uv coordinates

  ![](examples/results/image_map/depth.jpg) ![](examples/results/image_map/pncc.jpg) ![](examples/results/image_map/uv_coords.jpg)

* **uv map** `python examples/7_uv_map.py`

  render different attributes in uv coordinates.

  : colors(texture map), position(2d facial image & corresponding position map)

  ![](examples/results/uv_map/uv_texture_map.jpg) ![](examples/results/uv_map/image.jpg)![](examples/results/uv_map/uv_position_map.jpg) 



## Getting Started

### Prerequisite

- Python 3 (python 2 should work, but I haven't tested)

- Python packages:
  * numpy 
  * skimage (for reading&writing image)
  * scipy (for loading mat)
  * Cython (for compiling c++ files)

- **BFM Data**

  see [Data/BFM/readme.md](Data/BFM/readme.md)

### Usage

1. Clone the repository

```bash
git clone https://github.com/YadiraF/face3d
cd face3d
```

2. Compile c++ files(ignore if you use numpy version)

```bash
cd face3d/mesh_cython
python setup.py build_ext -i 
```

3. Run examples