Skip to content

Latest commit

 

History

History
592 lines (287 loc) · 19.9 KB

3D Face Papers.md

File metadata and controls

592 lines (287 loc) · 19.9 KB
Raw

#3D Face

Surveys & Doctoral Thesis

  • Face Image Analysis using a Multiple Features Fitting Strategy(2005, Basel)

  • 3D Face Modelling for 2D+3D Face Recognition(2007, Surrey)

  • Image Based 3D Face Reconstruction: A Survey(IJIG2009, Georgios Stylianou, Andreas Lanitis, EUC, CUT)

    early 3D facial acquisition approaches

  • animation reconstruction of deformable surfaces(2010, Hao Li, ETHz)

  • Inverse Rendering of Faces with a 3D Morphable Model(2012, Oswald Aldrian, York)

  • Digital Geometry Processing Theory and Applications(2012, Kun Zhou, Zhengjiang, 中文)

  • State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications

    State of the Art on 3D Reconstruction with RGB-D Cameras(EG2018, MZ, CT, MPI, Stanford, TUM, Disney, Technicolor, UEN) [talks]

Papers & Codes

Reconstruction&3D Alignment&Correspondences

1998 - 2015

  • A Morphable Model For The Synthesis Of 3D Faces

    (SIGGRAPH1998, V Blanz, T Vetter , MPI)

    3dmm,analysis-by-synthesis(cascaded, coarse to fine, using texture information), mid-detail

  • Efficient, Robust and Accurate Fitting of a 3D Morphable Model

    (ICCV2003, S Romdhani, T Vetter , Basel)

    3dmm, fitting Algorithm needs: Efficient, Robust, Accurate, Automatic. Mid-detail

  • Estimating 3D Shape and Texture Using Pixel Intensity, Edges, Specular Highlights, Texture Constraints and a Prior

    (CVPR2005, S Romdhani, T Vetter , Basel)

    3dmm, multiple features

  • A 3D Face Model for Pose and Illumination Invariant Face Recognition

    (AVSS2009, Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. , Basel) [data](https://faces.cs.unibas.ch/bfm/)

  • 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape

    (TPAMI2011, I Kemelmacher-Shlizerman, Basri R, UW)

    template, sfs, texture information, mid-detail

  • Face Reconstruction in the Wild

    (ICCV2011, Kemelmacher-Shlizerman I, Seitz S M , UW)

    collection, sparse correspondence, warp template, low-rank approximation(photometric stereo, for expression normalization), mid-detail

  • A FACS Valid 3D Dynamic Action Unit Database with Applications to 3D Dynamic Morphable Facial Modeling

    (ICCV2011, Cosker D, Krumhuber E, Hilton A. , UofSurrey)

    aam, expression

  • Viewing Real-World Faces in 3D

    (ICCV2013, T Hassner, Open U Israel)

    template, sparse correspondence, pose adjustment, depth optimization(SIFT)

  • Improving 3D Face Details based on Normal Map of Hetero-source Images

    (CVPRW2014, Yang, C., Chen, J., Su, N., & Su, G. , Tsinghua University)

  • Total Moving Face Reconstruction

    (LNCS2014, Suwajanakorn S, Kemelmacher-Shlizerman I, Seitz S M. , Washington)

    video(collections), template, average shape, pose estimation, 3d flow(correspondence), refinement, high-detail

  • FaceWarehouse: a 3D Facial Expression Database for Visual Computing

    (VCG2014, Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K., Zhejiang) [data]

  • Intrinsic Face Image Decomposition with Human Face Priors

    (ECCV2014, Li C, Zhou K, Lin S , Zhejiang)

  • Fitting 3D Morphable Models using Local Features

    (ICIP2015, Huber, P., Feng, Z. H., Christmas, W., Kittler, J., & Ratsch, M, Surrey)

    sparse correspondence, 3dmm, regression

  • What Makes Tom Hanks Look Like Tom Hanks

    (ICCV2015, Suwajanakorn S, Seitz S M, Kemelmacher-Shlizerman I. , Washington)

    collection, template, average model, 3D flow, correspondence, deformation vector, TPS, expression sililarity weighted, high-frequency details, Laplacian pyramid

  • Unconstrained Realtime Facial Performance Capture

    (CVPR2015, Hsieh, P. L., Ma, C., Yu, J., & Li, H. , USC)

    video,image collections, occlusion, segmentation, landmarks

  • Unconstrained 3D Face Reconstruction

    (CVPR2015, Roth, J., Tong, Y., & Liu, X, MSU)

    collection, sparse correspondence(landmarks), template, photometric stereo(SVD), matrix completion,

  • Pose-Invariant 3D Face Alignment

    (ICCV2015, Jourabloo, A., & Liu, X., MSU)

    alignment, dense correspondence, visibility, cascaded regressor, 3DPDM.

  • Discriminative 3D Morphable Model Fitting

    (CVPR2015)

2016

#CVPR

  • Large-pose Face Alignment via CNN-based Dense 3D Model Fitting

    (CVPR2016, Jourabloo, A., & Liu, X., MSU)

    alignment, 3dmm

  • Automated 3D Face Reconstruction from Multiple Images using Quality Measures

    (CVPR2016, Piotraschke, M., & Blanz, V , Siegen)

  • A Robust Multilinear Model Learning Framework for 3D Faces

    (CVPR2016, Bolkart, T., & Wuhrer, S., Saarland)

  • Face Alignment Across Large Poses: A 3D Solution

    (CVPR2016, Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. , MSU, CASIA)

  • Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

    (CVPR2016, Roth, J., Tong, Y., & Liu, X, MSU)

    landmarks, 3dmm, coarse-to-fine,photometric stereo, time: 7 minutes

  • Augmented Blendshapes for Real-time Simultaneous 3D Head Modeling and Facial Motion Capture

    (CVPR2016, Thomas, D., & Taniguchi, R. I. , Kyushu University)

  • A 3D Morphable Model learnt from 10,000 faces

    (CVPR2016, Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., & Dunaway, D., ICL)

#ECCV

  • Joint Face Alignment and 3D Face Reconstruction

    (ECCV2016, Liu, F., Zeng, D., Zhao, Q., & Liu, X, MSU, Sichuan U)

    alignment, landmarks

  • Real-Time Facial Segmentation and Performance Capture from RGB Input

    (ECCV2016, Saito, S., Li, T., & Li, H. , USC)

    occlusions, tracking

#Others

  • 3D Face Reconstruction by Learning from Synthetic Data

    (3DV2016, Richardson, E., Sela, M., & Kimmel, R, IIT)

    3dmm, cnn, regress 3dmm parameters, sfs

  • Face Reconstruction on Mobile Devices Using a Height Map Shape Model and Fast Regularization

    (3DV2016, Maninchedda, F., Häne, C., Oswald, M. R., & Pollefeys, M., ETH)

  • A Multiresolution 3D Morphable Face Model and Fitting Framework [code]

    (IJCV2016, Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, P., Christmas, W. J., ... & Kittler, J. ，Surrey)

  • Rapid Photorealistic Blendshape Modeling from RGB-D Sensors

    (2016, USC)

  • 3D Face Reconstruction with Region Based Best Fit Blending Using Mobile Phone for Virtual Reality Based Social Media

    (2016, Anbarjafari, G., Haamer, R. E., Lusi, I., Tikk, T., & Valgma, L. , Turkey)

    landmarks, uv texture, region

2017

#CVPR

  • 3D Face Morphable Models “In-the-Wild”

    (CVPR2017, Booth, J., Antonakos, E., Ploumpis, S., Trigeorgis, G., Panagakis, Y., & Zafeiriou, S., ICL)

    3dmm, register, UV

  • Face Normals “in-the-wild” using Fully Convolutional Networks

    (CVPR2017, Trigeorgis, G., Snape, P., Kokkinos, I., & Zafeiriou, S. , ICL)

  • Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network

    (CVPR2017, Tran, A. T., Hassner, T., Masi, I., & Medioni, G. , USC)

  • Fast 3D Reconstruction of Faces with Glasses

    (CVPR2017, Maninchedda, F., Oswald, M. R., & Pollefeys, M., ETH)

  • DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

    (CVPR2017, Güler, R. A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S., & Kokkinos, I. , ICL)

    dense correspondence, uv

  • Learning Detailed Face Reconstruction from a Single Image

    (CVPR2017, Richardson, E., Sela, M., Or-El, R., & Kimmel, R. , Washington)

  • End-to-end 3D face reconstruction with deep neural networks

    (CVPR2017, Dou, P., Shah, S. K., & Kakadiaris, I. A., UofHouston)

    3dmm, dl, directly learn 3dmm parameters

  • A Generative Model for Depth-based Robust 3D Facial Pose Tracking

    (CVPR2017, Cai, L. S. J., Pavlovic, T. J. C. V., & Ngan, K. N. , CUHK)

    occlusions

#ICCV

  • 3D Morphable Models as Spatial Transformer Networks

    (ICCV2017, Bas, A., Huber, P., Smith, W. A., Awais, M., & Kittler, J. , York, Surrey )

    dl, cnn, uv texture, landmarks, stn, 3dmm

  • Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

    (ICCV2017, Bhagavatula, C., Zhu, C., Luu, K., & Savvides, M., CMU)

  • Pose-Invariant Face Alignment with a Single CNN

    (ICCV2017, Jourabloo, A., Ye, M., Liu, X., & Ren, L. , MSU)

    alignment, 3dmm, dl, cnn

  • Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression code

    (ICCV2017, Jackson, A. S., Bulat, A., Argyriou, V., & Tzimiropoulos, G. , Nottingham)

    end-to-end, 3dmm, dl, cnn, landmarks, voxel

  • Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

    (ICCV2017, Sela, M., Richardson, E., & Kimmel, R. , IIT)

  • MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

    (ICCV2017, Tewari, A., Zollhöfer, M., Kim, H., Garrido, P., Bernard, F., Pérez, P., & Theobalt, C. , MPI)

  • Dense Face Alignment code

    (ICCVW2017, Liu, Y., Jourabloo, A., Ren, W., & Liu, X. , MSU)

  • Realtime Dynamic 3D Facial Reconstruction for Monocular Video In-the-Wild

    (ICCVW2017)

  • Learning Dense Facial Correspondences in Unconstrained Images

    (ICCV2017, Yu, R., Saito, S., Li, H., Ceylan, D., & Li, H.  , USC)

    dense correspondence

#others

  • Large Scale 3D Morphable Models [data](https://faces.cs.unibas.ch/bfm/)

    (IJCV2017, Booth, J., Roussos, A., Ponniah, A., Dunaway, D., & Zafeiriou, S. , ICL)

    for alignment, template, cnn, dl, sparse correspondence, landmarks, tps warping

  • What does 2D geometric information really tell us about 3D face shape? (2017, Bas, A., & Smith, W. A )

    shape from landmarks, shape from contours

  • Pix2Face: Direct 3D Face Model Estimation

    (2017)

    dense correspondence, 3dmm

  • 3D Face Reconstruction with Geometry Details from a Single Image

    (TIP2017, Jiang, L., Zhang, J., Deng, B., Li, H., & Liu, L. , USC, USTC)

    coarse-to-fine, landmarks, corrective deformatio, sfs

2018

#CVPR

  • Unsupervised Training for 3D Morphable Model Regression code

    (CVPR2018, Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., & Freeman, W. T , Google)

  • 4DFAB: A Large Scale 4D Database for Facial Expression Analysis and Biometric Applications data

    (CVPR2018, Cheng, S., Kotsia, I., Pantic, M., & Zafeiriou, S., ICL)

  • Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

    (CVPR2018, Cao, X., Chen, Z., Chen, A., Chen, X., Li, S., & Yu, J.  , shanghaitech)

    5 input images, 3dmm, shadow processing, light calibration, photometric stereo, denoising

  • Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition

    (CVPR2018, Liu, F., Zhu, R., Zeng, D., Zhao, Q., & Liu, X. , MSU, Sichuan)

  • Mesoscopic Facial Geometry Inference Using Deep Neural Networks

    (CVPR2018, Hao Li, USC)

    high-detail, dl, scan, uv texture, displacement

  • Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz

    (CVPR2018, Tewari, A., Zollhöfer, M., Garrido, P., Bernard, F., Kim, H., Pérez, P., & Theobalt, C., MPI)

  • SfSNet : Learning Shape, Reflectance and Illuminance of Faces in the Wild

    (CVPR2018, Sengupta, S., Kanazawa, A., Castillo, C. D., & Jacobs, D.  , Maryland, UCB )

  • Probabilistic Joint Face-Skull Modelling for Facial Reconstruction

    (CVPR2018, Madsen, D., Lüthi, M., Schneider, A., & Vetter, T. , Basel)

  • Alive Caricature from 2D to 3D

    (CVPR2018, Wu, Q., Zhang, J., Lai, Y. K., Zheng, J., & Cai, J, USTC)

  • Nonlinear 3D Face Morphable Model

    (CVPR2018, Tran, L., & Liu, X. , MSU)

  • InverseFaceNet: Deep Monocular Inverse Face Rendering

    (CVPR2018, Kim, H., Zollhöfer, M., Tewari, A., Thies, J., Richardt, C., & Theobalt, C. , MPI)

    Self-Supervised Bootstrapping

  • Extreme 3D Face Reconstruction: Looking Past Occlusions

    (CVPR2018, Tran, A. T., Hassner, T., Masi, I., Paz, E., Nirkin, Y., & Medioni, G. , USC)

  • Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

    (CVPR2018, best student paper, Joo, H., Simon, T., & Sheikh, Y. , CMU)

  • Modeling Facial Geometry using Compositional VAEs

    (CVPR2018, Bagautdinov, T., Wu, C., Saragih, J., Fua, P., & Sheikh, Y., EPEL, FRL )

#ECCV

  • 3D Face Reconstruction from Light Field Images: A Model-free Approach

    (ECCV2018, Feng, M., Gilani, S. Z., Wang, Y., & Mian, A., Western Australia, Hunan)

    epopolar plane images

  • Generating 3D Faces using Convolutional Mesh Autoencoders [code]

    (ECCV2018, Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J., MPI)

  • Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network [code]

    (ECCV2018, Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X., SJTU)

#others

  • Morphable Face Models - An Open Framework

    (FG2018, Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Schönborn, S., & Vetter, T. , Basel)

  • CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images [data&code]

    (TPAMI2018, Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang, Jianmin Zheng, USTC)

  • Multilinear Autoencoder for 3D Face Model Learning(WACV 2018, Universite Grenoble Alpes (LJK), France)

    3d scan to registered mesh. dl. height map

  • On Face Segmentation, Face Swapping, and Face Perception(AFGR,2018, HT)

  • Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild(FG2018) data

#arxiv

  • Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition(2017, Feng Liu, Xiaoming Liu)

  • Convolutional Point-set Representation: A Convolutional Bridge Between a Densely Annotated Image and 3D Face Alignment(20180317)

  • Unsupervised Depth Estimation, 3D Face Rotation and Replacement(20180325)

Production-level Reconstruction

more in computer graphics

  1. High-Quality Single-Shot Capture of Facial Geometry(TOG2010, ETHZ, Disney)

    cg, high-detail,stereo system, calibration, surface refinement, normal direction, mesoscopic

  2. Multiview Face Capture using Polarized Spherical Gradient Illumination(TOG2011)

    image collecitons

  3. High-Quality Passive Facial Performance Capture using Anchor Frames(SIGGRAPH2011, ETHZ, Disney)

    cg, stereo,anchor frame, tracking, mesh progration, physical movement, motion estimation, refinement

  4. Lightweight binocular facial perfor- mance capture under uncontrolled lighting(TOG2012, MPI)

    cg, high-detail, stereo, template,flow,data term, geometry term, smoothness term, mesh tracking, motion refinement, shape refinement, sfs

  5. Reconstructing Detailed Dynamic Face Geometry from Monocular Video(TOG2013, MPI)

    cg, dynamic, high-detail, blend model, sparse correspondence, dense correspondence(appearance matching, LBP), pose estimation , shape refinement, sfs

  6. 3D Shape Regression for Real-time Facial Animation(TOG2013, ZJU)

  7. Real-Time High-Fidelity Facial Performance Capture (TOG2015, ZJU)

    cg, landmarks, optical flow, train a regressor to learn detail

  8. Dynamic 3D Avatar Creation from Hand-held Video Input(TOG2015, EPEL)

    cg, dynamic, mobile, high-detail, avatar, 3dmm,sparse correspondence, eye mesh, tracking, refinement, sfs, detail map

  9. Reconstruction of Personalized 3D Face Rigs from Monocular Video(TOG2016, MPI)

    parametric shape prior, coarse-scale reconstruction, fine-scale(sfs), coase->medium->fine, 3dmm, corrective

  10. Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks（ASCA2017, USC)

  11. Multi-View Stereo on Consistent Face Topology(EG2017, USC)

    cg, high-detail, landmarks, template, pose estimation, refinement

  12. Avatar Digitization From a Single Image For Real-Time Rendering(SIGGRAPH Asia 2017, USC)

    cg, avatar, segmentation, head, hair, 3DMM, landmarks, texture completion

  13. Learning a model of facial shape and expression from 4D scans(TOG2017, USC, MPI)

  14. DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling(SIGGRAPH2017)

  15. High-Fidelity Facial Reflectance and Geometry Inference From an Unconstrained Image(SIGGRAPH2018, USC)

Texture

3D-aid texture generation/ UV texture completion
Keys: GAN

  1. Face Synthesis from Facial Identity Features(CVPR2017, google)

    3dmm, dl, landmarks

  2. Photorealistic Facial Texture Inference Using Deep Neural Networks(CVPR2017, Hao Li, USC)

    texture completion

  3. UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition(CVPR2018, SZ, ICL)

    gan, 3dmm, uv texture completion

  4. Multi-Attribute Robust Component Analysis for Facial UV Maps(2017, SZ, ICL)

  5. Realistic Dynamic Facial Textures from a Single Image using GANs(CVPR2017, Hao Li, USC, DeepMind)

  6. Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model(2018)

  7. Side Information for Face Completion: a Robust PCA Approach(20180120, SZ, ICL)

​

Transfer&Reenactment(Applications)

  1. Face Transfer with Multilinear Models (SIGGRAPH2005)

    Cartesian product(ID x EX x VI)

  2. Online Modeling For Realtime Facial Animation(TOG2013)

    rgbd, blendshape, corrective field

  3. Displaced Dynamic Expression Regression for Real-time Facial Tracking and Animation(SIGGRAPH2014)

  4. Real-time Expression Transfer for Facial Reenactment(SIGGRAPH AISA 2015)

  5. Face2Face: Real-time Face Capture and Reenactment of RGB Videos(CVPR2016)

    capture, transfer, 3dmm, landmarks, texture, expression, mouth retrieval

  6. Synthesizing Obama: Learning Lip Sync from Audio(SIGGRAPH2017)

  7. Deep Video Portrait(SIGGRAPH2018)

  8. HeadOn: Real-time Reenactment of Human Portrait Videos(SIGGRAPH2018)

3D-aid 2D face recognition

  1. Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification(ECCV2012, Columbia University)

  2. Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics(PAMI2006)

  3. 3D-aided face recognition robust to expression and pose variations (CVPR2014)

  4. Effective 3D based Frontalization for Unconstrained Face Recognition(ICPR2016, MICC, Florence)

  5. Effective Face Frontalization in Unconstrained Images(CVPR2015, TH, Israel)

  6. Do We Really Need to Collect Millions of Faces for Effective Face Recognition(ECCV2016, TH, USC, Israel)

  7. High-Fidelity Pose and Expression Normalization for Face Recognition in the Wild(CVPR2015)

  8. When 3D-Aided 2D Face Recognition Meets Deep Learning: An extended UR2D for Pose-Invariant Face Recognition(2017)

  9. Towards Large-Pose Face Frontalization in the Wild

  10. Fully Automatic Pose-Invariant Face Recognition via 3D Pose Normalization (ICCV2011, Cambridge, MA, USA)

3D face recognition

  1. Face Identification across Different Poses and Illuminations with a 3D Morphable Model(Automatic Face and Gesture Recognition2002, VB&TV)

  2. Preliminary Face Recognition Grand Challenge Results(2006)

  3. expression Invariant 3D Face Recognition with a Morphable Model(FG2008, TV, Basel)

  4. Bosphorus Database for 3D Face Analysis(2008)data

  5. Robust Learning from Normals for 3D face recognition(ECCV2012, SZ, ICL)

  6. Static and dynamic 3D facial expression recognition: A comprehensive survey(IVC2012, SZ, LijunYin)

  7. Deep 3D Face Identification(2017, USC)

  8. Robust Face Recognition with Deeply Normalized Depth Images (2018)

    depth image(front&neural)

  9. Learning from Millions of 3D Scans for Large-scale 3D Face Recognition(CVPR2018, Western Australia)