The technical details of EMICA?

[weihaosky]

Thanks for this excellent work!
In the readme, it says EMICA is a combination of DECA, EMOCA, SPECTRE, and MICA, but without any details.
May I ask how do you combine these methods to propose EMICA? Is there any documents (e.g., paper, technical report) introducing the EMICA?
Many thanks!

[radekd91]

In short, MICA's prediction for shape, another encoder that predicts camera and pose, and one more encoder that predicts the expression, that one is supervised with EMOCA's and SPECTRE's loss.

There is no paper or technical report. It's just a bit of engineering combining existing papers together. You can use it through the FaceReconstruction demos. If interested, you can step through the code.

[weihaosky]

Thanks for explaining! Did you train the encoder that predicts the expression from scratch?

[stas-polukeev]

@radekd91 Hey, thanks for the answer!
I wanted to clarify: If emoca's speed is limited by the forward passes of CNN, it can be sped up to somewhat real-time. However, MICA is optimization based approach, which is fundamentally offline. Is EMICA offline or online in that regard?

[radekd91]

MICA's identity prediction is not an optimization step but a simple forward pass through a net. An image is fed to an arcface face recognition net to get a feature. That one is then passed though an MLP to get flame identity.

…

On Thu, Nov 28, 2024, 07:39 stas-polukeev ***@***.***> wrote: @radekd91 <https://github.com/radekd91> Hey, thanks for the answer! I wanted to clarify: If emoca's speed is limited by the forward passes of CNN, it can be sped up to somewhat real-time. However, MICA is optimization based approach, which is fundamentally offline. Is EMICA offline or online in that regard? — Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7MKHPOAI25LKEQH6EKG4T2C23DTAVCNFSM6AAAAABQ3NNAM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBVGM3DKNRRHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[stas-polukeev]

@radekd91 Okay, thank you for a swift response!
also is it compatible with FLAME2023 model or is it based on 2020?

[radekd91]

If you downloaded all provided face reconstruction models, you should have a 2023 version in the assets. It's called EMICA-CVT_flame2023

[radekd91]

Thanks for explaining! Did you train the encoder that predicts the expression from scratch?

Correct. All other encoders (expect for MICA) were trained from scratch.