FaceReconstruction, error processing MICA image when running demo

[Daydreamer-f]

I encountered an error when running the provided example video (82-25-854x480_affwild2.mp4). The error occurred while processing the 2030th MICA image, even though the first 2029 images were processed without any issues. Could you please help me understand the cause of this problem?

[WARNING] Processing MICA image in forward pass. This is very inefficient for training. Please precompute the MICA images in the data loader.
 39%|███████████████████████████████████████████████████████████████▏                                                                                                   | 2029/5239 [04:56<07:48,  6.85it/s]
Traceback (most recent call last):
  File "demo_face_rec_on_video.py", line 174, in <module>
    main()
  File "demo_face_rec_on_video.py", line 169, in main
    reconstruct_video(args)
  File "demo_face_rec_on_video.py", line 85, in reconstruct_video
    vals = test(model, img)
  File "/vision/u/changan/projects/inferno/inferno_apps/FaceReconstruction/utils/test.py", line 7, in test
    values = model(batch, training=False, validation=False)
  File "/vision/u/changan/miniconda3/envs/work38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceRecBase.py", line 357, in forward
    batch = self.encode(batch, training=training)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceRecBase.py", line 522, in encode
    batch = self.face_encoder(batch, return_features=return_features)
  File "/vision/u/changan/miniconda3/envs/work38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceEncoder.py", line 18, in forward
    return self.encode(batch, return_features=return_features)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceEncoder.py", line 270, in encode
    batch = self.mica_deca_encoder.encode(batch, return_features=return_features)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceEncoder.py", line 211, in encode
    batch = self.mica_encoder.encode(batch, return_features=return_features)
  File "/vision/u/changan/projects/inferno/inferno/models/FaceReconstruction/FaceEncoder.py", line 173, in encode
    mica_image = self.mica_preprocessor(image, fan_landmarks, landmarks_validity=landmarks_validity)
  File "/vision/u/changan/projects/inferno/inferno/models/mica/MicaInputProcessing.py", line 58, in __call__
    mica_image = self._dirty_image_preprocessing(input_image)
  File "/vision/u/changan/projects/inferno/inferno/models/mica/MicaInputProcessing.py", line 179, in _dirty_image_preprocessing
    blob, _ = get_arcface_input(face, img)
  File "/vision/u/changan/projects/inferno/inferno/models/mica/detector.py", line 39, in get_arcface_input
    aimg = face_align.norm_crop(img, landmark=face.kps)
  File "/vision/u/changan/miniconda3/envs/work38/lib/python3.8/site-packages/insightface/utils/face_align.py", line 72, in norm_crop
    warped = cv2.warpAffine(img, M, (image_size, image_size), borderValue=0.0)
cv2.error: OpenCV(4.7.0) :-1: error: (-5:Bad argument) in function 'warpAffine'
> Overload resolution failed:
>  - M is not a numpy array, neither a scalar
>  - Expected Ptr<cv::UMat> for argument 'M'

[radekd91]

Not sure. Can you print print out what was different about that frame? Maybe some corner case and the function received bad inputs, wrong types?

[radekd91]

Pushed a fix. Please ty again.