Training

1. Multi-Scale Models
2. Holistic vs. Patch Features
3. Incremental Training
4. Training Example

1. Multi-Scale Models

All the methods implemented in menpofit are built in a multi-scale manner, i.e. in multiple resolutions. In all our core classes, this is controlled using the following three parameters:

reference_shape (PointCloud)
First, the size of the training images is normalized by rescaling them so that the scale of their ground truth shapes matches the scale of this reference shape. In case no reference shape is provided by the user, then the mean of the ground shapes is used. This step is essential in order to ensure consistency among the extracted features of the images.

diagonal (int)
This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. This rescaling takes place before normalizing the training images' size. Thus, this parameter controls the resolution of the model at the highest scale.

scales (tuple of float)
A tuple with the scale value at each level, provided in ascending order, i.e. from lowest to highest scale. These values are proportional to the final resolution achieved through the reference shape normalization.

The multi-scale training pattern followed by all menpofit methods is summarized in the following pseudocode:

REQUIRE: images, shapes
PARAMETERS: reference_shape, diagonal, scales
1. IF reference_shape IS None:
2.     reference_shape = mean_pointcloud(shapes)
3. reference_shape = reference_shape.rescale_to_diagonal(diagonal)
4. scaled_images, scaled_shapes = rescale_to_reference_shape(images, shapes,
5.                                                           reference_shape)
6. FOR scale IN scales:
7.     scaled_images, scaled_shapes = rescale(images, shapes, scale)
8.     scale_model = train_model(scaled_images, scaled_shapes)

2. Holistic vs. Patch Features

The methods implemented in menpofit can be roughly separated in two categories, based on the appearance representation approach:

• Holistic Models: These are models that use a holistic appearance, which means that the whole texture that lies within the deformable object is taken into account.
• Patch-based Models: These are models that employ a patch-based appearance representation, which means that rectangular patches are extracted that are centered around each landmark point.

Both hositic and patch-based methods have a holistic_features argument which expects the callable that will be used for extracting features from the training images. The patch-based methods build on top of the computed holistic features in order to extract patches and can have two behaviours depending on their nature:

1. Those that simply extract patches from the holistic_features representation of the training images, given a patch_shape.
2. Those that extract patches from the holistic_features representation of the training images, given a patch_shape, and additionally compute some patch_features.

3. Incremental Training

Most deformable models that exist in menpofit can be trained in an incremental fashion. This can be very useful in case you want to train a very powerful model on a large amount of training images, which would be impossible to do at once due to memory constraints. The incremental training can be performed by using the batch_size parameter, which will split the provided set of training images into batches of the specified size.

4. Training Example

Given the above basic assumptions and using the training_images loaded in the Importing Images section, an example of a typical call for training a deformable model using HolisticAAM is:

from menpofit.aam import HolisticAAM
from menpo.feature import igo

aam = HolisticAAM(training_images, reference_shape=None,
                  diagonal=150, scales=(0.5, 1.0),
                  holistic_features=igo, verbose=True)

The verbose flag allows printing the training progress:

- Computing reference shape                                                     Computing batch 0
- Building models
  - Scale 0: Done
  - Scale 1: Scaling images: [=         ] 14% (119/811) - 00:00:01 remaining

Information about any trained model can be retrieved by:

print(aam)

which returns:

Holistic Active Appearance Model
 - Images scaled to diagonal: 150.00
 - Images warped with DifferentiablePiecewiseAffine transform
 - Scales: [0.5, 1.0]
   - Scale 0.5
     - Holistic feature: igo
     - Appearance model class: PCAModel
       - 810 appearance components
     - Shape model class: OrthoPDM
       - 132 shape components
       - 4 similarity transform parameters
   - Scale 1.0
     - Holistic feature: igo
     - Appearance model class: PCAModel
       - 810 appearance components
     - Shape model class: OrthoPDM
       - 132 shape components
       - 4 similarity transform parameters