Welcome¶
Welcome to the MenpoFit documentation!
MenpoFit is a Python package for building, fitting and manipulating deformable models. It includes stateoftheart deformable modelling techniques implemented on top of the Menpo project. Currently, the techniques that have been implemented include:

LucasKanade Optimisation
CascadedRegression Optimisation
Active Pictorial Structures (APS)
Weighted GaussNewton Optimisation with fixed Jacobian and Hessian

LucasKanade Optimisation
LucasKanade Image Alignment (LK)
Forward Additive, Forward Compositional, Inverse Compositional
Residuals: SSD, Fourier SSD, ECC, Gradient Correlation, Gradient Images
Unified Active Appearance Model and Constrained Local Model (Unified AAMCLM)
Alternating/Project Out with Regularised Landmark Mean Shift

Active Shape Model
Regularised Landmark Mean Shift
Ensemble of Regression Trees (ERT) [provided by DLib]
Supervised Descent Method (SDM)
Non Parametric
Parametric Shape
Parametric Appearance
Fully Parametric
Please see the to References for an indicative list of papers that are relevant to the methods implemented in MenpoFit.
User Guide¶
The User Guide is designed to give you an overview of the key concepts within MenpoFit. In particular, we want to try and explain some of the design decisions that we made and demonstrate why we think they are powerful concepts for building, fitting and analysing deformable models.
Quick Start¶
Here we give a very quick rundown of the basic links and information sources for the project.
Basic Installation¶
In the Menpo Team, we strongly advocate the usage of conda for scientific Python, as it makes installation of compiled binaries much more simple. In particular, if you wish to use any of the related Menpo projects such as menpofit, menpo3d or menpodetect, you will not be able to easily do so without using conda. The installation of MenpoFit using conda is as easy as
$ conda install c menpo menpofit
Conda is able to work out all the requirements/dependencies of MenpoFit. You may for example notice that menpo is one of them. Please see the thorough installation instructions for each platform on the Menpo website.
API Documentation¶
MenpoFit is extensively documented on a permethod/class level and much of this documentation is reflected in the API Documentation. If any functions or classes are missing, please bring it to the attention of the developers on Github.
Notebooks¶
Explore the Menpo and MenpoFit Notebooks
For a more thorough set of examples, we provide a set of Jupyter notebooks that demonstrate common use cases of MenpoFit. The notebooks include extensive examples regarding all the stateoftheart deformable models that we provide. You may need to have a look at the Menpo notebooks in order to get an overview of the basic functionalities required by MenpoFit.
User Group and Issues¶
If you wish to get in contact with the Menpo developers, you can do so via various channels. If you have found a bug, or if any part of MenpoFit behaves in a way you do not expect, please raise an issue on Github.
If you want to ask a theoretical question, or are having problems installing or setting up MenpoFit, please visit the user group.
Introduction¶
This user guide is a general introduction to MenpoFit, aiming to provide a bird’s eye of MenpoFit’s design. After reading this guide you should be able to go explore MenpoFit’s extensive Notebooks and not be too surprised by what you see.
What makes MenpoFit better?¶
The vast majority of existing deformable modeling software suffers from one or more of the following important issues:
It is released in binary closedsource format
It does not come with training code; only pretrained models
It is not wellstructured which makes it very difficult to tweak and alter
It only focuses on a single method/model
MenpoFit overcomes the above issues by providing opensource training and fitting code for multiple stateoftheart deformable models under a unified protocol. We strongly believe that this is the only way towards reproducable and highquality research.
Core Interfaces¶
MenpoFit is an object oriented framework for building and fitting deformable models. It makes some basic assumptions that are common for all the implemented methods. For example, all deformable models are trained in multiple scales and the fitting procedure is, in most cases, iterative. MenpoFit’s key interfaces are:
MultiScaleNonParametricFitter
 multiscale fitting classMultiScaleParametricFitter
 multiscale fitting class that uses a parametric shape modelMultiScaleNonParametricIterativeResult
 multiscale result of an iterative fittingMultiScaleParametricIterativeResult
 multiscale result of an iterative fitting using a parametric shape model
Deformable Models¶
AAM
,LucasKanadeAAMFitter
,SupervisedDescentAAMFitter
 Active Appearance Model builder and fittersATM
,LucasKanadeATMFitter
 Active Template Model builder and fitterGenerativeAPS
,GaussNewtonAPSFitter
 Active Pictorial Structures builder and fitterCLM
,GradientDescentCLMFitter
 Constrained Local Model builder and fitterLucasKanadeFitter
 LucasKanade Image AlignmentSupervisedDescentFitter
 Supervised Descent Method builder and fitterDlibERT
 Ensemble of Regression Trees builder and fitter
Building Models¶
All MenpoFit’s models are built in a multiscale manner, i.e. in multiple resolutions. In all our core classes, this is controlled using the following three parameters:
 reference_shape (PointCloud)
First, the size of the training images is normalized by rescaling them so that the scale of their ground truth shapes matches the scale of this reference shape. In case no reference shape is provided, then the mean of the ground shapes is used. This step is essential in order to ensure consistency between the extracted features of the images.
 diagonal (int)
This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. This rescaling takes place before normalizing the training images’ size. Thus, diagonal controls the size of the model at the highest scale.
 scales (tuple of float)
A tuple with the scale value at each level, provided in ascending order, i.e. from lowest to highest scale. These values are proportional to the final resolution achieved through the reference shape normalization.
Additionally, all models have a holistic_features argument which expects the callable that will be used for extracting features from the training images.
Given the above assumptions, an example of a typical call for building a
deformable model using HolisticAAM
is:
from menpofit.aam import HolisticAAM
from menpo.feature import fast_dsift
aam = HolisticAAM(training_images, group='PTS', reference_shape=None,
diagonal=200, scales=(0.25, 0.5, 1.0),
holistic_features=fast_dsift, verbose=True)
Information about any kind of model can be retrieved by:
print(aam)
The next section (Fitting) explains the basics of fitting such a deformable model.
Fitting Models¶
Fitter Objects¶
MenpoFit has specialised classes for performing a fitting process that are
called Fitters. All Fitter objects are subclasses of
MultiScaleNonParametricFitter
and MultiScaleParametricFitter
.
The main difference between those two is that a MultiScaleParametricFitter
optimises over the parameters of a statistical shape model, whereas
MultiScaleNonParametricFitter
optimises directly the coordinates of a shape.
Their behaviour can differ depending on the deformable model. For example,
a LucasKanade AAM fitter (LucasKanadeAAMFitter
) assumes that you
have trained an AAM model (assume the aam we trained in the
Building section) and can be created as:
from menpofit.aam import LucasKanadeAAMFitter, WibergInverseCompositional
fitter = LucasKanadeAAMFitter(aam,
lk_algorithm_cls=WibergInverseCompositional,
n_shape=[5, 10, 15], n_appearance=150)
The constructor of the Fitter will set the active shape and appearance components based on n_shape and n_appearance respectively, and will also perform all the necessary precomputations based on the selected algorithm.
However, there are deformable models that are directly defined through a
Fitter object, which is responsible for training the model as well.
SupervisedDescentFitter
is a good example. The reason for that is that
the fitting process is utilised during the building procedure, thus the
functionality of a Fitter is required. Such models can be built as:
from menpofit.sdm import SupervisedDescentFitter, NonParametricNewton
fitter = SupervisedDescentFitter(training_images, group='PTS',
sd_algorithm_cls=NonParametricNewton,
verbose=True)
Information about a Fitter can be retrieved by:
print(fitter)
Fitting Methods¶
All the deformable models that are currently implemented in MenpoFit, which are the stateoftheart approaches in current literature, aim to find a local optimum of the cost function that they try to optimise, given an initialisation. The initialisation can be seen as an initial estimation of the target shape. MenpoFit’s Fitter objects provide two functions for fitting the model to an image:
result = fitter.fit_from_shape(image, initial_shape, max_iters=20, gt_shape=None,
return_costs=False, **kwargs)
or
result = fitter.fit_from_bb(image, bounding_box, max_iters=20, gt_shape=None,
return_costs=False, **kwargs)
They only differ on the type of initialisation. fit_from_shape
expects a
PointCloud as the initial_shape. On the other hand, the bounding_box
argument of fit_from_bb
is a PointDirectedGraph of 4 vertices that
represents the initial bounding box. The bounding box is used in order to
align the model’s reference shape and use the resulting PointCloud as the
initial shape. Such a bounding box can be retrieved using the detection
methods of menpodetect. The rest of the options are:
 max_iters (int or list of int)
Defines the maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then it specifies the maximum number of iterations per scale. Note that this does not apply on all deformable models. For example, it can control the number of iterations of a LucasKanade optimisation algorithm, but it does not affect the fitting of a cascadedregression method (e.g. SDM) which has a predefined number of cascades (iterations).
 gt_shape (PointCloud or None)
The ground truth shape associated to the image. This is only useful to compute the final fitting error. It is not used, of course, at any internal stage of the optimisation.
 return_costs (bool)
If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Thus, this option should only be used for research purposes. Finally, this argument does not apply to all deformable models. kwargs (dict)
Additional keyword arguments that can be passed to specific models.
The next section (Result) presents the basics of the fitting result.
Fitting Result¶
Objects¶
The fitting methods of the Fitters presented in the previous section return a result object. MenpoFit has three basic fitting result objects:
Result
: Basic fitting result object that holds the final shape, and optionally, the initial shape, ground truth shape and the image.MultiScaleNonParametricIterativeResult
: The result of a multiscale iterative fitting procedure. Apart from the final shape, it also stores the shapes acquired at each fitting iteration.MultiScaleParametricIterativeResult
: The same asMultiScaleNonParametricIterativeResult
with the difference that the optimisation was performed over the parameters of a statistical parametric shape model. Thus, apart from the actual shapes, it also stores the shape parameters acquired per iteration. Note that in this case, the initial shape that was provided by the user gets reconstructed using the shape model, i.e. it first gets projected in order to get the initial estimation of the shape parameters, and then gets reconstructed with those. The resulting shape is then used as initialisation for the iterative fitting process.
Attributes¶
The above result objects can provide some very useful information regarding the fitting procedure. For example, the various shapes can be retrieved as:
 result.final_shape
The final shape of the fitting procedure.
 result.initial_shape
The initial shape of the fitting procedure that was provided by the user.
 result.reconstructed_initial_shape
The reconstruction of the initial shape that was used to initialise the fitting procedure. It only applies for
MultiScaleParametricIterativeResult
. result.image
The image on which the fitting procedure was applied.
 result.gt_shape
The ground truth shape associated to the image.
 result.shapes
The list of shapes acquired at each fitting iteration. It only applies on
MultiScaleNonParametricIterativeResult
andMultiScaleParametricIterativeResult
. result.costs()
The cost values per iteration, if they were computed during fitting.
Also, a result can compute some error metrics, in case the gt_shape of the image exists:
 result.final_error()
The final fitting error.
 result.initial_error()
The initial fitting error.
 result.errors()
The list of errors acquired at each fitting iteration. It only applies on
MultiScaleNonParametricIterativeResult
andMultiScaleParametricIterativeResult
.
Visualizing Objects¶
In Menpo, we take an opinionated stance that visualization is a key part of generating research on deformable models. Therefore, we tried to make the mental overhead of visualizing objects as low as possible.
We also took a strong step towards simple visualization by integrating some of our objects with visualization widgets for the Jupyter notebook. Remember that our widgets live on their own repository, called menpowidgets.
Visualizing Models¶
Without further ado, a quick example of visualising the AAM trained in the Building section with an interactive widget:
%matplotlib inline # This is only needed if viewing in a Jupyter notebook
aam.view_aam_widget()
One can visualize the only the multiscale shape models:
%matplotlib inline
aam.view_shape_models_widget()
or the appearance models:
%matplotlib inline
import menpo.io as mio
aam.view_appearance_models_widget()
The same visualization widgets can be found in other models, such as ATM, CLM etc.
Visualizing Fitting Result¶
The fitting result objects shown in Building can be easily visualized. Specifically, the initial and final shapes can be rendered as:
%matplotlib inline
result.view(render_initial_shape=True)
Similarly, the shapes acquired at each iteration can be visualized as:
%matplotlib inline
fr.view_iterations()
and the corresponding errors as:
%matplotlib inline
fr.plot_errors()
Finally, a fitting result can also be analysed through an interactive widget as:
%matplotlib inline
fr.view_widget()
References¶
This is an indicative list of papers relevant to the methods that are implemented in MenpoFit. They are listed in alphabetical order of the first author’s surname.
J. AlabortiMedina, and S. Zafeiriou. “A Unified Framework for Compositional Fitting of Active Appearance Models”, arXiv:1601.00199.
J. AlabortiMedina, and S. Zafeiriou. “Bayesian Active Appearance Models”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
J. AlabortiMedina, and S. Zafeiriou. “Unifying Holistic and PartsBased Deformable Model Fitting”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
E. Antonakos, J. AlabortiMedina, G. Tzimiropoulos, and S. Zafeiriou. “Featurebased LucasKanade and Active Appearance Models”, IEEE Transactions on Image Processing, vol. 24, no. 9, pp. 26172632, 2015.
E. Antonakos, J. AlabortiMedina, G. Tzimiropoulos, and S. Zafeiriou. “HOG Active Appearance Models”, IEEE International Conference on Image Processing (ICIP), 2014.
E. Antonakos, J. AlabortiMedina, and S. Zafeiriou. “Active Pictorial Structures”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
A.B. Ashraf, S. Lucey, and T. Chen. “Fast Image Alignment in the Fourier Domain”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic. “Robust discriminative response map fitting with constrained local models”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.
S. Baker, and I. Matthews. “LucasKanade 20 years on: A unifying framework”, International Journal of Computer Vision, vol. 56, no. 3, pp. 221255, 2004.
P.N. Belhumeur, D.W. Jacobs, D.J. Kriegman, and N. Kumar. “Localizing parts of faces using a consensus of exemplars”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, pp. 29302940, 2013.
D.S. Bolme, J.R. Beveridge, B.A. Draper, and Y.M. Lui. “Visual Object Tracking using Adaptive Correlation Filters”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
T.F. Cootes, G.J. Edwards, and C.J. Taylor. “Active Appearance Models”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681–685, 2001.
T.F. Cootes, and C.J. Taylor. “Active shape models‘smart snakes’”, British Machine Vision Conference (BMVC), 1992.
T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham. “Active Shape Models  their training and application”, Computer Vision and Image Understanding, vol. 61, no. 1, pp. 3859, 1995.
D. Cristinacce, and T.F. Cootes. “Feature Detection and Tracking with Constrained Local Models”, British Machine Vision Conference (BMVC), 2006.
G.D. Evangelidis, and E.Z. Psarakis. “Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 18581865, 2008.
R. Gross, I. Matthews, and S. Baker. “Generic vs. person specific Active Appearance Models”, Image and Vision Computing, vol. 23, no. 12, pp. 10801093, 2005.
V. Kazemi, and J. Sullivan. “One millisecond face alignment with an ` `ensemble of regression trees”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
B.D. Lucas, and T. Kanade, “An iterative image registration technique with an application to stereo vision”, International Joint Conference on Artificial Intelligence, 1981.
I. Matthews, and S. Baker. “Active Appearance Models Revisited”, International Journal of Computer Vision, 60(2): 135164, 2004.
G. Papandreou, and P. Maragos. “Adaptive and constrained algorithms for ` `inverse compositional active appearance model fitting”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008.
D. Ross, J. Lim, R.S. Lin, and M.H. Yang. “Incremental Learning for Robust Visual Tracking”. International Journal on Computer Vision, vol. 77, no. 13, pp. 125141, 2007.
J.M. Saragih, S. Lucey, and J.F. Cohn. “Deformable model fitting by regularized landmark meanshift”, International Journal of Computer Vision, vol. 91, no. 2, pp. 200–215, 2011.
J.M. Saragih, and R. Goecke. “Learning AAM fitting through simulation”, Pattern Recognition, vol. 42, no. 11, pp. 2628–2636, 2009.
G. Tzimiropoulos, J. AlabortiMedina, S. Zafeiriou, and M. Pantic. “Active Orientation Models for Face Alignment inthewild”, IEEE Transactions on Information Forensics and Security, Special Issue on Facial Biometrics inthewild, vol. 9, no. 12, pp. 20242034, 2014.
G. Tzimiropoulos, and M. Pantic. “GaussNewton Deformable Part Models for Face Alignment IntheWild”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
G. Tzimiropoulos, J. AlabortiMedina, S. Zafeiriou, and M. Pantic. “Generic Active Appearance Models Revisited”, Asian Conference on Computer Vision, Springer, 2012.
G. Tzimiropoulos, M. Pantic. “Optimization problems for fast AAM fitting inthewild”, IEEE International Conference on Computer Vision (ICCV), 2013.
G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. “Robust and efficient parametric face alignment”, IEEE International Conference on Computer Vision (ICCV), 2011.
G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. “Subspace Learning from Image Gradient Orientations”, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol. 34, no. 12, pp. 24542466, 2012.
X. Xiong, and F. De la Torre. “Supervised descent method and its applications to face alignment”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.
The MenpoFit API¶
This section attempts to provide a simple browsing experience for the MenpoFit documentation. In MenpoFit, we use legible docstrings, and therefore, all documentation should be easily accessible in any sensible IDE (or IPython) via tab completion. However, this section should make most of the core classes available for viewing online.
Deformable Models¶
menpofit.aam
¶
Active Appearance Model¶
AAM is a generative model that consists of a statistical parametric model of the shape and the appearance of an object. MenpoFit has several AAMs which differ in the manner that they compute the warp (thus represent the appearance features).
AAM¶

class
menpofit.aam.base.
AAM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), transform=<class 'menpofit.transform.piecewiseaffine.DifferentiablePiecewiseAffine'>, shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, max_appearance_components=None, verbose=False, batch_size=None)[source]¶ Bases:
object
Class for training a multiscale holistic Active Appearance Model. Please see the references for a basic list of relevant papers.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the AAM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
transform (subclass of
DL
andDX
, optional) – A differential warp transform object, e.g.DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.max_appearance_components (int, float, list of those or
None
, optional) – The number of appearance components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.
References
 1
J. AlabortiMedina, and S. Zafeiriou. “A Unified Framework for Compositional Fitting of Active Appearance Models”, arXiv:1601.00199.
 2
T.F. Cootes, G.J. Edwards, and C.J. Taylor. “Active Appearance Models”, IEEE Transactions on Pattern Analysis & Machine Intelligence 6 (2001): 681685.
 3
I. Matthews, and S. Baker. “Active Appearance Models Revisited”, International Journal of Computer Vision, 60(2): 135164, 2004.
 4
G. Papandreou, and P. Maragos. “Adaptive and constrained algorithms for inverse compositional Active Appearance Model fitting”, IEEE Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18, June 2008.
 5
E. Antonakos, J. AlabortiMedina, G. Tzimiropoulos, and S. Zafeiriou. “FeatureBased LucasKanade and Active Appearance Models”, IEEE Transactions on Image Processing, 24(9): 26172632, 2015.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)[source]¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of
(n_params,)
ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of anAAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of menpo.image.Image) – List of the appearance reconstructions that correspond to the provided parameters.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface. It only applies in case you wish to fit the AAM with a LucasKanade algorithm (i.e.
LucasKanadeAAMFitter
). Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(images, group=None, shape_forgetting_factor=1.0, appearance_forgetting_factor=1.0, verbose=False, batch_size=None)[source]¶ Method to increment the trained AAM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.appearance_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the appearance model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, appearance_weights=None, scale_index=1)[source]¶ Generates a novel AAM instance given a set of shape and appearance weights. If no weights are provided, then the mean AAM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.appearance_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the appearance model that will be used to create a novel appearance instance. IfNone
, the weights are assumed to be zero, thus the mean appearance is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

random_instance
(scale_index=1)[source]¶ Generates a random instance of the AAM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

view_aam_widget
(n_shape_parameters=5, n_appearance_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the AAM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.n_appearance_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_appearance_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the appearance models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the shape models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
MaskedAAM¶

class
menpofit.aam.
MaskedAAM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, max_appearance_components=None, verbose=False, batch_size=None)[source]¶ Bases:
AAM
Class for training a multiscale patchbased Masked Active Appearance Model. The appearance of this model is formulated by simply masking an image with a patchbased mask.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the AAM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int), optional) – The size of the patches of the mask that is used to sample the appearance vectors.
shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.max_appearance_components (int, float, list of those or
None
, optional) – The number of appearance components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of
(n_params,)
ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of anAAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of menpo.image.Image) – List of the appearance reconstructions that correspond to the provided parameters.

build_fitter_interfaces
(sampling)¶ Method that builds the correct LucasKanade fitting interface. It only applies in case you wish to fit the AAM with a LucasKanade algorithm (i.e.
LucasKanadeAAMFitter
). Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(images, group=None, shape_forgetting_factor=1.0, appearance_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained AAM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.appearance_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the appearance model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, appearance_weights=None, scale_index=1)¶ Generates a novel AAM instance given a set of shape and appearance weights. If no weights are provided, then the mean AAM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.appearance_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the appearance model that will be used to create a novel appearance instance. IfNone
, the weights are assumed to be zero, thus the mean appearance is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the AAM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

view_aam_widget
(n_shape_parameters=5, n_appearance_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the AAM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.n_appearance_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_appearance_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the appearance models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
LinearAAM¶

class
menpofit.aam.
LinearAAM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), transform=<class 'menpofit.transform.thinsplatesplines.DifferentiableThinPlateSplines'>, shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, max_appearance_components=None, verbose=False, batch_size=None)[source]¶ Bases:
AAM
Class for training a multiscale Linear Active Appearance Model.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the AAM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
transform (subclass of
DL
andDX
, optional) – A differential warp transform object, e.g.DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.max_appearance_components (int, float, list of those or
None
, optional) – The number of appearance components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of
(n_params,)
ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of anAAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of menpo.image.Image) – List of the appearance reconstructions that correspond to the provided parameters.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface. It only applies in case you wish to fit the AAM with a LucasKanade algorithm (i.e.
LucasKanadeAAMFitter
). Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(images, group=None, shape_forgetting_factor=1.0, appearance_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained AAM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.appearance_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the appearance model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, appearance_weights=None, scale_index=1)¶ Generates a novel AAM instance given a set of shape and appearance weights. If no weights are provided, then the mean AAM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.appearance_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the appearance model that will be used to create a novel appearance instance. IfNone
, the weights are assumed to be zero, thus the mean appearance is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the AAM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

view_aam_widget
(n_shape_parameters=5, n_appearance_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the AAM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.n_appearance_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_appearance_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the appearance models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
LinearMaskedAAM¶

class
menpofit.aam.
LinearMaskedAAM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, max_appearance_components=None, verbose=False, batch_size=None)[source]¶ Bases:
AAM
Class for training a multiscale Linear Masked Active Appearance Model.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the AAM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int), optional) – The size of the patches of the mask that is used to sample the appearance vectors.
shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.max_appearance_components (int, float, list of those or
None
, optional) – The number of appearance components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of
(n_params,)
ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of anAAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of menpo.image.Image) – List of the appearance reconstructions that correspond to the provided parameters.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface. It only applies in case you wish to fit the AAM with a LucasKanade algorithm (i.e.
LucasKanadeAAMFitter
). Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(images, group=None, shape_forgetting_factor=1.0, appearance_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained AAM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.appearance_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the appearance model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, appearance_weights=None, scale_index=1)¶ Generates a novel AAM instance given a set of shape and appearance weights. If no weights are provided, then the mean AAM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.appearance_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the appearance model that will be used to create a novel appearance instance. IfNone
, the weights are assumed to be zero, thus the mean appearance is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the AAM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

view_aam_widget
(n_shape_parameters=5, n_appearance_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the AAM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.n_appearance_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_appearance_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the appearance models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
PatchAAM¶

class
menpofit.aam.
PatchAAM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), patch_normalisation=<function no_op>, shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, max_appearance_components=None, verbose=False, batch_size=None)[source]¶ Bases:
AAM
Class for training a multiscale PatchBased Active Appearance Model. The appearance of this model is formulated by simply sampling patches around the image’s landmarks.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the AAM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.
patch_normalisation (list of callable or a single callable, optional) – The normalisation function to be applied on the extracted patches. If list, then it must have length equal to the number of scales. If a single patch normalization callable, then this is the one applied to all scales.
shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.max_appearance_components (int, float, list of those or
None
, optional) – The number of appearance components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)[source]¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of a
AAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of ndarray) – List of the appearance reconstructions that correspond to the provided parameters.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface. It only applies in case you wish to fit the AAM with a LucasKanade algorithm (i.e.
LucasKanadeAAMFitter
). Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(images, group=None, shape_forgetting_factor=1.0, appearance_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained AAM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.appearance_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the appearance model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the AAM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, appearance_weights=None, scale_index=1)¶ Generates a novel AAM instance given a set of shape and appearance weights. If no weights are provided, then the mean AAM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.appearance_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the appearance model that will be used to create a novel appearance instance. IfNone
, the weights are assumed to be zero, thus the mean appearance is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the AAM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The AAM instance.

view_aam_widget
(n_shape_parameters=5, n_appearance_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the AAM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.n_appearance_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_appearance_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the appearance models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of appearance principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the AAM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
Fitters¶
An AAM can be optimised either in a gradient descent manner (LucasKanade) or using cascaded regression (Supervised Descent).
LucasKanadeAAMFitter¶

class
menpofit.aam.
LucasKanadeAAMFitter
(aam, lk_algorithm_cls=<class 'menpofit.aam.algorithm.lk.WibergInverseCompositional'>, n_shape=None, n_appearance=None, sampling=None)[source]¶ Bases:
AAMFitter
Class for defining an AAM fitter using the LucasKanade optimisation.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step takes place at each scale and it is not considered as an iteration, thus it is not counted for the provided max_iters.
 Parameters
aam (
AAM
or subclass) – The trained AAM model.lk_algorithm_cls (class, optional) –
The LukasKanade optimisation algorithm that will get applied. The possible algorithms are:
Class
Method
Alternating
Modified Alternating
ProjectOut
Simultaneous
Wiberg
n_shape (int or float or list of those or
None
, optional) – The number of shape components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_shape_components during training.n_appearance (int or float or list of those or
None
, optional) – The number of appearance components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_appearance_components during training.sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied.

appearance_reconstructions
(appearance_parameters, n_iters_per_scale)[source]¶ Method that generates the appearance reconstructions given a set of appearance parameters. This is to be combined with a
AAMResult
object, in order to generate the appearance reconstructions of a fitting procedure. Parameters
appearance_parameters (list of
(n_params,)
ndarray) – A set of appearance parameters per fitting iteration. It can be retrieved as a property of anAAMResult
object.n_iters_per_scale (list of int) – The number of iterations per scale. This is necessary in order to figure out which appearance parameters correspond to the model of each scale. It can be retrieved as a property of a
AAMResult
object.
 Returns
appearance_reconstructions (list of menpo.image.Image) – List of the appearance reconstructions that correspond to the provided parameters.

fit_from_bb
(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial bounding box.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

fit_from_shape
(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

warped_images
(image, shapes)[source]¶ Given an input test image and a list of shapes, it warps the image into the shapes. This is useful for generating the warped images of a fitting procedure stored within an
AAMResult
. Parameters
image (menpo.image.Image or subclass) – The input image to be warped.
shapes (list of menpo.shape.PointCloud) – The list of shapes in which the image will be warped. The shapes are obtained during the iterations of a fitting procedure.
 Returns
warped_images (list of menpo.image.MaskedImage or ndarray) – The warped images.

property
holistic_features
¶ The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of closure

property
n_scales
¶ Returns the number of scales.
 Type
int

property
reference_shape
¶ The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.
 Type
menpo.shape.PointCloud

property
scales
¶ The scale value of each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of int or float
SupervisedDescentAAMFitter¶

class
menpofit.aam.
SupervisedDescentAAMFitter
(images, aam, group=None, bounding_box_group_glob=None, n_shape=None, n_appearance=None, sampling=None, sd_algorithm_cls=<class 'menpofit.aam.algorithm.sd.ProjectOutNewton'>, n_iterations=6, n_perturbations=30, perturb_from_gt_bounding_box=<function noisy_shape_from_bounding_box>, batch_size=None, verbose=False)[source]¶ Bases:
SupervisedDescentFitter
Class for training a multiscale cascadedregression Supervised Descent AAM fitter.
 Parameters
images (list of menpo.image.Image) – The list of training images.
aam (
AAM
or subclass) – The trained AAM model.group (str or
None
, optional) – The landmark group that will be used to train the fitter. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.bounding_box_group_glob (glob or
None
, optional) – Glob that defines the bounding boxes to be used for training. IfNone
, then the bounding boxes of the ground truth shapes are used.n_shape (int or float or list of those or
None
, optional) – The number of shape components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_shape_components during training.n_appearance (int or float or list of those or
None
, optional) – The number of appearance components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_appearance_components during training.sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied.sd_algorithm_cls (class, optional) –
The Supervised Descent algorithm to be used. The possible algorithms are:
Class
Features
Regression
Mean Template
ProjectOut
App. Weights
n_iterations (int or list of int, optional) – The number of iterations (cascades) of each level. If list, it must specify a value per scale. If int, then it defines the total number of iterations (cascades) over all scales.
n_perturbations (int or
None
, optional) – The number of perturbations to be generated from the provided bounding boxes.perturb_from_gt_bounding_box (callable, optional) – The function that will be used to generate the perturbations.
batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.verbose (bool, optional) – If
True
, then the progress of training will be printed.

fit_from_bb
(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial bounding box.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

fit_from_shape
(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

increment
(images, group=None, bounding_box_group_glob=None, verbose=False, batch_size=None)¶ Method to increment the trained SDM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that corresponds to the ground truth shape of each image. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.bounding_box_group_glob (glob or
None
, optional) – Glob that defines the bounding boxes to be used for training. IfNone
, then the bounding boxes of the ground truth shapes are used.verbose (bool, optional) – If
True
, then the progress of training will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

warped_images
(image, shapes)[source]¶ Given an input test image and a list of shapes, it warps the image into the shapes. This is useful for generating the warped images of a fitting procedure stored within a
MultiScaleParametricIterativeResult
. Parameters
image (menpo.image.Image or subclass) – The input image to be warped.
shapes (list of menpo.shape.PointCloud) – The list of shapes in which the image will be warped. The shapes are obtained during the iterations of a fitting procedure.
 Returns
warped_images (list of menpo.image.MaskedImage or ndarray) – The warped images.

property
holistic_features
¶ The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of closure

property
n_scales
¶ Returns the number of scales.
 Type
int

property
reference_shape
¶ The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.
 Type
menpo.shape.PointCloud

property
scales
¶ The scale value of each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of int or float
LucasKanade Optimisation Algorithms¶
AlternatingForwardCompositional¶

class
menpofit.aam.
AlternatingForwardCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Alternating
Alternating Forward Compositional (AFC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

AlternatingInverseCompositional¶

class
menpofit.aam.
AlternatingInverseCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Alternating
Alternating Inverse Compositional (AIC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

ModifiedAlternatingForwardCompositional¶

class
menpofit.aam.
ModifiedAlternatingForwardCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
ModifiedAlternating
Modified Alternating Forward Compositional (MAFC) GaussNewton algorithm

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

ModifiedAlternatingInverseCompositional¶

class
menpofit.aam.
ModifiedAlternatingInverseCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
ModifiedAlternating
Modified Alternating Inverse Compositional (MAIC) GaussNewton algorithm

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

ProjectOutForwardCompositional¶

class
menpofit.aam.
ProjectOutForwardCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
ProjectOut
Projectout Forward Compositional (POFC) GaussNewton algorithm.

project_out
(J)¶ Projectsout the appearance subspace from a given vector or matrix.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

ProjectOutInverseCompositional¶

class
menpofit.aam.
ProjectOutInverseCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
ProjectOut
Projectout Inverse Compositional (POIC) GaussNewton algorithm.

project_out
(J)¶ Projectsout the appearance subspace from a given vector or matrix.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

SimultaneousForwardCompositional¶

class
menpofit.aam.
SimultaneousForwardCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Simultaneous
Simultaneous Forward Compositional (SFC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

SimultaneousInverseCompositional¶

class
menpofit.aam.
SimultaneousInverseCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Simultaneous
Simultaneous Inverse Compositional (SIC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

WibergForwardCompositional¶

class
menpofit.aam.
WibergForwardCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Wiberg
Wiberg Forward Compositional (WFC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

WibergInverseCompositional¶

class
menpofit.aam.
WibergInverseCompositional
(aam_interface, eps=1e05)[source]¶ Bases:
Wiberg
Wiberg Inverse Compositional (WIC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
template
¶ Returns the template of the AAM (usually the mean of the appearance model).
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

Supervised Descent Optimisation Algorithms¶
AppearanceWeightsNewton¶

class
menpofit.aam.
AppearanceWeightsNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, bias=True)[source]¶ Bases:
AppearanceWeights
Class for training a cascadedregression Newton algorithm using Incremental Regularized Linear Regression (
IRLRegression
) given a trained AAM model. The algorithm uses the projection weights of the appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

project
(J)¶ Projects a given vector or matrix onto the appearance subspace.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
AppearanceWeightsGaussNewton¶

class
menpofit.aam.
AppearanceWeightsGaussNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, alpha2=0, bias=True)[source]¶ Bases:
AppearanceWeights
Class for training a cascadedregression GaussNewton algorithm using Indirect Incremental Regularized Linear Regression (
IIRLRegression
) given a trained AAM model. The algorithm uses the projection weights of the appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
alpha2 (float, optional) – The regularization parameter of the Hessian matrix.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

project
(J)¶ Projects a given vector or matrix onto the appearance subspace.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
MeanTemplateNewton¶

class
menpofit.aam.
MeanTemplateNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, bias=True)[source]¶ Bases:
MeanTemplate
Class for training a cascadedregression Newton algorithm using Incremental Regularized Linear Regression (
IRLRegression
) given a trained AAM model. The algorithm uses the centered appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
MeanTemplateGaussNewton¶

class
menpofit.aam.
MeanTemplateGaussNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, alpha2=0, bias=True)[source]¶ Bases:
MeanTemplate
Class for training a cascadedregression GaussNewton algorithm using Indirect Incremental Regularized Linear Regression (
IIRLRegression
) given a trained AAM model. The algorithm uses the centered appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
alpha2 (float, optional) – The regularization parameter of the Hessian matrix.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
ProjectOutNewton¶

class
menpofit.aam.
ProjectOutNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, bias=True)[source]¶ Bases:
ProjectOut
Class for training a cascadedregression Newton algorithm using Incremental Regularized Linear Regression (
IRLRegression
) given a trained AAM model. The algorithm uses the projectedout appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
Class
AAM
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

project_out
(J)¶ Projectsout the appearance subspace from a given vector or matrix.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
ProjectOutGaussNewton¶

class
menpofit.aam.
ProjectOutGaussNewton
(aam_interface, n_iterations=3, compute_error=<function euclidean_bb_normalised_error>, alpha=0, alpha2=0, bias=True)[source]¶ Bases:
ProjectOut
Class for training a cascadedregression GaussNewton algorithm using Indirect Incremental Regularized Linear Regression (
IIRLRegression
) given a trained AAM model. The algorithm uses the projectedout appearance vectors as features in the regression. Parameters
aam_interface (The AAM interface class from menpofit.aam.algorithm.lk.) –
Existing interfaces include:
’LucasKanadeStandardInterface’
Suitable for holistic AAMs
’LucasKanadeLinearInterface’
Suitable for linear AAMs
’LucasKanadePatchInterface’
Suitable for patchbased AAMs
n_iterations (int, optional) – The number of iterations (cascades).
compute_error (callable, optional) – The function to be used for computing the fitting error when training each cascade.
alpha (float, optional) – The regularization parameter.
alpha2 (float, optional) – The regularization parameter of the Hessian matrix.
bias (bool, optional) – Flag that controls whether to use a bias term.

increment
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to increment the model with the set of current shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

project_out
(J)¶ Projectsout the appearance subspace from a given vector or matrix.
 Type
ndarray

run
(image, initial_shape, gt_shape=None, return_costs=False, **kwargs)¶ Run the algorithm to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the fitting procedure will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated to the image.return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that this argument currently has no effect and will raise a warning if set to ``True``. This is because it is not possible to evaluate the cost function of this algorithm.
 Returns
fitting_result (
AAMAlgorithmResult
) – The parametric iterative fitting result.

train
(images, gt_shapes, current_shapes, prefix='', verbose=False)¶ Method to train the model given a set of initial shapes.
 Parameters
images (list of menpo.image.Image) – The list of training images.
gt_shapes (list of menpo.shape.PointCloud) – The list of ground truth shapes that correspond to the images.
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images, which will be used as initial shapes.
prefix (str, optional) – The prefix to use when printing information.
verbose (bool, optional) – If
True
, then information is printed during training.
 Returns
current_shapes (list of menpo.shape.PointCloud) – The list of current shapes that correspond to the images.

property
appearance_model
¶ Returns the appearance model of the AAM.
 Type
menpo.model.PCAModel

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.
Fitting Result¶
AAMResult¶

class
menpofit.aam.result.
AAMResult
(results, scales, affine_transforms, scale_transforms, image=None, gt_shape=None)[source]¶ Bases:
MultiScaleParametricIterativeResult
Class for storing the multiscale iterative fitting result of an AAM. It holds the shapes, shape parameters, appearance parameters and costs per iteration.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step is not counted in the number of iterations.
 Parameters
results (list of
AAMAlgorithmResult
) – The list of optimization results per scale.scales (list or tuple) – The list of scale values per scale (low to high).
affine_transforms (list of menpo.transform.Affine) – The list of affine transforms per scale that transform the shapes into the original image space.
scale_transforms (list of menpo.shape.Scale) – The list of scaling transforms per scale.
image (menpo.image.Image or subclass or
None
, optional) – The image on which the fitting process was applied. Note that a copy of the image will be assigned as an attribute. IfNone
, then no image is assigned.gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated with the image. IfNone
, then no ground truth shape is assigned.

displacements
()¶ A list containing the displacement between the shape of each iteration and the shape of the previous one.
 Type
list of ndarray

displacements_stats
(stat_type='mean')¶ A list containing a statistical metric on the displacements between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type (
{'mean', 'median', 'min', 'max'}
, optional) – Specifies a statistic metric to be extracted from the displacements. Returns
displacements_stat (list of float) – The statistical metric on the points displacements for each iteration.
 Raises
ValueError – type must be ‘mean’, ‘median’, ‘min’ or ‘max’

errors
(compute_error=None)¶ Returns a list containing the error at each fitting iteration, if the ground truth shape exists.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
 Returns
errors (list of float) – The error at each iteration of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

final_error
(compute_error=None)¶ Returns the final error of the fitting process, if the ground truth shape exists. This is the error computed based on the final_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the fitted and ground truth shapes.
 Returns
final_error (float) – The final error at the end of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

initial_error
(compute_error=None)¶ Returns the initial error of the fitting process, if the ground truth shape and initial shape exist. This is the error computed based on the initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the initial and ground truth shapes.
 Returns
initial_error (float) – The initial error at the beginning of the fitting process.
 Raises
ValueError – Initial shape has not been set, so the initial error cannot be computed
ValueError – Ground truth shape has not been set, so the initial error cannot be computed

plot_costs
(figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the cost function evolution at each fitting iteration.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
, optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
line_style (
{'', '', '.', ':'}
, optional) – The style of the lines.line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (marker, optional) –
The style of the markers. Example marker options
{'.', ',', 'o', 'v', '^', '<', '>', '+', 'x', 'D', 'd', 's', 'p', '*', 'h', 'H', '1', '2', '3', '4', '8'}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers.If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See below, optional) –
The font of the axes. Example options
{'serif', 'sansserif', 'cursive', 'fantasy', 'monospace'}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{'normal', 'italic', 'oblique'}
, optional) – The font style of the axes.axes_font_weight (See below, optional) –
The font weight of the axes. Example options
{'ultralight', 'light', 'normal', 'regular', 'book', 'medium', 'roman', 'semibold', 'demibold', 'demi', 'bold', 'heavy', 'extra bold', 'black'}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_displacements
(stat_type='mean', figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of a statistical metric of the displacement between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type ({
mean
,median
,min
,max
}, optional) – Specifies a statistic metric to be extracted from the displacements (see also displacements_stats() method).figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_errors
(compute_error=None, figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the error evolution at each fitting iteration.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

reconstructed_initial_error
(compute_error=None)¶ Returns the error of the reconstructed initial shape of the fitting process, if the ground truth shape exists. This is the error computed based on the reconstructed_initial_shapes[0].
 Parameters
compute_error (callable, optional) – Callable that computes the error between the reconstructed initial and ground truth shapes.
 Returns
reconstructed_initial_error (float) – The error that corresponds to the initial shape’s reconstruction.
 Raises
ValueError – Ground truth shape has not been set, so the reconstructed initial error cannot be computed

to_result
(pass_image=True, pass_initial_shape=True, pass_gt_shape=True)¶ Returns a
Result
instance of the object, i.e. a fitting result object that does not store the iterations. This can be useful for reducing the size of saved fitting results. Parameters
pass_image (bool, optional) – If
True
, then the image will get passed (if it exists).pass_initial_shape (bool, optional) – If
True
, then the initial shape will get passed (if it exists).pass_gt_shape (bool, optional) – If
True
, then the ground truth shape will get passed (if it exists).
 Returns
result (
Result
) – The final “lightweight” fitting result.

view
(figure_id=None, new_figure=False, render_image=True, render_final_shape=True, render_initial_shape=False, render_gt_shape=False, subplots_enabled=True, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, final_marker_face_colour='r', final_marker_edge_colour='k', final_line_colour='r', initial_marker_face_colour='b', initial_marker_edge_colour='k', initial_line_colour='b', gt_marker_face_colour='y', gt_marker_edge_colour='k', gt_line_colour='y', render_lines=True, line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the fitting result. The method renders the final fitted shape and optionally the initial shape, ground truth shape and the image, id they were provided.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_image (bool, optional) – If
True
and the image exists, then it gets rendered.render_final_shape (bool, optional) – If
True
, then the final fitting shape gets rendered.render_initial_shape (bool, optional) – If
True
and the initial fitting shape exists, then it gets rendered.render_gt_shape (bool, optional) – If
True
and the ground truth shape exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (See Below, optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.final_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_line_colour (See Below, optional) –
The line colour of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_line_colour (See Below, optional) –
The line colour of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_line_colour (See Below, optional) –
The line colour of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.line_style (str or list of str, optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{'', '', '.', ':'}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.marker_style (str or list of str, optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (
{center, right, left}
, optional) – The horizontal alignment of the numbers’ texts.numbers_vertical_align (
{center, top, bottom, baseline}
, optional) – The vertical alignment of the numbers’ texts.numbers_font_name (See Below, optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (See Below, optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (
{normal, italic, oblique}
, optional) – The font style of the legend.legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (See Below, optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See Below, optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (See Below, optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_iterations
(figure_id=None, new_figure=False, iters=None, render_image=True, subplots_enabled=False, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, render_lines=True, line_style='', line_width=2, line_colour=None, render_markers=True, marker_edge_colour=None, marker_face_colour=None, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the iterations of the fitting process.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.iters (int or list of int or
None
, optional) –The iterations to be visualized. If
None
, then all the iterations are rendered.No.
Visualised shape
Description
0
self.initial_shape
Initial shape
1
self.reconstructed_initial_shape
Reconstructed initial
2
self.shapes[2]
Iteration 1
i
self.shapes[i]
Iteration i1
n_iters+1
self.final_shape
Final shape
render_image (bool, optional) – If
True
and the image exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (str (See Below), optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.line_style (str or list of str (See below), optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options:
{, , ., :}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
line_colour (colour or list of colour (See Below), optional) –
The colour of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.marker_style (str or `list of str (See below), optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
marker_edge_colour (colour or list of colour (See Below), optional) –
The edge colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_face_colour (colour or list of colour (See Below), optional) –
The face (filling) colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (str (See below), optional) –
The horizontal alignment of the numbers’ texts. Example options
{center, right, left}
numbers_vertical_align (str (See below), optional) –
The vertical alignment of the numbers’ texts. Example options
{center, top, bottom, baseline}
numbers_font_name (str (See below), optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (str (See below), optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (str (See below), optional) –
The font style of the legend. Example options
{normal, italic, oblique}
legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (str (See below), optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_widget
(figure_size=(7, 7))¶ Visualizes the result object using an interactive widget.
 Parameters
figure_size ((int, int), optional) – The initial size of the rendered figure.

property
appearance_parameters
¶ Returns the list of appearance parameters obtained at each iteration of the fitting process. The list includes the parameters of the initial_shape (if it exists) and final_shape.
 Type
list of
(n_params,)
ndarray

property
costs
¶ Returns a list with the cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
final_shape
¶ Returns the final shape of the fitting process.
 Type
menpo.shape.PointCloud

property
gt_shape
¶ Returns the ground truth shape associated with the image. In case there is not an attached ground truth shape, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
image
¶ Returns the image that the fitting was applied on, if it was provided. Otherwise, it returns
None
. Type
menpo.shape.Image or subclass or
None

property
initial_shape
¶ Returns the initial shape that was provided to the fitting method to initialise the fitting process. In case the initial shape does not exist, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
is_iterative
¶ Flag whether the object is an iterative fitting result.
 Type
bool

property
n_iters
¶ Returns the total number of iterations of the fitting process.
 Type
int

property
n_iters_per_scale
¶ Returns the number of iterations per scale of the fitting process.
 Type
list of int

property
n_scales
¶ Returns the number of scales used during the fitting process.
 Type
int

property
reconstructed_initial_shapes
¶ Returns the result of the reconstruction step that takes place at each scale before applying the iterative optimisation.
 Type
list of menpo.shape.PointCloud

property
shape_parameters
¶ Returns the list of shape parameters obtained at each iteration of the fitting process. The list includes the parameters of the initial_shape (if it exists) and final_shape.
 Type
list of
(n_params,)
ndarray

property
shapes
¶ Returns the list of shapes obtained at each iteration of the fitting process. The list includes the initial_shape (if it exists) and final_shape.
 Type
list of menpo.shape.PointCloud
AAMAlgorithmResult¶

class
menpofit.aam.result.
AAMAlgorithmResult
(shapes, shape_parameters, appearance_parameters, initial_shape=None, image=None, gt_shape=None, costs=None)[source]¶ Bases:
ParametricIterativeResult
Class for storing the iterative result of an AAM optimisation algorithm.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step is not counted in the number of iterations.
 Parameters
shapes (list of menpo.shape.PointCloud) – The list of shapes per iteration. The first and last members correspond to the initial and final shapes, respectively.
shape_parameters (list of
(n_shape_parameters,)
ndarray) – The list of shape parameters per iteration. The first and last members correspond to the initial and final shapes, respectively.appearance_parameters (list of
(n_appearance_parameters,)
ndarray) – The list of appearance parameters per iteration. The first and last members correspond to the initial and final shapes, respectively.initial_shape (menpo.shape.PointCloud or
None
, optional) – The initial shape from which the fitting process started. IfNone
, then no initial shape is assigned.image (menpo.image.Image or subclass or
None
, optional) – The image on which the fitting process was applied. Note that a copy of the image will be assigned as an attribute. IfNone
, then no image is assigned.gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated with the image. IfNone
, then no ground truth shape is assigned.costs (list of float or
None
, optional) – The list of cost per iteration. IfNone
, then it is assumed that the cost function cannot be computed for the specific algorithm.

displacements
()¶ A list containing the displacement between the shape of each iteration and the shape of the previous one.
 Type
list of ndarray

displacements_stats
(stat_type='mean')¶ A list containing a statistical metric on the displacements between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type (
{'mean', 'median', 'min', 'max'}
, optional) – Specifies a statistic metric to be extracted from the displacements. Returns
displacements_stat (list of float) – The statistical metric on the points displacements for each iteration.
 Raises
ValueError – type must be ‘mean’, ‘median’, ‘min’ or ‘max’

errors
(compute_error=None)¶ Returns a list containing the error at each fitting iteration, if the ground truth shape exists.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
 Returns
errors (list of float) – The error at each iteration of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

final_error
(compute_error=None)¶ Returns the final error of the fitting process, if the ground truth shape exists. This is the error computed based on the final_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the fitted and ground truth shapes.
 Returns
final_error (float) – The final error at the end of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

initial_error
(compute_error=None)¶ Returns the initial error of the fitting process, if the ground truth shape and initial shape exist. This is the error computed based on the initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the initial and ground truth shapes.
 Returns
initial_error (float) – The initial error at the beginning of the fitting process.
 Raises
ValueError – Initial shape has not been set, so the initial error cannot be computed
ValueError – Ground truth shape has not been set, so the initial error cannot be computed

plot_costs
(figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the cost function evolution at each fitting iteration.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
, optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
line_style (
{'', '', '.', ':'}
, optional) – The style of the lines.line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (marker, optional) –
The style of the markers. Example marker options
{'.', ',', 'o', 'v', '^', '<', '>', '+', 'x', 'D', 'd', 's', 'p', '*', 'h', 'H', '1', '2', '3', '4', '8'}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers.If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See below, optional) –
The font of the axes. Example options
{'serif', 'sansserif', 'cursive', 'fantasy', 'monospace'}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{'normal', 'italic', 'oblique'}
, optional) – The font style of the axes.axes_font_weight (See below, optional) –
The font weight of the axes. Example options
{'ultralight', 'light', 'normal', 'regular', 'book', 'medium', 'roman', 'semibold', 'demibold', 'demi', 'bold', 'heavy', 'extra bold', 'black'}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_displacements
(stat_type='mean', figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of a statistical metric of the displacement between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type ({
mean
,median
,min
,max
}, optional) – Specifies a statistic metric to be extracted from the displacements (see also displacements_stats() method).figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_errors
(compute_error=None, figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the error evolution at each fitting iteration.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

reconstructed_initial_error
(compute_error=None)¶ Returns the error of the reconstructed initial shape of the fitting process, if the ground truth shape exists. This is the error computed based on the reconstructed_initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the reconstructed initial and ground truth shapes.
 Returns
reconstructed_initial_error (float) – The error that corresponds to the initial shape’s reconstruction.
 Raises
ValueError – Ground truth shape has not been set, so the reconstructed initial error cannot be computed

to_result
(pass_image=True, pass_initial_shape=True, pass_gt_shape=True)¶ Returns a
Result
instance of the object, i.e. a fitting result object that does not store the iterations. This can be useful for reducing the size of saved fitting results. Parameters
pass_image (bool, optional) – If
True
, then the image will get passed (if it exists).pass_initial_shape (bool, optional) – If
True
, then the initial shape will get passed (if it exists).pass_gt_shape (bool, optional) – If
True
, then the ground truth shape will get passed (if it exists).
 Returns
result (
Result
) – The final “lightweight” fitting result.

view
(figure_id=None, new_figure=False, render_image=True, render_final_shape=True, render_initial_shape=False, render_gt_shape=False, subplots_enabled=True, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, final_marker_face_colour='r', final_marker_edge_colour='k', final_line_colour='r', initial_marker_face_colour='b', initial_marker_edge_colour='k', initial_line_colour='b', gt_marker_face_colour='y', gt_marker_edge_colour='k', gt_line_colour='y', render_lines=True, line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the fitting result. The method renders the final fitted shape and optionally the initial shape, ground truth shape and the image, id they were provided.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_image (bool, optional) – If
True
and the image exists, then it gets rendered.render_final_shape (bool, optional) – If
True
, then the final fitting shape gets rendered.render_initial_shape (bool, optional) – If
True
and the initial fitting shape exists, then it gets rendered.render_gt_shape (bool, optional) – If
True
and the ground truth shape exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (See Below, optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.final_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_line_colour (See Below, optional) –
The line colour of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_line_colour (See Below, optional) –
The line colour of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_line_colour (See Below, optional) –
The line colour of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.line_style (str or list of str, optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{'', '', '.', ':'}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.marker_style (str or list of str, optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (
{center, right, left}
, optional) – The horizontal alignment of the numbers’ texts.numbers_vertical_align (
{center, top, bottom, baseline}
, optional) – The vertical alignment of the numbers’ texts.numbers_font_name (See Below, optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (See Below, optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (
{normal, italic, oblique}
, optional) – The font style of the legend.legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (See Below, optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See Below, optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (See Below, optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_iterations
(figure_id=None, new_figure=False, iters=None, render_image=True, subplots_enabled=False, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, render_lines=True, line_style='', line_width=2, line_colour=None, render_markers=True, marker_edge_colour=None, marker_face_colour=None, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the iterations of the fitting process.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.iters (int or list of int or
None
, optional) –The iterations to be visualized. If
None
, then all the iterations are rendered.No.
Visualised shape
Description
0
self.initial_shape
Initial shape
1
self.reconstructed_initial_shape
Reconstructed initial
2
self.shapes[2]
Iteration 1
i
self.shapes[i]
Iteration i1
n_iters+1
self.final_shape
Final shape
render_image (bool, optional) – If
True
and the image exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (str (See Below), optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.line_style (str or list of str (See below), optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options:
{, , ., :}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
line_colour (colour or list of colour (See Below), optional) –
The colour of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.marker_style (str or `list of str (See below), optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
marker_edge_colour (colour or list of colour (See Below), optional) –
The edge colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_face_colour (colour or list of colour (See Below), optional) –
The face (filling) colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (str (See below), optional) –
The horizontal alignment of the numbers’ texts. Example options
{center, right, left}
numbers_vertical_align (str (See below), optional) –
The vertical alignment of the numbers’ texts. Example options
{center, top, bottom, baseline}
numbers_font_name (str (See below), optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (str (See below), optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (str (See below), optional) –
The font style of the legend. Example options
{normal, italic, oblique}
legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (str (See below), optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_widget
(figure_size=(7, 7))¶ Visualizes the result object using an interactive widget.
 Parameters
figure_size ((int, int), optional) – The initial size of the rendered figure.

property
appearance_parameters
¶ Returns the list of appearance parameters obtained at each iteration of the fitting process. The list includes the parameters of the initial_shape (if it exists) and final_shape.
 Type
list of
(n_params,)
ndarray

property
costs
¶ Returns a list with the cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
final_shape
¶ Returns the final shape of the fitting process.
 Type
menpo.shape.PointCloud

property
gt_shape
¶ Returns the ground truth shape associated with the image. In case there is not an attached ground truth shape, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
image
¶ Returns the image that the fitting was applied on, if it was provided. Otherwise, it returns
None
. Type
menpo.shape.Image or subclass or
None

property
initial_shape
¶ Returns the initial shape that was provided to the fitting method to initialise the fitting process. In case the initial shape does not exist, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
is_iterative
¶ Flag whether the object is an iterative fitting result.
 Type
bool

property
n_iters
¶ Returns the total number of iterations of the fitting process.
 Type
int

property
reconstructed_initial_shape
¶ Returns the initial shape’s reconstruction with the shape model that was used to initialise the iterative optimisation process.
 Type
menpo.shape.PointCloud

property
shape_parameters
¶ Returns the list of shape parameters obtained at each iteration of the fitting process. The list includes the parameters of the reconstructed_initial_shape and final_shape.
 Type
list of
(n_params,)
ndarray

property
shapes
¶ Returns the list of shapes obtained at each iteration of the fitting process. The list includes the initial_shape (if it exists), reconstructed_initial_shape and final_shape.
 Type
list of menpo.shape.PointCloud
PreTrained Model¶
load_balanced_frontal_face_fitter¶

menpofit.aam.
load_balanced_frontal_face_fitter
()[source]¶ Loads a frontal face patchbased AAM fitter that is a good compromise between model size, fitting time and fitting performance. The model returns 68 facial landmark points (the standard IBUG68 markup).
Note that the first time you invoke this function, menpofit will download the fitter from Menpo’s server. The fitter will then be stored locally for future use.
The model is a
PatchAAM
trained using the following parameters:Parameter
Value
diagonal
110
scales
(0.5, 1.0)
patch_shape
[(13, 13), (13, 13)]
holistic_features
menpo.feature.fast_dsift()
n_shape
[5, 20]
n_appearance
[30, 150]
lk_algorithm_cls
It is also using the following sampling grid:
import numpy as np patch_shape = (13, 13) sampling_step = 4 sampling_grid = np.zeros(patch_shape, dtype=np.bool) sampling_grid[::sampling_step, ::sampling_step] = True sampling = [sampling_grid, sampling_grid]
Additionally, it is trained on LFPW trainset, HELEN trainset, IBUG and AFW datasets (3283 images in total), which are hosted in http://ibug.doc.ic.ac.uk/resources/facialpointannotations/.
 Returns
fitter (
LucasKanadeAAMFitter
) – A pretrainedLucasKanadeAAMFitter
based on aPatchAAM
that performs facial landmark localization returning 68 points (iBUG68).
menpofit.aps
¶
Active Pictorial Structures¶
APS is a model that utilises a Gaussian Markov Random Field (GMRF) for learning an appearance model with pairwise distributions based on a graph. It also has a parametric statitical shape model (either using PCA or GMRF), as well as a springlike deformation prior term. The optimisation is performed using a weighted GaussNewton algorithm with fixed Jacobian and Hessian.
GenerativeAPS¶

class
menpofit.aps.
GenerativeAPS
(images, group=None, appearance_graph=None, shape_graph=None, deformation_graph=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), patch_normalisation=<function no_op>, use_procrustes=True, precision_dtype=<class 'numpy.float32'>, max_shape_components=None, n_appearance_components=None, can_be_incremented=False, verbose=False, batch_size=None)[source]¶ Bases:
object
Class for training a multiscale Generative Active Pictorial Structures model. Please see the references for a basic list of relevant papers.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the AAM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.appearance_graph (list of graphs or a single graph or
None
, optional) – The graph to be used for the appearance menpo.model.GMRFModel training. It must be a menpo.shape.UndirectedGraph. IfNone
, then a menpo.model.PCAModel is used instead.shape_graph (list of graphs or a single graph or
None
, optional) – The graph to be used for the shape menpo.model.GMRFModel training. It must be a menpo.shape.UndirectedGraph. IfNone
, then the shape model is built using menpo.model.PCAModel.deformation_graph (list of graphs or a single graph or
None
, optional) – The graph to be used for the deformation menpo.model.GMRFModel training. It must be either a menpo.shape.DirectedGraph or a menpo.shape.Tree. IfNone
, then the minimum spanning tree of the data is computed.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the APS. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.
patch_normalisation (list of callable or a single callable, optional) – The normalisation function to be applied on the extracted patches. If list, then it must have length equal to the number of scales. If a single patch normalization callable, then this is the one applied to all scales.
use_procrustes (bool, optional) – If
True
, then Generalized Procrustes Alignment is applied before building the deformation model.precision_dtype (numpy.dtype, optional) – The data type of the appearance GMRF’s precision matrix. For example, it can be set to numpy.float32 for single precision or to numpy.float64 for double precision. Even though the precision matrix is stored as a scipy.sparse matrix, this parameter has a big impact on the amount of memory required by the model.
max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.n_appearance_components (list of int or int or
None
, optional) – The number of appearance components used for building the appearance menpo.shape.GMRFModel. If list, then it must have length equal to the number of scales. If a single int, then this is the one applied to all scales. IfNone
, the covariance matrix of each edge is inverted using np.linalg.inv. If int, it is inverted using truncated SVD using the specified number of components.can_be_incremented (bool, optional) – In case you intend to incrementally update the model in the future, then this flag must be set to
True
from the first place. Note that ifTrue
, the appearance and deformation menpo.shape.GMRFModel models will occupy double memory.verbose (bool, optional) – If
True
, then the progress of building the APS will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.
References
 1
E. Antonakos, J. AlabortiMedina, and S. Zafeiriou, “Active Pictorial Structures”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 18721882, June 2015.

increment
(images, group=None, batch_size=None, verbose=False)[source]¶ Method that incrementally updates the APS model with a new batch of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the APS. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.verbose (bool, optional) – If
True
, then the progress of building the APS will be printed.

instance
(shape_weights=None, scale_index=1, as_graph=False)[source]¶ Generates an instance of the shape model.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
as_graph (bool, optional) – If
True
, then the instance will be returned as a menpo.shape.PointTree or a menpo.shape.PointDirectedGraph, depending on the type of the deformation graph.

random_instance
(scale_index=1, as_graph=False)[source]¶ Generates a random instance of the APS.
 Parameters
scale_index (int, optional) – The scale to be used.
as_graph (bool, optional) – If
True
, then the instance will be returned as a menpo.shape.PointTree or a menpo.shape.PointDirectedGraph, depending on the type of the deformation graph.

view_appearance_graph_widget
(scale_index=1, figure_size=(7, 7))[source]¶ Visualize the appearance graph using an interactive widget.
 Parameters
scale_index (int, optional) – The scale to be used.
figure_size ((int, int), optional) – The size of the rendered figure.
 Raises
ValueError – Scale level {scale_index} uses a PCA appearance model, so there is no graph

view_deformation_graph_widget
(scale_index=1, figure_size=(7, 7))[source]¶ Visualize the deformation graph using an interactive widget.
 Parameters
scale_index (int, optional) – The scale to be used.
figure_size ((int, int), optional) – The size of the rendered figure.

view_deformation_model
(scale_index=1, n_std=2, render_colour_bar=False, colour_map='jet', image_view=True, figure_id=None, new_figure=False, render_graph_lines=True, graph_line_colour='b', graph_line_style='', graph_line_width=1.0, ellipse_line_colour='r', ellipse_line_style='', ellipse_line_width=1.0, render_markers=True, marker_style='o', marker_size=5, marker_face_colour='k', marker_edge_colour='k', marker_edge_width=1.0, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', crop_proportion=0.1, figure_size=(7, 7))[source]¶ Visualize the deformation model by plotting a Gaussian ellipsis per graph edge.
 Parameters
scale_index (int, optional) – The scale to be used.
n_std (float, optional) – This defines the size of the ellipses in terms of number of standard deviations.
render_colour_bar (bool, optional) – If
True
, then the ellipses will be coloured based on their normalized standard deviations and a colour bar will also appear on the side. IfFalse
, then all the ellipses will have the same colour.colour_map (str, optional) – A valid Matplotlib colour map. For more info, please refer to matplotlib.cm.
image_view (bool, optional) – If
True
the ellipses will be rendered in the image coordinates system.figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_graph_lines (bool, optional) – Defines whether to plot the graph’s edges.
graph_line_colour (See Below, optional) –
The colour of the lines of the graph’s edges. Example options:
{r, g, b, c, m, k, w} or (3, ) ndarray
graph_line_style (
{, , ., :}
, optional) – The style of the lines of the graph’s edges.graph_line_width (float, optional) – The width of the lines of the graph’s edges.
ellipse_line_colour (See Below, optional) –
The colour of the lines of the ellipses. Example options:
{r, g, b, c, m, k, w} or (3, ) ndarray
ellipse_line_style (
{, , ., :}
, optional) – The style of the lines of the ellipses.ellipse_line_width (float, optional) – The width of the lines of the ellipses.
render_markers (bool, optional) – If
True
, the centers of the ellipses will be rendered.marker_style (See Below, optional) –
The style of the centers of the ellipses. Example options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the centers of the ellipses in points.
marker_face_colour (See Below, optional) –
The face (filling) colour of the centers of the ellipses. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (See Below, optional) –
The edge colour of the centers of the ellipses. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The edge width of the centers of the ellipses.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See Below, optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (See Below, optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold,demibold, demi, bold, heavy, extra bold, black}
crop_proportion (float, optional) – The proportion to be left around the centers’ pointcloud.
figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.

view_shape_graph_widget
(scale_index=1, figure_size=(7, 7))[source]¶ Visualize the shape graph using an interactive widget.
 Parameters
scale_index (int, optional) – The scale to be used.
figure_size ((int, int), optional) – The size of the rendered figure.
 Raises
ValueError – Scale level {scale_index} uses a PCA shape model, so there is no graph

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the shape models of the APS object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
Fitters¶
GaussNewtonAPSFitter¶

class
menpofit.aps.
GaussNewtonAPSFitter
(aps, gn_algorithm_cls=<class 'menpofit.aps.algorithm.gn.Inverse'>, n_shape=None, weight=200.0, sampling=None)[source]¶ Bases:
APSFitter
A class for fitting an APS model with GaussNewton optimization.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step takes place at each scale and it is not considered as an iteration, thus it is not counted for the provided max_iters.
 Parameters
aps (
GenerativeAPS
or subclass) – The trained model.gn_algorithm_cls (class, optional) – The GaussNewton optimisation algorithm that will get applied. The possible algorithms are
Inverse
andForward
. Note that theForward
algorithm is too slow. It is not recommended to be used for fitting an APS and is only included for comparison purposes.n_shape (int or float or list of those or
None
, optional) – The number of shape components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_shape_components during training.weight (float or list of float, optional) – The weight between the appearance cost and the deformation cost. The provided value gets multiplied with the deformation cost. If float, then the provided value will be used for all scales. If list, then it should define a value per scale.
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Note that depending on the model and the size of the appearance precision matrix, the subsampling may be impossible to be applied due to insufficient memory. This is because the subsampling of the appearance precision matrix involves converting it to scipy.sparse.lil_matrix, subsampling it and reconvert it back to scipy.sparse.bsr_matrix, which is a memory intensive procedure.

fit_from_bb
(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial bounding box.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

fit_from_shape
(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

warped_images
(image, shapes)¶ Given an input test image and a list of shapes, it warps the image into the shapes. This is useful for generating the warped images of a fitting procedure stored within an
APSResult
. Parameters
image (menpo.image.Image or subclass) – The input image to be warped.
shapes (list of menpo.shape.PointCloud) – The list of shapes in which the image will be warped. The shapes are obtained during the iterations of a fitting procedure.
 Returns
warped_images (list of menpo.image.MaskedImage or ndarray) – The warped images.

property
aps
¶ The trained APS model.
 Type
GenerativeAPS
or subclass

property
holistic_features
¶ The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of closure

property
n_scales
¶ Returns the number of scales.
 Type
int

property
reference_shape
¶ The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.
 Type
menpo.shape.PointCloud

property
scales
¶ The scale value of each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of int or float
GaussNewton Optimisation Algorithms¶
Inverse¶

class
menpofit.aps.
Inverse
(aps_interface, eps=1e05)[source]¶ Bases:
GaussNewton
Inverse GaussNewton algorithm for APS.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False)[source]¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.
 Returns
fitting_result (
APSAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance GMRF model.
 Type
menpo.model.GMRFModel

property
deformation_model
¶ Returns the deformation GMRF model.
 Type
menpo.model.GMRFModel

property
template
¶ Returns the template (usually the mean appearance).
 Type
menpo.image.Image

Forward¶

class
menpofit.aps.
Forward
(aps_interface, eps=1e05)[source]¶ Bases:
GaussNewton
Forward GaussNewton algorithm for APS.
Note
The Forward optimization is too slow. It is not recommended to be used for fitting an APS and is only included for comparison purposes. Use Inverse instead.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False)[source]¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.
 Returns
fitting_result (
APSAlgorithmResult
) – The parametric iterative fitting result.

property
appearance_model
¶ Returns the appearance GMRF model.
 Type
menpo.model.GMRFModel

property
deformation_model
¶ Returns the deformation GMRF model.
 Type
menpo.model.GMRFModel

property
template
¶ Returns the template (usually the mean appearance).
 Type
menpo.image.Image

Fitting Result¶
APSResult¶

class
menpofit.aps.result.
APSResult
(results, scales, affine_transforms, scale_transforms, image=None, gt_shape=None)[source]¶ Bases:
MultiScaleParametricIterativeResult
Class for storing the multiscale iterative fitting result of an APS. It holds the shapes, shape parameters, appearance parameters and costs per iteration.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step is not counted in the number of iterations.
 Parameters
results (list of
APSAlgorithmResult
) – The list of optimization results per scale.scales (list or tuple) – The list of scale values per scale (low to high).
affine_transforms (list of menpo.transform.Affine) – The list of affine transforms per scale that transform the shapes into the original image space.
scale_transforms (list of menpo.shape.Scale) – The list of scaling transforms per scale.
image (menpo.image.Image or subclass or
None
, optional) – The image on which the fitting process was applied. Note that a copy of the image will be assigned as an attribute. IfNone
, then no image is assigned.gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated with the image. IfNone
, then no ground truth shape is assigned.

displacements
()¶ A list containing the displacement between the shape of each iteration and the shape of the previous one.
 Type
list of ndarray

displacements_stats
(stat_type='mean')¶ A list containing a statistical metric on the displacements between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type (
{'mean', 'median', 'min', 'max'}
, optional) – Specifies a statistic metric to be extracted from the displacements. Returns
displacements_stat (list of float) – The statistical metric on the points displacements for each iteration.
 Raises
ValueError – type must be ‘mean’, ‘median’, ‘min’ or ‘max’

errors
(compute_error=None)¶ Returns a list containing the error at each fitting iteration, if the ground truth shape exists.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
 Returns
errors (list of float) – The error at each iteration of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

final_error
(compute_error=None)¶ Returns the final error of the fitting process, if the ground truth shape exists. This is the error computed based on the final_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the fitted and ground truth shapes.
 Returns
final_error (float) – The final error at the end of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

initial_error
(compute_error=None)¶ Returns the initial error of the fitting process, if the ground truth shape and initial shape exist. This is the error computed based on the initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the initial and ground truth shapes.
 Returns
initial_error (float) – The initial error at the beginning of the fitting process.
 Raises
ValueError – Initial shape has not been set, so the initial error cannot be computed
ValueError – Ground truth shape has not been set, so the initial error cannot be computed

plot_costs
(figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)[source]¶ Plot of the cost function evolution at each fitting iteration.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
, optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
line_style (
{'', '', '.', ':'}
, optional) – The style of the lines.line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (marker, optional) –
The style of the markers. Example marker options
{'.', ',', 'o', 'v', '^', '<', '>', '+', 'x', 'D', 'd', 's', 'p', '*', 'h', 'H', '1', '2', '3', '4', '8'}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers.If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See below, optional) –
The font of the axes. Example options
{'serif', 'sansserif', 'cursive', 'fantasy', 'monospace'}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{'normal', 'italic', 'oblique'}
, optional) – The font style of the axes.axes_font_weight (See below, optional) –
The font weight of the axes. Example options
{'ultralight', 'light', 'normal', 'regular', 'book', 'medium', 'roman', 'semibold', 'demibold', 'demi', 'bold', 'heavy', 'extra bold', 'black'}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_displacements
(stat_type='mean', figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of a statistical metric of the displacement between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type ({
mean
,median
,min
,max
}, optional) – Specifies a statistic metric to be extracted from the displacements (see also displacements_stats() method).figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_errors
(compute_error=None, figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the error evolution at each fitting iteration.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

reconstructed_initial_error
(compute_error=None)¶ Returns the error of the reconstructed initial shape of the fitting process, if the ground truth shape exists. This is the error computed based on the reconstructed_initial_shapes[0].
 Parameters
compute_error (callable, optional) – Callable that computes the error between the reconstructed initial and ground truth shapes.
 Returns
reconstructed_initial_error (float) – The error that corresponds to the initial shape’s reconstruction.
 Raises
ValueError – Ground truth shape has not been set, so the reconstructed initial error cannot be computed

to_result
(pass_image=True, pass_initial_shape=True, pass_gt_shape=True)¶ Returns a
Result
instance of the object, i.e. a fitting result object that does not store the iterations. This can be useful for reducing the size of saved fitting results. Parameters
pass_image (bool, optional) – If
True
, then the image will get passed (if it exists).pass_initial_shape (bool, optional) – If
True
, then the initial shape will get passed (if it exists).pass_gt_shape (bool, optional) – If
True
, then the ground truth shape will get passed (if it exists).
 Returns
result (
Result
) – The final “lightweight” fitting result.

view
(figure_id=None, new_figure=False, render_image=True, render_final_shape=True, render_initial_shape=False, render_gt_shape=False, subplots_enabled=True, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, final_marker_face_colour='r', final_marker_edge_colour='k', final_line_colour='r', initial_marker_face_colour='b', initial_marker_edge_colour='k', initial_line_colour='b', gt_marker_face_colour='y', gt_marker_edge_colour='k', gt_line_colour='y', render_lines=True, line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the fitting result. The method renders the final fitted shape and optionally the initial shape, ground truth shape and the image, id they were provided.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_image (bool, optional) – If
True
and the image exists, then it gets rendered.render_final_shape (bool, optional) – If
True
, then the final fitting shape gets rendered.render_initial_shape (bool, optional) – If
True
and the initial fitting shape exists, then it gets rendered.render_gt_shape (bool, optional) – If
True
and the ground truth shape exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (See Below, optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.final_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_line_colour (See Below, optional) –
The line colour of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_line_colour (See Below, optional) –
The line colour of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_line_colour (See Below, optional) –
The line colour of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.line_style (str or list of str, optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{'', '', '.', ':'}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.marker_style (str or list of str, optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (
{center, right, left}
, optional) – The horizontal alignment of the numbers’ texts.numbers_vertical_align (
{center, top, bottom, baseline}
, optional) – The vertical alignment of the numbers’ texts.numbers_font_name (See Below, optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (See Below, optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (
{normal, italic, oblique}
, optional) – The font style of the legend.legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (See Below, optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See Below, optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (See Below, optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_iterations
(figure_id=None, new_figure=False, iters=None, render_image=True, subplots_enabled=False, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, render_lines=True, line_style='', line_width=2, line_colour=None, render_markers=True, marker_edge_colour=None, marker_face_colour=None, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the iterations of the fitting process.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.iters (int or list of int or
None
, optional) –The iterations to be visualized. If
None
, then all the iterations are rendered.No.
Visualised shape
Description
0
self.initial_shape
Initial shape
1
self.reconstructed_initial_shape
Reconstructed initial
2
self.shapes[2]
Iteration 1
i
self.shapes[i]
Iteration i1
n_iters+1
self.final_shape
Final shape
render_image (bool, optional) – If
True
and the image exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (str (See Below), optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.line_style (str or list of str (See below), optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options:
{, , ., :}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
line_colour (colour or list of colour (See Below), optional) –
The colour of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.marker_style (str or `list of str (See below), optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
marker_edge_colour (colour or list of colour (See Below), optional) –
The edge colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_face_colour (colour or list of colour (See Below), optional) –
The face (filling) colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (str (See below), optional) –
The horizontal alignment of the numbers’ texts. Example options
{center, right, left}
numbers_vertical_align (str (See below), optional) –
The vertical alignment of the numbers’ texts. Example options
{center, top, bottom, baseline}
numbers_font_name (str (See below), optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (str (See below), optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (str (See below), optional) –
The font style of the legend. Example options
{normal, italic, oblique}
legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (str (See below), optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_widget
(figure_size=(7, 7))¶ Visualizes the result object using an interactive widget.
 Parameters
figure_size ((int, int), optional) – The initial size of the rendered figure.

property
appearance_costs
¶ Returns a list with the appearance cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
costs
¶ Returns a list with the cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
deformation_costs
¶ Returns a list with the deformation cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
final_shape
¶ Returns the final shape of the fitting process.
 Type
menpo.shape.PointCloud

property
gt_shape
¶ Returns the ground truth shape associated with the image. In case there is not an attached ground truth shape, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
image
¶ Returns the image that the fitting was applied on, if it was provided. Otherwise, it returns
None
. Type
menpo.shape.Image or subclass or
None

property
initial_shape
¶ Returns the initial shape that was provided to the fitting method to initialise the fitting process. In case the initial shape does not exist, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
is_iterative
¶ Flag whether the object is an iterative fitting result.
 Type
bool

property
n_iters
¶ Returns the total number of iterations of the fitting process.
 Type
int

property
n_iters_per_scale
¶ Returns the number of iterations per scale of the fitting process.
 Type
list of int

property
n_scales
¶ Returns the number of scales used during the fitting process.
 Type
int

property
reconstructed_initial_shapes
¶ Returns the result of the reconstruction step that takes place at each scale before applying the iterative optimisation.
 Type
list of menpo.shape.PointCloud

property
shape_parameters
¶ Returns the list of shape parameters obtained at each iteration of the fitting process. The list includes the parameters of the initial_shape (if it exists) and final_shape.
 Type
list of
(n_params,)
ndarray

property
shapes
¶ Returns the list of shapes obtained at each iteration of the fitting process. The list includes the initial_shape (if it exists) and final_shape.
 Type
list of menpo.shape.PointCloud
APSAlgorithmResult¶

class
menpofit.aps.result.
APSAlgorithmResult
(shapes, shape_parameters, initial_shape=None, image=None, gt_shape=None, appearance_costs=None, deformation_costs=None, costs=None)[source]¶ Bases:
ParametricIterativeResult
Class for storing the iterative result of an APS optimisation algorithm.
Note
When using a method with a parametric shape model, the first step is to reconstruct the initial shape using the shape model. The generated reconstructed shape is then used as initialisation for the iterative optimisation. This step is not counted in the number of iterations.
 Parameters
shapes (list of menpo.shape.PointCloud) – The list of shapes per iteration. The first and last members correspond to the initial and final shapes, respectively.
shape_parameters (list of
(n_shape_parameters,)
ndarray) – The list of shape parameters per iteration. The first and last members correspond to the initial and final shapes, respectively.initial_shape (menpo.shape.PointCloud or
None
, optional) – The initial shape from which the fitting process started. IfNone
, then no initial shape is assigned.image (menpo.image.Image or subclass or
None
, optional) – The image on which the fitting process was applied. Note that a copy of the image will be assigned as an attribute. IfNone
, then no image is assigned.gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape associated with the image. IfNone
, then no ground truth shape is assigned.appearance_costs (list of float or
None
, optional) – The list of the appearance cost per iteration. IfNone
, then it is assumed that the cost function cannot be computed for the specific algorithm.deformation_costs (list of float or
None
, optional) – The list of the deformation cost per iteration. IfNone
, then it is assumed that the cost function cannot be computed for the specific algorithm.costs (list of float or
None
, optional) – The list of the total cost per iteration. IfNone
, then it is assumed that the cost function cannot be computed for the specific algorithm.

displacements
()¶ A list containing the displacement between the shape of each iteration and the shape of the previous one.
 Type
list of ndarray

displacements_stats
(stat_type='mean')¶ A list containing a statistical metric on the displacements between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type (
{'mean', 'median', 'min', 'max'}
, optional) – Specifies a statistic metric to be extracted from the displacements. Returns
displacements_stat (list of float) – The statistical metric on the points displacements for each iteration.
 Raises
ValueError – type must be ‘mean’, ‘median’, ‘min’ or ‘max’

errors
(compute_error=None)¶ Returns a list containing the error at each fitting iteration, if the ground truth shape exists.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
 Returns
errors (list of float) – The error at each iteration of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

final_error
(compute_error=None)¶ Returns the final error of the fitting process, if the ground truth shape exists. This is the error computed based on the final_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the fitted and ground truth shapes.
 Returns
final_error (float) – The final error at the end of the fitting process.
 Raises
ValueError – Ground truth shape has not been set, so the final error cannot be computed

initial_error
(compute_error=None)¶ Returns the initial error of the fitting process, if the ground truth shape and initial shape exist. This is the error computed based on the initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the initial and ground truth shapes.
 Returns
initial_error (float) – The initial error at the beginning of the fitting process.
 Raises
ValueError – Initial shape has not been set, so the initial error cannot be computed
ValueError – Ground truth shape has not been set, so the initial error cannot be computed

plot_costs
(figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the cost function evolution at each fitting iteration.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
, optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
line_style (
{'', '', '.', ':'}
, optional) – The style of the lines.line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (marker, optional) –
The style of the markers. Example marker options
{'.', ',', 'o', 'v', '^', '<', '>', '+', 'x', 'D', 'd', 's', 'p', '*', 'h', 'H', '1', '2', '3', '4', '8'}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers.If
None
, the colour is sampled from the jet colormap. Example colour options are{'r', 'g', 'b', 'c', 'm', 'k', 'w'} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See below, optional) –
The font of the axes. Example options
{'serif', 'sansserif', 'cursive', 'fantasy', 'monospace'}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{'normal', 'italic', 'oblique'}
, optional) – The font style of the axes.axes_font_weight (See below, optional) –
The font weight of the axes. Example options
{'ultralight', 'light', 'normal', 'regular', 'book', 'medium', 'roman', 'semibold', 'demibold', 'demi', 'bold', 'heavy', 'extra bold', 'black'}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_displacements
(stat_type='mean', figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of a statistical metric of the displacement between the shape of each iteration and the shape of the previous one.
 Parameters
stat_type ({
mean
,median
,min
,max
}, optional) – Specifies a statistic metric to be extracted from the displacements (see also displacements_stats() method).figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

plot_errors
(compute_error=None, figure_id=None, new_figure=False, render_lines=True, line_colour='b', line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_face_colour='b', marker_edge_colour='k', marker_edge_width=1.0, render_axes=True, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=0.0, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(10, 6), render_grid=True, grid_line_style='', grid_line_width=0.5)¶ Plot of the error evolution at each fitting iteration.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the shape at each iteration and the ground truth shape.
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_lines (bool, optional) – If
True
, the line will be rendered.line_colour (colour or
None
(See below), optional) –The colour of the line. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
line_style (str (See below), optional) –
The style of the lines. Example options:
{, , ., :}
line_width (float, optional) – The width of the lines.
render_markers (bool, optional) – If
True
, the markers will be rendered.marker_style (str (See below), optional) –
The style of the markers. Example marker options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int, optional) – The size of the markers in points.
marker_face_colour (colour or
None
, optional) –The face (filling) colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_colour (colour or
None
, optional) –The edge colour of the markers. If
None
, the colour is sampled from the jet colormap. Example colour options are{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float, optional) – The width of the markers’ edge.
render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (str (See below), optional) –
The font style of the axes. Example options
{normal, italic, oblique}
axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the graph as a percentage of the curves’ width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits (float or (float, float) or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the graph as a percentage of the curves’ height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) or
None
, optional) – The size of the figure in inches.render_grid (bool, optional) – If
True
, the grid will be rendered.grid_line_style (
{'', '', '.', ':'}
, optional) – The style of the grid lines.grid_line_width (float, optional) – The width of the grid lines.
 Returns
renderer (menpo.visualize.GraphPlotter) – The renderer object.

reconstructed_initial_error
(compute_error=None)¶ Returns the error of the reconstructed initial shape of the fitting process, if the ground truth shape exists. This is the error computed based on the reconstructed_initial_shape.
 Parameters
compute_error (callable, optional) – Callable that computes the error between the reconstructed initial and ground truth shapes.
 Returns
reconstructed_initial_error (float) – The error that corresponds to the initial shape’s reconstruction.
 Raises
ValueError – Ground truth shape has not been set, so the reconstructed initial error cannot be computed

to_result
(pass_image=True, pass_initial_shape=True, pass_gt_shape=True)¶ Returns a
Result
instance of the object, i.e. a fitting result object that does not store the iterations. This can be useful for reducing the size of saved fitting results. Parameters
pass_image (bool, optional) – If
True
, then the image will get passed (if it exists).pass_initial_shape (bool, optional) – If
True
, then the initial shape will get passed (if it exists).pass_gt_shape (bool, optional) – If
True
, then the ground truth shape will get passed (if it exists).
 Returns
result (
Result
) – The final “lightweight” fitting result.

view
(figure_id=None, new_figure=False, render_image=True, render_final_shape=True, render_initial_shape=False, render_gt_shape=False, subplots_enabled=True, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, final_marker_face_colour='r', final_marker_edge_colour='k', final_line_colour='r', initial_marker_face_colour='b', initial_marker_edge_colour='k', initial_line_colour='b', gt_marker_face_colour='y', gt_marker_edge_colour='k', gt_line_colour='y', render_lines=True, line_style='', line_width=2, render_markers=True, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the fitting result. The method renders the final fitted shape and optionally the initial shape, ground truth shape and the image, id they were provided.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.render_image (bool, optional) – If
True
and the image exists, then it gets rendered.render_final_shape (bool, optional) – If
True
, then the final fitting shape gets rendered.render_initial_shape (bool, optional) – If
True
and the initial fitting shape exists, then it gets rendered.render_gt_shape (bool, optional) – If
True
and the ground truth shape exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (See Below, optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.final_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
final_line_colour (See Below, optional) –
The line colour of the final fitting shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
initial_line_colour (See Below, optional) –
The line colour of the initial shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_face_colour (See Below, optional) –
The face (filling) colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_marker_edge_colour (See Below, optional) –
The edge colour of the markers of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
gt_line_colour (See Below, optional) –
The line colour of the ground truth shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.line_style (str or list of str, optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{'', '', '.', ':'}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.marker_style (str or list of str, optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order. Example options:
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per shape in (final, initial, groundtruth) order.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (
{center, right, left}
, optional) – The horizontal alignment of the numbers’ texts.numbers_vertical_align (
{center, top, bottom, baseline}
, optional) – The vertical alignment of the numbers’ texts.numbers_font_name (See Below, optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (See Below, optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (
{normal, italic, oblique}
, optional) – The font style of the legend.legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (See Below, optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (See Below, optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (See Below, optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_iterations
(figure_id=None, new_figure=False, iters=None, render_image=True, subplots_enabled=False, channels=None, interpolation='bilinear', cmap_name=None, alpha=1.0, masked=True, render_lines=True, line_style='', line_width=2, line_colour=None, render_markers=True, marker_edge_colour=None, marker_face_colour=None, marker_style='o', marker_size=4, marker_edge_width=1.0, render_numbering=False, numbers_horizontal_align='center', numbers_vertical_align='bottom', numbers_font_name='sansserif', numbers_font_size=10, numbers_font_style='normal', numbers_font_weight='normal', numbers_font_colour='k', render_legend=True, legend_title='', legend_font_name='sansserif', legend_font_style='normal', legend_font_size=10, legend_font_weight='normal', legend_marker_scale=None, legend_location=2, legend_bbox_to_anchor=(1.05, 1.0), legend_border_axes_pad=None, legend_n_columns=1, legend_horizontal_spacing=None, legend_vertical_spacing=None, legend_border=True, legend_border_padding=None, legend_shadow=False, legend_rounded_corners=False, render_axes=False, axes_font_name='sansserif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', axes_x_limits=None, axes_y_limits=None, axes_x_ticks=None, axes_y_ticks=None, figure_size=(7, 7))¶ Visualize the iterations of the fitting process.
 Parameters
figure_id (object, optional) – The id of the figure to be used.
new_figure (bool, optional) – If
True
, a new figure is created.iters (int or list of int or
None
, optional) –The iterations to be visualized. If
None
, then all the iterations are rendered.No.
Visualised shape
Description
0
self.initial_shape
Initial shape
1
self.reconstructed_initial_shape
Reconstructed initial
2
self.shapes[2]
Iteration 1
i
self.shapes[i]
Iteration i1
n_iters+1
self.final_shape
Final shape
render_image (bool, optional) – If
True
and the image exists, then it gets rendered.subplots_enabled (bool, optional) – If
True
, then the requested final, initial and ground truth shapes get rendered on separate subplots.channels (int or list of int or
all
orNone
) – If int or list of int, the specified channel(s) will be rendered. Ifall
, all the channels will be rendered in subplots. IfNone
and the image is RGB, it will be rendered in RGB mode. IfNone
and the image is not RGB, it is equivalent toall
.interpolation (str (See Below), optional) –
The interpolation used to render the image. For example, if
bilinear
, the image will be smooth and ifnearest
, the image will be pixelated. Example options{none, nearest, bilinear, bicubic, spline16, spline36, hanning, hamming, hermite, kaiser, quadric, catrom, gaussian, bessel, mitchell, sinc, lanczos}
cmap_name (str, optional,) – If
None
, single channel and three channel images default to greyscale and rgb colormaps respectively.alpha (float, optional) – The alpha blending value, between 0 (transparent) and 1 (opaque).
masked (bool, optional) – If
True
, then the image is rendered as masked.render_lines (bool or list of bool, optional) – If
True
, the lines will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.line_style (str or list of str (See below), optional) –
The style of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options:
{, , ., :}
line_width (float or list of float, optional) – The width of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
line_colour (colour or list of colour (See Below), optional) –
The colour of the lines. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_markers (bool or list of bool, optional) – If
True
, the markers will be rendered. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.marker_style (str or `list of str (See below), optional) –
The style of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
marker_size (int or list of int, optional) – The size of the markers in points. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
marker_edge_colour (colour or list of colour (See Below), optional) –
The edge colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_face_colour (colour or list of colour (See Below), optional) –
The face (filling) colour of the markers. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
marker_edge_width (float or list of float, optional) – The width of the markers’ edge. You can either provide a single value that will be used for all shapes or a list with a different value per iteration shape.
render_numbering (bool, optional) – If
True
, the landmarks will be numbered.numbers_horizontal_align (str (See below), optional) –
The horizontal alignment of the numbers’ texts. Example options
{center, right, left}
numbers_vertical_align (str (See below), optional) –
The vertical alignment of the numbers’ texts. Example options
{center, top, bottom, baseline}
numbers_font_name (str (See below), optional) –
The font of the numbers. Example options
{serif, sansserif, cursive, fantasy, monospace}
numbers_font_size (int, optional) – The font size of the numbers.
numbers_font_style (
{normal, italic, oblique}
, optional) – The font style of the numbers.numbers_font_weight (str (See below), optional) –
The font weight of the numbers. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
numbers_font_colour (See Below, optional) –
The font colour of the numbers. Example options
{r, g, b, c, m, k, w} or (3, ) ndarray
render_legend (bool, optional) – If
True
, the legend will be rendered.legend_title (str, optional) – The title of the legend.
legend_font_name (See below, optional) –
The font of the legend. Example options
{serif, sansserif, cursive, fantasy, monospace}
legend_font_style (str (See below), optional) –
The font style of the legend. Example options
{normal, italic, oblique}
legend_font_size (int, optional) – The font size of the legend.
legend_font_weight (str (See below), optional) –
The font weight of the legend. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
legend_marker_scale (float, optional) – The relative size of the legend markers with respect to the original
legend_location (int, optional) –
The location of the legend. The predefined values are:
’best’
0
’upper right’
1
’upper left’
2
’lower left’
3
’lower right’
4
’right’
5
’center left’
6
’center right’
7
’lower center’
8
’upper center’
9
’center’
10
legend_bbox_to_anchor ((float, float) tuple, optional) – The bbox that the legend will be anchored.
legend_border_axes_pad (float, optional) – The pad between the axes and legend border.
legend_n_columns (int, optional) – The number of the legend’s columns.
legend_horizontal_spacing (float, optional) – The spacing between the columns.
legend_vertical_spacing (float, optional) – The vertical space between the legend entries.
legend_border (bool, optional) – If
True
, a frame will be drawn around the legend.legend_border_padding (float, optional) – The fractional whitespace inside the legend border.
legend_shadow (bool, optional) – If
True
, a shadow will be drawn behind legend.legend_rounded_corners (bool, optional) – If
True
, the frame’s corners will be rounded (fancybox).render_axes (bool, optional) – If
True
, the axes will be rendered.axes_font_name (str (See below), optional) –
The font of the axes. Example options
{serif, sansserif, cursive, fantasy, monospace}
axes_font_size (int, optional) – The font size of the axes.
axes_font_style (
{normal, italic, oblique}
, optional) – The font style of the axes.axes_font_weight (str (See below), optional) –
The font weight of the axes. Example options
{ultralight, light, normal, regular, book, medium, roman, semibold, demibold, demi, bold, heavy, extra bold, black}
axes_x_limits (float or (float, float) or
None
, optional) – The limits of the x axis. If float, then it sets padding on the right and left of the Image as a percentage of the Image’s width. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_y_limits ((float, float) tuple or
None
, optional) – The limits of the y axis. If float, then it sets padding on the top and bottom of the Image as a percentage of the Image’s height. If tuple or list, then it defines the axis limits. IfNone
, then the limits are set automatically.axes_x_ticks (list or tuple or
None
, optional) – The ticks of the x axis.axes_y_ticks (list or tuple or
None
, optional) – The ticks of the y axis.figure_size ((float, float) tuple or
None
optional) – The size of the figure in inches.
 Returns
renderer (class) – The renderer object.

view_widget
(figure_size=(7, 7))¶ Visualizes the result object using an interactive widget.
 Parameters
figure_size ((int, int), optional) – The initial size of the rendered figure.

property
appearance_costs
¶ Returns a list with the appearance cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
costs
¶ Returns a list with the cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
deformation_costs
¶ Returns a list with the deformation cost per iteration. It returns
None
if the costs are not computed. Type
list of float or
None

property
final_shape
¶ Returns the final shape of the fitting process.
 Type
menpo.shape.PointCloud

property
gt_shape
¶ Returns the ground truth shape associated with the image. In case there is not an attached ground truth shape, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
image
¶ Returns the image that the fitting was applied on, if it was provided. Otherwise, it returns
None
. Type
menpo.shape.Image or subclass or
None

property
initial_shape
¶ Returns the initial shape that was provided to the fitting method to initialise the fitting process. In case the initial shape does not exist, then
None
is returned. Type
menpo.shape.PointCloud or
None

property
is_iterative
¶ Flag whether the object is an iterative fitting result.
 Type
bool

property
n_iters
¶ Returns the total number of iterations of the fitting process.
 Type
int

property
reconstructed_initial_shape
¶ Returns the initial shape’s reconstruction with the shape model that was used to initialise the iterative optimisation process.
 Type
menpo.shape.PointCloud

property
shape_parameters
¶ Returns the list of shape parameters obtained at each iteration of the fitting process. The list includes the parameters of the reconstructed_initial_shape and final_shape.
 Type
list of
(n_params,)
ndarray

property
shapes
¶ Returns the list of shapes obtained at each iteration of the fitting process. The list includes the initial_shape (if it exists), reconstructed_initial_shape and final_shape.
 Type
list of menpo.shape.PointCloud
menpofit.atm
¶
Active Template Model¶
ATM is a generative model that performs deformable alignment between a template image and a test image with respect to a statistical parametric shape model. MenpoFit has several ATMs which differ in the manner that they compute the warp (thus represent the appearance features).
ATM¶

class
menpofit.atm.base.
ATM
(template, shapes, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), transform=<class 'menpofit.transform.piecewiseaffine.DifferentiablePiecewiseAffine'>, shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
object
Class for training a multiscale holistic Active Template Model.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the ATM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
transform (subclass of
DL
andDX
, optional) – A differential warp transform object, e.g.DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.
References
 1
S. Baker, and I. Matthews. “LucasKanade 20 years on: A unifying framework”, International Journal of Computer Vision, 56(3): 221255, 2004.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface.
 Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(template, shapes, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)[source]¶ Method to increment the trained ATM with a new set of training shapes and a new template.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, scale_index=1)[source]¶ Generates a novel ATM instance given a set of shape weights. If no weights are provided, the mean ATM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

random_instance
(scale_index=1)[source]¶ Generates a random instance of the ATM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

view_atm_widget
(n_shape_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the ATM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the shape models of the ATM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
MaskedATM¶

class
menpofit.atm.
MaskedATM
(template, shapes, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
ATM
Class for training a multiscale patchbased Masked Active Template Model. The appearance of this model is formulated by simply masking an image with a patchbased mask.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the ATM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int), optional) – The size of the patches of the mask that is used to sample the appearance vectors.
max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

build_fitter_interfaces
(sampling)¶ Method that builds the correct LucasKanade fitting interface.
 Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(template, shapes, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained ATM with a new set of training shapes and a new template.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, scale_index=1)¶ Generates a novel ATM instance given a set of shape weights. If no weights are provided, the mean ATM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the ATM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

view_atm_widget
(n_shape_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the ATM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the ATM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
LinearATM¶

class
menpofit.atm.
LinearATM
(template, shapes, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), transform=<class 'menpofit.transform.thinsplatesplines.DifferentiableThinPlateSplines'>, max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
ATM
Class for training a multiscale Linear Active Template Model.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the ATM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
transform (subclass of
DL
andDX
, optional) – A differential warp transform object, e.g.DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface.
 Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(template, shapes, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained ATM with a new set of training shapes and a new template.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, scale_index=1)¶ Generates a novel ATM instance given a set of shape weights. If no weights are provided, the mean ATM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the ATM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

view_atm_widget
(n_shape_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the ATM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the ATM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
LinearMaskedATM¶

class
menpofit.atm.
LinearMaskedATM
(template, shapes, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
ATM
Class for training a multiscale Linear Masked Active Template Model.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the ATM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches of the mask that is used to extract the appearance vectors. If a list is provided, then it defines a patch shape per scale.
max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface.
 Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(template, shapes, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained ATM with a new set of training shapes and a new template.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, scale_index=1)¶ Generates a novel ATM instance given a set of shape weights. If no weights are provided, the mean ATM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the ATM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

view_atm_widget
(n_shape_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the ATM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the ATM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
PatchATM¶

class
menpofit.atm.
PatchATM
(template, shapes, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), patch_normalisation=<function no_op>, max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
ATM
Class for training a multiscale PatchBased Active Template Model.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the ATM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.
max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

build_fitter_interfaces
(sampling)[source]¶ Method that builds the correct LucasKanade fitting interface.
 Parameters
sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied. Returns
fitter_interfaces (list) – The list of LucasKanade interface per scale.

increment
(template, shapes, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)¶ Method to increment the trained ATM with a new set of training shapes and a new template.
 Parameters
template (menpo.image.Image) – The template image.
shapes (list of menpo.shape.PointCloud) – The list of training shapes.
group (str or
None
, optional) – The landmark group of the template that will be used to train the ATM. IfNone
and the template only has a single landmark group, then that is the one that will be used.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the ATM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.

instance
(shape_weights=None, scale_index=1)¶ Generates a novel ATM instance given a set of shape weights. If no weights are provided, the mean ATM instance is returned.
 Parameters
shape_weights (
(n_weights,)
ndarray or list orNone
, optional) – The weights of the shape model that will be used to create a novel shape instance. IfNone
, the weights are assumed to be zero, thus the mean shape is used.scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

random_instance
(scale_index=1)¶ Generates a random instance of the ATM.
 Parameters
scale_index (int, optional) – The scale to be used.
 Returns
image (menpo.image.Image) – The ATM instance.

view_atm_widget
(n_shape_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))[source]¶ Visualizes the ATM using an interactive widget.
 Parameters
n_shape_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

view_shape_models_widget
(n_parameters=5, parameters_bounds=(3.0, 3.0), mode='multiple', figure_size=(7, 7))¶ Visualizes the shape models of the ATM object using an interactive widget.
 Parameters
n_parameters (int or list of int or
None
, optional) – The number of shape principal components to be used for the parameters sliders. If int, then the number of sliders per scale is the minimum between n_parameters and the number of active components per scale. If list of int, then a number of sliders is defined per scale. IfNone
, all the active components per scale will have a slider.parameters_bounds (
(float, float)
, optional) – The minimum and maximum bounds, in std units, for the sliders.mode ({
single
,multiple
}, optional) – If'single'
, only a single slider is constructed along with a drop down menu. If'multiple'
, a slider is constructed for each parameter.figure_size ((int, int), optional) – The size of the rendered figure.

property
n_scales
¶ Returns the number of scales.
 Type
int
Fitter¶
LucasKanadeATMFitter¶

class
menpofit.atm.
LucasKanadeATMFitter
(atm, lk_algorithm_cls=<class 'menpofit.atm.algorithm.InverseCompositional'>, n_shape=None, sampling=None)[source]¶ Bases:
MultiScaleParametricFitter
Class for defining an ATM fitter using the LucasKanade optimization.
 Parameters
atm (
ATM
or subclass) – The trained ATM model.lk_algorithm_cls (class, optional) –
The LukasKanade optimisation algorithm that will get applied. The possible algorithms are:
Class
Warp Direction
Warp Update
Forward
Compositional
Inverse
n_shape (int or float or list of those or
None
, optional) – The number of shape components that will be used. If int, then it defines the exact number of active components. If float, then it defines the percentage of variance to keep. If int or float, then the provided value will be applied for all scales. If list, then it defines a value per scale. IfNone
, then all the available components will be used. Note that this simply sets the active components without trimming the unused ones. Also, the available components may have already been trimmed to max_shape_components during training.sampling (list of int or ndarray or
None
) – It defines a sampling mask per scale. If int, then it defines the subsampling step of the sampling mask. If ndarray, then it explicitly defines the sampling mask. IfNone
, then no subsampling is applied.

fit_from_bb
(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial bounding box.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

fit_from_shape
(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multiscale fitter to an image given an initial shape.
 Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
 Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multiscale fitting result containing the result of the fitting procedure.

warped_images
(image, shapes)[source]¶ Given an input test image and a list of shapes, it warps the image into the shapes. This is useful for generating the warped images of a fitting procedure stored within a
MultiScaleParametricIterativeResult
. Parameters
image (menpo.image.Image or subclass) – The input image to be warped.
shapes (list of menpo.shape.PointCloud) – The list of shapes in which the image will be warped. The shapes are obtained during the iterations of a fitting procedure.
 Returns
warped_images (list of menpo.image.MaskedImage or ndarray) – The warped images.

property
holistic_features
¶ The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of closure

property
n_scales
¶ Returns the number of scales.
 Type
int

property
reference_shape
¶ The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.
 Type
menpo.shape.PointCloud

property
scales
¶ The scale value of each scale in ascending order, i.e. from lowest to highest scale.
 Type
list of int or float
LucasKanade Optimisation Algorithms¶
ForwardCompositional¶

class
menpofit.atm.
ForwardCompositional
(atm_interface, eps=1e05)[source]¶ Bases:
Compositional
Forward Compositional (FC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
ParametricIterativeResult
) – The parametric iterative fitting result.

property
template
¶ Returns the template of the ATM.
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

InverseCompositional¶

class
menpofit.atm.
InverseCompositional
(atm_interface, eps=1e05)[source]¶ Bases:
Compositional
Inverse Compositional (IC) GaussNewton algorithm.

run
(image, initial_shape, gt_shape=None, max_iters=20, return_costs=False, map_inference=False)¶ Execute the optimization algorithm.
 Parameters
image (menpo.image.Image) – The input test image.
initial_shape (menpo.shape.PointCloud) – The initial shape from which the optimization will start.
gt_shape (menpo.shape.PointCloud or
None
, optional) – The ground truth shape of the image. It is only needed in order to get passed in the optimization result object, which has the ability to compute the fitting error.max_iters (int, optional) – The maximum number of iterations. Note that the algorithm may converge, and thus stop, earlier.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.map_inference (bool, optional) – If
True
, then the solution will be given after performing MAP inference.
 Returns
fitting_result (
ParametricIterativeResult
) – The parametric iterative fitting result.

property
template
¶ Returns the template of the ATM.
 Type
menpo.image.Image or subclass

property
transform
¶ Returns the model driven differential transform object of the AAM, e.g.
DifferentiablePiecewiseAffine
orDifferentiableThinPlateSplines
.

menpofit.clm
¶
Constrained Local Model¶
Deformable model that consists of a generative parametric shape model and discriminatively trained experts per part.
CLM¶

class
menpofit.clm.
CLM
(images, group=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1), patch_shape=(17, 17), patch_normalisation=<function no_op>, context_shape=(34, 34), cosine_mask=True, sample_offsets=None, shape_model_cls=<class 'menpofit.modelinstance.OrthoPDM'>, expert_ensemble_cls=<class 'menpofit.clm.expert.ensemble.CorrelationFilterExpertEnsemble'>, max_shape_components=None, verbose=False, batch_size=None)[source]¶ Bases:
object
Class for training a multiscale holistic Constrained Local Model. Please see the references for a basic list of relevant papers.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the CLM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
reference_shape (menpo.shape.PointCloud or
None
, optional) – The reference shape that will be used for building the CLM. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. IfNone
, then the mean shape will be used.diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.
patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.
patch_normalisation (callable, optional) – The normalisation function to be applied on the extracted patches.
context_shape ((int, int) or list of (int, int), optional) – The context shape for the convolution. If a list is provided, then it defines a context shape per scale.
cosine_mask (bool, optional) – If
True
, then a cosine mask (Hanning function) will be applied on the extracted patches.sample_offsets (
(n_offsets, n_dims)
ndarray orNone
, optional) – The offsets to sample from within a patch. So(0, 0)
is the centre of the patch (no offset) and(1, 0)
would be sampling the patch from 1 pixel up the first axis away from the centre. IfNone
, then no offsets are applied.shape_model_cls (subclass of
PDM
, optional) – The class to be used for building the shape model. The most common choice isOrthoPDM
.expert_ensemble_cls (subclass of
ExpertEnsemble
, optional) – The class to be used for training the ensemble of experts. The most common choice isCorrelationFilterExpertEnsemble
.max_shape_components (int, float, list of those or
None
, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. IfNone
, then all the components are kept. Note that the unused components will be permanently trimmed.verbose (bool, optional) – If
True
, then the progress of building the CLM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the training is performed directly on the all the images.
References
 1
D. Cristinacce, and T. F. Cootes. “Feature Detection and Tracking with Constrained Local Models”, British Machine Vision Conference (BMVC), 2006.
 2
J.M. Saragih, S. Lucey, and J. F. Cohn. “Deformable model fitting by regularized landmark meanshift”, International Journal of Computer Vision (IJCV), 91(2): 200215, 2011.
 3
T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. “Active Shape Models  their training and application”, Computer Vision and Image Understanding (CVIU), 61(1): 3859, 1995.

increment
(images, group=None, shape_forgetting_factor=1.0, verbose=False, batch_size=None)[source]¶ Method to increment the trained CLM with a new set of training images.
 Parameters
images (list of menpo.image.Image) – The list of training images.
group (str or
None
, optional) – The landmark group that will be used to train the CLM. IfNone
and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.shape_forgetting_factor (
[0.0, 1.0]
float, optional) – Forgetting factor that weights the relative contribution of new samples vs old samples for the shape model. If1.0
, all samples are weighted equally and, hence, the result is the exact same as performing batch PCA on the concatenated list of old and new simples. If<1.0
, more emphasis is put on the new samples.verbose (bool, optional) – If
True
, then the progress of building the CLM will be printed.batch_size (int or
None
, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. IfNone
, then the traini