A.5 Modification of the average information matrix
To yield a search direction which is likely to improve the likelihood, or, equivalently,
decrease , the Hessian matrix or its approximation in a Newton type
optimisation strategy must be positive definite. While the AI matrix is a matrix
of sums of squares and crossproducts and thus virtually guaranteed to be
positive definite, it can have a relatively large condition number or minimum
eigenvalues close to zero. This can yield step sizes, calculated as the product of
the inverse of the AI matrix and the vector of first derivatives, which are
too large. Consequently, severe step size modifications may be required to
achieve an improvement . This may, at best, require several additional
likelihood evaluations or cause the algorithm to fail. Modification of the AI
matrix, to ensure that it is ‘safely’ positive definite and that its condition
number is not excessive, may improve performance of the AI algorithm in this
Several strategies are available. None has been found to be ‘best’.
- Schnabel and Estrow  described a modified Cholesky decomposition
of the Hessian matrix. This has been implemented using algorithm 695 of
the TOMS library (www.netlib.org/toms. This is the ) , but using
a factor of (where denotes machine precision) to determine the
critical size of pivots, which is intermediate to the original value of
and the value of suggested by Schnabel and Estrow .
- A partial Cholesky decomposition has been suggested by Forsgren
et al. . This has been implemented using a factor of .
- Modification strategies utilising the Cholesky decomposition have been
devised for scenarios where direct calculation of the eigenvalues is
impractical. For our applications,however, computational costs of an
eigenvalue decomposition of the AI matrix are negligible compared to
those of a likelihood evaluation. This allows a modification where we
know the minimum eigenvalue of the resulting matrix. Nocedahl and
Wright [37, Chapter 6] described two variations, which have been
- Set all eigenvalues less than a value of to , and construct the
modified AI matrix by pre- and postmultiplying the diagonal matrix
of eigenvalues with the matrix of eigenvectors and its transpose,
- Add a diagonal matrix
to the AI matrix, with and the
smallest eigenvalue of the AI matrix. This has been chosen as the
default procedure, with bigger than , and the
largest eigenvalue of the AI matrix.
Choice of the modification can have a substantial effect on the efficiency of the AI
algorithm. In particular, too large a modification can slow convergence rates
unnecessarily. Further experience is necessary to determine which is a good choice of
modification for specific cases.