What is the cause of the error message "Too many iterations in tqli"?

The error message "Too many iterations in tqli" is sometimes encountered when performing analyses such as Minimum Noise Fraction transform, Principal Components transform, or Maximum Likelihood classification on spectral datasets. This Help Article describes the most common situations that cause this error, and how to avoid them.

Each of the algorithms that can produce this error requires ENVI to invert the variance/covariance matrix for the input image. Inverting a matrix is a computationally intensive process, so computers must use an iterative method to perform the inversion. The subroutine that performs these iterations is called TQLI and typically, the process converges on a solution in 3 or 4 iterations. However, sometimes the characteristics of the image data introduce what is called a 'singularity' into the covariance matrix, and this singularity prevents a solution from converging (i.e., it makes the covariance matrix impossible to invert). When this happens, the TQLI subroutine reaches a maximum number of iterations without a solution and you get the error, "Too Many Iterations in TQLI".

What would cause a singularity in your image's covariance matrix?

There are 2 common causes of the problem:

1. Singularities in the covariance matrix are often caused by one image band being a perfect linear combination of other image bands. This is not uncommon if the instrument had any bad bands (e.g., bands that the instrument wasn't able to measure correctly). In many cases, the bad bands are set to a constant value (like zero, or the largest negative or positive number the instrument can register), are set to the same value as an adjacent band, or are interpolated from nearby bands. All of these cases will cause a singularity because the bad bands become a linear combination of other image bands.
   
2. The first PCA rotation performed during an MNF transform is based on the statistics of the image noise. If you are estimating the noise from the image data using the shift differencing approach (ENVI's default method), it is recommended that you choose a spatial subset of your image that is spectrally uniform to estimate the noise. In this case, you could get a singularity if the shift difference noise subset has zero variance in any band. For example, if you use a deep placid lake for your noise subset, most bands will have very little variation in the subset (which is what you want). However, if there is one band where there is absolutely no variation at all (that is, every pixel in that band in the subset has the same value), then you get zero variance for that band. This introduces a zero into the covariance matrix, causing a singularity, and the matrix can't be inverted. A similar problem is encountered if a region of interest specified as training set for Maximum Likelihood classification includes only pixels that all have the same value in one band.

What can you do to search for bad bands or zero-variance spatial subsets?   

• You can find the bands that contain unusual values (zero variance, large outlier numbers, or identical statistics to that of an adjacent band) by calculating basic statistics for your full image (under Basic Tools -> Statistics -> Compute Statistics). You can then add the bad bands to the Bad Bands list in the ENVI header (File -> Edit ENVI Header -> Edit Attributes -> Bad Bands List) so that they are excluded from numerical processing. Another option to exclude specific bands is to choose "Spectral Subset" in the File Selection dialog when using the MNF, PCA, or Maximum Likelihood tools.    
  
• You can compute basic statistics on the same shift-difference subset of your input image (under Basic Tools -> Statistics -> Compute Statistics -> Spatial subset) that you plan to choose for an MNF transform. In the statistics report, check to see if any band has zero variance. If this is the source of the problem, you should be able to correct for it by choosing a slightly different or larger subset for your noise estimate. (NOTE: The shift difference spatial subset must contain more pixels than you have bands in the image you are transforming.)

If possible, we recommend contacting the data provider that developed or flew the instrument to see if they have a noise model (dark current data) for the instrument. This data provides the best estimate of noise for the MNF transform and should be used whenever available.

Review on 12/31/2013 MM