Occasionally when running PROC LOESS, a warning about the singularity of the sums of squares and cross products (SSCP) matrix is produced in the SAS log. For example:

WARNING: At the selected smoothing parameter 0.073, the local SSCP matrix for 5 fit point(s) is numerically singular. The fitted value and standard errors at those points are not uniquely defined.

Following are some possible causes of the warning message and ways to avoid the warning.

Measurement scales of dependent and independent variable(s)

If the measurement scales of the response variable (Y) and the explanatory variables (X's) are vastly different then any regression routine can have problems. In such cases running PROC STDIZE on Y or X may help. The SSCP warning message has been known to occur in cases where the Y values were extremely large in relation to the X's and scaling Y (for instance, dividing Y by 1,000,000) removed the singularity.

Smoothing parameter too small

When the smoothing parameter (S) is too small, collinearity problems may occur.

• If defining the smoothing parameter with the SMOOTH= option, increase this value.
• There is no problem if an automatic smoothing parameter selection criterion results in, say, S=.4 but during the search PROC LOESS evaluates S=.1 which, though not selected, gives the singular SSCP warning. However, if the selected S produces the warning then try a larger smoothing parameter.

Equally spaced data

Related to a too small smoothing parameter, the warning has occurred in cases of equally spaced data when fitting at the central X value in a neighborhood of equally spaced X values. Note that all regressions in the non-equally spaced case are full rank.

Consider a case where there are three fit points with one regressor, X. Label the points from smallest to largest as points 1, 2, and 3. A separate weighted regression is done at each fit point. When doing the regression centered at point 2, point 1 and point 3 are on the boundary of the local neighborhood and are assigned a weight of zero by the tri-cube weight function as detailed below. So, the fit at point 2 corresponds to a linear regression (intercept and slope) with one data point and hence is singular. When the regression is centered at point 1, point 3 is the boundary point assigned a weight of zero, but points 1 and 2 have non-zero weights so this regression is non-singular. Similarly, the regression centered at point 3 is non-singular. In this case of three local regressions done at three equally spaced points, the fit centered at one of the fit points is singular and the warning is issued.

The distances of points X=1, X=2, and X=3 from point X=2 are d₁=1, d₂=0, and d₃=1. Ignoring the constant multiplier which is inconsequential for singularity considerations, the tri-cube weight function in this case is w = [ 1 - (d_i/d₃)³ ]³ , for i=1, 2, 3. Hence the weights computed at each of the three points are:

X=1:	w = [ 1 - (1/1)3 ]3 = 0
X=2:	w = [ 1 - (0/1)3 ]3 = 1
X=3:	w = [ 1 - (1/1)3 ]3 = 0

This shows that the boundary points, X=1 and X=3, have zero weights when fitting at X=2 resulting in a singular regression.

Many replicates

The LOESS algorithm devised by W. S. Cleveland generally, not just specifically as implemented in PROC LOESS, is not as good when there are many repeated observations (replicates) at the X values. The singular SSCP matrix warning can occur when there are numerous replicates at a few or many of the regressor values. Suppose X ranges from 1 to 10 but has n=50 measurements at x=5. With, say, 10 points in the local neighborhood, PROC LOESS will choose the 10 closest points at X=5. This will result in an X matrix that corresponds to a single point at X=5 in turn giving rise to a singular X'X which produces the warning. In cases with many repeated observations, try a smoothing parameter for which the number of points in the local neighborhood is greater than the largest number of replicates at any X location.

SSCP warning is not that bad

PROC LOESS uses a generalized inverse and in the case of a singular SSCP matrix this causes non-unique fitted values and standard errors of the fitted values. That is, if a different generalized inverse were used then different fitted values and standard errors would result. However, this does not mean the fitted values produced are necessarily uninformative.

 

References

Cleveland, W. S. (1993), Visualizing Data, Summit, NJ: Hobart Press.

Cleveland, W. S., Devlin, S. J., and Grosse, E. (1988), "Regression by Local Fitting," Journal of Econometrics, 37, 87-114.

Cleveland, W. S. and Grosse, E. (1991), "Computational Methods for Local Regression," Statistics and Computing, 1, 47-62.