A nonpositive definite Hessian matrix can indicate a surface saddlepoint or linear dependencies in the parameters. If PROC MIXED has converged to a saddlepoint, then the final solution it provides is not the optimal solution. To get around this problem, run the model again using different starting values. Try adding a PARMS statement to your PROC MIXED code. Either use the OLS option to specify ordinary least squares starting values (rather than the default MIVQUE0 values) or specify your own starting values. The syntax requesting the OLS starting values is:

parms / ols;

The syntax for specifying your own starting values is a little more complex. See the syntax section of the PROC MIXED chapter in the SAS/STAT User's Guide.

If you still get this message with new starting values, then there are probably linear dependencies in the parameters. This means that the model is too complex for your data to support. Try simplifying the covariance structure until you no longer receive this message. Other possible causes might include largely varying scales for some independent variables. You might rescale some of the independent variables so the values are ideally within 1 to 10.

The results from a model that has a nonpositive definite Hessian are not reliable and should be interpreted with extreme caution.