After using a variable or effect selection method to automatically select variables or effects for a final model, if you then fit the final model without using the selection method you may see differences in the results or the number of observations used. Several procedures offer effect or variable selection methods. For example, the SELECTION= option available in the REG, LOGISTIC, GLMSELECT, and PHREG procedures implements effect selection methods in those procedures ^[NOTE].

The differences you see when comparing the results (parameter estimates, predicted values, number of observations used, etc.) of the final model chosen by the selection method to the results of the final model from a separate procedure run are usually due to missing values causing more observations to be omitted in the fit using the selection method than in the fit without the method.

Note that an observation that is missing on any variable listed in the MODEL statement is ignored when fitting the model. When using the selection method, if an observation is missing on a variable listed in the MODEL statement it is omitted even if this variable is not included in the final model. However, if you refit the model without using the selection method and specify only the variables in the final model, then the observation will not be ignored because it is missing on a variable that is no longer listed in the MODEL statement. The differences you see result from the same model being fit using different sets of observations.

An easy way to determine if the analyses are based on different numbers of observations is to check the "Number of Observations Used" at the beginning of the results from each run of the procedure.

For example, suppose you fit a logistic model to 100 observations in data set MYDATA and submit the following statements which use stepwise selection to choose a final model.

  proc logistic data=mydata;
     model y = x1 x2 x3 x4 x5 / selection=stepwise;
     run;

If the final model chosen by the stepwise method contains only X1 and X3, then you could refit the final model using these statements.

  proc logistic data=mydata;
     model y = x1 x3;
     run;

If you compare the results of the final model from these two analyses they may differ. If some of the 100 observations in MYDATA have missing values on X2, X4 and/or X5, then those observations will be omitted in the first model fit. However, none of those observations will be omitted from the second analysis because X2, X4, and X5 are not in the MODEL statement. Any observations that have missing values on X1 or X3 will be omitted from both analyses.

________

NOTE: This also applies to the variable selection methods in PROC STEPDISC. Observations having missing values on any of the variables listed in the VAR statement are omitted from the analysis. A subsequent analysis in PROC DISCRIM using only the variables selected by PROC STEPDISC will use more observations if some observations were omitted because of missing values on variables in the VAR statement of PROC STEPDISC and if those variables are not specified in the VAR statement of PROC DISCRIM.