The APPEND procedure adds all row data to the BASE= data set before integrity constraint validation is performed and before data values are added to indexes. This approach of appending data is called the fast-append method, and it is the default method for PROC APPEND.
If a row’s data values violate one of the defined integrity constraints, the row must be removed from the BASE= data set. When a large number of rows fails the integrity constraint validation step, the time required for removing each row from the BASE= data set uses significant computer resources. Removing large numbers of rows can cause the SAS job to seem to stop responding.
In most cases, using PROC APPEND to append millions of invalid rows of data to a BASE= data set should still complete successfully with a warning in the SAS® log. A message similar to the following indicates how many invalid rows of data were not added to the BASE= data set.
However, this warning does not appear if the THREADS system option is in effect. When multiple threads are used during the PROC APPEND process, the SAS job seems to stop responding, rather than to end with a warning or error.
Two approaches can address this PROC APPEND issue.
The APPENDVER=V6 option appends one observation at a time to the BASE= data set. The integrity constraint validation step is performed on each row before the row is added to the data set. This option prevents the rows from having to be removed from the data set if they fail the integrity constraint validation step. Using the APPENDVER=V6 option prevents the default fast-append method from being used.
To work around issues that occur when multiple threads are used during PROC APPEND processing, use the NOTHREADS system option to turn threading off.