How to increase performance using SAS/ACCESS® to Teradata when loading a SAS data set into Teradata


The SAS/ACCESS Interface to Teradata supports a bulk loading capability, called Fastload, which greatly speeds insertion of data into empty Teradata tables.

The SAS/ACCESS Fastload facility is similar to the native Teradata Fastload Utility, and they share the following limitations:

1)-- Fastload can load only empty tables. It cannot append to a table that already contains data. If you attempt to use Fastload when appending to a table that contains rows, your append step will fail.

2)-- Both the Teradata Fastload Utility and the SAS/ACCESS Interface Fastload logs data errors to tables. Error recovery can be difficult: you must refer to Teradata's Fastload documentation to find the error that corresponds to the code stored in the error table.

3)-- Fastload will not load duplicate rows (rows where all corresponding fields contain identical data) into a Teradata table. If your SAS data set contains duplicate rows, you can use the normal insert (load) process.

4)-- If you do not specify Fastload, your Teradata tables are loaded normally (slowly). To invoke Fastload in the SAS/ACCESS Interface, simply add the BULKLOAD=YES option to a processing step that populates an empty Teradata table. (Or use the Teradata alias, which is FASTLOAD=.). The following data set options are available: BL_LOG= Teradata Details: Specifies the names of the error tables created when using the SAS/ACCESS Fastload facility. By default, Fastload errors are logged in Teradata tables named SAS_FASTLOAD_ERRS1_randnum and SAS_FASTLOAD_ERRS2_randnum, where randnum is a randomly generated number.

For example, if you specify BL_LOG=my_load_errors, errors are logged in tables my_load_errors1 and my_load_errors2. If you specify BL_LOG=errtab, errors are logged in tables name errtab1 and errtab2.

Note: SAS/ACCESS automatically deletes the error tables if no errors are logged. If there are errors, the tables are retained, and SAS/ACCESS issues a warning message that includes the names of the error tables.

DBCOMMIT= 
Teradata Details: This option causes a Teradata "checkpoint" after each group of n rows are transmitted. Checkpointing slows performance but provides known synchronization points if there is a failure during the loading process. If BULKLOAD=YES, and DBCOMMIT= is not explicitly set, then the default is "no checkpointing."

The Teradata alias for this option is CHECKPOINT=.

Examples:

The following example invokes the Fastload facility.

libname fload teradata user=testuser password=testpass;

  data fload.nffloat(bulkload=yes);
     do x=1 to 1000000;
        output;
     end;
  run;

The following example uses Fastload to append SAS data to an empty Teradata table and specifies the BL_LOG= option to name the error tables append_err1 and append_err2. (In practice, applications typically append many rows.)

/* create the empty Teradata table */

proc sql;
    connect to teradata as tera(user=testuser password=testpass);
    execute (create table performers
            (userid int, salary decimal(10,2), job_desc char(50)))
             by tera;
    execute (commit) by tera;
  quit;

/* create the SAS data to be loaded */

data local;
     input userid 5. salary 9. job_desc $50.;
           cards;
             0433 35993.00 grounds keeper
             4432 44339.92 code groomer
             3288 59000.00 manager
             ;

/* append the SAS data and name the Teradata error tables */

libname tera teradata user=testuser password=testpass;

proc append data=local base=tera.performers (bulkload=yes bl_log=append_err);
run;