Metadata is generated for all information map data items that contain the Metadata custom property set to either Counts or Values.

Generating Metadata

You can generate metadata by using the dm.sh gmd command. The following examples show how to generate metadata at different information map levels:

Data item:

$ ./dm.sh gmd --character SQL --dataitems "root.OCCPCD"

 

All data items in tables:

$ ./dm.sh gmd --tables ORADM.CLIENTS --character SQL --maxconcurrent 5 --bcName <yourBCname>

 

All character data items.

$ ./dm.sh gmd --character SQL --maxconcurrent 5 --bcName <yourBCname>

 

General Tips

Here are some general tips about generating metadata:

• Minimize the setting by using the custom property Metadata=Values/Counts on data items that generate more than 1,000 unique values.
  
  
• If you encounter an error during metadata generation, check for on-premises issues since those are usually the source rather than the SAS CI360 gateway.
  
  
• You should always review the <DM-agent-install-directory>/logs/onprem_direct.log file if an error occurs while generating metadata.
  
  
• If you made relationship changes to the Information Map, run the dm.sh cim command:

$ ./dm.sh cim --bcName <yourBCname>

This command is necessary so that queries are generated with the most recent Information Map joins.

Common Errors

Sometimes you encounter a problem with the SAS^® 9.4 platform and not with the metadata generation itself. To verify the issue, check whether the SAS 9.4 Platform is in stable condition:

$ dm.sh gmd --character SQL --maxconcurrent 2

[main] WARN org.springframework.web.client.RestTemplate  - POST request for "http://localhost:10080/CIOnPremDirect/rest/command/metadata" resulted in 500 (Internal Server Error); invoking error handler

Error completing request: org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error

Error: Problem running metadata generation

---

Sometimes you encounter an error similar to the following in the console:

$ dm.sh gmd --character SQL --maxconcurrent 2

Error completing request: org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error

Error: Problem running metadata generation

 

1. If this error occurs, review the onprem_direct.log and search for “CommandController – handleMetadataRequest” to find the start metadata generation section in the log file:

2023-04-12T16:04:06,599 INFO  [http-nio-10080-exec-3] SID[] USER[] CC[] []
cs.crm.rest.controller.CommandController - handleMetadataRequest {"charMethod":"SQL","tableNames":
["ORADM.CLIENTS"],"maxConcurrent":5,"generateMetadata":true,"maxNominalValues":1000,
"type":"metadata","bcName":"citesusp"}

2. After that, search for “ERROR” and review the ones you find, which might include the following:

• permission issues for the MAMETA library directory
• no join between tables in the information map
• database tables are not visible
• out of disk space because you are generating metadata on a field with a large number of unique values

 

3. Add the following logger to the <DM-agent-directory>/log_config/ SASCustIntelOnPremDirect-log4j.xml file if you want to see the metadata generated query:

    <logger name="com.sas.analytics.crm.custdata.datapattern.metadata" additivity="false">

        <level value="DEBUG"/>

        <appender-ref ref="SAS_FILE"/>

    </logger>

 

The onprem_direct.log file should contain a message similar to the following, which should help identify the issue:

2021-03-04 09:19:39,831 DEBUG [CIExec-6] SID[9d90e95e5729056c:-279cafff:177fd955232:-79fe] USER[SAS Demo User] CC[] [] taSQLCountsGenerator$CountQueryGenerator - TID[CIExec-6]

**NOTICE** THIS IS PRE-EXECUTION HIGH DETAIL DEBUG LOGGING **NOTICE**

 

/* clear syscc before new query */

%let syscc=0;

%let sysrc=0;

 

options Locale=en_US;

LIBNAME MAORA ORACLE  PATH=ORA1212  SCHEMA=ma  USER=mauser  PASSWORD=******** ;

 

libname MAMeta '/install/ma/MAMeta/mafunc';

libname MetaTMP '/install/ma/MATables/SAS/DEHAXLBL1RH1S5FM';

 

PROC SQL;

Create table MetaTMP.MAM_06300000DEFEYNTMD3EWW4SD as

SELECT

        table0.STATE AS UNFORMATTED LABEL='State' FORMAT=$char.,

        table0.STATE AS FORMATTED LABEL='State' FORMAT=$2.,

         ( count(table0.CLIENTID) )  AS COUNT1 LABEL='count1'

FROM

        MAORA.CUSTOMER1 table0

GROUP BY

        1,

        2;

quit;

4. If you are still unsure why the error occurs, contact SAS Technical Support and provide the onprem_direct.log log.