Indexing jobs in SAS Visual Investigator fail with an error message similar to the following:

The index for the object <objectName> could not be created.

In addition, in the svi-indexer logs, error messages similar to the following occur:

Encountered an error executing step createNewIndex in job objectTypeChildJob\ncom.sas.svi.indexer.exception.IndexerException: The index for the object \"PTY\" could not be created.
Caused by: org.springframework.web.client.HttpServerErrorException$InternalServerError: 500  on POST request for \"/svi-sand/admin/indices/PTY\": \"{\"errorCode\":95001,\"message\":\"An error occurred during a request to a different service. Please contact your application administrator if this issue persists.\"

In svi-sand logs, error messages similar to the following also occur:

[REQUEST_HANDLING_ERROR] [d0bad391-b80e-445d-89b5-1e506bf6b38a] The request to the URI /svi-sand/admin/indices/PTY resulted in an error. Refer to the related log entries for more information.
[REQUEST_HANDLING_ERROR_DETAILS] [d0bad391-b80e-445d-89b5-1e506bf6b38a] Error: key: OPENSEARCH_ERROR.msg resource bundle: svi-sand-opensearch message: OpenSearch returned an error: Code: 429. Error: circuit_breaking_exception:[parent] Data too large, data for [indices:admin/aliases] would be [2046553436/1.9gb], which is larger than the limit of [2040109465/1.8gb], real usage: [2046552864/1.9gb], new bytes reserved: [572/572b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=786/786b, neural_search=0/0b].
[REQUEST_HANDLING_ERROR_CAUSED_BY] [d0bad391-b80e-445d-89b5-1e506bf6b38a] Caused by: exception: OpenSearchException message: OpenSearch returned an error: Code: 429. Error: circuit_breaking_exception:[parent] Data too large, data for [indices:admin/aliases] would be [2046553436/1.9gb], which is larger than the limit of [2040109465/1.8gb], real usage: [2046552864/1.9gb], new bytes reserved: [572/572b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=786/786b, neural_search=0/0b].

Cause

429 errors from OpenSearch (or ElasticSearch for older versions) suggest that the OpenSearch (or ElasticSearch) service is overwhelmed and struggling to accept all of the information that the Indexer provides. From the logs, the default setting is - 1.8GB in the example above.

Usually, this default setting is set to a percentage of the total memory available for the OpenSearch or ElasticSearch JVM.

See the OpenSearch and ElasticSearch documentation for additional information.

Resolution

To resolve this issue, do either of the following:

• Increase the amount of memory that is available to OpenSearch or ElasticSearch through the SAS^® Environment Manager JVM setting (that is, set java_option_xmx to a higher value). You can also increase the memory size by setting the heap size value for the OpenSearch master node.
• Slow down the Indexer. If memory is an issue, or speed does not feel like a problem, you can do as follows:
  • Reduce the number of dataIndexer threads, which is part of the sas.svi-indexer.tuning configuration setting that is accessible on the SAS Visual Investigator Indexer service through SAS Environment Manager.
  • Reduce the default chunkSize or those of the entities whose jobs fail through the "Partition Tuning" settings in SAS Visual Investigator - Admin.