A "data could not be deleted" error occurs during incremental indexing in SAS® Visual Investigator


The following error occurs on the Jobs tab during incremental indexing in SAS Visual Investigator:

SANDDeleteByQueryException: The data could not be deleted.

When you execute the incremental index job, the LinkReferenceDeleter generates and queries for each documentId × relationshipName combination. LinkReferenceDeleter then batches the combination into OR queries of up to maxQueryConditionLimit (default 512) before sending to SAND/OpenSearch as a _delete_by_query. However, OpenSearch rejects these complex queries with search_phase_execution_exception: all shards failed because the query exceeds its internal complexity limits.

Workaround

To circumvent this issue, adjust the maxQueryConditionLimit configuration setting by completing the following steps:

  1. In SAS® Environment Manager, go to Configuration (the Wrench icon) in the left pane.
  2. Select Definitions from the View drop-down menu.
  3. Search for sas.svi-indexer.opensearch in the Search filter.
  4. In the right pane, click the New configuration button.
    sas.svi-indexer.opensearch in the Search filter ►in the right pane, click the New configuration button.

  5. A pop-up window is displayed for the maxQueryConditionLimit setting. The default value is 512. Decrease the setting and click Save.
    The maxQueryConditionLimit setting pop-up window.

  6. Restart the svi-indexer pod for the adjusted setting to take effect.

Note: You might need to readjust the setting to locate the correct value. Start with 256.