Stopping Apache HTTP Servers in a SAS® Viya® 3.5 environment that are configured with High Availability returns a 401 error or 500 error


You might encounter an error in the SAS Viya 3.5 environment when your setup includes high availability that contains two or more Apache HTTP Servers. If you stop one of the Apache HTTP servers, you receive a 500 error or a 401 error in the user interface for various web applications.

This issue returns the following errors in the logs:

org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : "{"errorCode":0,"message":"Couldn't retrieve remote JWK set: Connection refused (Connection refused)","details":["traceId: 04ce3faf9fcece3d","path: /identities/users/username/memberships","traceId: a336dcc45911130e","path: /authorization/decision"],"links":[],"version":2,"httpStatusCode":401}"
[{"errorCode":0,"message":"Couldn't retrieve remote JWK set: Connection refused (Connection refused)","details":["traceId: 16aa7f44b2e6e379","path: /themes/themes/@defaultApplicationTheme"],"remediation":null,"links":[],"version":2,"httpStatusCode":401}]
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 : [{"errorCode":0,"message":"Couldn't retrieve remote JWK set: Connection refused (Connection refused)","details":["traceId: 16aa7f44b2e6e379","path: /themes/themes/@defaultApplicationTheme"],"remediation":null,"links":[],"version":2,"httpStatusCode":401}]

Cause

These errors occur because of the third-party code that SAS uses to validate tokens. To validate the signature, the third-party code retrieves the RSA public key by calling an endpoint on SAS Logon Manager. This endpoint must be initialized with a URL in order to retrieve the key. To obtain that URL, SAS uses the service resolver, which in turn uses the service information in the consul for HTTPD. The address for HTTPD changes after a failover, but the third-party library still uses the old, invalid URL that was resolved before a failover.

Workaround

To circumvent this problem, complete the steps below:

  1. Make sure that all the configurations required for high availability are completed as listed in the following resources: High Availability with SAS Viya 3.x: Front-end Load-Balancing Considerations and Configure External Reverse Proxy.
  2. Then complete the additional steps below to circumvent this issue:
  1. Create a backup of the sas-java-services file that resides in the /opt/sas/viya/config/etc/sysconfig/sas-javaesntl/ directory and save it in the /tmp/ directory. Note: Do not back up or create another copy of the sas-java-services file under the /opt/sas/viya/config/etc/sysconfig/sas-javaesntl/ directory, because all files will be read regardless of their names.
  2. Edit the sas-java-services file that resides in the /opt/sas/viya/config/etc/sysconfig/sas-javaesntl/ directory. This file exports environment variables with a name prefixed with "java_global_option_".
  3. In the file, find the java_global_option_internal_proxies variable. Directly below it, add another variable similar to the one below but specify the URL to your load balancer:
export java_global_option_httpd="-Dsas.url.httpd=https://load-balancer.sas.example.com"
  1. ​To pick up the variable, restart all SAS Viya microservices. After the restart, you can use the following command to verify that the service was started with the Java property: ps -fe | grep service. Replace service with the name of the microservice. For example, ps -fe | grep saslogon.