SAS® Viya® administrative tasks fail with "version difference between client (x) and server (x) exceeds the supported minor version skew of +/-1"


A SAS Viya license update initiated with the sas-orchestration deploy command can fail if the version of kubectl in the orchestration container is not compatible with the Kubernetes API server version.

This compatibility requirement is part of the kubectl and Kubernetes design. It is not a version-skew restriction created or imposed by SAS. Kubernetes supports kubectl within one minor version, older or newer, of the Kubernetes API server. See Version Skew Policy for additional information.

Symptoms

When you apply an updated SAS Viya license with the sas-orchestration deploy command, the deployment assessment fails during the Kubernetes version-alignment check.

As a result, the log contains messages similar to the following:

version difference between client (1.33.8) and server
(1.35.6-eks-bca9cf6) exceeds the supported minor
version skew of +/-1
 
Step failed: kubectl and server version combination
exceed the supported skew window
 
Operation 'kubernetes-kubectl-server-version-alignment-check' failed
 
Operation 'deploy' failed

In the above example, the orchestration container used kubectl 1.33.8, but the Kubernetes API server reported version 1.35.6. The two-minor-version difference was outside the supported kubectl version-skew window. The version-skew window is enforced on all combinations of the kubectl client and Kubernetes API server, so this behavior might occur on any version combination.

Cause

The sas-orchestration deploy command executes tasks using the kubectl client, which is embedded inside the sas-orchestration docker container. Kubectl performs assessment checks before completing most actions. One of these checks compares the kubectl client version with the version of the Kubernetes API server.

Kubernetes defines the supported compatibility range for its components. Under the Kubernetes version-skew policy, kubectl is supported within one minor version, older or newer, of the kube-apiserver. For example, a Kubernetes 1.35 API server supports kubectl versions 1.34, 1.35, and 1.36. This policy is defined by the Kubernetes project and is not a restriction defined by SAS or specific to the sas-orchestration container image.

The sas-orchestration container image is associated with a specific SAS Viya release and its deployment assets. It is not possible to update the kubectl version included inside the sas-orchestration container image without also updating the SAS Viya version. As a result, circumstances can occur where the sas-orchestration container image attempts to use a kubectl client, which has become outdated and is disallowed from interacting with the Kubernetes API of a cluster where SAS Viya is deployed.

Resolution

If a kubectl version-skew occurs, you must install and use a standalone kubectl client that is compatible with the Kubernetes API server to apply the SAS Viya license. You will follow the "Kubernetes Commands" methods, sometimes referred to as "Manual Deployment" methods, outlined in SAS documentation. You can freely switch between using the sas-orchestration container image and standard Kubernetes Commands without needing to perform modifications to SAS Viya.

In order to successfully execute standard Kubernetes Commands, ensure that you have the following:

To perform the license update, complete the following steps, as outlined in the SAS Viya Platform Administration documentation:

  1. Download the updated SAS Viya license from the my.sas.com portal.
  2. Store the updated SAS Viya license (.jwt file) on the server where you configured the standalone kubectl client.
  3. Modify the kustomization.yaml file to reference the new SAS Viya license in the appropriate secretGenerator block.
  4. Confirm that any other changes are complete, as noted in How to Apply a New License: Update using Kubernetes Commands.
  5. Deploy SAS Viya using standard Kubernetes commands (kubectl apply).

If a situation occurs where these manual actions are required, it is possible that your SAS Viya version, Kubernetes version, or both, are nearing Limited Support. You should check the resources available within the latest version of SAS documentation, support policy, and Kubernetes release history to confirm.