Windows Server 2022 SAS connections to TLS 1.3 hosts are suspended or fail when the primary ECC Curve is not supported by the target server


When you run SAS on Windows Server 2022 and try to connect to a server using TLS 1.3, the TLS negotiation might fail. As a result, the attempt is suspended or produces an error.

With a LDAPS_OPEN call, a generic error similar to the following occurs:

ERROR: Unable to contact the LDAP server.
ERROR: LDAP SSL Message ldapsNegotiate() failed -2143305722.
ERROR: Possible cause: Server certificate not found, port not SSL enabled

With the HTTP procedure, an error similar to the following occurs:

ERROR: The TCP/IP tcpSockEstablishSSL() support routine failed with error 10054
       (The connection was reset by a peer.).
ERROR: The tcpSockRead call failed.
       The system error is 'The connection was reset by a peer.'.
ERROR: Call to tcpSockContinueSSL failed.

Cause

The Windows Server 2022 SChannel implementation that SAS uses sends a single ECC Curve as part of the TLS 1.3 Client Hello. If the selected curve is not supported by the server that is contacted, the server responds with a Hello Retry Request. However, this response is not handled, which causes the connection to fail or be suspended.

Workaround

To work around this issue, change the ECC Curve Order in the Windows group policy so that a curve supported by the destination server is specified first, which prevents the Hello Retry Request.

Note: Changing the ECC Curve Order group policy requires a system restart.