Fail-Open Vulnerability in set_tlsext_servername_callback allows Security Bypass via Unhandled Exceptions

Summary

The pyopenssl library exhibits a Fail-Open vulnerability in its handling of the Server Name Indication (SNI) callback (set_tlsext_servername_callback). The internal wrapper for this callback catches all Python exceptions raised by user code but returns 0 (Success/SSL_TLSEXT_ERR_OK) to the underlying OpenSSL engine.

This behavior allows a TLS connection to be successfully established even when the security validation logic inside the callback crashes or raises an exception, potentially bypassing critical access controls or authentication mechanisms implemented at the SNI layer.

Vulnerability Description

In src/OpenSSL/SSL.py, the set_tlsext_servername_callback method wraps the user-provided function with a decorator that does not explicitly handle exceptions to signal failure: