libvncserver: serialize WebSocket SSL teardown

[marcofortina]

Summary

This is a conservative WSS disconnect race mitigation.

The issue reports intermittent crashes when a secure WebSocket client disconnects while the output thread is still sending framebuffer updates. The reported OpenSSL backtrace shows SSL_write() using SSL state that was already freed by rfbCloseClient() from the input thread. The GnuTLS backtrace shows the same class of issue around rfbssl_destroy() / credential teardown.

Changes

• Serialize WebSocket SSL teardown with rfbWriteExact() by taking cl->outputMutex while destroying cl->sslctx.
• Make OpenSSL rfbssl_destroy() idempotent by returning early for NULL context and clearing cl->sslctx before freeing backend objects.
• Make GnuTLS rfbssl_destroy() idempotent in the same way.
• Free the backend SSL context wrapper after destroying its backend-specific resources.
• Clear cl->wspath after freeing it.

Validation

Local build and tests without TLS:

cmake -S . -B build-665-patch \
  -DWITH_EXAMPLES=OFF \
  -DWITH_TESTS=ON \
  -DWITH_OPENSSL=OFF \
  -DWITH_GNUTLS=OFF \
  -DWITH_GCRYPT=OFF \
  -DWITH_SDL=OFF \
  -DWITH_GTK=OFF \
  -DWITH_QT=OFF \
  -DWITH_FFMPEG=OFF \
  -DWITH_XCB=OFF \
  -DWITH_LIBSSHTUNNEL=OFF \
  -DWITH_SYSTEMD=OFF \
  -DCMAKE_BUILD_TYPE=Debug
cmake --build build-665-patch --parallel 1
ctest --test-dir build-665-patch --output-on-failure

Result:

100% tests passed, 0 tests failed out of 5

Local build and tests with OpenSSL enabled:

cmake -S . -B build-665-openssl \
  -DWITH_EXAMPLES=OFF \
  -DWITH_TESTS=ON \
  -DWITH_OPENSSL=ON \
  -DWITH_GNUTLS=OFF \
  -DWITH_GCRYPT=OFF \
  -DWITH_SDL=OFF \
  -DWITH_GTK=OFF \
  -DWITH_QT=OFF \
  -DWITH_FFMPEG=OFF \
  -DWITH_XCB=OFF \
  -DWITH_LIBSSHTUNNEL=OFF \
  -DWITH_SYSTEMD=OFF \
  -DCMAKE_BUILD_TYPE=Debug
cmake --build build-665-openssl --parallel 1
ctest --test-dir build-665-openssl --output-on-failure

Result:

100% tests passed, 0 tests failed out of 5

WSS disconnect stress smoke

Validated with an OpenSSL-enabled AddressSanitizer build using
examples/server/camera with WSS enabled and a Python stress client that performs
TLS connect, WebSocket upgrade, RFB 3.8 handshake, SetEncodings, framebuffer
update requests, and abrupt disconnects in parallel.

Stress result:

• 8 client threads
• 75 loops per thread
• 600 total WSS connection attempts
• 587 completed client sessions
• server stayed alive
• no AddressSanitizer crash
• no SEGV/ABORTING
• no observed SSL teardown use-after-free

Notes

Closes #665.