Segfault on Node 22.19.0 (most likely issue with OpenSSL 3.0.17)

[kastov]

Version

v22.19.0

Platform

Linux remnawave 5.15.0-139-generic #149-Ubuntu SMP Fri Apr 11 22:06:13 UTC 2025 x86_64 Linux

Subsystem

Official alpine linux image, node:22-alpine

What steps will reproduce the bug?

This problem is quite difficult to reproduce, it appears rather spontaneously and I unfortunately cannot trigger it deliberately.
Below I'm attaching the stacktrace (made with node-segfault-handler).

The problem started manifesting exclusively on version 22.19.0. On version 22.18.0, this problem was not observed.

=========== Caught a Segmentation Fault [pid=1570] ===========
-----[ Native Stacktraces ]-----
[pc=0x00007f27e28d87ce, sp=0x00007f255e922c60] in segfault_handler(int)+0x4e
[pc=0x00007f27e599d5a4, sp=0x00007f255e922c80] in +0x4e
[pc=0x000055e05741573c, sp=0x00007f255e923a60] in OSSL_STORE_load+0x11c
[pc=0x000055e057425acb, sp=0x00007f255e923ab0] in by_store_subject+0xcb
[pc=0x000055e057444f6d, sp=0x00007f255e923b60] in X509_STORE_CTX_get_by_subject+0x15d
[pc=0x000055e057445ea3, sp=0x00007f255e923be0] in X509_STORE_CTX_get1_issuer+0x83
[pc=0x000055e05744a821, sp=0x00007f255e923c50] in build_chain+0x1c1
[pc=0x000055e05744c848, sp=0x00007f255e923ce0] in verify_chain+0x28
[pc=0x000055e05744db33, sp=0x00007f255e923d50] in X509_verify_cert+0xc3
[pc=0x000055e057219ee0, sp=0x00007f255e923d80] in ssl_verify_cert_chain+0x290
[pc=0x000055e05725e5a2, sp=0x00007f255e923db0] in tls_post_process_server_certificate+0x42
[pc=0x000055e05725a415, sp=0x00007f255e923df0] in state_machine+0x775
[pc=0x00007f255f48bc3c, sp=0x00007f255e923eb0] in +0x775
[pc=0x00007f255f4868a8, sp=0x00007f255e9241b0] in +0x775
[pc=0x00007f255f48501f, sp=0x00007f255e928720] in +0x775
[pc=0x00007f255f483e8d, sp=0x00007f255e929300] in +0x775
[pc=0x00007f255f55b85f, sp=0x00007f255e929eb0] in +0x775
[pc=0x00007f255f55362b, sp=0x00007f255e92c610] in +0x775
[pc=0x00007f255f8654f1, sp=0x00007f255e933650] in +0x775
[pc=0x00007f255f8619cb, sp=0x00007f255e936500] in +0x775
[pc=0x00007f255f86188f, sp=0x00007f255e9366e0] in +0x775
[pc=0x00007f255f884cdc, sp=0x00007f255e936830] in +0x775
[pc=0x00007f255f8b3632, sp=0x00007f255e936970] in +0x775
[pc=0x00007f255f8b1bb7, sp=0x00007f255e937050] in +0x775
[pc=0x00007f255f8b5a58, sp=0x00007f255e937ba0] in +0x775
[pc=0x00007f255ef51cbe, sp=0x00007f255e938260] in +0x775
[pc=0x00007f255ef1b740, sp=0x00007f255e9383f0] in +0x775
[pc=0x00007f255ef17dfe, sp=0x00007f255e938680] in +0x775
[pc=0x00007f255ef55a55, sp=0x00007f255e938b30] in +0x775
[pc=0x00007f255ef397bb, sp=0x00007f255e939000] in +0x775
[pc=0x00007f255ef47276, sp=0x00007f255e939110] in +0x775
[pc=0x00007f255f8fd520, sp=0x00007f255e93a1d0] in +0x775
[pc=0x00007f255f907357, sp=0x00007f255e93a210] in +0x775
[pc=0x00007f255f8fb8b1, sp=0x00007f255e93a380] in +0x775
[pc=0x00007f255f8fb321, sp=0x00007f255e93a3a0] in +0x775
[pc=0x00007f255f9002bd, sp=0x00007f255e93a480] in +0x775
[pc=0x00007f255f8e2f11, sp=0x00007f255e93a540] in +0x775
[pc=0x00007f27e59a59d2, sp=0x00007f255e93a570] in +0x775
---[ V8 JavaScript Stacktraces ]---

Sometimes, stacktrace can be empty for some reasons.

=========== Caught a Segmentation Fault [pid=448] ===========
-----[ Native Stacktraces ]-----
[pc=0x00007f98978727ce, sp=0x00007f9613119ce0] in segfault_handler(int)+0x4e
[pc=0x00007f989a8025a4, sp=0x00007f9613119d00] in +0x4e

---[ V8 JavaScript Stacktraces ]---

Apparently, the problem is related to the recent OpenSSL update to version 3.0.17 – #58097

The problem is widespread and there are already multiple issues:

• concurrent TLS connection segfault in x509 storage (regression on 3.0.17) openssl/openssl#28171
• segfault with debian-bookworm and openssl 3.0.17 prisma/prisma#27785
• Segmentation faults in SSL/TLS connections on Debian Bookworm with OpenSSL 3.0.17 confluentinc/librdkafka#5159
• Segmentation fault with Debian Bookworm and OpenSSL 3.0.17 karafka/rdkafka-ruby#667
• Segmentation fault at address X oven-sh/bun#21515

I also checked the OpenSSL versions with the command node -p "process.versions.openssl" inside an Alpine container:
Node 22.18.0 - 3.0.16
Node 22.19.0 - 3.0.17

Accordingly, I also tried rolling back to Node version 22.18.0 and the problem did not recur, from which I can conclude that it is most likely related to the OpenSSL version.

How often does it reproduce? Is there a required condition?

Depends of load.

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

No response

[anshul119]

I had a similar problem recently. I observed high usage on my node app (using node:22-alpine and v22.19.0). Sometimes, the app would not start without any error or exit.

I struggled for a few days trying to debug it, and reverting to an older node version helped fix the issue.

[siddarth]

seeing the same on our end. any updates on triage, @aduh95?

[ronking-zhu]

can you check this issue? #60240

[brianbauer42]

Just adding a +1 that we were experiencing segfaults reliably enough that we couldn't successfully deploy to k8s. They appeared to originate in some calls from Prisma orm to our postgres db. We didn't troubleshoot further, but downgrading node resolved the issue.