-
Notifications
You must be signed in to change notification settings - Fork 1
[BOT ISSUE] OpenAI: Responses API connect() WebSocket transport not instrumented #188
Description
Summary
The OpenAI Python SDK's client.responses.connect() method — the WebSocket-based real-time streaming transport for the Responses API — is not instrumented. Calls to it fall through via NamedWrapper.__getattr__ to the unwrapped SDK, producing zero Braintrust tracing.
This is OpenAI's Realtime API surface integrated into the Responses endpoint. It enables bidirectional WebSocket streaming for audio, text, and tool use in real-time applications, and is the SDK entrypoint for voice AI and real-time agent use cases.
What is missing
ResponsesV1Wrapper (line 873 of oai.py) and AsyncResponsesV1Wrapper (line 892) only define create() and parse(). The connect() method is not wrapped.
| Method | Instrumented? |
|---|---|
responses.create() |
Yes |
responses.parse() |
Yes |
responses.stream() |
No (tracked in #170) |
responses.retrieve() |
No (tracked in #187) |
responses.connect() |
No |
The OpenAI SDK's Responses.connect() returns a ResponsesConnectionManager (sync) or AsyncResponsesConnectionManager (async) that manages a WebSocket connection. Through this connection, users can:
- Send audio, text, and tool results in real-time
- Receive streaming model output (audio and text)
- Handle function/tool calls mid-session
- Manage conversation turns bidirectionally
At minimum, instrumentation should provide:
- Session-level span: Created when
connect()opens a session, capturing model config, total session duration, and aggregate token usage - Turn-level spans: Child spans for each send/receive exchange, capturing input content, output content, and per-turn token usage
- Tool use spans: Nested spans for function calls and results within a session
Analogy with existing issues
This is the OpenAI equivalent of issue #125 (Google GenAI Live API not instrumented). Both are WebSocket-based real-time multimodal streaming APIs from major providers that produce zero tracing today.
Braintrust docs status
not_found — The OpenAI integration page does not mention connect(), WebSocket transport, or Realtime API instrumentation. The page does reference a "Evaluating audio with the OpenAI Realtime API" cookbook, suggesting awareness of the feature but no SDK-level tracing support.
Upstream sources
- OpenAI Python SDK
Responsesclass:openai/resources/responses/responses.py— definesconnect()returningResponsesConnectionManager/AsyncResponsesConnectionManager - The SDK imports
websockets.sync.client.ClientConnectionfor the WebSocket transport - OpenAI Realtime API documentation (accessible via platform.openai.com/docs/guides/realtime)
Local files inspected
py/src/braintrust/oai.py:ResponsesV1Wrapper(line 873) — only definescreate()andparse()AsyncResponsesV1Wrapper(line 892) — only defines asynccreate()andparse()NamedWrapper.__getattr__(line 24) — delegates unwrapped attrs to original client, soconnect()silently bypasses tracing
py/src/braintrust/wrappers/test_openai.py— no tests forconnect(), WebSocket, or Realtime APIpy/noxfile.py— no realtime-specific test session