Skip to content

[BOT ISSUE] OpenAI: Responses API connect() WebSocket transport not instrumented #188

@braintrust-bot

Description

@braintrust-bot

Summary

The OpenAI Python SDK's client.responses.connect() method — the WebSocket-based real-time streaming transport for the Responses API — is not instrumented. Calls to it fall through via NamedWrapper.__getattr__ to the unwrapped SDK, producing zero Braintrust tracing.

This is OpenAI's Realtime API surface integrated into the Responses endpoint. It enables bidirectional WebSocket streaming for audio, text, and tool use in real-time applications, and is the SDK entrypoint for voice AI and real-time agent use cases.

What is missing

ResponsesV1Wrapper (line 873 of oai.py) and AsyncResponsesV1Wrapper (line 892) only define create() and parse(). The connect() method is not wrapped.

Method Instrumented?
responses.create() Yes
responses.parse() Yes
responses.stream() No (tracked in #170)
responses.retrieve() No (tracked in #187)
responses.connect() No

The OpenAI SDK's Responses.connect() returns a ResponsesConnectionManager (sync) or AsyncResponsesConnectionManager (async) that manages a WebSocket connection. Through this connection, users can:

  • Send audio, text, and tool results in real-time
  • Receive streaming model output (audio and text)
  • Handle function/tool calls mid-session
  • Manage conversation turns bidirectionally

At minimum, instrumentation should provide:

  • Session-level span: Created when connect() opens a session, capturing model config, total session duration, and aggregate token usage
  • Turn-level spans: Child spans for each send/receive exchange, capturing input content, output content, and per-turn token usage
  • Tool use spans: Nested spans for function calls and results within a session

Analogy with existing issues

This is the OpenAI equivalent of issue #125 (Google GenAI Live API not instrumented). Both are WebSocket-based real-time multimodal streaming APIs from major providers that produce zero tracing today.

Braintrust docs status

not_found — The OpenAI integration page does not mention connect(), WebSocket transport, or Realtime API instrumentation. The page does reference a "Evaluating audio with the OpenAI Realtime API" cookbook, suggesting awareness of the feature but no SDK-level tracing support.

Upstream sources

  • OpenAI Python SDK Responses class: openai/resources/responses/responses.py — defines connect() returning ResponsesConnectionManager / AsyncResponsesConnectionManager
  • The SDK imports websockets.sync.client.ClientConnection for the WebSocket transport
  • OpenAI Realtime API documentation (accessible via platform.openai.com/docs/guides/realtime)

Local files inspected

  • py/src/braintrust/oai.py:
    • ResponsesV1Wrapper (line 873) — only defines create() and parse()
    • AsyncResponsesV1Wrapper (line 892) — only defines async create() and parse()
    • NamedWrapper.__getattr__ (line 24) — delegates unwrapped attrs to original client, so connect() silently bypasses tracing
  • py/src/braintrust/wrappers/test_openai.py — no tests for connect(), WebSocket, or Realtime API
  • py/noxfile.py — no realtime-specific test session

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions