Skip to content

Anthropic: stop_reason and stop_sequence not captured in span output #175

@braintrust-bot

Description

@braintrust-bot

Summary

The Anthropic integration's _log_message_to_span function extracts only role and content from the response message, dropping stop_reason and stop_sequence. Every other comparable wrapper in this repo captures the model's stop/finish reason in the span output.

Inconsistency across wrappers

Wrapper Captures stop/finish reason? Location
OpenAI (chat completions) Yesfinish_reason in output choices oai.py ~line 354 (streaming), ~line 204 (non-streaming)
OpenAI (responses) Yesstatus in response object oai.py response wrapper
Google GenAI Yesfinish_reason per candidate google_genai/tracing.py ~line 313
Agno Yesfinish_reason in aggregated output agno/tracing.py ~line 293
Anthropic No anthropic/tracing.py ~line 445

What is dropped

The Anthropic Message response object includes:

  • stop_reason: One of "end_turn", "max_tokens", "stop_sequence", "tool_use" — tells users why the model stopped generating
  • stop_sequence: The specific stop sequence matched (when stop_reason is "stop_sequence")
  • model: The resolved model name (e.g., claude-sonnet-4-5-20250929 when the request used claude-sonnet-4-5-latest)

The current code at _log_message_to_span (line ~445 of tracing.py) only extracts:

output = {
    k: v
    for k, v in {"role": getattr(message, "role", None), "content": getattr(message, "content", None)}.items()
    if v
} or None

Why this matters

  • stop_reason: "max_tokens" signals output truncation — a common debugging need
  • stop_reason: "tool_use" signals the model wants tool execution — critical for agent workflows
  • The resolved model name is needed for accurate cost tracking when using model aliases
  • Other wrappers in this repo already capture the equivalent field, so users may expect it from Anthropic too

Braintrust docs status

The Anthropic integration docs describe streaming and metric collection but do not mention stop_reason capture: not_found.

Upstream source

Anthropic Messages API reference documents stop_reason as a top-level field on every Message response.

Local files inspected

  • py/src/braintrust/integrations/anthropic/tracing.py_log_message_to_span (line ~445) only extracts role and content
  • py/src/braintrust/integrations/anthropic/_utils.pyextract_anthropic_usage extracts token metrics but not response-level fields
  • py/src/braintrust/oai.py — OpenAI wrapper captures finish_reason for comparison
  • py/src/braintrust/integrations/google_genai/tracing.py — Google GenAI captures finish_reason for comparison

Metadata

Metadata

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions