DSPy: adapter format and parse callbacks not implemented

[braintrust-bot]

Summary

DSPy's BaseCallback interface defines 6 callback pairs, but BraintrustDSpyCallback implements only 4 of 6. The two missing pairs provide visibility into DSPy's adapter layer — the critical step where user inputs are formatted into LM prompts and where raw LM outputs are parsed back into structured fields.

Missing callbacks

Callback pair	Purpose
on_adapter_format_start / on_adapter_format_end	Fired when a dspy.Adapter subclass formats the input into prompt messages for the LM. Shows how signatures, demos, and instructions are assembled into the actual prompt.
on_adapter_parse_start / on_adapter_parse_end	Fired when a dspy.Adapter subclass parses the raw LM text output into structured fields (e.g., extracting typed fields from completion text).

What is already implemented

The Braintrust callback implements on_lm_start/end, on_module_start/end, on_tool_start/end, and on_evaluate_start/end. These cover the outer module execution and the inner LM call, but not the adapter transformation steps between them.

Why this matters

The adapter layer is where DSPy's core abstraction happens — converting high-level signatures into prompts and parsing completions back into structured outputs. Without these callbacks, users can see what went into the module and what came out of the LM, but not how the adapter formatted the prompt or parsed the output. This is particularly valuable when debugging few-shot demo selection, instruction formatting, or output parsing failures.

Braintrust docs status

not_found — The DSPy integration page documents tracing of "DSPy module executions, LLM calls with token counts, tool invocations, hierarchical span relationships" but does not mention adapter-level tracing.

Upstream sources

• DSPy BaseCallback class: dspy/utils/callback.py — defines all 6 callback pairs including on_adapter_format_* and on_adapter_parse_*
• DSPy observability tutorial: dspy.ai/tutorials/observability

Local files inspected

• py/src/braintrust/integrations/dspy/tracing.py — BraintrustDSpyCallback class (implements 4 of 6 pairs)
• py/src/braintrust/integrations/dspy/patchers.py — patcher setup
• py/src/braintrust/integrations/dspy/test_dspy.py — tests (no adapter callback coverage)