Skip to content

[codex] fix MCP output schema contracts#851

Merged
RhysSullivan merged 8 commits into
mainfrom
codex/mcp-output-schema-regression-tests
May 20, 2026
Merged

[codex] fix MCP output schema contracts#851
RhysSullivan merged 8 commits into
mainfrom
codex/mcp-output-schema-regression-tests

Conversation

@RhysSullivan
Copy link
Copy Markdown
Owner

@RhysSullivan RhysSullivan commented May 20, 2026

Summary

Fixes the MCP output schema/display mismatch and keeps regression coverage around the full contract.

For MCP tools, upstream outputSchema describes the MCP result's structuredContent. Executor invocation data, however, is the full successful MCP CallToolResult object under ToolResult.ok.data: content, optional structuredContent, optional _meta, and optional isError. The MCP plugin now registers tool output schemas as that CallToolResult shape, nesting the upstream MCP schema under structuredContent.

The CallToolResult wrapper is derived from @modelcontextprotocol/sdk's CallToolResultSchema rather than hand-written. Executor only substitutes structuredContent with the per-tool upstream output schema and narrows successful results to isError: false, so content block details and future MCP schema changes track the SDK. Top-level extra properties remain permitted because MCP's Result type is intentionally extensible.

The PR also adds reusable output contract coverage through typeCheckOutputTypeScript, which compiles a described tool output TypeScript type against an actual runtime output value. MCP tests use it against real invocation data, and the existing core tool-invoker tests now share the helper too.

Root Cause

MCP discovery copied the upstream MCP outputSchema directly into Executor's registered tool schema. That made the schema panel and generated output TypeScript describe data as the upstream structured payload, while the actual invocation returned data as the full MCP call result envelope.

Validation

  • bun run --cwd packages/core/sdk typecheck passes with existing Effect language-service warnings
  • bun run --cwd packages/core/execution typecheck passes with existing src/promise.ts warning
  • bun run --cwd packages/plugins/mcp typecheck passes with existing Effect language-service warnings
  • ./node_modules/.bin/oxlint -c .oxlintrc.jsonc packages/plugins/mcp/src/sdk/plugin.ts packages/plugins/mcp/src/sdk/elicitation.test.ts packages/plugins/mcp/package.json bun.lock --deny-warnings
  • bun run --cwd packages/core/sdk test -- src/testing/tool-output-contract.test.ts
  • bun run --cwd packages/plugins/mcp test -- src/sdk/elicitation.test.ts -t "outputSchema|CallToolResult|refreshSource|structured MCP result"
  • bun run --cwd packages/plugins/mcp test -- src/sdk/elicitation.test.ts

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 20, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 5e7f807 Commit Preview URL

Branch Preview URL
May 20 2026, 07:05 PM

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 20, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 5e7f807 May 20 2026, 07:05 PM

@RhysSullivan RhysSullivan changed the title [codex] add MCP output schema regression tests [codex] add MCP output contract regression tests May 20, 2026
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented May 20, 2026

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@851

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@851

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@851

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@851

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@851

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@851

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@851

@executor-js/plugin-google-discovery

npm i https://pkg.pr.new/@executor-js/plugin-google-discovery@851

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@851

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@851

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@851

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@851

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@851

executor

npm i https://pkg.pr.new/executor@851

commit: 5e7f807

@RhysSullivan RhysSullivan changed the title [codex] add MCP output contract regression tests [codex] fix MCP output schema contracts May 20, 2026
@RhysSullivan RhysSullivan marked this pull request as ready for review May 20, 2026 19:55
@RhysSullivan RhysSullivan merged commit 44a02ad into main May 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant