Add experimental browser CDP tools without a bundled spec#1163
Merged
threepointone merged 8 commits intomainfrom Apr 14, 2026
Merged
Add experimental browser CDP tools without a bundled spec#1163threepointone merged 8 commits intomainfrom
threepointone merged 8 commits intomainfrom
Conversation
🦋 Changeset detectedLatest commit: 1dc6687 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
jonnyparris
added a commit
to jonnyparris/dodo
that referenced
this pull request
Apr 8, 2026
Replace the single browser_navigate tool (text extraction only) with two code-mode tools that give sessions full CDP access: - browser_search: query the ~1.7MB CDP spec server-side (56 domains, 664 commands, 237 events). The spec never enters the context window. - browser_execute: run CDP commands against a live headless Chrome via WebSocket. Screenshots, DOM manipulation, network interception, etc. Uses Dodo's existing DynamicWorkerExecutor + BROWSER binding. The LLM writes JS that executes in a sandbox with a cdp helper (send, attachToTarget, getDebugLog). Adapted from cloudflare/agents#1163. Also: - Remove @cloudflare/puppeteer dependency (no longer needed) - Update system prompt with CDP usage patterns - Fix stale MCP catalog test (sentry entry was removed)
whoiskatrin
changed the title
Introduce experimental "Browse the Web" browser tools that expose Chrome DevTools Protocol (CDP) access (browser_search and browser_execute). Adds documentation, an examples/ai-chat integration (env types, wrangler config, start script, server & UI updates), a CDP spec build script (scripts/build-cdp-spec.ts) that generates spec data, new browser package exports and build targets, and end-to-end browser tests with Vitest. Also updates .gitignore and package metadata to include new optional dependencies and test scripts. This enables agents to run sandboxed JS that queries the CDP spec and executes CDP commands against a host browser for scraping, inspection, and debugging.
Update and sync dependency versions across example and package package.json files: bump ai to ^6.0.159, @cloudflare/vite-plugin to ^1.32.2, @cloudflare/workers-types to ^4.20260414.1, wrangler to ^4.82.2, react/react-dom and types to ^19.2.5, bump @x402 packages to 2.10.0 and viem to ^2.47.17, and other minor patch updates. Also rename a codemode test file to a .backup.ts. These changes align examples and packages with recent upstream patch releases and keep dev tooling consistent.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Add @tanstack/ai@^0.10.2 to packages/agents dependencies and bump its peerDependency to ^0.10.2. Remove the @ai-sdk/openai peer entry and update package-lock.json to reflect the new packages (including @tanstack/ai, @tanstack/ai-event-client and partial-json) and clean up duplicate entries, keeping manifest and lockfile consistent.
Introduce first-class browser tools (@cloudflare/think/tools/browser) that expose two tools: browser_search and browser_execute for CDP-based web automation. Implements createBrowserTools (packages/think/src/tools/browser.ts) which delegates to agents/browser createBrowserToolHandlers; exports SEARCH_DESCRIPTION and EXECUTE_DESCRIPTION from packages/agents/src/browser, and adds the new entry to package exports and the build script. Documentation and design are updated to describe the tools and usage, a changeset is added, and unit tests (packages/think/src/tests/browser-tools.test.ts) validate the toolset shape. Requires a Browser Rendering binding and a WorkerLoader (or a cdpUrl override) at runtime.
When a baseUrl is provided, preserve its protocol on the parsed URL instead of always mapping wss->https or ws->http; only apply the wss->https / ws->http mapping for the non-baseUrl (Workers) path that requires fetch+Upgrade. Also harden truncateResponse by coalescing JSON.stringify results to the string 'null' to avoid returning undefined for non-serializable inputs.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Pass through any provided options.headers into the WebSocket upgrade fetch in cdp-session so custom headers (e.g. auth) are preserved. In shared.ts wrap the cleanup DELETE request in a try/catch so failures during cleanup don't mask the original result or error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
agents/browser,agents/browser/ai,agents/browser/tanstack-ai).wrangler dev, without a separate Chrome process.@cloudflare/think/tools/browser), following the same pattern as workspace, execute, and extensions.What Changed
Browser tools (agents package)
browser_searchfetches the live protocol from the Browser Rendering devtools endpoints, normalizes it into a tool-facing search shape, and caches it briefly in memory.browser_executeconnects through the Browser Rendering devtools WebSocket endpoint and cleans up sessions with the correspondingDELETEendpoint.build-cdp-specbuild step.wrangler devinstance and exercise simple search and execution scenarios.ai-chatexample to use the Browser Rendering binding directly in local development.design/browser-tools.md) and usage doc (docs/browse-the-web.md).Think integration
@cloudflare/think/tools/browserwithcreateBrowserTools()— wrapsagents/browseras a first-class Think tool alongside workspace, execute, and extensions.docs/think/tools.md) and design doc (design/think.md) with browser tools section.Bug fixes (from Devin review)
truncateResponsethrowingTypeErrorwhen sandbox code returnsundefined(no explicit return statement).connectUrlusing wrong protocol when rewriting localhost URLs to an HTTPS base URL (tunnel/proxy case).Testing
npm run test:browser(agents browser e2e)npm run test -w @cloudflare/think(Think unit tests including browser tools)npm run -w agents buildandnpm run -w @cloudflare/think buildNotes
CDP_BASE_URLor a separately managed Chrome process.docs/browse-the-web.mdanddesign/browser-tools.md.