Skip to content

Add AWS Bedrock provider (Converse + ConverseStream)#461

Open
ricardosllm wants to merge 1 commit into
editor-code-assistant:masterfrom
ricardosllm:bedrock-provider
Open

Add AWS Bedrock provider (Converse + ConverseStream)#461
ricardosllm wants to merge 1 commit into
editor-code-assistant:masterfrom
ricardosllm:bedrock-provider

Conversation

@ricardosllm
Copy link
Copy Markdown
Contributor

@ricardosllm ricardosllm commented May 14, 2026

Closes #254.

Adds a bedrock provider/API type using AWS Bedrock's native Converse and ConverseStream APIs.

Why

Bedrock's OpenAI-compatible endpoint only serves openai.* models — Claude (and other) inference profiles return 404 there, so the existing OpenAI-compat custom-provider path can't reach them. This adds a dedicated handler. (Also supersedes the partial workaround discussed in #329.)

Scope

  • Converse (non-streaming) and ConverseStream (streaming) — includes a binary vnd.amazon.eventstream frame decoder
  • Bearer-token auth (Authorization: Bearer <token>) — sidesteps SigV4; pairs with AWS_BEARER_TOKEN_BEDROCK
  • Tool callingtoolConfig / toolSpec / toolUse / toolResult blocks
  • ReasoningreasoningContent blocks; reasoning_config via additionalModelRequestFields when reason?
  • Region is part of the runtime URL; model ids / inference-profile ids are user-configured
  • extraPayload is filtered to Converse-valid top-level keys (verified against the botocore service model) so variant payloads aimed at other APIs don't 400
  • Retries on transient statuses reuse the existing error classification; modeled stream error frames terminate the turn cleanly (no spurious premature-stop)

SigV4 auth and a model-catalog endpoint are intentionally out of scope for this first cut.

Config

{
  "providers": {
    "bedrock": {
      "api": "bedrock",
      "url": "https://bedrock-runtime.us-east-1.amazonaws.com",
      "key": "${env:AWS_BEARER_TOKEN_BEDROCK}",
      "models": {
        "us.anthropic.claude-sonnet-4-5-20250929-v1:0": {}
      }
    }
  }
}

Also reachable via the /login flow (provider bedrock).

How to verify / test

Automatedbb test (suite green, 523 tests, 2881 assertions). Bedrock-specific: clojure -M:test --focus eca.llm-providers.bedrock-test covers the binary event-stream decoder (crafted frames, truncated/malformed), message normalization, extraPayload filtering, request/response shapes, both tool-call loops, streaming callbacks, and error paths.

Manual against live AWS:

  1. Generate a Bedrock API key in the AWS console (Bedrock → API keys) and export it: export AWS_BEARER_TOKEN_BEDROCK=...
  2. Build: bb debug-cli./eca
  3. Add the config above (adjust region in the URL and the model id to one enabled in your account), or use /loginbedrock.
  4. Point your editor/client at the built ./eca, select a bedrock/... model, and run a chat. Verify:
    • streaming text renders incrementally
    • a prompt that triggers a tool call round-trips (tool runs, model continues)
    • reasoning content shows when using a reasoning-capable model

This branch was integrated and smoke-tested live (non-streaming text, ConverseStream streaming, multi-turn tool-call round-trip) by a downstream consumer that embeds ECA as a library.


  • I added an entry in the changelog under the unreleased section.
  • This is not an AI slop.

🤖 Generated with Claude Code

Bedrock's OpenAI-compatible endpoint only serves openai.* models; Claude
and other inference profiles are reachable only through the native
Converse API. This adds a dedicated `bedrock` API type implementing both
Converse (non-streaming) and ConverseStream (binary event-stream framing),
with bearer-token auth, tool calling and reasoning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ricardosllm
Copy link
Copy Markdown
Contributor Author

I wrote this with Claude Opus 4.7 and reviewed as well with OpenAI GPT 5.5

It works for my usecase, here's what I tested:

Configuration. Bedrock configured as a standard provider entry:

  {:providers {"bedrock" {:api    "bedrock"
                          :url    "https://bedrock-runtime.ap-southeast-1.amazonaws.com"
                          :key    "<bearer token>"
                          :models {"global.anthropic.claude-sonnet-4-6" {}}}}}
  • Region ap-southeast-1, runtime URL as above.
  • Bearer-token auth (Authorization: Bearer) via config :key.
  • Model: global.anthropic.claude-sonnet-4-6 (also configured global.anthropic.claude-opus-4-7).
  • Driven through eca.llm-api/sync-or-async-prompt! with :provider "bedrock", the per-call :config above, and :model-capabilities — no
    :models model-config entry needed; the generic branch in llm-api/prompt! keyed on (or model-config model-capabilities) handles it.

Proven working — live against AWS Bedrock:

  • Non-streaming Converse — text response, usage parsing.
  • ConverseStream — token-by-token streaming, messageStop/end_turn, finish + usage callbacks.
  • Tool calling — multi-turn round-trip: model emits toolUse, on-tools-called returns {:new-messages :tools}, ECA re-issues, model incorporates the tool result into the final answer. The on-tools-called contract ([{:id :full-name :arguments}] in, {:new-messages :tools} out) round-trips with no adapter glue.
  • provider->api-handler's :else branch correctly routes :api "bedrock" to the native handler.
  • Streaming chat and agentic tool workstreams confirmed end-to-end.

Not covered:

  • Reasoning/thinking path (reasoningContent / redactedContent)
  • Image input (supports-image?)
  • :stream false via extra-payload (default streaming + the non-stream handler path were tested)
  • Retry/error classification under real 429/503 from AWS
  • Mid-stream cancellation
  • Parallel tool calls (one tool call per turn tested)
  • Long multi-turn conversations
  • Non-Claude Bedrock models / inference profiles beyond the global Anthropic ones

If you think this is in the right direction I can try to test what was not covered yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New LLM provider AWS bedrock - support converse and conversestream

1 participant