Light, fluffy, and always free - The AWS Local Emulator alternative
feat(bedrock): add proxy backend for real LLM responses via OpenAI-compatible API (#1789)
Oleh Melnyk875e0f2d3b63
diff --git a/docs/services/bedrock-runtime.md b/docs/services/bedrock-runtime.mdindex bd1fdbd372..43bfbca00c 100644--- a/docs/services/bedrock-runtime.md+++ b/docs/services/bedrock-runtime.md@@ -3,7 +3,12 @@**Protocol:** REST JSON**Endpoint:** `POST http://localhost:4566/model/{modelId}/...`-Floci emulates the AWS Bedrock Runtime data-plane API with a dummy response stub. The response shape matches the real AWS Converse and InvokeModel contracts so AWS SDK and CLI clients accept the reply without error. No real model inference is performed: every call returns a fixed assistant turn plus synthetic token usage metadata.+Floci emulates the AWS Bedrock Runtime data-plane API. The response shape matches the real AWS Converse and InvokeModel contracts so AWS SDK and CLI clients accept the reply without error.++Two backends are available, selected via `floci.services.bedrock-runtime.backend`:++- `stub` (default) — no real model inference: every call returns a fixed assistant turn plus synthetic token usage metadata.+- `proxy` — forwards `Converse` requests to any OpenAI-compatible `/chat/completions` endpoint (Ollama, OpenRouter, LiteLLM, vLLM, ...), translating request/response shapes including tool use. `InvokeModel` still falls back to the stub response in this mode. See [Proxy Backend](#proxy-backend) below.The Bedrock management plane (`aws bedrock ...`: `ListFoundationModels`, `GetFoundationModel`, customization) is not yet emulated.@@ -11,22 +16,41 @@ The Bedrock management plane (`aws bedrock ...`: `ListFoundationModels`, `GetFou| Operation | Endpoint | Notes ||-----------|----------|-------|-| `Converse` | `POST /model/{modelId}/converse` | Returns static assistant message |-| `InvokeModel` | `POST /model/{modelId}/invoke` | Returns Anthropic-shaped body for `anthropic.*` and `*.anthropic.*` model ids; generic `{"outputs": [...]}` shape otherwise |+| `Converse` | `POST /model/{modelId}/converse` | `stub`: returns a static assistant message. `proxy`: forwards to the configured OpenAI-compatible backend and translates the reply. |+| `InvokeModel` | `POST /model/{modelId}/invoke` | Returns Anthropic-shaped body for `anthropic.*` and `*.anthropic.*` model ids; generic `{"outputs": [...]}` shape otherwise. Not proxied yet — always the stub response, regardless of backend. || `ConverseStream` | `POST /model/{modelId}/converse-stream` | Returns 501 `UnsupportedOperationException` || `InvokeModelWithResponseStream` | `POST /model/{modelId}/invoke-with-response-stream` | Returns 501 `UnsupportedOperationException` |`modelId` is URL-decoded by JAX-RS and echoed verbatim. Plain model ids (e.g. `anthropic.claude-3-haiku-20240307-v1:0`), inference-profile ids (e.g. `us.anthropic.claude-3-5-sonnet-20241022-v2:0`), and full ARNs containing slashes (e.g. `arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0`) are all accepted.-Converse accepts `messages`, `system`, `inferenceConfig`, and `toolConfig` fields. Only `messages` is validated (non-empty array). Other fields are accepted and ignored. Tool-use round-tripping is not implemented.+Converse accepts `messages`, `system`, `inferenceConfig`, and `toolConfig` fields. Only `messages` is validated (non-empty array). With the `stub` backend, other fields are accepted and ignored, and tool-use round-tripping is not implemented. With the `proxy` backend, `system`, `toolConfig.tools` and `toolConfig.toolChoice`, and `inferenceConfig` (`maxTokens`, `temperature`, `topP`, `stopSequences`) are translated and forwarded (see below).-InvokeModel bodies are passed through as opaque bytes; the stub does not parse request payloads.+InvokeModel bodies are passed through as opaque bytes; neither backend parses request payloads.## Configuration| Variable | Default | Description ||---|---|---|-| `FLOCI_SERVICES_BEDROCKRUNTIME_ENABLED` | `true` | Enable or disable the service |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_ENABLED` | `true` | Enable or disable the service |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_BACKEND` | `stub` | `stub` or `proxy` |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_URL` | _(none)_ | Base URL of the OpenAI-compatible backend, e.g. `http://localhost:11434/v1`. Required when `backend=proxy`. |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_API_KEY` | _(none)_ | Sent as `Authorization: Bearer {value}` when set |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_DEFAULT_MODEL` | _(none)_ | Fallback OpenAI-side model id when no mapping matches and passthrough is disabled |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_MODEL_MAPPING` | _(none)_ | Comma-separated `bedrockModelId=openaiModelId` pairs, e.g. `anthropic.claude-3-sonnet-20240229-v1:0=claude-3-sonnet` |+| `FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_PASSTHROUGH` | `false` | When true, forward the raw Bedrock model id as-is if no mapping matches |++## Proxy Backend++Model resolution order for an incoming `modelId`: explicit `model-mapping` entry, then (if `passthrough=true`) the raw `modelId` as-is, then `default-model`, else a `400 ValidationException`.++```bash+# Ollama (local, no API key needed)+export FLOCI_SERVICES_BEDROCK_RUNTIME_BACKEND=proxy+export FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_URL=http://localhost:11434/v1+export FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_DEFAULT_MODEL=llama3.2+```++Useful for integration testing: applications that call Bedrock directly don't need to stand up their own stub/mock of Bedrock — point the existing SDK client at Floci and get real responses from whatever OpenAI-compatible backend is already available, with the stubbing concern handled by Floci instead of the application under test.## Examples@@ -62,8 +86,8 @@ print(resp["output"]["message"]["content"][0]["text"])## Out of Scope-- Real model inference (always returns a fixed string).-- Streaming (`ConverseStream`, `InvokeModelWithResponseStream`) return 501.+- Real model inference with the `stub` backend (always returns a fixed string).+- `InvokeModel` against the `proxy` backend (still returns the stub response).+- Streaming (`ConverseStream`, `InvokeModelWithResponseStream`) return 501, regardless of backend.- Bedrock management plane (`ListFoundationModels`, `GetFoundationModel`, model customisation).- Bedrock Agents, Knowledge Bases, Guardrails, provisioned throughput.-- Tool-use round-tripping in Converse.diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.javaindex 04ffa2f705..7cc50c1f9f 100644--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java@@ -1164,6 +1164,58 @@ public interface EmulatorConfig {interface BedrockRuntimeServiceConfig {@WithDefault("true")boolean enabled();++ /**+ * Converse/InvokeModel backend: "stub" (default, hardcoded response, no+ * external calls) or "proxy" (forwards Converse to an OpenAI-compatible+ * /chat/completions endpoint; see {@link BedrockProxyConfig}).+ */+ @WithDefault("stub")+ String backend();++ BedrockProxyConfig proxy();+ }++ interface BedrockProxyConfig {+ /**+ * Base URL of the OpenAI-compatible backend (Ollama, OpenRouter, LiteLLM,+ * vLLM), e.g. "http://localhost:11434/v1". Required when backend=proxy;+ * requests are POSTed to "{url}/chat/completions".+ */+ Optional<String> url();++ /** Sent as "Authorization: Bearer {apiKey}" when present. */+ Optional<String> apiKey();++ /**+ * Fallback OpenAI-side model id used when no explicit mapping matches+ * and passthrough is disabled.+ */+ Optional<String> defaultModel();++ /**+ * Comma-separated {@code bedrockModelId=openaiModelId} pairs, e.g.+ * {@code "anthropic.claude-3-sonnet-20240229-v1:0=claude-3-sonnet"}.+ * A delimited string rather than a native Map config property: Bedrock+ * model ids contain '.' and ':', which collide with SmallRye's per-key+ * env-var naming convention for maps.+ */+ Optional<String> modelMapping();++ /**+ * When true, and no explicit mapping matches, forward the raw Bedrock+ * model id as-is instead of requiring a mapping or defaultModel.+ */+ @WithDefault("false")+ boolean passthrough();++ /**+ * How long to wait for the backend to finish generating a response before+ * failing the request with ModelTimeoutException. Larger models on+ * CPU-backed backends (e.g. Ollama) may need more than the default.+ */+ @WithDefault("60")+ int requestTimeoutSeconds();}interface TextractServiceConfig {diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeController.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeController.javaindex 78c9c97103..2884bc4f8f 100644--- a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeController.java+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeController.java@@ -4,6 +4,7 @@ import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.ObjectNode;import io.github.hectorvent.floci.core.common.AwsException;+import io.smallrye.common.annotation.Blocking;import jakarta.inject.Inject;import jakarta.ws.rs.Consumes;import jakarta.ws.rs.POST;@@ -41,6 +42,7 @@ public class BedrockRuntimeController {}@POST+ @Blocking@Path("/{modelId:.+}/converse")public Response converse(@PathParam("modelId") String modelId, String body) {if (modelId == null || modelId.isBlank()) {@@ -62,12 +64,13 @@ public class BedrockRuntimeController {"messages is required and must be a non-empty array.", 400);}- ObjectNode response = service.buildConverseResponse(modelId);+ ObjectNode response = service.buildConverseResponse(modelId, (ObjectNode) request);LOG.debugv("Bedrock Converse: modelId={0}, messages={1}", modelId, messages.size());return Response.ok(response).build();}@POST+ @Blocking@Path("/{modelId:.+}/invoke")@Consumes(MediaType.WILDCARD)public Response invokeModel(@PathParam("modelId") String modelId, byte[] body) {@@ -75,7 +78,7 @@ public class BedrockRuntimeController {throw new AwsException("ValidationException", "modelId is required.", 400);}// Bedrock InvokeModel bodies are model-specific opaque blobs; do not parse.- byte[] response = service.buildInvokeModelResponse(modelId);+ byte[] response = service.buildInvokeModelResponse(modelId, body);LOG.debugv("Bedrock InvokeModel: modelId={0}, bodyBytes={1}",modelId, body == null ? 0 : body.length);return Response.ok(response, MediaType.APPLICATION_JSON).build();@@ -86,7 +89,7 @@ public class BedrockRuntimeController {@Consumes(MediaType.WILDCARD)public Response invokeModelWithResponseStream(@PathParam("modelId") String modelId) {throw new AwsException("UnsupportedOperationException",- "InvokeModelWithResponseStream is not supported by the Floci stub. "+ "InvokeModelWithResponseStream is not supported by Floci yet. "+ "Use InvokeModel or Converse instead.", 501);}@@ -95,7 +98,7 @@ public class BedrockRuntimeController {@Consumes(MediaType.WILDCARD)public Response converseStream(@PathParam("modelId") String modelId) {throw new AwsException("UnsupportedOperationException",- "ConverseStream is not supported by the Floci stub. "+ "ConverseStream is not supported by Floci yet. "+ "Use Converse instead.", 501);}}diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeService.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeService.javaindex fc41d8d490..58ea435708 100644--- a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeService.java+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockRuntimeService.java@@ -1,75 +1,51 @@package io.github.hectorvent.floci.services.bedrockruntime;-import com.fasterxml.jackson.databind.ObjectMapper;-import com.fasterxml.jackson.databind.node.ArrayNode;import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.services.bedrockruntime.backend.BedrockBackend;+import io.github.hectorvent.floci.services.bedrockruntime.backend.ProxyBackend;+import io.github.hectorvent.floci.services.bedrockruntime.backend.StubBackend;import jakarta.enterprise.context.ApplicationScoped;import jakarta.inject.Inject;+import org.jboss.logging.Logger;/**- * Dummy response builder for Bedrock Runtime. Stateless.- * No real model inference: returns a fixed assistant turn plus token usage metadata.+ * Thin orchestration layer for Bedrock Runtime: delegates Converse/InvokeModel to the+ * backend selected by {@code floci.services.bedrock-runtime.backend}.*/@ApplicationScopedpublic class BedrockRuntimeService {- private final ObjectMapper objectMapper;+ private static final Logger LOG = Logger.getLogger(BedrockRuntimeService.class);++ private final EmulatorConfig config;+ private final StubBackend stubBackend;+ private final ProxyBackend proxyBackend;@Inject- public BedrockRuntimeService(ObjectMapper objectMapper) {- this.objectMapper = objectMapper;+ public BedrockRuntimeService(EmulatorConfig config, StubBackend stubBackend, ProxyBackend proxyBackend) {+ this.config = config;+ this.stubBackend = stubBackend;+ this.proxyBackend = proxyBackend;}- public ObjectNode buildConverseResponse(String modelId) {- ObjectNode root = objectMapper.createObjectNode();-- ObjectNode output = root.putObject("output");- ObjectNode message = output.putObject("message");- message.put("role", "assistant");- ArrayNode content = message.putArray("content");- ObjectNode textBlock = content.addObject();- textBlock.put("text", "Floci stub response for model=" + modelId);-- root.put("stopReason", "end_turn");-- ObjectNode usage = root.putObject("usage");- usage.put("inputTokens", 10);- usage.put("outputTokens", 12);- usage.put("totalTokens", 22);-- ObjectNode metrics = root.putObject("metrics");- metrics.put("latencyMs", 1);+ public ObjectNode buildConverseResponse(String modelId, ObjectNode bedrockRequest) {+ return backend().converse(modelId, bedrockRequest);+ }- return root;+ public byte[] buildInvokeModelResponse(String modelId, byte[] body) {+ return backend().invokeModel(modelId, body);}- public byte[] buildInvokeModelResponse(String modelId) {- ObjectNode root = objectMapper.createObjectNode();- String lower = modelId == null ? "" : modelId.toLowerCase();- if (lower.startsWith("anthropic.") || lower.contains(".anthropic.")) {- root.put("id", "msg_stub");- root.put("type", "message");- root.put("role", "assistant");- ArrayNode content = root.putArray("content");- ObjectNode block = content.addObject();- block.put("type", "text");- block.put("text", "Floci stub response");- root.put("model", modelId);- root.put("stop_reason", "end_turn");- ObjectNode usage = root.putObject("usage");- usage.put("input_tokens", 10);- usage.put("output_tokens", 12);- } else {- // Generic minimal shape for Meta, Mistral, Titan and others.- // Bedrock returns provider-specific bodies; callers parse by model family.- ArrayNode outputs = root.putArray("outputs");- ObjectNode item = outputs.addObject();- item.put("text", "Floci stub response");+ private BedrockBackend backend() {+ String backend = config.services().bedrockRuntime().backend();+ if ("proxy".equalsIgnoreCase(backend)) {+ return proxyBackend;}- try {- return objectMapper.writeValueAsBytes(root);- } catch (Exception e) {- throw new RuntimeException("Failed to serialize InvokeModel response", e);+ if (!"stub".equalsIgnoreCase(backend)) {+ LOG.warnv("Unrecognized floci.services.bedrock-runtime.backend value \"{0}\"; falling back to stub. "+ + "Expected \"stub\" or \"proxy\".", backend);}+ return stubBackend;}}diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockBackend.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockBackend.javanew file mode 100644index 0000000000..a4211fb833--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockBackend.java@@ -0,0 +1,14 @@+package io.github.hectorvent.floci.services.bedrockruntime.backend;++import com.fasterxml.jackson.databind.node.ObjectNode;++/**+ * Executes Bedrock Runtime Converse/InvokeModel requests against a concrete backend+ * (a hardcoded stub, or a proxy to a real OpenAI-compatible model endpoint).+ */+public interface BedrockBackend {++ ObjectNode converse(String modelId, ObjectNode bedrockRequest);++ byte[] invokeModel(String modelId, byte[] body);+}diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockOpenAiTranslator.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockOpenAiTranslator.javanew file mode 100644index 0000000000..183d472d76--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/BedrockOpenAiTranslator.java@@ -0,0 +1,279 @@+package io.github.hectorvent.floci.services.bedrockruntime.backend;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ArrayNode;+import com.fasterxml.jackson.databind.node.ObjectNode;+import org.jboss.logging.Logger;++/**+ * Translates between the Bedrock Runtime Converse wire shape and the OpenAI+ * Chat Completions wire shape used by Ollama, OpenRouter, LiteLLM, vLLM, etc.+ */+final class BedrockOpenAiTranslator {++ private static final Logger LOG = Logger.getLogger(BedrockOpenAiTranslator.class);++ private BedrockOpenAiTranslator() {+ }++ static ObjectNode toOpenAiRequest(ObjectMapper mapper, ObjectNode bedrockRequest, String resolvedModel) {+ ObjectNode openAi = mapper.createObjectNode();+ openAi.put("model", resolvedModel);+ openAi.put("stream", false);+ ArrayNode openAiMessages = openAi.putArray("messages");++ JsonNode system = bedrockRequest.path("system");+ if (system.isArray()) {+ for (JsonNode block : system) {+ String text = block.path("text").asText(null);+ if (text != null) {+ ObjectNode msg = openAiMessages.addObject();+ msg.put("role", "system");+ msg.put("content", text);+ }+ }+ }++ JsonNode messages = bedrockRequest.path("messages");+ if (messages.isArray()) {+ for (JsonNode message : messages) {+ translateMessage(mapper, openAiMessages, message);+ }+ }++ JsonNode toolConfig = bedrockRequest.path("toolConfig");+ JsonNode tools = toolConfig.path("tools");+ if (tools.isArray() && !tools.isEmpty()) {+ ArrayNode openAiTools = openAi.putArray("tools");+ for (JsonNode tool : tools) {+ JsonNode toolSpec = tool.path("toolSpec");+ if (!toolSpec.isObject()) {+ continue;+ }+ ObjectNode function = putFunctionRef(openAiTools.addObject(), toolSpec.path("name").asText(""));+ if (toolSpec.hasNonNull("description")) {+ function.put("description", toolSpec.path("description").asText());+ }+ JsonNode schema = toolSpec.path("inputSchema").path("json");+ if (schema.isObject()) {+ function.set("parameters", schema.deepCopy());+ }+ }++ // "auto" needs no explicit output: it's OpenAI's default whenever tools are present.+ JsonNode toolChoice = toolConfig.path("toolChoice");+ if (toolChoice.hasNonNull("any")) {+ openAi.put("tool_choice", "required");+ } else if (toolChoice.hasNonNull("tool")) {+ String toolName = toolChoice.path("tool").path("name").asText("");+ if (!toolName.isBlank()) {+ putFunctionRef(openAi.putObject("tool_choice"), toolName);+ }+ }+ }++ JsonNode inferenceConfig = bedrockRequest.path("inferenceConfig");+ JsonNode maxTokens = inferenceConfig.path("maxTokens");+ if (maxTokens.isNumber()) {+ openAi.put("max_tokens", maxTokens.asInt());+ }+ JsonNode temperature = inferenceConfig.path("temperature");+ if (temperature.isNumber()) {+ openAi.put("temperature", temperature.asDouble());+ }+ JsonNode topP = inferenceConfig.path("topP");+ if (topP.isNumber()) {+ openAi.put("top_p", topP.asDouble());+ }+ JsonNode stopSequences = inferenceConfig.path("stopSequences");+ if (stopSequences.isArray() && !stopSequences.isEmpty()) {+ openAi.set("stop", stopSequences.deepCopy());+ }++ return openAi;+ }++ /** Builds the OpenAI {@code {type: "function", function: {name}}} shape shared by tools[] entries and tool_choice. */+ private static ObjectNode putFunctionRef(ObjectNode target, String name) {+ target.put("type", "function");+ return target.putObject("function").put("name", name);+ }++ /**+ * A Bedrock message's content[] blocks map to OpenAI in three different ways:+ * plain text accumulates into the message's content string, toolUse blocks become+ * entries in that same message's tool_calls[], and toolResult blocks become their+ * own standalone OpenAI message with role=tool — so one Bedrock message can expand+ * into zero, one, or several OpenAI messages.+ */+ private static void translateMessage(ObjectMapper mapper, ArrayNode openAiMessages, JsonNode message) {+ String role = message.path("role").asText("user");+ JsonNode content = message.path("content");+ StringBuilder text = new StringBuilder();+ ArrayNode toolCalls = null;++ if (content.isArray()) {+ for (JsonNode block : content) {+ if (block.has("toolResult")) {+ JsonNode toolResult = block.path("toolResult");+ ObjectNode toolMsg = openAiMessages.addObject();+ toolMsg.put("role", "tool");+ toolMsg.put("tool_call_id", toolResult.path("toolUseId").asText(""));+ toolMsg.put("content", extractToolResultText(toolResult.path("content")));+ continue;+ }+ if (block.has("toolUse")) {+ JsonNode toolUse = block.path("toolUse");+ if (toolCalls == null) {+ toolCalls = mapper.createArrayNode();+ }+ ObjectNode toolCall = toolCalls.addObject();+ toolCall.put("id", toolUse.path("toolUseId").asText(""));+ toolCall.put("type", "function");+ ObjectNode function = toolCall.putObject("function");+ function.put("name", toolUse.path("name").asText(""));+ function.put("arguments", toJsonString(mapper, toolUse.path("input")));+ continue;+ }+ String blockText = block.path("text").asText(null);+ if (blockText != null) {+ if (text.length() > 0) {+ text.append('\n');+ }+ text.append(blockText);+ }+ }+ }++ if (text.length() > 0 || toolCalls != null) {+ ObjectNode msg = openAiMessages.addObject();+ msg.put("role", role);+ if (text.length() > 0) {+ msg.put("content", text.toString());+ } else {+ // OpenAI's spec treats content as optional when tool_calls is present; some+ // strict backends reject an empty string here instead of null.+ msg.putNull("content");+ }+ if (toolCalls != null) {+ msg.set("tool_calls", toolCalls);+ }+ }+ }++ private static String extractToolResultText(JsonNode toolResultContent) {+ if (!toolResultContent.isArray()) {+ return "";+ }+ StringBuilder sb = new StringBuilder();+ for (JsonNode part : toolResultContent) {+ String text = part.path("text").asText(null);+ if (text == null && part.has("json")) {+ text = part.path("json").toString();+ }+ if (text != null) {+ if (sb.length() > 0) {+ sb.append('\n');+ }+ sb.append(text);+ }+ }+ return sb.toString();+ }++ private static String toJsonString(ObjectMapper mapper, JsonNode node) {+ try {+ return mapper.writeValueAsString(node);+ } catch (Exception e) {+ LOG.warnv("Failed to serialize toolUse input as JSON: {0}", e.getMessage());+ return "{}";+ }+ }++ static ObjectNode toBedrockResponse(ObjectMapper mapper, JsonNode openAiResponse, long latencyMs) {+ JsonNode choice = openAiResponse.path("choices").path(0);+ JsonNode message = choice.path("message");+ String finishReason = choice.path("finish_reason").asText("stop");+ JsonNode toolCallsNode = message.path("tool_calls");+ boolean hasToolCalls = toolCallsNode.isArray() && !toolCallsNode.isEmpty();++ ObjectNode root = mapper.createObjectNode();+ ObjectNode output = root.putObject("output");+ ObjectNode outMessage = output.putObject("message");+ outMessage.put("role", "assistant");+ ArrayNode content = outMessage.putArray("content");++ if (hasToolCalls) {+ // Some OpenAI-compatible backends emit assistant text alongside tool_calls (e.g. a+ // preamble like "Let me check that for you") — preserve it as a leading text block+ // instead of dropping it.+ String leadingText = extractMessageText(message);+ if (!leadingText.isBlank()) {+ content.addObject().put("text", leadingText);+ }+ for (JsonNode toolCall : toolCallsNode) {+ ObjectNode toolUse = content.addObject().putObject("toolUse");+ toolUse.put("toolUseId", toolCall.path("id").asText(""));+ JsonNode function = toolCall.path("function");+ toolUse.put("name", function.path("name").asText(""));+ toolUse.set("input", parseToolArguments(mapper, function.path("arguments").asText("{}")));+ }+ } else {+ content.addObject().put("text", extractMessageText(message));+ }++ // stopReason is derived from whether we actually emitted toolUse content, not from+ // finish_reason alone: a backend can report finish_reason=tool_calls without a usable+ // tool_calls array, which must not produce a Converse response claiming tool_use with+ // no toolUse blocks in it.+ root.put("stopReason", hasToolCalls ? "tool_use" : mapFinishReason(finishReason));++ JsonNode usage = openAiResponse.path("usage");+ ObjectNode usageOut = root.putObject("usage");+ usageOut.put("inputTokens", usage.path("prompt_tokens").asInt(0));+ usageOut.put("outputTokens", usage.path("completion_tokens").asInt(0));+ usageOut.put("totalTokens", usage.path("total_tokens").asInt(0));++ root.putObject("metrics").put("latencyMs", latencyMs);++ return root;+ }++ private static String extractMessageText(JsonNode message) {+ JsonNode content = message.path("content");+ if (content.isTextual()) {+ return content.asText("");+ }+ if (content.isArray()) {+ StringBuilder sb = new StringBuilder();+ for (JsonNode part : content) {+ String text = part.path("text").asText(null);+ if (text != null) {+ if (sb.length() > 0) {+ sb.append('\n');+ }+ sb.append(text);+ }+ }+ return sb.toString();+ }+ return "";+ }++ private static ObjectNode parseToolArguments(ObjectMapper mapper, String argumentsJson) {+ try {+ JsonNode parsed = mapper.readTree(argumentsJson);+ if (parsed.isObject()) {+ return (ObjectNode) parsed;+ }+ } catch (Exception e) {+ LOG.warnv("Failed to parse tool_call arguments as JSON: {0}", e.getMessage());+ }+ return mapper.createObjectNode();+ }++ private static String mapFinishReason(String finishReason) {+ return "length".equals(finishReason) ? "max_tokens" : "end_turn";+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/ProxyBackend.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/ProxyBackend.javanew file mode 100644index 0000000000..6d9436c3a5--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/ProxyBackend.java@@ -0,0 +1,165 @@+package io.github.hectorvent.floci.services.bedrockruntime.backend;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.core.common.AwsException;+import jakarta.enterprise.context.ApplicationScoped;+import jakarta.inject.Inject;+import org.jboss.logging.Logger;++import java.net.URI;+import java.net.http.HttpClient;+import java.net.http.HttpRequest;+import java.net.http.HttpResponse;+import java.net.http.HttpTimeoutException;+import java.time.Duration;+import java.util.LinkedHashMap;+import java.util.Map;++/**+ * Forwards Bedrock Converse requests to any OpenAI-compatible {@code /chat/completions}+ * endpoint (Ollama, OpenRouter, LiteLLM, vLLM, ...). InvokeModel is not yet supported and+ * fails fast rather than returning a fabricated response.+ */+@ApplicationScoped+public class ProxyBackend implements BedrockBackend {++ private static final Logger LOG = Logger.getLogger(ProxyBackend.class);++ private final ObjectMapper objectMapper;+ private final EmulatorConfig config;++ // Config is immutable for the process's lifetime, so the mapping string is parsed+ // once here rather than on every Converse request.+ private final Map<String, String> modelMapping;++ private final HttpClient httpClient = HttpClient.newBuilder()+ .version(HttpClient.Version.HTTP_1_1)+ .connectTimeout(Duration.ofSeconds(10))+ .build();++ @Inject+ public ProxyBackend(ObjectMapper objectMapper, EmulatorConfig config) {+ this.objectMapper = objectMapper;+ this.config = config;+ this.modelMapping = parseModelMapping(config.services().bedrockRuntime().proxy().modelMapping().orElse(""));+ }++ @Override+ public ObjectNode converse(String modelId, ObjectNode bedrockRequest) {+ EmulatorConfig.BedrockProxyConfig proxyConfig = config.services().bedrockRuntime().proxy();+ String resolvedModel = resolveModel(modelId, proxyConfig);+ String baseUrl = proxyConfig.url()+ .filter(url -> !url.isBlank())+ .orElseThrow(() -> new AwsException("ValidationException",+ "floci.services.bedrock-runtime.proxy.url is required when backend=proxy.", 400));++ ObjectNode openAiRequest = BedrockOpenAiTranslator.toOpenAiRequest(objectMapper, bedrockRequest, resolvedModel);++ URI uri;+ HttpRequest.Builder builder;+ try {+ uri = URI.create(stripTrailingSlash(baseUrl) + "/chat/completions");+ builder = HttpRequest.newBuilder()+ .uri(uri)+ .timeout(Duration.ofSeconds(proxyConfig.requestTimeoutSeconds()))+ .header("Content-Type", "application/json");+ } catch (IllegalArgumentException e) {+ throw new AwsException("ValidationException",+ "floci.services.bedrock-runtime.proxy.url is not a valid URL: " + e.getMessage(), 400);+ }+ proxyConfig.apiKey()+ .filter(key -> !key.isBlank())+ .ifPresent(key -> builder.header("Authorization", "Bearer " + key));++ String requestBody;+ try {+ requestBody = objectMapper.writeValueAsString(openAiRequest);+ } catch (Exception e) {+ throw new AwsException("InternalServerException", "Failed to serialize proxy request: " + e.getMessage(), 500);+ }+ builder.POST(HttpRequest.BodyPublishers.ofString(requestBody));++ long start = System.nanoTime();+ HttpResponse<String> response;+ try {+ response = httpClient.send(builder.build(), HttpResponse.BodyHandlers.ofString());+ } catch (HttpTimeoutException e) {+ LOG.warnv("Bedrock proxy backend timed out: modelId={0}, url={1}, error={2}", modelId, uri, e.getMessage());+ throw new AwsException("ModelTimeoutException", "Proxy backend timed out: " + e.getMessage(), 408);+ } catch (Exception e) {+ LOG.warnv("Bedrock proxy backend call failed: modelId={0}, url={1}, error={2}", modelId, uri, e.getMessage());+ throw new AwsException("ModelErrorException", "Failed to reach proxy backend: " + e.getMessage(), 424);+ }+ long latencyMs = (System.nanoTime() - start) / 1_000_000;++ if (response.statusCode() >= 300) {+ LOG.warnv("Bedrock proxy backend returned HTTP {0}: {1}", response.statusCode(), response.body());+ throw new AwsException("ModelErrorException",+ "Proxy backend returned HTTP " + response.statusCode() + ": " + truncate(response.body(), 512),+ 424);+ }++ JsonNode openAiResponse;+ try {+ openAiResponse = objectMapper.readTree(response.body());+ } catch (Exception e) {+ throw new AwsException("ModelErrorException", "Proxy backend returned malformed JSON: " + e.getMessage(), 424);+ }++ return BedrockOpenAiTranslator.toBedrockResponse(objectMapper, openAiResponse, latencyMs);+ }++ @Override+ public byte[] invokeModel(String modelId, byte[] body) {+ throw new AwsException("ValidationException",+ "InvokeModel is not supported by the bedrock-runtime proxy backend; use Converse.", 400);+ }++ String resolveModel(String bedrockModelId, EmulatorConfig.BedrockProxyConfig proxyConfig) {+ String mapped = modelMapping.get(bedrockModelId);+ if (mapped != null) {+ return mapped;+ }+ if (proxyConfig.passthrough()) {+ return bedrockModelId;+ }+ if (proxyConfig.defaultModel().isPresent()) {+ return proxyConfig.defaultModel().get();+ }+ throw new AwsException("ValidationException",+ "No model mapping found for: " + bedrockModelId+ + ". Set FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_MODEL_MAPPING or "+ + "FLOCI_SERVICES_BEDROCK_RUNTIME_PROXY_DEFAULT_MODEL", 400);+ }++ static Map<String, String> parseModelMapping(String raw) {+ if (raw == null || raw.isBlank()) {+ return Map.of();+ }+ Map<String, String> result = new LinkedHashMap<>();+ for (String pair : raw.split(",")) {+ String trimmed = pair.trim();+ if (trimmed.isEmpty()) {+ continue;+ }+ int eq = trimmed.indexOf('=');+ if (eq <= 0 || eq == trimmed.length() - 1) {+ LOG.warnv("Ignoring malformed bedrock-runtime proxy model-mapping entry: {0}", trimmed);+ continue;+ }+ result.put(trimmed.substring(0, eq).trim(), trimmed.substring(eq + 1).trim());+ }+ return result;+ }++ private static String stripTrailingSlash(String url) {+ return url.endsWith("/") ? url.substring(0, url.length() - 1) : url;+ }++ private static String truncate(String value, int maxLength) {+ return value != null && value.length() > maxLength ? value.substring(0, maxLength) + "…" : value;+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/StubBackend.java b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/StubBackend.javanew file mode 100644index 0000000000..cb2e84d9fd--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/bedrockruntime/backend/StubBackend.java@@ -0,0 +1,77 @@+package io.github.hectorvent.floci.services.bedrockruntime.backend;++import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ArrayNode;+import com.fasterxml.jackson.databind.node.ObjectNode;+import jakarta.enterprise.context.ApplicationScoped;+import jakarta.inject.Inject;++/**+ * Dummy response builder for Bedrock Runtime. Stateless.+ * No real model inference: returns a fixed assistant turn plus token usage metadata.+ */+@ApplicationScoped+public class StubBackend implements BedrockBackend {++ private final ObjectMapper objectMapper;++ @Inject+ public StubBackend(ObjectMapper objectMapper) {+ this.objectMapper = objectMapper;+ }++ @Override+ public ObjectNode converse(String modelId, ObjectNode bedrockRequest) {+ ObjectNode root = objectMapper.createObjectNode();++ ObjectNode output = root.putObject("output");+ ObjectNode message = output.putObject("message");+ message.put("role", "assistant");+ ArrayNode content = message.putArray("content");+ ObjectNode textBlock = content.addObject();+ textBlock.put("text", "Floci stub response for model=" + modelId);++ root.put("stopReason", "end_turn");++ ObjectNode usage = root.putObject("usage");+ usage.put("inputTokens", 10);+ usage.put("outputTokens", 12);+ usage.put("totalTokens", 22);++ ObjectNode metrics = root.putObject("metrics");+ metrics.put("latencyMs", 1);++ return root;+ }++ @Override+ public byte[] invokeModel(String modelId, byte[] body) {+ ObjectNode root = objectMapper.createObjectNode();+ String lower = modelId == null ? "" : modelId.toLowerCase();+ if (lower.startsWith("anthropic.") || lower.contains(".anthropic.")) {+ root.put("id", "msg_stub");+ root.put("type", "message");+ root.put("role", "assistant");+ ArrayNode content = root.putArray("content");+ ObjectNode block = content.addObject();+ block.put("type", "text");+ block.put("text", "Floci stub response");+ root.put("model", modelId);+ root.put("stop_reason", "end_turn");+ ObjectNode usage = root.putObject("usage");+ usage.put("input_tokens", 10);+ usage.put("output_tokens", 12);+ } else {+ // Generic minimal shape for Meta, Mistral, Titan and others.+ // Bedrock returns provider-specific bodies; callers parse by model family.+ ArrayNode outputs = root.putArray("outputs");+ ObjectNode item = outputs.addObject();+ item.put("text", "Floci stub response");+ }+ try {+ return objectMapper.writeValueAsBytes(root);+ } catch (Exception e) {+ throw new RuntimeException("Failed to serialize InvokeModel response", e);+ }+ }+}diff --git a/src/main/resources/application.yml b/src/main/resources/application.ymlindex 0d3171b465..c21932c621 100644--- a/src/main/resources/application.yml+++ b/src/main/resources/application.yml@@ -366,6 +366,13 @@ floci:keep-running-on-shutdown: falsebedrock-runtime:enabled: true+ backend: stub+ proxy:+ passthrough: false+ # url: http://localhost:11434/v1+ # api-key: sk-...+ # default-model: llama3+ # model-mapping: "anthropic.claude-3-sonnet-20240229-v1:0=claude-3-sonnet"eks:enabled: truemock: falsediff --git a/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyIntegrationTest.javanew file mode 100644index 0000000000..67ca258a21--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyIntegrationTest.java@@ -0,0 +1,721 @@+package io.github.hectorvent.floci.services.bedrockruntime;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.sun.net.httpserver.Headers;+import com.sun.net.httpserver.HttpServer;+import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.QuarkusTestProfile;+import io.quarkus.test.junit.TestProfile;+import jakarta.inject.Inject;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.io.IOException;+import java.net.InetSocketAddress;+import java.nio.charset.StandardCharsets;+import java.util.Map;+import java.util.concurrent.atomic.AtomicReference;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.equalTo;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNotNull;+import static org.junit.jupiter.api.Assertions.assertTrue;++/**+ * Verifies the Bedrock Runtime {@code proxy} backend translates Converse requests to+ * OpenAI Chat Completions and translates the response back to the Bedrock shape,+ * against a real (mock) OpenAI-compatible backend.+ */+@QuarkusTest+@TestProfile(BedrockProxyIntegrationTest.ProxyBackendProfile.class)+class BedrockProxyIntegrationTest {++ private static final String AUTH_HEADER =+ "AWS4-HMAC-SHA256 Credential=AKID/20260101/us-east-1/bedrock/aws4_request";+ private static final int PORT = 18930;+ private static final String MAPPED_MODEL_ID = "anthropic.claude-3-haiku-20240307-v1:0";++ @Inject+ ObjectMapper objectMapper;++ private HttpServer backend;+ private final AtomicReference<String> nextResponseBody = new AtomicReference<>();+ private final AtomicReference<Integer> nextResponseStatus = new AtomicReference<>(200);+ private final AtomicReference<RecordedRequest> received = new AtomicReference<>();++ private record RecordedRequest(String path, Headers headers, byte[] body) {}++ @BeforeEach+ void setUp() throws IOException {+ backend = HttpServer.create(new InetSocketAddress("127.0.0.1", PORT), 0);+ backend.createContext("/v1/chat/completions", exchange -> {+ byte[] reqBody = exchange.getRequestBody().readAllBytes();+ received.set(new RecordedRequest(exchange.getRequestURI().getPath(), exchange.getRequestHeaders(), reqBody));++ byte[] resp = nextResponseBody.get().getBytes(StandardCharsets.UTF_8);+ exchange.getResponseHeaders().add("Content-Type", "application/json");+ exchange.sendResponseHeaders(nextResponseStatus.get(), resp.length);+ exchange.getResponseBody().write(resp);+ exchange.close();+ });+ backend.start();+ }++ @AfterEach+ void tearDown() {+ backend.stop(0);+ }++ @Test+ void converse_basicTextResponse() {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "Hello from the proxy backend!"}+ }],+ "usage": {"prompt_tokens": 7, "completion_tokens": 9, "total_tokens": 16}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("output.message.role", equalTo("assistant"))+ .body("output.message.content[0].text", equalTo("Hello from the proxy backend!"))+ .body("stopReason", equalTo("end_turn"))+ .body("usage.inputTokens", equalTo(7))+ .body("usage.outputTokens", equalTo(9))+ .body("usage.totalTokens", equalTo(16));+ }++ @Test+ void converse_toolCallsTranslateToToolUse() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "tool_calls",+ "message": {+ "role": "assistant",+ "tool_calls": [{+ "id": "call_abc123",+ "type": "function",+ "function": {+ "name": "get_weather",+ "arguments": "{\\"city\\":\\"Berlin\\"}"+ }+ }]+ }+ }],+ "usage": {"prompt_tokens": 5, "completion_tokens": 8, "total_tokens": 13}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "weather in Berlin?"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "description": "Get current weather for a city",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }]+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("output.message.role", equalTo("assistant"))+ .body("output.message.content[0].toolUse.toolUseId", equalTo("call_abc123"))+ .body("output.message.content[0].toolUse.name", equalTo("get_weather"))+ .body("output.message.content[0].toolUse.input.city", equalTo("Berlin"))+ .body("stopReason", equalTo("tool_use"))+ .body("usage.inputTokens", equalTo(5))+ .body("usage.outputTokens", equalTo(8));++ RecordedRequest r = received.get();+ assertNotNull(r);+ assertEquals("Bearer test-key", r.headers().getFirst("Authorization"));+ JsonNode sentRequest = objectMapper.readTree(r.body());+ assertEquals("claude-3-haiku", sentRequest.path("model").asText());+ assertEquals("get_weather", sentRequest.path("tools").get(0).path("function").path("name").asText());+ assertEquals("object", sentRequest.path("tools").get(0).path("function").path("parameters").path("type").asText());+ }++ @Test+ void converse_mixedTextAndToolCallsPreservesBothBlocks() {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "tool_calls",+ "message": {+ "role": "assistant",+ "content": "Let me check the weather for you.",+ "tool_calls": [{+ "id": "call_abc123",+ "type": "function",+ "function": {+ "name": "get_weather",+ "arguments": "{\\"city\\":\\"Kyiv\\"}"+ }+ }]+ }+ }],+ "usage": {"prompt_tokens": 5, "completion_tokens": 8, "total_tokens": 13}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "weather in Kyiv?"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "description": "Get current weather for a city",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }]+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("output.message.content[0].text", equalTo("Let me check the weather for you."))+ .body("output.message.content[1].toolUse.toolUseId", equalTo("call_abc123"))+ .body("output.message.content[1].toolUse.name", equalTo("get_weather"))+ .body("output.message.content[1].toolUse.input.city", equalTo("Kyiv"))+ .body("stopReason", equalTo("tool_use"));+ }++ @Test+ void converse_toolCallsFinishReasonWithoutArray_fallsBackToText() {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "tool_calls",+ "message": {"role": "assistant"}+ }],+ "usage": {"prompt_tokens": 2, "completion_tokens": 2, "total_tokens": 4}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("stopReason", equalTo("end_turn"))+ .body("output.message.content[0].text", equalTo(""));+ }++ @Test+ void converse_arrayContentResponse_extractsText() {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": [{"type": "text", "text": "array-shaped reply"}]}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("output.message.content[0].text", equalTo("array-shaped reply"));+ }++ @Test+ void converse_multiTurnToolResult_forwardsAsOpenAiToolMessage() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "it is 18C and sunny"}+ }],+ "usage": {"prompt_tokens": 3, "completion_tokens": 3, "total_tokens": 6}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [+ {"role": "user", "content": [{"text": "weather in Berlin?"}]},+ {"role": "assistant", "content": [{"toolUse": {"toolUseId": "call_1", "name": "get_weather", "input": {"city": "Berlin"}}}]},+ {"role": "user", "content": [{"toolResult": {"toolUseId": "call_1", "content": [{"text": "18C sunny"}], "status": "success"}}]}+ ]+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentMessages = objectMapper.readTree(received.get().body()).path("messages");+ boolean hasAssistantToolCall = false;+ boolean hasToolResultMessage = false;+ for (JsonNode m : sentMessages) {+ if ("assistant".equals(m.path("role").asText()) && m.path("tool_calls").isArray()+ && "get_weather".equals(m.path("tool_calls").get(0).path("function").path("name").asText())) {+ hasAssistantToolCall = true;+ }+ if ("tool".equals(m.path("role").asText()) && "call_1".equals(m.path("tool_call_id").asText())+ && m.path("content").asText().contains("18C sunny")) {+ hasToolResultMessage = true;+ }+ }+ assertTrue(hasAssistantToolCall, "expected an assistant message with tool_calls forwarded");+ assertTrue(hasToolResultMessage, "expected a tool-role message with the toolResult content forwarded");+ }++ @Test+ void converse_assistantToolCallOnlyMessageHasNullContent() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [+ {"role": "user", "content": [{"text": "weather in Berlin?"}]},+ {"role": "assistant", "content": [{"toolUse": {"toolUseId": "call_1", "name": "get_weather", "input": {"city": "Berlin"}}}]}+ ]+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentMessages = objectMapper.readTree(received.get().body()).path("messages");+ JsonNode assistantMessage = null;+ for (JsonNode m : sentMessages) {+ if ("assistant".equals(m.path("role").asText())) {+ assistantMessage = m;+ }+ }+ assertNotNull(assistantMessage, "expected an assistant message with the tool call");+ assertTrue(assistantMessage.path("content").isNull(),+ "content should be null (not empty string) when the message has only tool calls");+ }++ @Test+ void converse_proxyErrorBodyIsTruncated() {+ nextResponseBody.set("x".repeat(1000));+ nextResponseStatus.set(500);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(424)+ .body("message", org.hamcrest.Matchers.allOf(+ org.hamcrest.Matchers.containsString("…"),+ org.hamcrest.Matchers.not(org.hamcrest.Matchers.containsString("x".repeat(1000)))));+ }++ @Test+ void converse_omitsToolsWhenNoToolConfig() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "no tools here"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertTrue(sentRequest.path("tools").isMissingNode());+ }++ @Test+ void converse_forwardsForcedToolChoice() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "tool_calls",+ "message": {+ "role": "assistant",+ "tool_calls": [{+ "id": "call_1",+ "type": "function",+ "function": {"name": "get_weather", "arguments": "{\\"city\\":\\"Berlin\\"}"}+ }]+ }+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "weather in Berlin?"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }],+ "toolChoice": {"tool": {"name": "get_weather"}}+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertEquals("function", sentRequest.path("tool_choice").path("type").asText());+ assertEquals("get_weather", sentRequest.path("tool_choice").path("function").path("name").asText());+ }++ @Test+ void converse_forwardsAnyToolChoiceAsRequired() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }],+ "toolChoice": {"any": {}}+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertEquals("required", sentRequest.path("tool_choice").asText());+ }++ @Test+ void converse_omitsToolChoiceWhenNotSpecified() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }]+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertFalse(sentRequest.has("tool_choice"));+ }++ @Test+ void converse_forwardsInferenceConfig() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "inferenceConfig": {"maxTokens": 100, "temperature": 0.3, "topP": 0.9}+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertEquals(100, sentRequest.path("max_tokens").asInt());+ assertEquals(0.3, sentRequest.path("temperature").asDouble());+ assertEquals(0.9, sentRequest.path("top_p").asDouble());+ }++ @Test+ void converse_omitsInferenceConfigFieldsWhenAbsent() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertFalse(sentRequest.has("max_tokens"));+ assertFalse(sentRequest.has("temperature"));+ assertFalse(sentRequest.has("top_p"));+ }++ @Test+ void converse_nullOrWrongTypeInferenceConfigFieldsAreSkipped() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "inferenceConfig": {"maxTokens": null, "temperature": "high", "topP": true}+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertFalse(sentRequest.has("max_tokens"));+ assertFalse(sentRequest.has("temperature"));+ assertFalse(sentRequest.has("top_p"));+ }++ @Test+ void converse_forwardsStopSequences() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "inferenceConfig": {"stopSequences": ["</tool>", "STOP"]}+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertEquals("</tool>", sentRequest.path("stop").get(0).asText());+ assertEquals("STOP", sentRequest.path("stop").get(1).asText());+ }++ @Test+ void converse_nullOrMissingToolChoiceNameIsIgnored() throws IOException {+ nextResponseBody.set("""+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """);++ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {+ "messages": [{"role": "user", "content": [{"text": "hi"}]}],+ "toolConfig": {+ "tools": [{+ "toolSpec": {+ "name": "get_weather",+ "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}}}+ }+ }],+ "toolChoice": {"tool": {}}+ }+ }+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(200);++ JsonNode sentRequest = objectMapper.readTree(received.get().body());+ assertFalse(sentRequest.has("tool_choice"));+ }++ @Test+ void invokeModel_notSupportedByProxyBackend_returns400() {+ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("{}")+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/invoke")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ValidationException"));+ }++ @Test+ void converse_noMappingNoDefaultNoPassthrough_returns400() {+ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/unmapped.model-v1:0/converse")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ValidationException"));+ }++ public static final class ProxyBackendProfile implements QuarkusTestProfile {+ @Override+ public Map<String, String> getConfigOverrides() {+ return Map.of(+ "floci.services.bedrock-runtime.backend", "proxy",+ "floci.services.bedrock-runtime.proxy.url", "http://127.0.0.1:" + PORT + "/v1",+ "floci.services.bedrock-runtime.proxy.api-key", "test-key",+ "floci.services.bedrock-runtime.proxy.model-mapping",+ MAPPED_MODEL_ID + "=claude-3-haiku"+ );+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyInvalidUrlIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyInvalidUrlIntegrationTest.javanew file mode 100644index 0000000000..913693bf5e--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyInvalidUrlIntegrationTest.java@@ -0,0 +1,49 @@+package io.github.hectorvent.floci.services.bedrockruntime;++import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.QuarkusTestProfile;+import io.quarkus.test.junit.TestProfile;+import org.junit.jupiter.api.Test;++import java.util.Map;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.equalTo;++/**+ * Verifies a syntactically-invalid {@code proxy.url} surfaces as a clean+ * 400 ValidationException rather than an unmapped IllegalArgumentException.+ */+@QuarkusTest+@TestProfile(BedrockProxyInvalidUrlIntegrationTest.InvalidUrlProfile.class)+class BedrockProxyInvalidUrlIntegrationTest {++ private static final String AUTH_HEADER =+ "AWS4-HMAC-SHA256 Credential=AKID/20260101/us-east-1/bedrock/aws4_request";++ @Test+ void converse_malformedProxyUrl_returns400() {+ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/anthropic.claude-3-haiku-20240307-v1:0/converse")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ValidationException"));+ }++ public static final class InvalidUrlProfile implements QuarkusTestProfile {+ @Override+ public Map<String, String> getConfigOverrides() {+ return Map.of(+ "floci.services.bedrock-runtime.backend", "proxy",+ "floci.services.bedrock-runtime.proxy.url", "not a valid url",+ "floci.services.bedrock-runtime.proxy.passthrough", "true"+ );+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyPassthroughIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyPassthroughIntegrationTest.javanew file mode 100644index 0000000000..65df8e3fb4--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyPassthroughIntegrationTest.java@@ -0,0 +1,101 @@+package io.github.hectorvent.floci.services.bedrockruntime;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.sun.net.httpserver.HttpServer;+import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.QuarkusTestProfile;+import io.quarkus.test.junit.TestProfile;+import jakarta.inject.Inject;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.io.IOException;+import java.net.InetSocketAddress;+import java.nio.charset.StandardCharsets;+import java.util.Map;+import java.util.concurrent.atomic.AtomicReference;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.equalTo;+import static org.junit.jupiter.api.Assertions.assertEquals;++/**+ * Verifies that with {@code proxy.passthrough=true} the raw Bedrock model id is forwarded+ * as-is to the OpenAI-compatible backend, bypassing the mapping/default-model requirement.+ * Kept in a separate class/profile since {@code @TestProfile} config overrides require a+ * fresh Quarkus application context, one per distinct override set.+ */+@QuarkusTest+@TestProfile(BedrockProxyPassthroughIntegrationTest.PassthroughProfile.class)+class BedrockProxyPassthroughIntegrationTest {++ private static final String AUTH_HEADER =+ "AWS4-HMAC-SHA256 Credential=AKID/20260101/us-east-1/bedrock/aws4_request";+ private static final int PORT = 18931;+ private static final String RAW_MODEL_ID = "meta.llama3-8b-instruct-v1:0";++ @Inject+ ObjectMapper objectMapper;++ private HttpServer backend;+ private final AtomicReference<byte[]> receivedBody = new AtomicReference<>();++ @BeforeEach+ void setUp() throws IOException {+ backend = HttpServer.create(new InetSocketAddress("127.0.0.1", PORT), 0);+ backend.createContext("/v1/chat/completions", exchange -> {+ receivedBody.set(exchange.getRequestBody().readAllBytes());++ byte[] resp = """+ {+ "choices": [{+ "finish_reason": "stop",+ "message": {"role": "assistant", "content": "passthrough ok"}+ }],+ "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}+ }+ """.getBytes(StandardCharsets.UTF_8);+ exchange.getResponseHeaders().add("Content-Type", "application/json");+ exchange.sendResponseHeaders(200, resp.length);+ exchange.getResponseBody().write(resp);+ exchange.close();+ });+ backend.start();+ }++ @AfterEach+ void tearDown() {+ backend.stop(0);+ }++ @Test+ void converse_passthroughSendsRawModelId() throws IOException {+ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + RAW_MODEL_ID + "/converse")+ .then()+ .statusCode(200)+ .body("output.message.content[0].text", equalTo("passthrough ok"));++ JsonNode sentRequest = objectMapper.readTree(receivedBody.get());+ assertEquals(RAW_MODEL_ID, sentRequest.path("model").asText());+ }++ public static final class PassthroughProfile implements QuarkusTestProfile {+ @Override+ public Map<String, String> getConfigOverrides() {+ return Map.of(+ "floci.services.bedrock-runtime.backend", "proxy",+ "floci.services.bedrock-runtime.proxy.url", "http://127.0.0.1:" + PORT + "/v1",+ "floci.services.bedrock-runtime.proxy.passthrough", "true"+ );+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyTimeoutIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyTimeoutIntegrationTest.javanew file mode 100644index 0000000000..bf31b5e009--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/bedrockruntime/BedrockProxyTimeoutIntegrationTest.java@@ -0,0 +1,87 @@+package io.github.hectorvent.floci.services.bedrockruntime;++import com.sun.net.httpserver.HttpServer;+import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.QuarkusTestProfile;+import io.quarkus.test.junit.TestProfile;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.io.IOException;+import java.net.InetSocketAddress;+import java.nio.charset.StandardCharsets;+import java.util.Map;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.equalTo;++/**+ * Verifies that a slow backend response is surfaced as ModelTimeoutException (408),+ * not the generic ModelErrorException (424) - the AWS SDKs' default retry strategy+ * treats 408 as retryable and 424 as terminal.+ */+@QuarkusTest+@TestProfile(BedrockProxyTimeoutIntegrationTest.SlowProxyBackendProfile.class)+class BedrockProxyTimeoutIntegrationTest {++ private static final String AUTH_HEADER =+ "AWS4-HMAC-SHA256 Credential=AKID/20260101/us-east-1/bedrock/aws4_request";+ private static final int PORT = 18931;+ private static final String MAPPED_MODEL_ID = "anthropic.claude-3-haiku-20240307-v1:0";++ private HttpServer backend;++ @BeforeEach+ void setUp() throws IOException {+ backend = HttpServer.create(new InetSocketAddress("127.0.0.1", PORT), 0);+ backend.createContext("/v1/chat/completions", exchange -> {+ try {+ Thread.sleep(2000);+ } catch (InterruptedException interrupted) {+ Thread.currentThread().interrupt();+ exchange.close();+ return;+ }+ byte[] resp = "{}".getBytes(StandardCharsets.UTF_8);+ exchange.getResponseHeaders().add("Content-Type", "application/json");+ exchange.sendResponseHeaders(200, resp.length);+ exchange.getResponseBody().write(resp);+ exchange.close();+ });+ backend.start();+ }++ @AfterEach+ void tearDown() {+ backend.stop(0);+ }++ @Test+ void converse_backendTimesOut_returnsModelTimeoutException() {+ given()+ .contentType("application/json")+ .header("Authorization", AUTH_HEADER)+ .body("""+ {"messages": [{"role": "user", "content": [{"text": "hi"}]}]}+ """)+ .when()+ .post("/model/" + MAPPED_MODEL_ID + "/converse")+ .then()+ .statusCode(408)+ .body("__type", equalTo("ModelTimeoutException"));+ }++ public static final class SlowProxyBackendProfile implements QuarkusTestProfile {+ @Override+ public Map<String, String> getConfigOverrides() {+ return Map.of(+ "floci.services.bedrock-runtime.backend", "proxy",+ "floci.services.bedrock-runtime.proxy.url", "http://127.0.0.1:" + PORT + "/v1",+ "floci.services.bedrock-runtime.proxy.model-mapping",+ MAPPED_MODEL_ID + "=claude-3-haiku",+ "floci.services.bedrock-runtime.proxy.request-timeout-seconds", "1"+ );+ }+ }+}