Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(bedrock, cognito): improved parity (#1590)

Nahuel Nucera6ecdfc4da973
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5d8cb6ed..f4184fe6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,8 +7,16 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Added
+- **Bedrock — Knowledge Base ingestion reads the bucket, and `Retrieve` retrieves** — `StartIngestionJob` answered `COMPLETE` with an all-zero statistics block without touching the data source, and `Retrieve` answered `[]` forever, so a green ingestion run was indistinguishable from a working one. The job now reads the S3 data source (honouring `inclusionPrefixes`) and reports real statistics — scanned, new versus modified, and non-UTF-8 documents as failed with a reason each; a source it cannot read at all (missing bucket, non-S3 type) lands `FAILED` with `failureReasons` instead of a fabricated success. `Retrieve` searches the ingested documents lexically and returns content, the `s3Location` of each hit and a score, `ResourceNotFoundException` for an unknown knowledge base. Reported by @bradleyhet.
+- **KMS — multi-Region keys and `ReplicateKey`** — `CreateKey` with `MultiRegion` mints an `mrk-` key carrying `MultiRegionConfiguration`, and `ReplicateKey` creates a same-id replica in another region sharing the key material, so a ciphertext from the primary decrypts against the replica. Invalid sources and occupied regions are refused as on AWS, and deleting a primary with live replicas lands `PendingReplicaDeletion`. Reported by @cringdahl.
+
### Fixed
-- **Cognito — `PreventUserExistenceErrors=ENABLED` now hides an unknown user** — an app client created with the setting still answered `UserNotFoundException` from the flows a browser reaches, so any sign-in or password-reset form backed by MiniStack told an attacker which addresses have an account. The setting was accepted, stored and echoed back by `DescribeUserPoolClient`, so the client looked correctly configured while nothing read it. The flows AWS names for the setting — `USER_PASSWORD_AUTH`, `USER_SRP_AUTH` and `ADMIN_USER_PASSWORD_AUTH` — now answer `NotAuthorizedException` with the wrong-password message, at `RespondToAuthChallenge` as well as at `InitiateAuth`, which is what closes SRP: its `InitiateAuth` returns a `PASSWORD_VERIFIER` challenge without resolving the user, so the leak sat one step later, in the step every browser SDK makes. `ForgotPassword` and `ResendConfirmationCode` answer a normal `CodeDeliveryDetails` while delivering nothing and changing no state, and `ConfirmForgotPassword` answers `CodeMismatchException`. The `Destination` those two report is now masked as AWS reports it (`j****@e****`) on the real-user path too, since a full address there would have restored the same oracle in another field. `LEGACY` clients are unchanged, and the admin *directory* operations such as `AdminGetUser` keep reporting `UserNotFoundException` — only the admin auth flow is covered, as AWS documents. `CUSTOM_AUTH` is untouched: AWS handles it by passing `UserNotFound: true` to the challenge Lambda, which MiniStack does not model yet. Reported by @fhfournier.
+- **Bedrock — guardrails guard** — `ApplyGuardrail` evaluated nothing: every input answered `action: NONE` with an empty assessment, a nonexistent guardrail id answered HTTP 200, and `Converse` ignored `guardrailConfig` entirely, so a guardrail test suite went green while letting everything through. The deterministic policies are now enforced — word policy, the sensitive-information regexes, and the pattern-matchable PII types (EMAIL, PHONE, IP/MAC address, URL, SSN, card numbers, AWS keys) with `BLOCK` and `ANONYMIZE` honoured per input/output stage — with the documented response anatomy: `GUARDRAIL_INTERVENED`, the blocked messaging as output, and the per-policy assessment. `Converse` and `ConverseStream` apply the same evaluation to both stages (`stopReason: guardrail_intervened`, guardrail trace on request), and an unknown id answers `ResourceNotFoundException` everywhere. The model-graded policies (topic, content, contextual grounding, `PROFANITY`) still need a classifier and are not evaluated. Reported by @bradleyhet.
+- **Bedrock — `InvokeAgent` no longer fabricates an `ORCHESTRATE` trace** — with `enableTrace` the canned reply was preceded by an `orchestrationTrace` describing reasoning that never ran, which made the absence of orchestration harder to detect from the one artifact a developer would check. No trace events are emitted. Reported by @bradleyhet.
+- **Cognito — `PreventUserExistenceErrors=ENABLED` hides an unknown user** — `USER_PASSWORD_AUTH`, `USER_SRP_AUTH` and `ADMIN_USER_PASSWORD_AUTH` answer `NotAuthorizedException` (at `RespondToAuthChallenge` too, which is what closes SRP), `ForgotPassword` and `ResendConfirmationCode` answer a simulated `CodeDeliveryDetails` with the masked `Destination` (`j****@e****`), and `ConfirmForgotPassword` answers `CodeMismatchException`. Admin *directory* operations keep `UserNotFoundException`, and `CUSTOM_AUTH` is untouched. Contributed by @fhfournier.
+- **Lambda — SQS event source mapping records carry `AWSTraceHeader` and the FIFO attributes** — the trace header and `MessageGroupId`/`MessageDeduplicationId`/`SequenceNumber` were dropped from the event's `attributes` map; each now rides the record when set, per the documented event shape. Reported by @future-h-takeda-g3.
+- **Request bodies assemble in linear time** — the ASGI body reader, the aws-chunked decoder, S3 `CompleteMultipartUpload` and the DSQL proxy's row serialization each rebuilt an immutable `bytes` per chunk; all four now collect and join once, taking a 95 MB `PutObject` from ~23 s to ~0.3 s. Reported by @vernonhaughton.
## [1.5.4] — 2026-08-31
diff --git a/ministack/services/bedrock_agent.py b/ministack/services/bedrock_agent.py
index eed20fd0..cb2b6955 100644
--- a/ministack/services/bedrock_agent.py
+++ b/ministack/services/bedrock_agent.py
@@ -186,6 +186,7 @@ _knowledge_bases = AccountRegionScopedDict() # kb_id -> KnowledgeBase
_data_sources = AccountRegionScopedDict() # f"{kb_id}/{ds_id}" -> DataSource
_ingestion_jobs = AccountRegionScopedDict() # f"{kb_id}/{ds_id}/{job_id}" -> IngestionJob
_kb_documents = AccountRegionScopedDict() # f"{kb_id}/{ds_id}/{doc_id}" -> Doc
+_kb_document_texts = AccountRegionScopedDict() # f"{kb_id}/{ds_id}/{doc_id}" -> {Text, Uri}
_flows = AccountRegionScopedDict() # flow_id -> Flow
_flow_versions = AccountRegionScopedDict() # f"{flow_id}/{version}" -> FlowVersion
_flow_aliases = AccountRegionScopedDict() # f"{flow_id}/{alias_id}" -> FlowAlias
@@ -196,7 +197,8 @@ _tags = AccountRegionScopedDict() # arn -> {key: value}
_ALL_STORES = [_agents, _agent_versions, _agent_aliases, _agent_action_groups,
_agent_collaborators, _agent_knowledge_bases, _knowledge_bases,
- _data_sources, _ingestion_jobs, _kb_documents, _flows,
+ _data_sources, _ingestion_jobs, _kb_documents,
+ _kb_document_texts, _flows,
_flow_versions, _flow_aliases, _prompts, _prompt_versions, _tags]
@@ -213,6 +215,7 @@ def get_state():
"agent_knowledge_bases": _agent_knowledge_bases,
"knowledge_bases": _knowledge_bases, "data_sources": _data_sources,
"ingestion_jobs": _ingestion_jobs, "kb_documents": _kb_documents,
+ "kb_document_texts": _kb_document_texts,
"flows": _flows, "flow_versions": _flow_versions,
"flow_aliases": _flow_aliases, "prompts": _prompts,
"prompt_versions": _prompt_versions, "tags": _tags,
@@ -232,6 +235,7 @@ def restore_state(data):
_data_sources.update(data.get("data_sources", {}))
_ingestion_jobs.update(data.get("ingestion_jobs", {}))
_kb_documents.update(data.get("kb_documents", {}))
+ _kb_document_texts.update(data.get("kb_document_texts", {}))
_flows.update(data.get("flows", {}))
_flow_versions.update(data.get("flow_versions", {}))
_flow_aliases.update(data.get("flow_aliases", {}))
@@ -854,9 +858,20 @@ def _update_ds(kb_id: str, ds_id: str, body) -> tuple:
def _delete_ds(kb_id: str, ds_id: str) -> tuple:
key = f"{kb_id}/{ds_id}"
- if key not in _data_sources:
+ rec = _data_sources.get(key)
+ if rec is None:
return _not_found(f"Data source {ds_id} not found.")
del _data_sources[key]
+ # dataDeletionPolicy DELETE removes the vector-store data on AWS; RETAIN
+ # (the default) leaves the ingested documents retrievable.
+ if rec.get("DataDeletionPolicy") == "DELETE":
+ prefix = f"{kb_id}/{ds_id}/"
+ for doc_key in [k for k in _kb_document_texts.keys()
+ if k.startswith(prefix)]:
+ del _kb_document_texts[doc_key]
+ for doc_key in [k for k in _kb_documents.keys()
+ if k.startswith(prefix)]:
+ del _kb_documents[doc_key]
return _json({"KnowledgeBaseId": kb_id, "DataSourceId": ds_id,
"Status": "DELETING"}, status=202)
@@ -866,33 +881,104 @@ def _delete_ds(kb_id: str, ds_id: str) -> tuple:
# ===========================================================================
+def _run_s3_ingestion(kb_id: str, ds_id: str, ds_rec: dict) -> tuple[dict, list]:
+ """Read the data source's S3 bucket and index its text documents.
+
+ Returns (statistics, failure_reasons). The statistics are real counts of
+ what was read and stored — never a fabricated success.
+ """
+ from ministack.services import s3 as s3_svc
+
+ stats = {"NumberOfDocumentsScanned": 0,
+ "NumberOfNewDocumentsIndexed": 0,
+ "NumberOfModifiedDocumentsIndexed": 0,
+ "NumberOfDocumentsDeleted": 0,
+ "NumberOfDocumentsFailed": 0,
+ "NumberOfMetadataDocumentsScanned": 0,
+ "NumberOfMetadataDocumentsModified": 0}
+ failures = []
+
+ config = ds_rec.get("DataSourceConfiguration") or {}
+ if config.get("type") != "S3":
+ failures.append(
+ f"Data source type {config.get('type')} is not ingested by "
+ "MiniStack; only S3 data sources are read.")
+ return stats, failures
+ s3_config = config.get("s3Configuration") or {}
+ bucket_arn = s3_config.get("bucketArn") or ""
+ bucket_name = bucket_arn.rsplit(":", 1)[-1]
+ prefixes = s3_config.get("inclusionPrefixes") or []
+ bucket = s3_svc._buckets.get(bucket_name)
+ if bucket is None:
+ failures.append(f"S3 bucket {bucket_name} does not exist.")
+ return stats, failures
+
+ for key in sorted(bucket["objects"].keys()):
+ if key.endswith("/"):
+ continue
+ if prefixes and not any(key.startswith(p) for p in prefixes):
+ continue
+ if key.endswith(".metadata.json"):
+ stats["NumberOfMetadataDocumentsScanned"] += 1
+ continue
+ stats["NumberOfDocumentsScanned"] += 1
+ data = s3_svc._get_object_data(bucket_name, key)
+ try:
+ text = (data or b"").decode("utf-8")
+ except UnicodeDecodeError:
+ stats["NumberOfDocumentsFailed"] += 1
+ failures.append(f"Document s3://{bucket_name}/{key} is not UTF-8 "
+ "text and was not indexed.")
+ continue
+ doc_key = f"{kb_id}/{ds_id}/{key}"
+ existing = doc_key in _kb_document_texts
+ uri = f"s3://{bucket_name}/{key}"
+ _kb_document_texts[doc_key] = {"Text": text, "Uri": uri}
+ _kb_documents[doc_key] = {
+ "DocumentIdentifier": {"dataSourceType": "S3", "s3": {"uri": uri}},
+ "Status": "INDEXED",
+ "StatusReason": "",
+ "UpdatedAt": _now_iso(),
+ }
+ if existing:
+ stats["NumberOfModifiedDocumentsIndexed"] += 1
+ else:
+ stats["NumberOfNewDocumentsIndexed"] += 1
+ return stats, failures
+
+
def _start_ingestion_job(kb_id: str, ds_id: str, body) -> tuple:
body_obj, err = _parse_body(body)
if err:
return err
- if f"{kb_id}/{ds_id}" not in _data_sources:
+ ds_rec = _data_sources.get(f"{kb_id}/{ds_id}")
+ if ds_rec is None:
return _not_found(f"Data source {ds_id} not found.")
job_id = _id("IJ")
now = _now_iso()
+ stats, failures = _run_s3_ingestion(kb_id, ds_id, ds_rec)
+ indexed = (stats["NumberOfNewDocumentsIndexed"]
+ + stats["NumberOfModifiedDocumentsIndexed"])
+ # A job that could not read its source at all is FAILED, loudly; a job that
+ # read the bucket reports what it actually indexed.
+ status = "FAILED" if (failures and not indexed
+ and not stats["NumberOfDocumentsScanned"]) else "COMPLETE"
rec = {
"DataSourceId": ds_id,
"KnowledgeBaseId": kb_id,
"IngestionJobId": job_id,
- "Status": "COMPLETE",
+ "Status": status,
"StartedAt": now,
"UpdatedAt": now,
"Description": body_obj.get("description", ""),
- "Statistics": {"NumberOfDocumentsScanned": 0,
- "NumberOfNewDocumentsIndexed": 0,
- "NumberOfModifiedDocumentsIndexed": 0,
- "NumberOfDocumentsDeleted": 0,
- "NumberOfDocumentsFailed": 0,
- "NumberOfMetadataDocumentsScanned": 0,
- "NumberOfMetadataDocumentsModified": 0},
- "FailureReasons": [],
+ "Statistics": stats,
+ "FailureReasons": failures,
}
_ingestion_jobs[f"{kb_id}/{ds_id}/{job_id}"] = rec
- return _json({"IngestionJob": rec}, status=202)
+ # The start response reports STARTING, as on AWS; the work is already done
+ # synchronously, so the first GetIngestionJob sees the final state.
+ started_view = dict(rec, Status="STARTING" if status == "COMPLETE" else status)
+ return _json({"IngestionJob": started_view}, status=202)
def _get_ingestion_job(kb_id: str, ds_id: str, job_id: str) -> tuple:
diff --git a/ministack/services/bedrock_agent_runtime.py b/ministack/services/bedrock_agent_runtime.py
index e4f7f0f6..97857eeb 100644
--- a/ministack/services/bedrock_agent_runtime.py
+++ b/ministack/services/bedrock_agent_runtime.py
@@ -235,17 +235,11 @@ def _invoke_agent(agent_id: str, alias_id: str, session_id: str, body) -> tuple:
prompt = body_obj.get("inputText", "")
end_session = bool(body_obj.get("endSession"))
reply = _mock_reply(agent_id, prompt)
- # Stream: trace -> chunk(s) -> (optional end-of-session)
+ # Stream: chunk(s) -> (optional end-of-session). No trace events are
+ # emitted, even with enableTrace: no orchestration happens here (no model
+ # call, no action-group selection), and a fabricated ORCHESTRATE trace is
+ # exactly the artifact a developer would read to conclude that it did.
stream = b""
- if body_obj.get("enableTrace"):
- stream += _es_event("trace", {
- "agentId": agent_id,
- "agentAliasId": alias_id,
- "sessionId": session_id,
- "trace": {"orchestrationTrace": {
- "modelInvocationInput": {"text": prompt, "type": "ORCHESTRATE"},
- }},
- })
# Chunk(s) — body is base64 of bytes per AWS, but for InvokeAgent the inner
# `bytes` field is the raw response text bytes encoded as a Blob (boto3
# handles base64 transparently in protocol layer for blob members).
@@ -296,10 +290,34 @@ def _retrieve(kb_id: str, body) -> tuple:
body_obj, err = _parse_body(body)
if err:
return err
+ query = (body_obj.get("retrievalQuery") or {}).get("text")
if not body_obj.get("retrievalQuery"):
return _validation("retrievalQuery is required.")
- # Return empty results — shape-correct
- return _json({"RetrievalResults": [], "NextToken": None})
+ from ministack.services import bedrock_agent as agent_svc
+ if kb_id not in agent_svc._knowledge_bases:
+ return _not_found(f"Knowledge base {kb_id} not found.")
+ number_of_results = (((body_obj.get("retrievalConfiguration") or {})
+ .get("vectorSearchConfiguration") or {})
+ .get("numberOfResults")) or 5
+ # Lexical retrieval over the documents ingestion actually stored — no
+ # embeddings locally, so scoring is term overlap: the fraction of query
+ # terms present in the document.
+ terms = {t for t in re.findall(r"[a-z0-9]+", (query or "").lower()) if t}
+ scored = []
+ for key, doc in agent_svc._kb_document_texts.items():
+ if not key.startswith(f"{kb_id}/"):
+ continue
+ doc_terms = set(re.findall(r"[a-z0-9]+", doc["Text"].lower()))
+ hits = len(terms & doc_terms)
+ if hits:
+ scored.append((hits / len(terms), doc))
+ scored.sort(key=lambda pair: pair[0], reverse=True)
+ results = [{
+ "Content": {"Text": doc["Text"], "Type": "TEXT"},
+ "Location": {"Type": "S3", "S3Location": {"Uri": doc["Uri"]}},
+ "Score": round(score, 4),
+ } for score, doc in scored[:number_of_results]]
+ return _json({"RetrievalResults": results, "NextToken": None})
def _retrieve_and_generate(body) -> tuple:
diff --git a/ministack/services/bedrock_runtime.py b/ministack/services/bedrock_runtime.py
index a07f8b57..766266d5 100644
--- a/ministack/services/bedrock_runtime.py
+++ b/ministack/services/bedrock_runtime.py
@@ -172,6 +172,16 @@ def _messages_text(messages) -> str:
return "\n".join(chunks)
+def _last_user_text(messages) -> str:
+ """The text of the newest user message — what a guardrail's INPUT stage sees."""
+ for m in reversed(messages or []):
+ if m.get("role") == "user":
+ return "\n".join(
+ c["text"] for c in m.get("content") or []
+ if isinstance(c, dict) and "text" in c)
+ return ""
+
+
def _system_text(system) -> str:
chunks = []
for s in system or []:
@@ -329,10 +339,58 @@ def _converse(model_id: str, headers, body) -> tuple:
system = body_obj.get("system", [])
inference_config = body_obj.get("inferenceConfig") or {}
started = int(time.time() * 1000)
+
+ guardrail_config = body_obj.get("guardrailConfig")
+ guardrail = None
+ if guardrail_config:
+ identifier = guardrail_config.get("guardrailIdentifier", "")
+ guardrail = _resolve_guardrail(
+ identifier, guardrail_config.get("guardrailVersion", "DRAFT"))
+ if guardrail is None:
+ return _guardrail_not_found(identifier)
+ input_result = _evaluate_guardrail(
+ guardrail, "INPUT", _last_user_text(messages))
+ if input_result["blocked"]:
+ response = _build_converse_response(
+ model_id, messages, system, started,
+ guardrail.get("BlockedInputMessaging") or "Blocked by guardrail.")
+ response["stopReason"] = "guardrail_intervened"
+ if guardrail_config.get("trace") in ("enabled", "enabled_full"):
+ response["trace"] = {"guardrail": {"inputAssessment": {
+ guardrail["GuardrailId"]: input_result["assessment"],
+ }}}
+ return 200, {
+ "Content-Type": "application/json",
+ "x-amzn-bedrock-input-token-count": str(response["usage"]["inputTokens"]),
+ "x-amzn-bedrock-output-token-count": str(response["usage"]["outputTokens"]),
+ }, json.dumps(response).encode()
+
text = _proxy_to_openai_chat(model_id, messages, system, inference_config)
if text is None:
text = _mock_reply(model_id, messages, system)
+
+ output_assessment = None
+ if guardrail is not None:
+ output_result = _evaluate_guardrail(guardrail, "OUTPUT", text)
+ output_assessment = output_result["assessment"]
+ if output_result["blocked"]:
+ text = guardrail.get("BlockedOutputsMessaging") or "Blocked by guardrail."
+ elif output_result["anonymized"]:
+ text = output_result["masked_text"]
+
response = _build_converse_response(model_id, messages, system, started, text)
+ if guardrail is not None:
+ if output_assessment and any(
+ f.get("action") == "BLOCKED"
+ for policy in output_assessment.values()
+ for entries in policy.values()
+ for f in entries
+ ):
+ response["stopReason"] = "guardrail_intervened"
+ if guardrail_config.get("trace") in ("enabled", "enabled_full"):
+ response["trace"] = {"guardrail": {"outputAssessments": {
+ guardrail["GuardrailId"]: [output_assessment or {}],
+ }}}
resp_headers = {
"Content-Type": "application/json",
"x-amzn-bedrock-input-token-count": str(response["usage"]["inputTokens"]),
@@ -376,7 +434,8 @@ def _es_event(event_type: str, payload: dict) -> bytes:
)
-def _build_converse_stream(model_id: str, messages, system, started_at_ms: int, text: str) -> bytes:
+def _build_converse_stream(model_id: str, messages, system, started_at_ms: int, text: str,
+ stop_reason: str = "end_turn") -> bytes:
"""Emit the AWS ConverseStream event sequence:
messageStart -> contentBlockDelta* -> contentBlockStop -> messageStop -> metadata
All events under :event-type, payload is application/json per AWS wire trace.
@@ -399,7 +458,7 @@ def _build_converse_stream(model_id: str, messages, system, started_at_ms: int,
"delta": {"text": ""},
})
stream += _es_event("contentBlockStop", {"contentBlockIndex": 0})
- stream += _es_event("messageStop", {"stopReason": "end_turn"})
+ stream += _es_event("messageStop", {"stopReason": stop_reason})
input_tokens = _estimate_tokens(_system_text(system) + _messages_text(messages))
output_tokens = _estimate_tokens(text)
latency_ms = max(1, int(time.time() * 1000) - started_at_ms)
@@ -429,10 +488,37 @@ def _converse_stream(model_id: str, headers, body) -> tuple:
system = body_obj.get("system", [])
inference_config = body_obj.get("inferenceConfig") or {}
started = int(time.time() * 1000)
- text = _proxy_to_openai_chat(model_id, messages, system, inference_config)
- if text is None:
- text = _mock_reply(model_id, messages, system)
- stream_bytes = _build_converse_stream(model_id, messages, system, started, text)
+
+ guardrail_config = body_obj.get("guardrailConfig")
+ guardrail = None
+ stop_reason = None
+ if guardrail_config:
+ identifier = guardrail_config.get("guardrailIdentifier", "")
+ guardrail = _resolve_guardrail(
+ identifier, guardrail_config.get("guardrailVersion", "DRAFT"))
+ if guardrail is None:
+ return _guardrail_not_found(identifier)
+ input_result = _evaluate_guardrail(
+ guardrail, "INPUT", _last_user_text(messages))
+ if input_result["blocked"]:
+ text = guardrail.get("BlockedInputMessaging") or "Blocked by guardrail."
+ stop_reason = "guardrail_intervened"
+ guardrail = None # no output pass on a blocked input
+
+ if stop_reason is None:
+ text = _proxy_to_openai_chat(model_id, messages, system, inference_config)
+ if text is None:
+ text = _mock_reply(model_id, messages, system)
+ if guardrail is not None:
+ output_result = _evaluate_guardrail(guardrail, "OUTPUT", text)
+ if output_result["blocked"]:
+ text = guardrail.get("BlockedOutputsMessaging") or "Blocked by guardrail."
+ stop_reason = "guardrail_intervened"
+ elif output_result["anonymized"]:
+ text = output_result["masked_text"]
+
+ stream_bytes = _build_converse_stream(model_id, messages, system, started, text,
+ stop_reason=stop_reason or "end_turn")
resp_headers = {
"Content-Type": "application/vnd.amazon.eventstream",
"x-amzn-bedrock-content-type": "application/json",
@@ -723,6 +809,161 @@ def _invoke_model_with_response_stream(model_id: str, headers, body) -> tuple:
}, stream_bytes
+# ---------------------------------------------------------------------------
+# Guardrail evaluation
+#
+# The deterministic policies are enforced for real: word policy (exact
+# words/phrases), the sensitive-information regexes, and the PII entity types
+# that reduce to a pattern. The model-graded policies (topic, content,
+# contextual grounding, the managed PROFANITY word list) need a classifier
+# real Bedrock runs server-side and are not evaluated — they are reported as
+# not-assessed rather than passed.
+# ---------------------------------------------------------------------------
+
+# PII entity types that are deterministically matchable. The remaining types
+# (NAME, ADDRESS, AGE, ...) need entity recognition and are not evaluated.
+_PII_PATTERNS = {
+ "EMAIL": r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}",
+ "PHONE": r"(?<![\d.])(?:\+?\d{1,3}[ .-]?)?(?:\(\d{2,4}\)[ .-]?)?\d{3}[ .-]?\d{3,4}[ .-]?\d{0,4}(?<![ .-])(?![\d.])",
+ "IP_ADDRESS": r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
+ "MAC_ADDRESS": r"\b(?:[0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}\b",
+ "URL": r"https?://[^\s<>\"]+",
+ "US_SOCIAL_SECURITY_NUMBER": r"\b\d{3}-\d{2}-\d{4}\b",
+ "CREDIT_DEBIT_CARD_NUMBER": r"\b(?:\d[ -]?){13,16}\b",
+ "AWS_ACCESS_KEY": r"\b(?:AKIA|ASIA)[0-9A-Z]{16}\b",
+ "AWS_SECRET_KEY": r"\b[A-Za-z0-9/+=]{40}\b",
+}
+
+
+def _resolve_guardrail(identifier: str, version: str):
+ """Return the guardrail record, or None. Identifier may be an ID or ARN."""
+ from ministack.services import bedrock as bedrock_svc
+ rec = bedrock_svc._guardrails.get(identifier)
+ if rec is None:
+ for r in bedrock_svc._guardrails.values():
+ if r.get("GuardrailArn") == identifier:
+ rec = r
+ break
+ if rec is None:
+ return None
+ if version and version != "DRAFT":
+ return bedrock_svc._guardrail_versions.get(
+ f"{rec['GuardrailId']}/{version}")
+ return rec
+
+
+def _guardrail_filter_applies(cfg: dict, source: str) -> tuple:
+ """Resolve a policy entry's (enabled, action) for INPUT or OUTPUT."""
+ if source == "INPUT":
+ enabled = cfg.get("inputEnabled", True)
+ action = cfg.get("inputAction") or cfg.get("action") or "BLOCK"
+ else:
+ enabled = cfg.get("outputEnabled", True)
+ action = cfg.get("outputAction") or cfg.get("action") or "BLOCK"
+ return enabled, action
+
+
+# Config action -> assessment filter action (per botocore enums)
+_GUARDRAIL_ACTION_WIRE = {"BLOCK": "BLOCKED", "ANONYMIZE": "ANONYMIZED", "NONE": "NONE"}
+
+
+def _evaluate_guardrail(rec: dict, source: str, text: str) -> dict:
+ """Evaluate the deterministic policies of a guardrail against one text.
+
+ Returns {assessment, blocked, anonymized, masked_text, word_chars,
+ sip_chars} where masked_text carries ANONYMIZE replacements.
+ """
+ custom_words = []
+ pii_findings = []
+ regex_findings = []
+ blocked = False
+ masked = text
+ word_cfgs = ((rec.get("WordPolicy") or {}).get("wordsConfig")) or []
+ for cfg in word_cfgs:
+ word = cfg.get("text") or ""
+ if not word:
+ continue
+ enabled, action = _guardrail_filter_applies(cfg, source)
+ if not enabled or action == "NONE":
+ continue
+ if re.search(rf"(?<![\w]){re.escape(word)}(?![\w])", text, re.IGNORECASE):
+ custom_words.append(
+ {"match": word, "action": "BLOCKED", "detected": True})
+ blocked = True
+
+ sip = rec.get("SensitiveInformationPolicy") or {}
+ for cfg in sip.get("regexesConfig") or []:
+ pattern = cfg.get("pattern") or ""
+ enabled, action = _guardrail_filter_applies(cfg, source)
+ if not pattern or not enabled:
+ continue
+ try:
+ matches = list(re.finditer(pattern, masked))
+ except re.error:
+ continue
+ if not matches:
+ continue
+ name = cfg.get("name") or "regex"
+ wire_action = _GUARDRAIL_ACTION_WIRE.get(action, "BLOCKED")
+ regex_findings.append({
+ "name": name,
+ "match": matches[0].group(0),
+ "regex": pattern,
+ "action": wire_action,
+ "detected": True,
+ })
+ if action == "BLOCK":
+ blocked = True
+ elif action == "ANONYMIZE":
+ masked = re.sub(pattern, "{" + name + "}", masked)
+ for cfg in sip.get("piiEntitiesConfig") or []:
+ pii_type = cfg.get("type") or ""
+ pattern = _PII_PATTERNS.get(pii_type)
+ enabled, action = _guardrail_filter_applies(cfg, source)
+ if not pattern or not enabled:
+ continue
+ matches = list(re.finditer(pattern, masked))
+ if not matches:
+ continue
+ wire_action = _GUARDRAIL_ACTION_WIRE.get(action, "BLOCKED")
+ pii_findings.append({
+ "match": matches[0].group(0),
+ "type": pii_type,
+ "action": wire_action,
+ "detected": True,
+ })
+ if action == "BLOCK":
+ blocked = True
+ elif action == "ANONYMIZE":
+ masked = re.sub(pattern, "{" + pii_type + "}", masked)
+
+ assessment = {}
+ if custom_words:
+ assessment["wordPolicy"] = {"customWords": custom_words}
+ if pii_findings or regex_findings:
+ sip_assessment = {}
+ if pii_findings:
+ sip_assessment["piiEntities"] = pii_findings
+ if regex_findings:
+ sip_assessment["regexes"] = regex_findings
+ assessment["sensitiveInformationPolicy"] = sip_assessment
+ units = max(1, -(-len(text) // 1000)) if text else 1
+ return {
+ "assessment": assessment,
+ "blocked": blocked,
+ "anonymized": masked != text,
+ "masked_text": masked,
+ "word_units": units if word_cfgs else 0,
+ "sip_units": units if (sip.get("regexesConfig")
+ or sip.get("piiEntitiesConfig")) else 0,
+ }
+
+
+def _guardrail_not_found(identifier: str) -> tuple:
+ return _error("ResourceNotFoundException",
+ f"Guardrail with identifier {identifier} not found.", 404)
+
+
# ---------------------------------------------------------------------------
# ApplyGuardrail
# ---------------------------------------------------------------------------
@@ -742,32 +983,50 @@ def _apply_guardrail(guardrail_id: str, version: str, body) -> tuple:
"source must be one of: INPUT, OUTPUT.", 400)
if not isinstance(content, list):
return _error("ValidationException", "content must be an array.", 400)
- # Mock evaluation: pass everything through unchanged.
+ rec = _resolve_guardrail(guardrail_id, version)
+ if rec is None:
+ return _guardrail_not_found(guardrail_id)
outputs = []
+ assessment = {}
+ blocked = False
+ anonymized = False
total_chars = 0
+ word_units = 0
+ sip_units = 0
for block in content:
- if isinstance(block, dict) and "text" in block:
- text = block["text"].get("text", "")
- outputs.append({"text": text})
- total_chars += len(text)
+ if not (isinstance(block, dict) and "text" in block):
+ continue
+ text = block["text"].get("text", "")
+ total_chars += len(text)
+ result = _evaluate_guardrail(rec, source, text)
+ blocked = blocked or result["blocked"]
+ anonymized = anonymized or result["anonymized"]
+ word_units += result["word_units"]
+ sip_units += result["sip_units"]
+ for policy, findings in result["assessment"].items():
+ merged = assessment.setdefault(policy, {})
+ for key, entries in findings.items():
+ merged.setdefault(key, []).extend(entries)
+ outputs.append({"text": result["masked_text"]})
+ if blocked:
+ message = (rec.get("BlockedInputMessaging") if source == "INPUT"
+ else rec.get("BlockedOutputsMessaging"))
+ outputs = [{"text": message or "Blocked by guardrail."}]
+ # Detection-only findings (config action NONE) are reported in the
+ # assessment without intervening, as on AWS.
+ intervened = blocked or anonymized
response = {
"usage": {
"topicPolicyUnits": 0,
"contentPolicyUnits": 0,
- "wordPolicyUnits": 0,
- "sensitiveInformationPolicyUnits": 0,
+ "wordPolicyUnits": word_units,
+ "sensitiveInformationPolicyUnits": sip_units,
"sensitiveInformationPolicyFreeUnits": 0,
"contextualGroundingPolicyUnits": 0,
},
- "action": "NONE",
+ "action": "GUARDRAIL_INTERVENED" if intervened else "NONE",
"outputs": outputs,
- "assessments": [{
- "topicPolicy": None,
- "contentPolicy": None,
- "wordPolicy": None,
- "sensitiveInformationPolicy": None,
- "contextualGroundingPolicy": None,
- }] if outputs else [],
+ "assessments": [assessment] if assessment else ([{}] if outputs else []),
"guardrailCoverage": {
"textCharacters": {"guarded": total_chars, "total": total_chars},
},
diff --git a/ministack/services/cognito.py b/ministack/services/cognito.py
index 21cf2360..21220314 100644
--- a/ministack/services/cognito.py
+++ b/ministack/services/cognito.py
@@ -1198,7 +1198,7 @@ def _hides_user_existence(pool: dict, client_id: str) -> bool:
does not affect.
"""
client = (pool.get("_clients") or {}).get(client_id) or {}
- return client.get("PreventUserExistenceErrors", "ENABLED") == "ENABLED"
+ return client.get("PreventUserExistenceErrors", "LEGACY") == "ENABLED"
def _hidden_user_error(pool: dict, client_id: str, err):
@@ -2463,7 +2463,7 @@ def _create_user_pool_client(data):
"AllowedOAuthScopes": data.get("AllowedOAuthScopes", []),
"AllowedOAuthFlowsUserPoolClient": data.get("AllowedOAuthFlowsUserPoolClient", False),
"AnalyticsConfiguration": data.get("AnalyticsConfiguration"),
- "PreventUserExistenceErrors": data.get("PreventUserExistenceErrors", "ENABLED"),
+ "PreventUserExistenceErrors": data.get("PreventUserExistenceErrors", "LEGACY"),
"EnableTokenRevocation": data.get("EnableTokenRevocation", True),
"EnablePropagateAdditionalUserContextData": data.get("EnablePropagateAdditionalUserContextData", False),
"AuthSessionValidity": data.get("AuthSessionValidity", 3),
diff --git a/tests/test_bedrock.py b/tests/test_bedrock.py
index 67efff1e..14e42a75 100644
--- a/tests/test_bedrock.py
+++ b/tests/test_bedrock.py
@@ -778,10 +778,20 @@ def test_bedrock_invoke_model_with_response_stream_titan_envelope():
# ---------------------------------------------------------------------------
+def _make_guardrail(name, **policies):
+ return _bedrock().create_guardrail(
+ name=name,
+ blockedInputMessaging="Input blocked by guardrail.",
+ blockedOutputsMessaging="Output blocked by guardrail.",
+ **policies,
+ )["guardrailId"]
+
+
def test_bedrock_apply_guardrail_returns_required_fields():
+ gid = _make_guardrail("gr-apply-shape")
client = make_client("bedrock-runtime")
resp = client.apply_guardrail(
- guardrailIdentifier="gr-anything",
+ guardrailIdentifier=gid,
guardrailVersion="DRAFT",
source="INPUT",
content=[{"text": {"text": "hello world"}}],
@@ -795,6 +805,125 @@ def test_bedrock_apply_guardrail_returns_required_fields():
assert resp["guardrailCoverage"]["textCharacters"]["total"] == len("hello world")
+def test_bedrock_apply_guardrail_unknown_id_is_not_found():
+ client = make_client("bedrock-runtime")
+ try:
+ client.apply_guardrail(
+ guardrailIdentifier="gr-does-not-exist",
+ guardrailVersion="DRAFT",
+ source="INPUT",
+ content=[{"text": {"text": "hello"}}],
+ )
+ raise AssertionError("expected ResourceNotFoundException")
+ except botocore.exceptions.ClientError as e:
+ assert e.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
+def test_bedrock_apply_guardrail_blocks_configured_word():
+ gid = _make_guardrail(
+ "gr-word-block",
+ wordPolicyConfig={"wordsConfig": [{"text": "pineapple"}]},
+ )
+ client = make_client("bedrock-runtime")
+ resp = client.apply_guardrail(
+ guardrailIdentifier=gid, guardrailVersion="DRAFT", source="INPUT",
+ content=[{"text": {"text": "I love Pineapple on pizza"}}],
+ )
+ assert resp["action"] == "GUARDRAIL_INTERVENED"
+ assert resp["outputs"] == [{"text": "Input blocked by guardrail."}]
+ words = resp["assessments"][0]["wordPolicy"]["customWords"]
+ assert words[0]["match"] == "pineapple"
+ assert words[0]["action"] == "BLOCKED"
+ assert resp["usage"]["wordPolicyUnits"] >= 1
+ # A clean input passes the same guardrail
+ clean = client.apply_guardrail(
+ guardrailIdentifier=gid, guardrailVersion="DRAFT", source="INPUT",
+ content=[{"text": {"text": "I love mango on pizza"}}],
+ )
+ assert clean["action"] == "NONE"
+ assert clean["outputs"][0]["text"] == "I love mango on pizza"
+
+
+def test_bedrock_apply_guardrail_anonymizes_pii_and_regex():
+ gid = _make_guardrail(
+ "gr-pii-anon",
+ sensitiveInformationPolicyConfig={
+ "piiEntitiesConfig": [{"type": "EMAIL", "action": "ANONYMIZE"}],
+ "regexesConfig": [{"name": "ticket", "pattern": r"TICKET-\d+",
+ "action": "ANONYMIZE"}],
+ },
+ )
+ client = make_client("bedrock-runtime")
+ resp = client.apply_guardrail(
+ guardrailIdentifier=gid, guardrailVersion="DRAFT", source="OUTPUT",
+ content=[{"text": {"text": "Contact [email protected] about TICKET-42"}}],
+ )
+ assert resp["action"] == "GUARDRAIL_INTERVENED"
+ assert resp["outputs"][0]["text"] == "Contact {EMAIL} about {ticket}"
+ sip = resp["assessments"][0]["sensitiveInformationPolicy"]
+ assert sip["piiEntities"][0]["type"] == "EMAIL"
+ assert sip["piiEntities"][0]["action"] == "ANONYMIZED"
+ assert sip["regexes"][0]["name"] == "ticket"
+ assert sip["regexes"][0]["action"] == "ANONYMIZED"
+
+
+def test_bedrock_apply_guardrail_blocking_pii_blocks():
+ gid = _make_guardrail(
+ "gr-pii-block",
+ sensitiveInformationPolicyConfig={
+ "piiEntitiesConfig": [{"type": "US_SOCIAL_SECURITY_NUMBER",
+ "action": "BLOCK"}],
+ },
+ )
+ client = make_client("bedrock-runtime")
+ resp = client.apply_guardrail(
+ guardrailIdentifier=gid, guardrailVersion="DRAFT", source="INPUT",
+ content=[{"text": {"text": "my ssn is 123-45-6789"}}],
+ )
+ assert resp["action"] == "GUARDRAIL_INTERVENED"
+ assert resp["outputs"] == [{"text": "Input blocked by guardrail."}]
+
+
+def test_bedrock_converse_honours_guardrail_config():
+ gid = _make_guardrail(
+ "gr-converse",
+ wordPolicyConfig={"wordsConfig": [{"text": "forbidden"}]},
+ )
+ client = make_client("bedrock-runtime")
+ blocked = client.converse(
+ modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
+ messages=[{"role": "user", "content": [{"text": "say the forbidden word"}]}],
+ guardrailConfig={"guardrailIdentifier": gid, "guardrailVersion": "DRAFT",
+ "trace": "enabled"},
+ )
+ assert blocked["stopReason"] == "guardrail_intervened"
+ assert blocked["output"]["message"]["content"][0]["text"] == \
+ "Input blocked by guardrail."
+ assessment = blocked["trace"]["guardrail"]["inputAssessment"][gid]
+ assert assessment["wordPolicy"]["customWords"][0]["match"] == "forbidden"
+
+ clean = client.converse(
+ modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
+ messages=[{"role": "user", "content": [{"text": "hello there"}]}],
+ guardrailConfig={"guardrailIdentifier": gid, "guardrailVersion": "DRAFT"},
+ )
+ assert clean["stopReason"] == "end_turn"
+
+
+def test_bedrock_converse_unknown_guardrail_is_not_found():
+ client = make_client("bedrock-runtime")
+ try:
+ client.converse(
+ modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
+ messages=[{"role": "user", "content": [{"text": "hello"}]}],
+ guardrailConfig={"guardrailIdentifier": "gr-nope",
+ "guardrailVersion": "DRAFT"},
+ )
+ raise AssertionError("expected ResourceNotFoundException")
+ except botocore.exceptions.ClientError as e:
+ assert e.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
def test_bedrock_apply_guardrail_rejects_invalid_source():
import botocore.exceptions
diff --git a/tests/test_bedrock_agent.py b/tests/test_bedrock_agent.py
index 84e9648c..26f77507 100644
--- a/tests/test_bedrock_agent.py
+++ b/tests/test_bedrock_agent.py
@@ -221,19 +221,28 @@ def test_bedrock_agent_data_source_crud():
def test_bedrock_agent_ingestion_job_lifecycle():
+ s3 = make_client("s3")
+ s3.create_bucket(Bucket="kb-ij-docs")
+ s3.put_object(Bucket="kb-ij-docs", Key="a.txt", Body=b"MiniStack runs on port 4566.")
+ s3.put_object(Bucket="kb-ij-docs", Key="b.txt", Body=b"Ingestion stores documents.")
kb = _make_kb("kb-ij")
ds = _agent().create_data_source(
knowledgeBaseId=kb["knowledgeBaseId"], name="ds-ij",
- dataSourceConfiguration={"type": "S3", "s3Configuration": {"bucketArn": "arn:aws:s3:::b"}},
+ dataSourceConfiguration={"type": "S3", "s3Configuration": {"bucketArn": "arn:aws:s3:::kb-ij-docs"}},
)["dataSource"]
job = _agent().start_ingestion_job(
knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
)["ingestionJob"]
+ assert job["status"] == "STARTING"
g = _agent().get_ingestion_job(
knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
ingestionJobId=job["ingestionJobId"],
)
- assert g["ingestionJob"]["status"] in ("STARTING", "IN_PROGRESS", "COMPLETE")
+ assert g["ingestionJob"]["status"] == "COMPLETE"
+ stats = g["ingestionJob"]["statistics"]
+ assert stats["numberOfDocumentsScanned"] == 2
+ assert stats["numberOfNewDocumentsIndexed"] == 2
+ assert stats["numberOfDocumentsFailed"] == 0
lst = _agent().list_ingestion_jobs(
knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
)
@@ -241,6 +250,59 @@ def test_bedrock_agent_ingestion_job_lifecycle():
for j in lst["ingestionJobSummaries"])
+def test_bedrock_agent_ingestion_job_missing_bucket_fails_loudly():
+ kb = _make_kb("kb-ij-missing")
+ ds = _agent().create_data_source(
+ knowledgeBaseId=kb["knowledgeBaseId"], name="ds-ij-missing",
+ dataSourceConfiguration={"type": "S3", "s3Configuration": {
+ "bucketArn": "arn:aws:s3:::kb-ij-no-such-bucket"}},
+ )["dataSource"]
+ job = _agent().start_ingestion_job(
+ knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
+ )["ingestionJob"]
+ assert job["status"] == "FAILED"
+ g = _agent().get_ingestion_job(
+ knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
+ ingestionJobId=job["ingestionJobId"],
+ )["ingestionJob"]
+ assert g["status"] == "FAILED"
+ assert g["failureReasons"]
+ assert "kb-ij-no-such-bucket" in g["failureReasons"][0]
+ assert g["statistics"]["numberOfDocumentsScanned"] == 0
+
+
+def test_bedrock_agent_ingestion_reindex_counts_modified_and_failed():
+ s3 = make_client("s3")
+ s3.create_bucket(Bucket="kb-ij-mixed")
+ s3.put_object(Bucket="kb-ij-mixed", Key="doc.txt", Body=b"Readable text.")
+ s3.put_object(Bucket="kb-ij-mixed", Key="blob.bin", Body=b"\xff\xfe\x00\x01binary")
+ kb = _make_kb("kb-ij-mixed")
+ ds = _agent().create_data_source(
+ knowledgeBaseId=kb["knowledgeBaseId"], name="ds-ij-mixed",
+ dataSourceConfiguration={"type": "S3", "s3Configuration": {"bucketArn": "arn:aws:s3:::kb-ij-mixed"}},
+ )["dataSource"]
+
+ def run():
+ job = _agent().start_ingestion_job(
+ knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
+ )["ingestionJob"]
+ return _agent().get_ingestion_job(
+ knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"],
+ ingestionJobId=job["ingestionJobId"],
+ )["ingestionJob"]
+
+ first = run()
+ assert first["status"] == "COMPLETE"
+ assert first["statistics"]["numberOfDocumentsScanned"] == 2
+ assert first["statistics"]["numberOfNewDocumentsIndexed"] == 1
+ assert first["statistics"]["numberOfDocumentsFailed"] == 1
+ assert any("blob.bin" in r for r in first["failureReasons"])
+
+ second = run()
+ assert second["statistics"]["numberOfNewDocumentsIndexed"] == 0
+ assert second["statistics"]["numberOfModifiedDocumentsIndexed"] == 1
+
+
# ---------------------------------------------------------------------------
# Flows
# ---------------------------------------------------------------------------
diff --git a/tests/test_bedrock_agent_runtime.py b/tests/test_bedrock_agent_runtime.py
index ff27329c..afde0aac 100644
--- a/tests/test_bedrock_agent_runtime.py
+++ b/tests/test_bedrock_agent_runtime.py
@@ -19,15 +19,78 @@ def _ar():
# ---------------------------------------------------------------------------
+def _make_kb(name):
+ return make_client("bedrock-agent").create_knowledge_base(
+ name=name,
+ roleArn="arn:aws:iam::000000000000:role/kb-role",
+ knowledgeBaseConfiguration={
+ "type": "VECTOR",
+ "vectorKnowledgeBaseConfiguration": {
+ "embeddingModelArn":
+ "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0",
+ },
+ },
+ storageConfiguration={
+ "type": "OPENSEARCH_SERVERLESS",
+ "opensearchServerlessConfiguration": {
+ "collectionArn": "arn:aws:aoss:us-east-1:000000000000:collection/c",
+ "vectorIndexName": "idx",
+ "fieldMapping": {"vectorField": "v", "textField": "t",
+ "metadataField": "m"},
+ },
+ },
+ )["knowledgeBase"]
+
+
def test_bedrock_ar_retrieve_returns_results_envelope():
+ kb = _make_kb("kb-ar-envelope")
resp = _ar().retrieve(
- knowledgeBaseId="KBTEST1234",
+ knowledgeBaseId=kb["knowledgeBaseId"],
retrievalQuery={"text": "what is AWS?"},
)
assert "retrievalResults" in resp
assert isinstance(resp["retrievalResults"], list)
+def test_bedrock_ar_retrieve_unknown_kb_is_not_found():
+ try:
+ _ar().retrieve(knowledgeBaseId="KBNOPE1234",
+ retrievalQuery={"text": "anything"})
+ raise AssertionError("expected ResourceNotFoundException")
+ except botocore.exceptions.ClientError as e:
+ assert e.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
+def test_bedrock_ar_retrieve_returns_ingested_documents():
+ s3 = make_client("s3")
+ s3.create_bucket(Bucket="kb-ar-docs")
+ s3.put_object(Bucket="kb-ar-docs", Key="ports.txt",
+ Body=b"MiniStack serves every service on port 4566.")
+ s3.put_object(Bucket="kb-ar-docs", Key="pets.txt",
+ Body=b"Cats sleep most of the day.")
+ kb = _make_kb("kb-ar-retrieve")
+ agent_api = make_client("bedrock-agent")
+ ds = agent_api.create_data_source(
+ knowledgeBaseId=kb["knowledgeBaseId"], name="ds-ar",
+ dataSourceConfiguration={"type": "S3", "s3Configuration": {
+ "bucketArn": "arn:aws:s3:::kb-ar-docs"}},
+ )["dataSource"]
+ agent_api.start_ingestion_job(
+ knowledgeBaseId=kb["knowledgeBaseId"], dataSourceId=ds["dataSourceId"])
+
+ resp = _ar().retrieve(
+ knowledgeBaseId=kb["knowledgeBaseId"],
+ retrievalQuery={"text": "which port does ministack serve on"},
+ )
+ results = resp["retrievalResults"]
+ assert results
+ top = results[0]
+ assert "4566" in top["content"]["text"]
+ assert top["location"]["s3Location"]["uri"] == "s3://kb-ar-docs/ports.txt"
+ assert top["location"]["type"] == "S3"
+ assert 0 < top["score"] <= 1
+
+
def test_bedrock_ar_retrieve_validation_missing_query():
try:
_ar().retrieve(knowledgeBaseId="kb-x", retrievalQuery={})
@@ -265,6 +328,24 @@ def test_bedrock_ar_invoke_agent_returns_chunk_eventstream():
assert "chunk" in names
+def test_bedrock_ar_invoke_agent_emits_no_fabricated_trace():
+ # No orchestration happens locally, so enableTrace must not produce an
+ # ORCHESTRATE trace describing reasoning that never ran.
+ resp = _ar().invoke_agent(
+ agentId="AG-2",
+ agentAliasId="AL-2",
+ sessionId="sess-trace",
+ inputText="what is the weather",
+ enableTrace=True,
+ )
+ names = []
+ for e in resp["completion"]:
+ if isinstance(e, dict):
+ names.extend(e.keys())
+ assert "chunk" in names
+ assert "trace" not in names
+
+
def test_bedrock_ar_get_and_delete_agent_memory():
resp = _ar().get_agent_memory(
agentId="AG-1", agentAliasId="AL-1",