Reversible PII de-identification round trip (T1–T7) #1
17
.gitignore
vendored
17
.gitignore
vendored
@@ -37,3 +37,20 @@ override.tf.json
|
|||||||
.terraformrc
|
.terraformrc
|
||||||
terraform.rc
|
terraform.rc
|
||||||
|
|
||||||
|
|
||||||
|
# Terraform Lambda build artifacts
|
||||||
|
terraform/*.zip
|
||||||
|
|
||||||
|
# Local venvs
|
||||||
|
.venv*/
|
||||||
|
|
||||||
|
# AgentCore generated (env-specific)
|
||||||
|
agent/.bedrock_agentcore.yaml
|
||||||
|
agent/Dockerfile
|
||||||
|
.bedrock_agentcore/
|
||||||
|
|
||||||
|
# Python / OS noise
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.DS_Store
|
||||||
|
agent/.dockerignore
|
||||||
|
|||||||
125
CLAUDE.md
125
CLAUDE.md
@@ -60,30 +60,107 @@ fusion/ POLICY_SETUP.md (SaaS console config — human does this, not y
|
|||||||
## Task backlog (pick these up)
|
## Task backlog (pick these up)
|
||||||
Each task: keep it demo-grade, add a note in the file, and update this list.
|
Each task: keep it demo-grade, add a note in the file, and update this list.
|
||||||
|
|
||||||
- **T1 — `/tokenize` endpoint** (`gateway_api/tokenize/`). HTTPS endpoint Fusion
|
- **T1 — `/tokenize` endpoint** ✅ **done + live-verified** (`gateway_api/tokenize/handler.py`,
|
||||||
calls with `{query}`. Steps: POST query to Presidio `/analyze`; for each finding
|
terraform `terraform/gateway.tf`). Lambda behind a **public API Gateway HTTP API**
|
||||||
mint a random token (`CUST_<rand>` for PERSON→customer, `TW_<rand>` for TW_ROC_ID);
|
(not a Function URL — this account's SCP blocks unauthenticated Function URLs),
|
||||||
write vault items `{token, type, value, session_id, expires_at}`; splice tokens
|
shared-secret auth (`tokenize_api_key`, sent as `x-api-key` / bearer, checked
|
||||||
into the text. Return `{deidentified_prompt, session_id}`.
|
in-handler). Calls Presidio `/analyze`, resolves overlapping findings (specific
|
||||||
*Accept:* real name in → tokenized text + session_id out; vault rows written;
|
entity wins, so the ROC ID stays `TW_ROC_ID` not `PERSON`), mints random
|
||||||
same input yields different tokens on repeat calls.
|
`CUST_<rand>`/`TW_<rand>` tokens, writes vault rows
|
||||||
- **T2 — `/restore` endpoint** (`gateway_api/restore/`). Input `{session_id, text}`.
|
`{token, type, value, session_id, expires_at [, customer_id]}` (`value` = original
|
||||||
Look up this session's tokens in the vault, re-attach identity (envelope-level:
|
PII for /restore; `customer_id` best-effort resolved for the RAG tool), splices
|
||||||
prepend the real name; also swap any inline tokens). Return `{final}`.
|
right-to-left, returns `{deidentified_prompt, session_id}`. **Verified live** in
|
||||||
*Accept:* tokenized answer + session_id → answer with `王小明` restored.
|
`ap-southeast-1`: `王小明`→`CUST_*`, `A123456789`→`TW_*`, clean prompt, vault rows
|
||||||
- **T3 — reachability from SaaS.** Make Presidio (or fold detection into T1 so only
|
written with resolved `customer_id`, repeat→different tokens, no/bad key→401.
|
||||||
`/tokenize` + `/restore` are public) reachable from Fusion SaaS with auth.
|
Infra changes made while deploying: Presidio task bumped to **2GB** (1GB OOM'd on
|
||||||
*Accept:* Fusion SaaS can call the endpoints over HTTPS; nothing else is public.
|
the 603MB zh model → connection refused), **awslogs** added, **ARM64/Graviton**
|
||||||
- **T4 — AgentCore Gateway target.** Wire the RAG Lambda as the MCP tool; confirm
|
runtime (built on Apple Silicon via podman). All AWS objects tagged
|
||||||
the agent can call `get_customer_activity_summary(CUST_000123)` and get the package.
|
`Owner="conan hncb demo"`. *Caveats:* Presidio has no stable endpoint — its
|
||||||
*Accept:* a manual `agentcore` invoke returns the evidence package.
|
Fargate public IP changes per task launch, so `presidio_url` in
|
||||||
- **T5 — Agent runtime.** Pin deps, `agentcore launch`, capture the Runtime ARN.
|
`terraform/local.auto.tfvars` must be refreshed and the tokenize Lambda re-applied
|
||||||
*Accept:* invoking the runtime with a tokenized prompt returns talking points.
|
(an ALB/Cloud Map would fix this; out of scope for the demo). The default VPC was
|
||||||
- **T6 — UI wiring.** Point `ui/index.html` GATEWAY_URL at the Fusion SaaS entry
|
created by hand and tagged `maintenance=manual-cleanup-required` (Terraform doesn't
|
||||||
(or `/tokenize` for a Fusion-less dry run); host on S3+CloudFront.
|
own it, so `teardown.sh` won't remove it).
|
||||||
*Accept:* the split view renders restored vs tokenized.
|
- **T2 — `/restore` endpoint** ✅ **done + live-verified** (`gateway_api/restore/handler.py`,
|
||||||
- **T7 — E2E rehearsal.** Run the full script in README; confirm the Bedrock/
|
terraform in `terraform/gateway.tf`). `POST /restore` on the same API Gateway,
|
||||||
AgentCore trace shows only tokens. Then verify `teardown.sh` leaves nothing paid-for.
|
same shared-secret auth; IAM is read-only `dynamodb:Scan` on the vault. Input
|
||||||
|
`{session_id, text}` → scans the vault for THIS session's tokens (scoped, so one
|
||||||
|
session can't restore another's), swaps inline tokens back to the original PII
|
||||||
|
(longest-token-first), and prepends the customer's real name at the envelope
|
||||||
|
level (`（客戶：王小明）`). Returns `{final, restored}`. **Verified live** with the
|
||||||
|
full T1→T2 round trip: tokenized prompt out → simulated token-only cloud answer →
|
||||||
|
`王小明` + `A123456789` restored, no tokens left; no/bad key → 401; unknown
|
||||||
|
session → text unchanged, `restored:0` (no cross-session leak). `restore_url`
|
||||||
|
is a Terraform output.
|
||||||
|
- **T3 — reachability from SaaS** ✅ **done + live-verified** (`terraform/ecs.tf`,
|
||||||
|
`terraform/gateway.tf`). Presidio is now private: its SG allows 5001 only from the
|
||||||
|
tokenize Lambda's SG (no `0.0.0.0/0`). The tokenize Lambda runs **in the VPC**
|
||||||
|
(`vpc_config` + `AWSLambdaVPCAccessExecutionRole`) and reaches Presidio on its
|
||||||
|
**private** IP; DynamoDB is reached via a **gateway VPC endpoint** (no NAT). The
|
||||||
|
Fargate task keeps a public IP only to pull from ECR, but all inbound is SG-locked.
|
||||||
|
Only the API Gateway (`/tokenize` + `/restore`, shared-secret auth) is public.
|
||||||
|
**Verified live:** Presidio's public IP `:5001` now times out from the internet;
|
||||||
|
the full round trip still works through the private path. *Caveat update:* the
|
||||||
|
tokenize Lambda now targets Presidio's **private** IP (`172.31.2.216` currently),
|
||||||
|
which still changes per task launch — refresh `presidio_url` in
|
||||||
|
`terraform/local.auto.tfvars` and re-apply `aws_lambda_function.tokenize` if the
|
||||||
|
task restarts (Cloud Map/ALB would give a stable name; out of scope for the demo).
|
||||||
|
- **T4 — AgentCore Gateway target** ✅ **done + live-verified** (`terraform/agentcore.tf`,
|
||||||
|
`scripts/agentcore_setup.sh`, `scripts/gateway_invoke_test.py`). Created an MCP
|
||||||
|
Gateway (`hncb-rag-gateway`, `authorizerType=AWS_IAM` so inbound is SigV4 — no
|
||||||
|
Cognito) with the RAG Lambda registered as a `lambda` MCP target; the Gateway
|
||||||
|
assumes a Terraform-managed IAM role that can only `lambda:InvokeFunction` the RAG
|
||||||
|
tool. Built the Gateway/target with `aws bedrock-agentcore-control` (Terraform
|
||||||
|
lags here). **Verified live:** `gateway_invoke_test.py` (SigV4 MCP
|
||||||
|
initialize→tools/list→tools/call) returned the full de-identified evidence package
|
||||||
|
for `CUST_000123` through the Gateway; the tool shows up namespaced as
|
||||||
|
`rag___get_customer_activity_summary`. Gotchas handled: the Gateway `inputSchema`
|
||||||
|
is a restricted JSON-Schema subset (strip `default`/unknown keys); MCP responses
|
||||||
|
can be SSE; tool names are target-prefixed. Gateway tagged `Owner="conan hncb demo"`;
|
||||||
|
`teardown.sh` deletes it (not in Terraform state). MCP URL is printed by the setup
|
||||||
|
script. *Note:* `.venv-t4/` (gitignored) holds boto3 for the invoke test.
|
||||||
|
- **T5 — Agent runtime** ✅ **done + live-verified** (`agent/agent.py`,
|
||||||
|
`agent/requirements.txt`, `scripts/agentcore_setup.sh` T5 section). Pinned deps
|
||||||
|
(bedrock-agentcore 1.16.0 / strands-agents 1.45.0 / mcp 1.28.1 / boto3 1.43.38),
|
||||||
|
`agentcore configure --non-interactive` + `agentcore launch` (cloud CodeBuild,
|
||||||
|
auto-created execution role + ECR + STM memory). **Runtime ARN:**
|
||||||
|
`arn:aws:bedrock-agentcore:ap-southeast-1:286171702468:runtime/hncb_advisor_agent-duWZOT5Far`.
|
||||||
|
**Verified live:** `agentcore invoke {"prompt": "...CUST_000123..."}` returns
|
||||||
|
numbered talking points built only from the de-identified evidence — no name, no
|
||||||
|
leaked token. Three gotchas fixed (all captured in code/scripts):
|
||||||
|
1. In `ap-southeast-1` the model is **INFERENCE_PROFILE-only** → use
|
||||||
|
`apac.anthropic.claude-3-5-sonnet-20241022-v2:0`, not the raw id.
|
||||||
|
2. Our Gateway is **AWS_IAM**, so `agent.py` SigV4-signs MCP requests (httpx.Auth)
|
||||||
|
instead of using a bearer token.
|
||||||
|
3. The auto-created runtime **execution role can't call the Gateway** by default →
|
||||||
|
must attach `bedrock-agentcore:InvokeGateway` on the gateway ARN (the launch
|
||||||
|
step in `agentcore_setup.sh` shows the exact command). Runtime + memory added to
|
||||||
|
`teardown.sh`; agent logic was de-risked locally before launch.
|
||||||
|
- **T6 — UI wiring** ✅ **done + live-verified** (`ui/index.html`, `terraform/ui.tf`,
|
||||||
|
`gateway_api/orchestrator/`). Hosted on **S3 + CloudFront** (private bucket, OAC).
|
||||||
|
For the Fusion-less dry run I added a thin **demo orchestrator** (`POST /demo`, a
|
||||||
|
Lambda on the same API) that runs the full chain server-side —
|
||||||
|
`/tokenize` → SigV4 `InvokeAgentRuntime` → `/restore` — and returns
|
||||||
|
`{final, deidentified_prompt, agent_tokenized}` (this stands in for Fusion ONLY
|
||||||
|
for the dry run; we still never build/host Fusion). `index.html` reads the live
|
||||||
|
endpoint from a generated `config.js` (so no ephemeral URL is committed) and
|
||||||
|
renders the split view. CORS enabled on the API. **UI:**
|
||||||
|
`https://d3n89cj9w7ynf0.cloudfront.net`. **Verified live:** CloudFront serves
|
||||||
|
`index.html` + `config.js` (200); `/demo` returns `王小明`→`CUST_*` on the cloud
|
||||||
|
side and `王小明`-restored talking points on the advisor side. Only the browser
|
||||||
|
render itself is left as a human eyeball check. All resources tagged.
|
||||||
|
- **T7 — E2E rehearsal** ✅ **done + live-verified** (README demo/deploy sections
|
||||||
|
rewritten with the real commands). Ran the full round trip via `/demo`
|
||||||
|
(`王小明`→`CUST_317499` out, agent talking points on tokens only, `王小明` restored).
|
||||||
|
**Money shot proven:** in the cloud runtime CloudWatch trace for that session the
|
||||||
|
token appears in **13 events** while `王小明` and `A123456789` appear in **0** —
|
||||||
|
the Bedrock/AgentCore trace is token-only. **Teardown verified (non-destructively):**
|
||||||
|
`terraform destroy` targets 43 resources (Fargate, CloudFront, S3, DynamoDB, all
|
||||||
|
Lambdas, API GW, VPC endpoint, SGs, IAM) and `teardown.sh` additionally deletes the
|
||||||
|
Gateway/targets, Runtime, Memory, and both ECR repos — i.e. every billed resource.
|
||||||
|
Residual is free-only: the hand-made default VPC, the AgentCore SDK exec role, an
|
||||||
|
idle CodeBuild project. The actual `teardown.sh` run is left for the human to fire
|
||||||
|
after recording (didn't auto-destroy the live demo).
|
||||||
|
|
||||||
## Known caveats (don't "fix" these silently)
|
## Known caveats (don't "fix" these silently)
|
||||||
- zh-TW detection is demo-narrow (tuned to the scripted entities), not production recall.
|
- zh-TW detection is demo-narrow (tuned to the scripted entities), not production recall.
|
||||||
|
|||||||
57
README.md
57
README.md
@@ -62,30 +62,53 @@ fusion/ POLICY_SETUP.md (SaaS console config — the manual part)
|
|||||||
```
|
```
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
`aws-cli` configured (creds + region), `terraform >= 1.5`, `docker`, `python3`,
|
`aws-cli` (creds + region — this demo runs in `ap-southeast-1`), `terraform >= 1.5`,
|
||||||
the AgentCore CLI (`npm i -g @aws/agentcore`), Bedrock model access enabled for
|
`podman` or `docker` (Presidio image), `python3` + a venv with `boto3`,
|
||||||
`bedrock_model_id`, and access to the shared **Fusion SaaS** instance.
|
`bedrock-agentcore-starter-toolkit` (`pip install`, provides `agentcore`), Bedrock
|
||||||
|
access for the **apac** Claude 3.5 Sonnet v2 *inference profile*
|
||||||
|
(`apac.anthropic.claude-3-5-sonnet-20241022-v2:0` — the raw id is not on-demand in
|
||||||
|
this region), and — for the production path — the shared **Fusion SaaS** instance.
|
||||||
|
|
||||||
## Deploy
|
## Deploy
|
||||||
```bash
|
The live stack was brought up in this order (authoritative runbook: the task notes
|
||||||
bash scripts/deploy.sh # infra -> presidio image -> ECS -> seed -> AgentCore
|
in `CLAUDE.md`, which record the exact sequencing and gotchas):
|
||||||
# then: build /tokenize + /restore (gateway_api/, tasks T1/T2)
|
1. `terraform -chdir=terraform apply` core infra (DynamoDB, RAG + tokenize/restore/
|
||||||
# then: configure the shared Fusion SaaS instance (fusion/POLICY_SETUP.md)
|
orchestrator Lambdas, API Gateway, Presidio on Fargate, UI on S3+CloudFront).
|
||||||
# then: point ui/index.html GATEWAY_URL at the Fusion SaaS entrypoint and open it
|
Presidio needs its image in ECR and `presidio_url` set to the task's **private** IP.
|
||||||
```
|
2. Seed the synthetic customer + the demo vault token (`seed/seed.py`).
|
||||||
|
3. `scripts/agentcore_setup.sh` → MCP Gateway + RAG tool target.
|
||||||
|
4. `agentcore configure` + `agentcore launch` → Runtime ARN; then grant the runtime
|
||||||
|
execution role `bedrock-agentcore:InvokeGateway` (command in `agentcore_setup.sh`).
|
||||||
|
5. Put the Runtime ARN in `terraform/local.auto.tfvars` (`agent_runtime_arn`) and
|
||||||
|
re-apply so `/demo` can invoke it.
|
||||||
|
|
||||||
## Demo script (maps to the 8 steps)
|
## Demo script (Fusion-less dry run — maps to the 8 steps)
|
||||||
1. Advisor UI: submit *"請幫我整理王小明最近三個月的理財往來，並給我下次拜訪話術。"*
|
```bash
|
||||||
2. Fusion (via `/tokenize`) detects `王小明` + `A123456789`, tokenizes, logs tokens only.
|
cd terraform && DEMO=$(terraform output -raw demo_url) && UI=$(terraform output -raw ui_url); cd ..
|
||||||
3. Show the Bedrock/AgentCore trace — the prompt the cloud saw contains `CUST_000123`.
|
|
||||||
4-6. Agent tool-calls back on-prem; RAG resolves the token, returns a summary.
|
# 1-2, 4-8: advisor query -> tokenize -> agent (on tokens) -> restore, in one call:
|
||||||
7. Agent writes talking points (no PII).
|
curl -s -X POST "$DEMO" -H 'content-type: application/json' \
|
||||||
8. Fusion (via `/restore`) restores `王小明`; the UI shows restored beside tokenized.
|
-d '{"query":"請幫我整理王小明最近三個月的理財往來，並給我下次拜訪話術。"}' | python3 -m json.tool
|
||||||
|
# -> deidentified_prompt: "...CUST_xxxxxx..." (what left for the cloud)
|
||||||
|
# agent_tokenized: talking points, tokens only
|
||||||
|
# final: "（客戶：王小明）..." (identity restored on-prem)
|
||||||
|
|
||||||
|
# 3. money shot — the cloud runtime trace only ever shows the token:
|
||||||
|
LG=/aws/bedrock-agentcore/runtimes/<runtime-id>-DEFAULT
|
||||||
|
aws logs filter-log-events --log-group-name "$LG" --filter-pattern '"王小明"' --query 'length(events)' # 0
|
||||||
|
aws logs filter-log-events --log-group-name "$LG" --filter-pattern '"CUST_"' --query 'length(events)' # >0
|
||||||
|
|
||||||
|
# 8 (visual): open the split-view UI and submit the same query
|
||||||
|
echo "$UI"
|
||||||
|
```
|
||||||
|
|
||||||
## Teardown
|
## Teardown
|
||||||
```bash
|
```bash
|
||||||
bash scripts/teardown.sh # stop paying for Fargate / AgentCore
|
bash scripts/teardown.sh # deletes Gateway/Runtime/Memory + `terraform destroy` + ECR repos
|
||||||
```
|
```
|
||||||
|
Leftover-but-free after teardown: the hand-made default VPC, the
|
||||||
|
`AmazonBedrockAgentCoreSDKRuntime-*` role, and a CodeBuild project (delete by hand
|
||||||
|
if you want it spotless).
|
||||||
|
|
||||||
## Honest caveats
|
## Honest caveats
|
||||||
- **zh-TW detection is demo-narrow** — tuned to the scripted entities, not
|
- **zh-TW detection is demo-narrow** — tuned to the scripted entities, not
|
||||||
|
|||||||
@@ -6,10 +6,19 @@ CUST_000123, never a name), reasons with a Bedrock model, calls the RAG tool
|
|||||||
through AgentCore Gateway (MCP), and returns de-identified talking points.
|
through AgentCore Gateway (MCP), and returns de-identified talking points.
|
||||||
Fusion restores the real identity on the way back out -- not this agent.
|
Fusion restores the real identity on the way back out -- not this agent.
|
||||||
|
|
||||||
Framework: Strands. Deploy target: AgentCore Runtime (see scripts/agentcore_setup.py).
|
Framework: Strands. Deploy target: AgentCore Runtime (see scripts/agentcore_setup.sh).
|
||||||
Pin versions in requirements.txt; SDK surfaces move quickly.
|
Pin versions in requirements.txt; SDK surfaces move quickly.
|
||||||
|
|
||||||
|
Auth to the Gateway: our Gateway uses AWS_IAM, so every MCP request is SigV4-signed
|
||||||
|
with the Runtime's execution-role credentials (service `bedrock-agentcore`). If a
|
||||||
|
GATEWAY_TOKEN is provided instead, we fall back to bearer auth (CUSTOM_JWT gateways).
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from botocore.auth import SigV4Auth
|
||||||
|
from botocore.awsrequest import AWSRequest
|
||||||
|
from botocore.session import Session
|
||||||
from bedrock_agentcore.runtime import BedrockAgentCoreApp
|
from bedrock_agentcore.runtime import BedrockAgentCoreApp
|
||||||
from strands import Agent
|
from strands import Agent
|
||||||
from strands.models import BedrockModel
|
from strands.models import BedrockModel
|
||||||
@@ -18,9 +27,12 @@ from mcp.client.streamable_http import streamablehttp_client
|
|||||||
|
|
||||||
app = BedrockAgentCoreApp()
|
app = BedrockAgentCoreApp()
|
||||||
|
|
||||||
GATEWAY_URL = os.environ["AGENTCORE_GATEWAY_URL"] # set by agentcore_setup.py
|
GATEWAY_URL = os.environ["AGENTCORE_GATEWAY_URL"] # set by agentcore_setup.sh
|
||||||
GATEWAY_TOKEN = os.environ.get("AGENTCORE_GATEWAY_TOKEN", "")
|
GATEWAY_TOKEN = os.environ.get("AGENTCORE_GATEWAY_TOKEN", "")
|
||||||
MODEL_ID = os.environ.get("BEDROCK_MODEL_ID", "anthropic.claude-3-5-sonnet-20241022-v2:0")
|
REGION = os.environ.get("AWS_REGION", "ap-southeast-1")
|
||||||
|
# In non-US regions Claude 3.5 Sonnet v2 is INFERENCE_PROFILE-only, so default to
|
||||||
|
# the APAC cross-region profile. Override with BEDROCK_MODEL_ID at launch.
|
||||||
|
MODEL_ID = os.environ.get("BEDROCK_MODEL_ID", "apac.anthropic.claude-3-5-sonnet-20241022-v2:0")
|
||||||
|
|
||||||
SYSTEM_PROMPT = (
|
SYSTEM_PROMPT = (
|
||||||
"You are a financial-advisor assistant. You will be given a customer reference "
|
"You are a financial-advisor assistant. You will be given a customer reference "
|
||||||
@@ -32,9 +44,34 @@ SYSTEM_PROMPT = (
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _SigV4Auth(httpx.Auth):
|
||||||
|
"""SigV4-sign each MCP request with the Runtime's execution-role credentials."""
|
||||||
|
|
||||||
|
requires_request_body = True
|
||||||
|
|
||||||
|
def __init__(self, service, region):
|
||||||
|
self._creds = Session().get_credentials()
|
||||||
|
self._service = service
|
||||||
|
self._region = region
|
||||||
|
|
||||||
|
def auth_flow(self, request):
|
||||||
|
aws_req = AWSRequest(
|
||||||
|
method=request.method,
|
||||||
|
url=str(request.url),
|
||||||
|
data=request.content,
|
||||||
|
headers=dict(request.headers),
|
||||||
|
)
|
||||||
|
SigV4Auth(self._creds, self._service, self._region).add_auth(aws_req)
|
||||||
|
request.headers.update(dict(aws_req.headers))
|
||||||
|
yield request
|
||||||
|
|
||||||
|
|
||||||
def _mcp_client():
|
def _mcp_client():
|
||||||
headers = {"Authorization": f"Bearer {GATEWAY_TOKEN}"} if GATEWAY_TOKEN else {}
|
if GATEWAY_TOKEN:
|
||||||
return MCPClient(lambda: streamablehttp_client(GATEWAY_URL, headers=headers))
|
headers = {"Authorization": f"Bearer {GATEWAY_TOKEN}"}
|
||||||
|
return MCPClient(lambda: streamablehttp_client(GATEWAY_URL, headers=headers))
|
||||||
|
auth = _SigV4Auth("bedrock-agentcore", REGION)
|
||||||
|
return MCPClient(lambda: streamablehttp_client(GATEWAY_URL, auth=auth))
|
||||||
|
|
||||||
|
|
||||||
@app.entrypoint
|
@app.entrypoint
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
bedrock-agentcore
|
# Pinned before `agentcore launch` (versions resolved 2026-07-01). These SDKs move
|
||||||
strands-agents
|
# fast; re-resolve and re-pin if you rebuild.
|
||||||
mcp
|
bedrock-agentcore==1.16.0
|
||||||
boto3
|
strands-agents==1.45.0
|
||||||
|
mcp==1.28.1
|
||||||
|
boto3==1.43.38
|
||||||
|
|||||||
@@ -8,10 +8,18 @@ directly. Instead it calls two **public HTTPS endpoints** this repo exposes
|
|||||||
writes, and restore on the AWS side. Fusion owns the orchestration and routing.
|
writes, and restore on the AWS side. Fusion owns the orchestration and routing.
|
||||||
|
|
||||||
## Endpoints Fusion calls
|
## Endpoints Fusion calls
|
||||||
- `TOKENIZE_URL` = `https://<...>/tokenize` (ingress: detect + mint + vault-write + splice)
|
- `TOKENIZE_URL` = `https://<api-id>.execute-api.<region>.amazonaws.com/tokenize`
|
||||||
- `RESTORE_URL` = `https://<...>/restore` (egress: vault lookup + re-attach identity)
|
(ingress: detect + mint + vault-write + splice). Get the live value with
|
||||||
|
`terraform -chdir=terraform output -raw tokenize_url`.
|
||||||
|
- `RESTORE_URL` = `https://<...>/restore` (egress: vault lookup + re-attach identity — T2)
|
||||||
- `AGENT_RUNTIME_ARN` (or its HTTPS invoke endpoint) = printed by `scripts/agentcore_setup.sh`
|
- `AGENT_RUNTIME_ARN` (or its HTTPS invoke endpoint) = printed by `scripts/agentcore_setup.sh`
|
||||||
- Secure all three with an API key / OAuth from the Fusion outbound config.
|
- **Auth (T1, live):** the endpoint is a public API Gateway HTTP API; it requires a
|
||||||
|
shared secret in the **`x-api-key`** header (a `Authorization: Bearer <secret>`
|
||||||
|
header also works). Configure Fusion's outbound request to send it. The secret is
|
||||||
|
the Terraform `tokenize_api_key` var (kept in gitignored `terraform/local.auto.tfvars`,
|
||||||
|
never committed) — hand it to the Fusion console operator out of band.
|
||||||
|
- *Note:* a Lambda Function URL was the first choice, but this account's SCP blocks
|
||||||
|
unauthenticated Function URLs, so the public front door is API Gateway instead.
|
||||||
|
|
||||||
## Ingress policy (advisor request → cloud)
|
## Ingress policy (advisor request → cloud)
|
||||||
1. **Authenticate** the advisor; apply the RBAC / business-purpose check.
|
1. **Authenticate** the advisor; apply the RBAC / business-purpose check.
|
||||||
@@ -27,10 +35,17 @@ writes, and restore on the AWS side. Fusion owns the orchestration and routing.
|
|||||||
3. Return `{ "final": ..., "deidentified_prompt": ... }` so the UI shows the split view.
|
3. Return `{ "final": ..., "deidentified_prompt": ... }` so the UI shows the split view.
|
||||||
|
|
||||||
## Vault item shape (DynamoDB, written by /tokenize)
|
## Vault item shape (DynamoDB, written by /tokenize)
|
||||||
|
One row per detected entity. `value` is the original PII (so `/restore` can put it
|
||||||
|
back); a resolvable PERSON also gets a `customer_id` so the RAG tool can turn the
|
||||||
|
token into a de-identified evidence package.
|
||||||
```
|
```
|
||||||
{ "token": "CUST_000123", "type": "CUSTOMER", "value": "cust-0001",
|
{ "token": "CUST_863651", "type": "PERSON", "value": "王小明",
|
||||||
|
"session_id": "<conv id>", "expires_at": <epoch+ttl>, "customer_id": "cust-0001" }
|
||||||
|
{ "token": "TW_683250", "type": "TW_ROC_ID", "value": "A123456789",
|
||||||
"session_id": "<conv id>", "expires_at": <epoch+ttl> }
|
"session_id": "<conv id>", "expires_at": <epoch+ttl> }
|
||||||
```
|
```
|
||||||
|
(The seed's `--with-demo-token` writes a different `type=CUSTOMER, value=cust-0001`
|
||||||
|
row; that's only a standalone RAG test aid, not what `/tokenize` mints.)
|
||||||
|
|
||||||
## Why this split
|
## Why this split
|
||||||
Keeping detection, minting, and the vault behind `/tokenize` and `/restore` means
|
Keeping detection, minting, and the vault behind `/tokenize` and `/restore` means
|
||||||
|
|||||||
100
gateway_api/orchestrator/handler.py
Normal file
100
gateway_api/orchestrator/handler.py
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
"""
|
||||||
|
Demo orchestrator (T6) -- Fusion-less dry-run entrypoint for the UI.
|
||||||
|
|
||||||
|
Fusion is shared SaaS and configured by a human, so for a self-contained demo this
|
||||||
|
tiny endpoint stands in for Fusion's orchestration ONLY: it runs the full round trip
|
||||||
|
server-side so the static UI has one URL to call. In production Fusion does this; we
|
||||||
|
never build/host Fusion itself.
|
||||||
|
|
||||||
|
Chain (mirrors the 8 steps):
|
||||||
|
{query}
|
||||||
|
-> POST /tokenize -> {deidentified_prompt, session_id} (identity leaves as tokens)
|
||||||
|
-> InvokeAgentRuntime -> token-only talking points (cloud sees tokens only)
|
||||||
|
-> POST /restore -> {final} (identity restored on-prem)
|
||||||
|
return {final, deidentified_prompt, agent_tokenized, session_id}
|
||||||
|
|
||||||
|
The UI renders `final` (advisor view) vs `deidentified_prompt`+`agent_tokenized`
|
||||||
|
(what the cloud actually saw). Stdlib + botocore (bundled in Lambda) only.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
from botocore.auth import SigV4Auth
|
||||||
|
from botocore.awsrequest import AWSRequest
|
||||||
|
from botocore.session import Session
|
||||||
|
|
||||||
|
REGION = os.environ["REGION"]
|
||||||
|
TOKENIZE_URL = os.environ["TOKENIZE_URL"]
|
||||||
|
RESTORE_URL = os.environ["RESTORE_URL"]
|
||||||
|
API_KEY = os.environ.get("TOKENIZE_API_KEY", "")
|
||||||
|
RUNTIME_ARN = os.environ.get("AGENT_RUNTIME_ARN", "")
|
||||||
|
|
||||||
|
_creds = Session().get_credentials()
|
||||||
|
|
||||||
|
|
||||||
|
def _post_json(url, obj):
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=json.dumps(obj).encode(),
|
||||||
|
headers={"Content-Type": "application/json", "x-api-key": API_KEY}, method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=50) as r:
|
||||||
|
return json.loads(r.read().decode())
|
||||||
|
|
||||||
|
|
||||||
|
def _invoke_runtime(prompt):
|
||||||
|
"""SigV4-signed InvokeAgentRuntime data-plane call; returns the agent's text."""
|
||||||
|
url = (f"https://bedrock-agentcore.{REGION}.amazonaws.com"
|
||||||
|
f"/runtimes/{urllib.parse.quote(RUNTIME_ARN, safe='')}/invocations?qualifier=DEFAULT")
|
||||||
|
body = json.dumps({"prompt": prompt}).encode()
|
||||||
|
headers = {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": secrets.token_hex(20),
|
||||||
|
}
|
||||||
|
aws_req = AWSRequest(method="POST", url=url, data=body, headers=headers)
|
||||||
|
SigV4Auth(_creds, "bedrock-agentcore", REGION).add_auth(aws_req)
|
||||||
|
req = urllib.request.Request(url, data=body, headers=dict(aws_req.headers), method="POST")
|
||||||
|
with urllib.request.urlopen(req, timeout=120) as r:
|
||||||
|
raw = r.read().decode()
|
||||||
|
try:
|
||||||
|
obj = json.loads(raw)
|
||||||
|
return obj.get("result", raw) if isinstance(obj, dict) else raw
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def _reply(status, obj):
|
||||||
|
return {
|
||||||
|
"statusCode": status,
|
||||||
|
"headers": {"Content-Type": "application/json"},
|
||||||
|
"body": json.dumps(obj, ensure_ascii=False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def lambda_handler(event, _context):
|
||||||
|
try:
|
||||||
|
payload = json.loads(event.get("body") or "{}") if "body" in event else event
|
||||||
|
query = (payload.get("query") or "").strip()
|
||||||
|
if not query:
|
||||||
|
return _reply(400, {"error": "query is required"})
|
||||||
|
|
||||||
|
tok = _post_json(TOKENIZE_URL, {"query": query})
|
||||||
|
deid = tok["deidentified_prompt"]
|
||||||
|
session_id = tok["session_id"]
|
||||||
|
|
||||||
|
# Cloud reasoning on tokens only (skip gracefully if no runtime configured).
|
||||||
|
agent_tokenized = _invoke_runtime(deid) if RUNTIME_ARN else deid
|
||||||
|
|
||||||
|
# Restore identity into the agent's answer (envelope + inline).
|
||||||
|
res = _post_json(RESTORE_URL, {"session_id": session_id, "text": agent_tokenized})
|
||||||
|
|
||||||
|
return _reply(200, {
|
||||||
|
"final": res["final"],
|
||||||
|
"deidentified_prompt": deid,
|
||||||
|
"agent_tokenized": agent_tokenized,
|
||||||
|
"session_id": session_id,
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
return _reply(502, {"error": f"{type(e).__name__}: {e}"})
|
||||||
98
gateway_api/restore/handler.py
Normal file
98
gateway_api/restore/handler.py
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
"""
|
||||||
|
/restore endpoint (T2) -- egress re-identification for the HNCB demo (step 8).
|
||||||
|
|
||||||
|
Fusion SaaS POSTs the agent's token-only answer back here with the session_id.
|
||||||
|
We:
|
||||||
|
1. read THIS session's tokens from the on-prem vault (scoped by session_id, so
|
||||||
|
one session can never restore another's identities)
|
||||||
|
2. swap any inline tokens in the text back to the original PII
|
||||||
|
3. re-attach identity at the envelope level too: prepend the real customer name,
|
||||||
|
so the advisor sees who the answer is about even if no token appears inline
|
||||||
|
4. return {final} -- the only step where a real name comes back into view
|
||||||
|
|
||||||
|
The reversible map lives only in the vault (on-prem zone); the cloud never saw it.
|
||||||
|
Runs as a Lambda behind the same public API Gateway as /tokenize, guarded by the
|
||||||
|
same shared secret. Stdlib + boto3 only, per repo conventions.
|
||||||
|
"""
|
||||||
|
import base64
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import boto3
|
||||||
|
from boto3.dynamodb.conditions import Attr
|
||||||
|
|
||||||
|
ddb = boto3.resource("dynamodb")
|
||||||
|
VAULT = ddb.Table(os.environ["VAULT_TABLE"])
|
||||||
|
|
||||||
|
# Shared secret Fusion presents (x-api-key / bearer). Unset -> allow (dev only).
|
||||||
|
API_KEY = os.environ.get("TOKENIZE_API_KEY")
|
||||||
|
|
||||||
|
|
||||||
|
def _authorized(headers):
|
||||||
|
if not API_KEY:
|
||||||
|
return True
|
||||||
|
presented = headers.get("x-api-key") or ""
|
||||||
|
if not presented:
|
||||||
|
auth = headers.get("authorization") or ""
|
||||||
|
if auth.lower().startswith("bearer "):
|
||||||
|
presented = auth[7:]
|
||||||
|
return bool(presented) and hmac.compare_digest(presented, API_KEY)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse(event):
|
||||||
|
if "body" in event and not isinstance(event.get("body"), dict):
|
||||||
|
raw = event.get("body") or "{}"
|
||||||
|
if event.get("isBase64Encoded"):
|
||||||
|
raw = base64.b64decode(raw).decode("utf-8")
|
||||||
|
payload = json.loads(raw) if raw else {}
|
||||||
|
headers = {k.lower(): v for k, v in (event.get("headers") or {}).items()}
|
||||||
|
return payload, headers
|
||||||
|
return event, {}
|
||||||
|
|
||||||
|
|
||||||
|
def _reply(status, obj):
|
||||||
|
return {
|
||||||
|
"statusCode": status,
|
||||||
|
"headers": {"Content-Type": "application/json"},
|
||||||
|
"body": json.dumps(obj, ensure_ascii=False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _session_tokens(session_id):
|
||||||
|
# Demo-grade: scan the vault filtered by session_id (small table, no GSI).
|
||||||
|
# A production vault would query a session_id index instead.
|
||||||
|
items, kwargs = [], {"FilterExpression": Attr("session_id").eq(session_id)}
|
||||||
|
while True:
|
||||||
|
res = VAULT.scan(**kwargs)
|
||||||
|
items.extend(res.get("Items", []))
|
||||||
|
lek = res.get("LastEvaluatedKey")
|
||||||
|
if not lek:
|
||||||
|
return items
|
||||||
|
kwargs["ExclusiveStartKey"] = lek
|
||||||
|
|
||||||
|
|
||||||
|
def lambda_handler(event, _context):
|
||||||
|
payload, headers = _parse(event)
|
||||||
|
|
||||||
|
if not _authorized(headers):
|
||||||
|
return _reply(401, {"error": "unauthorized"})
|
||||||
|
|
||||||
|
session_id = (payload.get("session_id") or "").strip()
|
||||||
|
text = payload.get("text")
|
||||||
|
if not session_id or text is None:
|
||||||
|
return _reply(400, {"error": "session_id and text are required"})
|
||||||
|
|
||||||
|
tokens = _session_tokens(session_id)
|
||||||
|
|
||||||
|
# Inline: swap every token for this session back to its original value.
|
||||||
|
# Longest token first so no token is a prefix of another mid-replace.
|
||||||
|
restored = text
|
||||||
|
for it in sorted(tokens, key=lambda t: len(t["token"]), reverse=True):
|
||||||
|
restored = restored.replace(it["token"], it["value"])
|
||||||
|
|
||||||
|
# Envelope: prepend the customer's real name (the PERSON mapping), if any.
|
||||||
|
name = next((it["value"] for it in tokens if it.get("type") == "PERSON"), None)
|
||||||
|
final = f"（客戶：{name}）\n{restored}" if name else restored
|
||||||
|
|
||||||
|
return _reply(200, {"final": final, "restored": len(tokens)})
|
||||||
187
gateway_api/tokenize/handler.py
Normal file
187
gateway_api/tokenize/handler.py
Normal file
@@ -0,0 +1,187 @@
|
|||||||
|
"""
|
||||||
|
/tokenize endpoint (T1) -- ingress de-identification for the HNCB demo (step 2).
|
||||||
|
|
||||||
|
Fusion SaaS POSTs the advisor's raw query here. We:
|
||||||
|
1. send the text to the Presidio detector (/analyze) -> typed findings
|
||||||
|
2. mint a RANDOM token per finding (CUST_<rand> for a PERSON, TW_<rand> for a
|
||||||
|
Taiwan ROC ID) -- detection != redaction: the detector never sees a token,
|
||||||
|
WE own minting + the reversible map
|
||||||
|
3. write each mapping into the on-prem vault, keyed by a fresh session_id
|
||||||
|
4. splice the tokens back into the text (right-to-left, so offsets stay valid)
|
||||||
|
5. return {deidentified_prompt, session_id} -- the only thing that leaves for the cloud
|
||||||
|
|
||||||
|
Vault row shape (kept compatible with lambda_rag/handler.py):
|
||||||
|
{token, type, value, session_id, expires_at [, customer_id]}
|
||||||
|
- `value` is the ORIGINAL pii string -> lets /restore (T2) put identity back
|
||||||
|
- `customer_id` is added for a PERSON we can resolve, so the RAG tool (T4)
|
||||||
|
can turn a token into a de-identified evidence package. The RAG handler reads
|
||||||
|
entry.get("customer_id") for non-"CUSTOMER" types, so this Just Works.
|
||||||
|
|
||||||
|
Runs as a Lambda behind a Function URL (public HTTPS). Auth is a shared secret
|
||||||
|
(Fusion sends it as a bearer token / x-api-key); see TOKENIZE_API_KEY below.
|
||||||
|
Stdlib + boto3 only, per repo conventions.
|
||||||
|
"""
|
||||||
|
import base64
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import time
|
||||||
|
import urllib.request
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import boto3
|
||||||
|
|
||||||
|
ddb = boto3.resource("dynamodb")
|
||||||
|
VAULT = ddb.Table(os.environ["VAULT_TABLE"])
|
||||||
|
# Optional: resolving PERSON -> customer_id makes the vault row usable by the RAG
|
||||||
|
# tool. If the customers table isn't configured we still tokenize (best effort).
|
||||||
|
_CUSTOMERS_TABLE = os.environ.get("CUSTOMERS_TABLE")
|
||||||
|
CUSTOMERS = ddb.Table(_CUSTOMERS_TABLE) if _CUSTOMERS_TABLE else None
|
||||||
|
|
||||||
|
PRESIDIO_URL = os.environ.get("PRESIDIO_URL", "http://localhost:5001").rstrip("/")
|
||||||
|
TTL_SECONDS = int(os.environ.get("VAULT_TTL_SECONDS", "3600"))
|
||||||
|
# Shared secret Fusion presents. If unset we allow all callers (local dev only)
|
||||||
|
# and say so in the response so it's obvious this isn't locked down.
|
||||||
|
API_KEY = os.environ.get("TOKENIZE_API_KEY")
|
||||||
|
|
||||||
|
# entity type -> token prefix. Only these are tokenized in the demo.
|
||||||
|
_PREFIX = {"PERSON": "CUST", "TW_ROC_ID": "TW"}
|
||||||
|
# When Presidio returns overlapping spans (e.g. the ROC ID digits flagged as BOTH
|
||||||
|
# TW_ROC_ID and PERSON), the more specific entity wins. Higher = kept.
|
||||||
|
_PRIORITY = {"TW_ROC_ID": 2, "PERSON": 1}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_overlaps(findings):
|
||||||
|
# Keep only demo-scoped entities, then greedily drop any finding that overlaps
|
||||||
|
# one we've already kept. Best-first order = specific entity, then score, then
|
||||||
|
# longer span -- so the ROC ID stays TW_ROC_ID and the spurious PERSON on the
|
||||||
|
# same digits is discarded (no double-tokenising, no corrupted splice).
|
||||||
|
cand = [f for f in findings if f["entity_type"] in _PREFIX]
|
||||||
|
cand.sort(
|
||||||
|
key=lambda f: (_PRIORITY.get(f["entity_type"], 0), f["score"], f["end"] - f["start"]),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
kept = []
|
||||||
|
for f in cand:
|
||||||
|
if any(f["start"] < k["end"] and f["end"] > k["start"] for k in kept):
|
||||||
|
continue # overlaps a higher-priority finding already kept
|
||||||
|
kept.append(f)
|
||||||
|
return kept
|
||||||
|
|
||||||
|
|
||||||
|
def _mint(prefix):
|
||||||
|
# Random per request -> the same query yields different tokens every call.
|
||||||
|
# 6 digits mirrors the seeded demo token (CUST_000123) for a familiar look.
|
||||||
|
return f"{prefix}_{secrets.randbelow(1_000_000):06d}"
|
||||||
|
|
||||||
|
|
||||||
|
def _analyze(text, language="zh"):
|
||||||
|
body = json.dumps({"text": text, "language": language}).encode("utf-8")
|
||||||
|
req = urllib.request.Request(
|
||||||
|
f"{PRESIDIO_URL}/analyze",
|
||||||
|
data=body,
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||||
|
return json.loads(resp.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_customer_id(name):
|
||||||
|
# Demo-grade name -> customer_id resolution (one seeded customer, so a scan is
|
||||||
|
# fine). Best effort: if it fails we still tokenize, just without a RAG link.
|
||||||
|
if not CUSTOMERS:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from boto3.dynamodb.conditions import Attr
|
||||||
|
|
||||||
|
res = CUSTOMERS.scan(FilterExpression=Attr("name").eq(name), Limit=1)
|
||||||
|
items = res.get("Items", [])
|
||||||
|
return items[0]["customer_id"] if items else None
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _authorized(headers):
|
||||||
|
if not API_KEY:
|
||||||
|
return True # local dev / unset -> allow (flagged in the response)
|
||||||
|
presented = headers.get("x-api-key") or ""
|
||||||
|
if not presented:
|
||||||
|
auth = headers.get("authorization") or ""
|
||||||
|
if auth.lower().startswith("bearer "):
|
||||||
|
presented = auth[7:]
|
||||||
|
return bool(presented) and hmac.compare_digest(presented, API_KEY)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse(event):
|
||||||
|
"""Accept a Lambda Function URL / API Gateway event, or a raw dict (tests)."""
|
||||||
|
if "body" in event and not isinstance(event.get("body"), dict):
|
||||||
|
raw = event.get("body") or "{}"
|
||||||
|
if event.get("isBase64Encoded"):
|
||||||
|
raw = base64.b64decode(raw).decode("utf-8")
|
||||||
|
payload = json.loads(raw) if raw else {}
|
||||||
|
headers = {k.lower(): v for k, v in (event.get("headers") or {}).items()}
|
||||||
|
return payload, headers
|
||||||
|
# Direct invoke: the event itself is the payload; no HTTP headers.
|
||||||
|
return event, {}
|
||||||
|
|
||||||
|
|
||||||
|
def _reply(status, obj):
|
||||||
|
return {
|
||||||
|
"statusCode": status,
|
||||||
|
"headers": {"Content-Type": "application/json"},
|
||||||
|
"body": json.dumps(obj, ensure_ascii=False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def lambda_handler(event, _context):
|
||||||
|
payload, headers = _parse(event)
|
||||||
|
|
||||||
|
if not _authorized(headers):
|
||||||
|
return _reply(401, {"error": "unauthorized"})
|
||||||
|
|
||||||
|
query = (payload.get("query") or "").strip()
|
||||||
|
if not query:
|
||||||
|
return _reply(400, {"error": "query is required"})
|
||||||
|
language = payload.get("language", "zh")
|
||||||
|
|
||||||
|
try:
|
||||||
|
findings = _analyze(query, language)
|
||||||
|
except Exception as e: # detector unreachable -> fail closed, never leak raw text
|
||||||
|
return _reply(502, {"error": f"detector unavailable: {e}"})
|
||||||
|
|
||||||
|
session_id = uuid.uuid4().hex
|
||||||
|
expires_at = int(time.time()) + TTL_SECONDS
|
||||||
|
|
||||||
|
# Drop overlapping/duplicate spans, then tokenize right-to-left so earlier
|
||||||
|
# offsets stay valid as we splice.
|
||||||
|
findings = _resolve_overlaps(findings)
|
||||||
|
findings.sort(key=lambda f: f["start"], reverse=True)
|
||||||
|
text = query
|
||||||
|
minted = []
|
||||||
|
for f in findings:
|
||||||
|
prefix = _PREFIX[f["entity_type"]] # guaranteed present by _resolve_overlaps
|
||||||
|
original = f["text"]
|
||||||
|
token = _mint(prefix)
|
||||||
|
|
||||||
|
item = {
|
||||||
|
"token": token,
|
||||||
|
"type": f["entity_type"],
|
||||||
|
"value": original, # original PII -> lets /restore put it back
|
||||||
|
"session_id": session_id,
|
||||||
|
"expires_at": expires_at,
|
||||||
|
}
|
||||||
|
if f["entity_type"] == "PERSON":
|
||||||
|
customer_id = _resolve_customer_id(original)
|
||||||
|
if customer_id:
|
||||||
|
item["customer_id"] = customer_id # RAG tool resolves via this
|
||||||
|
VAULT.put_item(Item=item)
|
||||||
|
|
||||||
|
text = text[: f["start"]] + token + text[f["end"] :]
|
||||||
|
minted.append({"token": token, "type": f["entity_type"]})
|
||||||
|
|
||||||
|
resp = {"deidentified_prompt": text, "session_id": session_id, "tokens": minted}
|
||||||
|
if not API_KEY:
|
||||||
|
resp["warning"] = "TOKENIZE_API_KEY unset -- endpoint is unauthenticated (dev only)"
|
||||||
|
return _reply(200, resp)
|
||||||
@@ -1,41 +1,117 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# AgentCore setup (steps 3-4-7). Terraform's AgentCore coverage lags, so this
|
# AgentCore setup (steps 3-4-7). Terraform's AgentCore coverage lags, so this
|
||||||
# stage uses the AgentCore CLI / SDK. Commands below reflect the current CLI
|
# stage uses the AWS CLI's `bedrock-agentcore-control` API. Verified against the
|
||||||
# shape -- CHECK against `agentcore --help`, the surface moves fast.
|
# CLI on 2026-07-01; the surface moves fast, so re-check with
|
||||||
|
# `aws bedrock-agentcore-control help` if a call rejects.
|
||||||
|
#
|
||||||
|
# T4 (implemented below): create an MCP Gateway and register the RAG Lambda as a
|
||||||
|
# tool. Inbound auth is AWS_IAM (SigV4) -- no Cognito needed. The Gateway assumes
|
||||||
|
# the IAM role from Terraform (`terraform output -raw gateway_role_arn`) to invoke
|
||||||
|
# the Lambda.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")"
|
||||||
|
|
||||||
: "${REGION:?set REGION}"
|
: "${REGION:?set REGION}"
|
||||||
: "${RAG_LAMBDA_ARN:?set RAG_LAMBDA_ARN (from: terraform output -raw rag_lambda_arn)}"
|
: "${RAG_LAMBDA_ARN:?set RAG_LAMBDA_ARN (from: terraform -chdir=terraform output -raw rag_lambda_arn)}"
|
||||||
|
: "${GATEWAY_ROLE_ARN:?set GATEWAY_ROLE_ARN (from: terraform -chdir=terraform output -raw gateway_role_arn)}"
|
||||||
: "${BEDROCK_MODEL_ID:=anthropic.claude-3-5-sonnet-20241022-v2:0}"
|
: "${BEDROCK_MODEL_ID:=anthropic.claude-3-5-sonnet-20241022-v2:0}"
|
||||||
|
GW_NAME="${GW_NAME:-hncb-rag-gateway}"
|
||||||
|
TAG='conan hncb demo'
|
||||||
|
export AWS_PAGER=""
|
||||||
|
|
||||||
echo "==> 1. Install the AgentCore CLI (recommended tooling)"
|
echo "==> 1. Create the MCP Gateway (AWS_IAM inbound auth)"
|
||||||
echo " npm install -g @aws/agentcore # or: pip install bedrock-agentcore-starter-toolkit"
|
GW_JSON="$(aws bedrock-agentcore-control create-gateway \
|
||||||
|
--region "$REGION" \
|
||||||
|
--name "$GW_NAME" \
|
||||||
|
--role-arn "$GATEWAY_ROLE_ARN" \
|
||||||
|
--protocol-type MCP \
|
||||||
|
--authorizer-type AWS_IAM \
|
||||||
|
--tags "Owner=$TAG" 2>/dev/null \
|
||||||
|
|| aws bedrock-agentcore-control list-gateways --region "$REGION" \
|
||||||
|
--query "items[?name=='$GW_NAME']|[0]" --output json)"
|
||||||
|
GW_ID="$(echo "$GW_JSON" | python3 -c 'import sys,json;print(json.load(sys.stdin)["gatewayId"])')"
|
||||||
|
echo " gatewayId=$GW_ID"
|
||||||
|
|
||||||
echo "==> 2. Create a Gateway that exposes the RAG Lambda as an MCP tool"
|
echo "==> 2. Wait for the Gateway to be READY"
|
||||||
echo " AgentCore Gateway turns a Lambda into an MCP tool with no code changes."
|
for i in $(seq 1 20); do
|
||||||
|
ST="$(aws bedrock-agentcore-control get-gateway --region "$REGION" --gateway-identifier "$GW_ID" --query status --output text)"
|
||||||
|
echo " [$i] status=$ST"
|
||||||
|
[ "$ST" = "READY" ] && break
|
||||||
|
sleep 6
|
||||||
|
done
|
||||||
|
GW_URL="$(aws bedrock-agentcore-control get-gateway --region "$REGION" --gateway-identifier "$GW_ID" --query gatewayUrl --output text)"
|
||||||
|
|
||||||
|
echo "==> 3. Register the RAG Lambda as an MCP tool target"
|
||||||
|
# Build the target config: mcp.lambda with the tool schema from agent/tool_schema.json
|
||||||
|
# (wrapped in the inlinePayload list). The Gateway's inputSchema is a RESTRICTED
|
||||||
|
# JSON-Schema subset (only type/properties/required/items/description), so we strip
|
||||||
|
# anything else (e.g. the "default" on period_days) recursively.
|
||||||
|
python3 - "$RAG_LAMBDA_ARN" > /tmp/hncb_target.json <<'PY'
|
||||||
|
import json, sys, os
|
||||||
|
ALLOWED = {"type", "properties", "required", "items", "description"}
|
||||||
|
def clean(s):
|
||||||
|
if not isinstance(s, dict):
|
||||||
|
return s
|
||||||
|
out = {k: v for k, v in s.items() if k in ALLOWED}
|
||||||
|
if "properties" in out:
|
||||||
|
out["properties"] = {k: clean(v) for k, v in out["properties"].items()}
|
||||||
|
if "items" in out:
|
||||||
|
out["items"] = clean(out["items"])
|
||||||
|
return out
|
||||||
|
lambda_arn = sys.argv[1]
|
||||||
|
schema = json.load(open(os.path.join("..", "agent", "tool_schema.json")))
|
||||||
|
schema["inputSchema"] = clean(schema["inputSchema"])
|
||||||
|
if "outputSchema" in schema:
|
||||||
|
schema["outputSchema"] = clean(schema["outputSchema"])
|
||||||
|
print(json.dumps({
|
||||||
|
"mcp": {"lambda": {"lambdaArn": lambda_arn, "toolSchema": {"inlinePayload": [schema]}}}
|
||||||
|
}))
|
||||||
|
PY
|
||||||
|
if aws bedrock-agentcore-control list-gateway-targets --region "$REGION" \
|
||||||
|
--gateway-identifier "$GW_ID" --query 'items[?name==`rag`]' --output text | grep -q rag; then
|
||||||
|
echo " target 'rag' already exists -- reusing"
|
||||||
|
else
|
||||||
|
aws bedrock-agentcore-control create-gateway-target \
|
||||||
|
--region "$REGION" \
|
||||||
|
--gateway-identifier "$GW_ID" \
|
||||||
|
--name "rag" \
|
||||||
|
--target-configuration file:///tmp/hncb_target.json \
|
||||||
|
--credential-provider-configurations '[{"credentialProviderType":"GATEWAY_IAM_ROLE"}]' \
|
||||||
|
--query '{target:targetId,status:status}' --output json
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Gateway ready. MCP URL:"
|
||||||
|
echo " $GW_URL"
|
||||||
|
echo "Export it for the agent + the invoke test:"
|
||||||
|
echo " export AGENTCORE_GATEWAY_URL=$GW_URL"
|
||||||
|
echo ""
|
||||||
|
echo "Verify T4 (manual invoke returns the evidence package):"
|
||||||
|
echo " REGION=$REGION AGENTCORE_GATEWAY_URL=$GW_URL python3 scripts/gateway_invoke_test.py CUST_000123"
|
||||||
|
|
||||||
|
########################################
|
||||||
|
# T5: deploy the agent to AgentCore Runtime (uses the starter toolkit CLI).
|
||||||
|
# In ap-southeast-1 Claude 3.5 Sonnet v2 is INFERENCE_PROFILE-only -> use the
|
||||||
|
# apac.* profile id, not the raw model id.
|
||||||
|
########################################
|
||||||
|
APAC_MODEL="apac.anthropic.claude-3-5-sonnet-20241022-v2:0"
|
||||||
|
echo ""
|
||||||
|
echo "==> (T5) Deploy the agent to AgentCore Runtime:"
|
||||||
cat <<EOF
|
cat <<EOF
|
||||||
agentcore gateway create \\
|
cd ../agent # requirements.txt is version-pinned
|
||||||
--name hncb-rag-gateway \\
|
pip install bedrock-agentcore-starter-toolkit # provides 'agentcore'
|
||||||
--region ${REGION} \\
|
agentcore configure -e agent.py -n hncb_advisor_agent -r ${REGION} \\
|
||||||
--target-type lambda \\
|
-rf requirements.txt --non-interactive
|
||||||
--target-arn ${RAG_LAMBDA_ARN} \\
|
agentcore launch --env AGENTCORE_GATEWAY_URL=${GW_URL} --env BEDROCK_MODEL_ID=${APAC_MODEL}
|
||||||
--tool-name get_customer_activity_summary \\
|
# -> note the printed Runtime ARN (Fusion / the UI will invoke this).
|
||||||
--tool-description "Return a de-identified activity summary for a customer TOKEN" \\
|
|
||||||
--tool-schema file://../agent/tool_schema.json
|
# REQUIRED post-launch: the auto-created execution role can't call the Gateway
|
||||||
# Note the printed Gateway MCP URL -> export AGENTCORE_GATEWAY_URL=...
|
# yet. Grant it (role name is printed by launch as AmazonBedrockAgentCoreSDKRuntime-*):
|
||||||
|
ROLE=\$(aws iam list-roles --query "Roles[?starts_with(RoleName,'AmazonBedrockAgentCoreSDKRuntime-${REGION}')].RoleName | [0]" --output text)
|
||||||
|
GW_ARN=\$(aws bedrock-agentcore-control get-gateway --region ${REGION} --gateway-identifier ${GW_ID} --query gatewayArn --output text)
|
||||||
|
aws iam put-role-policy --role-name "\$ROLE" --policy-name hncb-gateway-invoke \\
|
||||||
|
--policy-document "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":[\"bedrock-agentcore:InvokeGateway\"],\"Resource\":[\"\$GW_ARN\",\"\$GW_ARN/*\"]}]}"
|
||||||
|
|
||||||
|
# Verify: agentcore invoke '{"prompt":"talking points for CUST_000123"}'
|
||||||
EOF
|
EOF
|
||||||
|
echo "Fill AGENTCORE_GATEWAY_URL + Runtime ARN into fusion/POLICY_SETUP.md."
|
||||||
echo "==> 3. Deploy the agent to AgentCore Runtime"
|
|
||||||
cat <<EOF
|
|
||||||
cd ../agent
|
|
||||||
agentcore configure --entrypoint agent.py --name hncb-advisor-agent --region ${REGION}
|
|
||||||
AGENTCORE_GATEWAY_URL=<from step 2> BEDROCK_MODEL_ID=${BEDROCK_MODEL_ID} \\
|
|
||||||
agentcore launch
|
|
||||||
# Note the printed Runtime ARN -> Fusion will invoke this.
|
|
||||||
EOF
|
|
||||||
|
|
||||||
echo "==> 4. (recommended) Lock the Runtime to accept calls only from the Gateway"
|
|
||||||
echo " Use a resource policy (SigV4) or allowedWorkloadConfiguration (JWT)."
|
|
||||||
echo " See AgentCore 'Runtime targets' docs."
|
|
||||||
|
|
||||||
echo "Done printing steps. Fill AGENTCORE_GATEWAY_URL + Runtime ARN into fusion/POLICY_SETUP.md."
|
|
||||||
|
|||||||
@@ -16,7 +16,8 @@ terraform -chdir=terraform apply -target=aws_lambda_function.rag -auto-approve
|
|||||||
|
|
||||||
echo "==> Step 2: build + push the Presidio detector image"
|
echo "==> Step 2: build + push the Presidio detector image"
|
||||||
aws ecr describe-repositories --repository-names hncb-presidio --region "$REGION" >/dev/null 2>&1 \
|
aws ecr describe-repositories --repository-names hncb-presidio --region "$REGION" >/dev/null 2>&1 \
|
||||||
|| aws ecr create-repository --repository-name hncb-presidio --region "$REGION" >/dev/null
|
|| aws ecr create-repository --repository-name hncb-presidio --region "$REGION" \
|
||||||
|
--tags Key=Owner,Value="conan hncb demo" >/dev/null
|
||||||
aws ecr get-login-password --region "$REGION" | docker login --username AWS --password-stdin "$ECR"
|
aws ecr get-login-password --region "$REGION" | docker login --username AWS --password-stdin "$ECR"
|
||||||
docker build -t "$ECR/hncb-presidio:latest" ./presidio
|
docker build -t "$ECR/hncb-presidio:latest" ./presidio
|
||||||
docker push "$ECR/hncb-presidio:latest"
|
docker push "$ECR/hncb-presidio:latest"
|
||||||
|
|||||||
104
scripts/gateway_invoke_test.py
Normal file
104
scripts/gateway_invoke_test.py
Normal file
@@ -0,0 +1,104 @@
|
|||||||
|
"""
|
||||||
|
Manual T4 verification: call the RAG tool THROUGH the AgentCore Gateway (MCP) and
|
||||||
|
print the de-identified evidence package. This is the "manual agentcore invoke"
|
||||||
|
the task's acceptance asks for -- it exercises the real Gateway -> Lambda path,
|
||||||
|
not the Lambda directly.
|
||||||
|
|
||||||
|
Auth is AWS_IAM: every MCP request is SigV4-signed for the `bedrock-agentcore`
|
||||||
|
service with your local creds. MCP uses streamable HTTP (JSON-RPC, responses may
|
||||||
|
come back as SSE), so we do initialize -> tools/list -> tools/call.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
REGION=ap-southeast-1 AGENTCORE_GATEWAY_URL=<url> \
|
||||||
|
python3 scripts/gateway_invoke_test.py CUST_000123
|
||||||
|
|
||||||
|
Requires botocore (pip install boto3). Depends on nothing else.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
from botocore.auth import SigV4Auth
|
||||||
|
from botocore.awsrequest import AWSRequest
|
||||||
|
from botocore.session import Session
|
||||||
|
|
||||||
|
REGION = os.environ["REGION"]
|
||||||
|
URL = os.environ["AGENTCORE_GATEWAY_URL"]
|
||||||
|
TOKEN = sys.argv[1] if len(sys.argv) > 1 else "CUST_000123"
|
||||||
|
|
||||||
|
_creds = Session().get_credentials()
|
||||||
|
_session_id = None # set from the initialize response
|
||||||
|
|
||||||
|
|
||||||
|
def _rpc(method, params=None, notify=False):
|
||||||
|
"""One SigV4-signed MCP JSON-RPC call. Returns the parsed result (or None)."""
|
||||||
|
global _session_id
|
||||||
|
body = {"jsonrpc": "2.0", "method": method}
|
||||||
|
if not notify:
|
||||||
|
body["id"] = 1
|
||||||
|
if params is not None:
|
||||||
|
body["params"] = params
|
||||||
|
data = json.dumps(body).encode()
|
||||||
|
|
||||||
|
headers = {"Content-Type": "application/json", "Accept": "application/json, text/event-stream"}
|
||||||
|
if _session_id:
|
||||||
|
headers["Mcp-Session-Id"] = _session_id
|
||||||
|
|
||||||
|
req = AWSRequest(method="POST", url=URL, data=data, headers=headers)
|
||||||
|
SigV4Auth(_creds, "bedrock-agentcore", REGION).add_auth(req)
|
||||||
|
|
||||||
|
try:
|
||||||
|
resp = urllib.request.urlopen(urllib.request.Request(URL, data=data, headers=dict(req.headers)), timeout=40)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
print(f"HTTP {e.code} on {method}: {e.read().decode()[:400]}", file=sys.stderr)
|
||||||
|
raise
|
||||||
|
|
||||||
|
sid = resp.headers.get("Mcp-Session-Id")
|
||||||
|
if sid:
|
||||||
|
_session_id = sid
|
||||||
|
raw = resp.read().decode()
|
||||||
|
if notify:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Response is either JSON or SSE (event-stream). Extract the JSON-RPC object.
|
||||||
|
payload = raw
|
||||||
|
if "text/event-stream" in (resp.headers.get("Content-Type") or "") or raw.lstrip().startswith("event:"):
|
||||||
|
for line in raw.splitlines():
|
||||||
|
if line.startswith("data:"):
|
||||||
|
payload = line[5:].strip()
|
||||||
|
obj = json.loads(payload)
|
||||||
|
if "error" in obj:
|
||||||
|
raise RuntimeError(f"MCP error on {method}: {obj['error']}")
|
||||||
|
return obj.get("result")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
_rpc("initialize", {
|
||||||
|
"protocolVersion": "2025-06-18",
|
||||||
|
"capabilities": {},
|
||||||
|
"clientInfo": {"name": "hncb-t4-test", "version": "1.0"},
|
||||||
|
})
|
||||||
|
_rpc("notifications/initialized", notify=True)
|
||||||
|
|
||||||
|
tools = _rpc("tools/list").get("tools", [])
|
||||||
|
names = [t["name"] for t in tools]
|
||||||
|
print("tools exposed by gateway:", names)
|
||||||
|
# Gateway namespaces tool names (e.g. rag___get_customer_activity_summary).
|
||||||
|
tool = next((n for n in names if n.endswith("get_customer_activity_summary")), None)
|
||||||
|
if not tool:
|
||||||
|
sys.exit("get_customer_activity_summary not found on the gateway")
|
||||||
|
|
||||||
|
result = _rpc("tools/call", {"name": tool, "arguments": {"customer_token": TOKEN}})
|
||||||
|
print(f"\n=== evidence package for {TOKEN} (via Gateway MCP) ===")
|
||||||
|
# MCP tool results come back as content blocks; print text blocks.
|
||||||
|
for block in result.get("content", []):
|
||||||
|
if block.get("type") == "text":
|
||||||
|
try:
|
||||||
|
print(json.dumps(json.loads(block["text"]), indent=2, ensure_ascii=False))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
print(block["text"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -6,14 +6,47 @@ cd "$(dirname "$0")/.."
|
|||||||
|
|
||||||
REGION="${REGION:-us-east-1}"
|
REGION="${REGION:-us-east-1}"
|
||||||
|
|
||||||
echo "==> Delete AgentCore resources (adjust names to what agentcore_setup created)"
|
echo "==> Delete AgentCore Gateway + targets (not in Terraform state)"
|
||||||
echo " agentcore runtime delete --name hncb-advisor-agent --region ${REGION} || true"
|
export AWS_PAGER=""
|
||||||
echo " agentcore gateway delete --name hncb-rag-gateway --region ${REGION} || true"
|
GW_ID="$(aws bedrock-agentcore-control list-gateways --region "$REGION" \
|
||||||
|
--query "items[?name=='hncb-rag-gateway'].gatewayId | [0]" --output text 2>/dev/null || echo None)"
|
||||||
|
if [ "$GW_ID" != "None" ] && [ -n "$GW_ID" ]; then
|
||||||
|
for TID in $(aws bedrock-agentcore-control list-gateway-targets --region "$REGION" \
|
||||||
|
--gateway-identifier "$GW_ID" --query 'items[].targetId' --output text 2>/dev/null); do
|
||||||
|
aws bedrock-agentcore-control delete-gateway-target --region "$REGION" \
|
||||||
|
--gateway-identifier "$GW_ID" --target-id "$TID" || true
|
||||||
|
done
|
||||||
|
aws bedrock-agentcore-control delete-gateway --region "$REGION" --gateway-identifier "$GW_ID" || true
|
||||||
|
echo " deleted gateway $GW_ID"
|
||||||
|
fi
|
||||||
|
echo "==> Delete AgentCore Runtime + Memory (T5; not in Terraform state)"
|
||||||
|
for RT in $(aws bedrock-agentcore-control list-agent-runtimes --region "$REGION" \
|
||||||
|
--query "agentRuntimes[?starts_with(agentRuntimeName,'hncb')].agentRuntimeId" --output text 2>/dev/null); do
|
||||||
|
aws bedrock-agentcore-control delete-agent-runtime --region "$REGION" --agent-runtime-id "$RT" || true
|
||||||
|
echo " deleted runtime $RT"
|
||||||
|
done
|
||||||
|
for MEM in $(aws bedrock-agentcore-control list-memories --region "$REGION" \
|
||||||
|
--query "memories[?starts_with(id,'hncb')].id" --output text 2>/dev/null); do
|
||||||
|
aws bedrock-agentcore-control delete-memory --region "$REGION" --memory-id "$MEM" || true
|
||||||
|
echo " deleted memory $MEM"
|
||||||
|
done
|
||||||
|
echo " (agentcore launch also auto-created an ECR repo, a CodeBuild project, and an"
|
||||||
|
echo " AmazonBedrockAgentCoreSDKRuntime-* execution role -- delete by hand if desired.)"
|
||||||
|
|
||||||
echo "==> Terraform destroy (DynamoDB, Lambda, ECS, IAM)"
|
echo "==> Terraform destroy (DynamoDB, Lambda, ECS, IAM)"
|
||||||
terraform -chdir=terraform destroy -auto-approve
|
terraform -chdir=terraform destroy -auto-approve
|
||||||
|
|
||||||
echo "==> (optional) delete the Presidio ECR repo"
|
echo "==> Delete ECR repos (Presidio + the agentcore-built agent image; not in TF state)"
|
||||||
echo " aws ecr delete-repository --repository-name hncb-presidio --force --region ${REGION} || true"
|
for REPO in hncb-presidio bedrock-agentcore-hncb_advisor_agent; do
|
||||||
|
aws ecr delete-repository --repository-name "$REPO" --force --region "$REGION" 2>/dev/null \
|
||||||
|
&& echo " deleted ECR $REPO" || true
|
||||||
|
done
|
||||||
|
echo " (also free/leftover: the AmazonBedrockAgentCoreSDKRuntime-* role and a CodeBuild"
|
||||||
|
echo " project agentcore created -- no cost idle; delete by hand if you want it spotless.)"
|
||||||
|
|
||||||
echo "Teardown complete (mind any S3/CloudFront you created for the UI)."
|
echo "==> (manual) default VPC created by hand for the T1 live deploy (ap-southeast-1)"
|
||||||
|
echo " Not in Terraform state; tagged maintenance=manual-cleanup-required. Free while"
|
||||||
|
echo " empty, so usually leave it. To remove: aws ec2 delete-vpc --vpc-id vpc-0e97f4fdb643c3e26"
|
||||||
|
|
||||||
|
echo "Teardown complete. (UI S3+CloudFront are Terraform-managed now, so the destroy"
|
||||||
|
echo " above removes them; the S3 bucket has force_destroy=true.)"
|
||||||
|
|||||||
46
terraform/.terraform.lock.hcl
generated
Normal file
46
terraform/.terraform.lock.hcl
generated
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
# This file is maintained automatically by "terraform init".
|
||||||
|
# Manual edits may be lost in future updates.
|
||||||
|
|
||||||
|
provider "registry.terraform.io/hashicorp/archive" {
|
||||||
|
version = "2.8.0"
|
||||||
|
constraints = "~> 2.0"
|
||||||
|
hashes = [
|
||||||
|
"h1:WB6H5ksIZiyq1lQlD/PWeh+tn4FLsbSjVnRW3+4xe2Y=",
|
||||||
|
"zh:0d14713fdc259fb377d0b899ad3c650a34194bd52194c863303ef22a65a580e2",
|
||||||
|
"zh:369b56040c7a8085d04e7e8ffac1e2b321a3170e502f788819bc34b868ec016f",
|
||||||
|
"zh:4d1a3b983ed6af5a52bfe12794674ae55cbadfa6021b37106ade68b433ad216a",
|
||||||
|
"zh:5c547549e26e083573c78a966ca68ce6d7df6bb8f3948f66a575f07da46b74ea",
|
||||||
|
"zh:6de093e62a975eb19a5e3017ce38e6e3cb639c17b79648d2000e0a8348f0e997",
|
||||||
|
"zh:7267936c2cdbc448efeb594d73e6b56a53d6a7ae14fe88cdd2a4133adc3302f0",
|
||||||
|
"zh:7482f023050ed426b4b45116e1761643bc33b1fd4ce4a6fab207ae2571f35940",
|
||||||
|
"zh:76bbd93b234e5a2927d98b511d86565700f549b570871a194c35f944b96cefb7",
|
||||||
|
"zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3",
|
||||||
|
"zh:c6afc4bc1f002bac9c173007dd4da05fde788cd14c2916089f958c33fedb0dfa",
|
||||||
|
"zh:d3ba40bd806a3a08e9237dece679193c99afb2085de6b45d7f5d1f673cfcd368",
|
||||||
|
"zh:e1ad7ded53ecd6f0e5b473a3b44eae2b2e885653a56050ab583d387332be02e4",
|
||||||
|
"zh:e93e78575ce82be6084cc153c24ba8f385dc8d6880888ee66e918460c870953d",
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
provider "registry.terraform.io/hashicorp/aws" {
|
||||||
|
version = "5.100.0"
|
||||||
|
constraints = "~> 5.0"
|
||||||
|
hashes = [
|
||||||
|
"h1:Ijt7pOlB7Tr7maGQIqtsLFbl7pSMIj06TVdkoSBcYOw=",
|
||||||
|
"zh:054b8dd49f0549c9a7cc27d159e45327b7b65cf404da5e5a20da154b90b8a644",
|
||||||
|
"zh:0b97bf8d5e03d15d83cc40b0530a1f84b459354939ba6f135a0086c20ebbe6b2",
|
||||||
|
"zh:1589a2266af699cbd5d80737a0fe02e54ec9cf2ca54e7e00ac51c7359056f274",
|
||||||
|
"zh:6330766f1d85f01ae6ea90d1b214b8b74cc8c1badc4696b165b36ddd4cc15f7b",
|
||||||
|
"zh:7c8c2e30d8e55291b86fcb64bdf6c25489d538688545eb48fd74ad622e5d3862",
|
||||||
|
"zh:99b1003bd9bd32ee323544da897148f46a527f622dc3971af63ea3e251596342",
|
||||||
|
"zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425",
|
||||||
|
"zh:9f8b909d3ec50ade83c8062290378b1ec553edef6a447c56dadc01a99f4eaa93",
|
||||||
|
"zh:aaef921ff9aabaf8b1869a86d692ebd24fbd4e12c21205034bb679b9caf883a2",
|
||||||
|
"zh:ac882313207aba00dd5a76dbd572a0ddc818bb9cbf5c9d61b28fe30efaec951e",
|
||||||
|
"zh:bb64e8aff37becab373a1a0cc1080990785304141af42ed6aa3dd4913b000421",
|
||||||
|
"zh:dfe495f6621df5540d9c92ad40b8067376350b005c637ea6efac5dc15028add4",
|
||||||
|
"zh:f0ddf0eaf052766cfe09dea8200a946519f653c384ab4336e2a4a64fdd6310e9",
|
||||||
|
"zh:f1b7e684f4c7ae1eed272b6de7d2049bb87a0275cb04dbb7cda6636f600699c9",
|
||||||
|
"zh:ff461571e3f233699bf690db319dfe46aec75e58726636a0d97dd9ac6e32fb70",
|
||||||
|
]
|
||||||
|
}
|
||||||
41
terraform/agentcore.tf
Normal file
41
terraform/agentcore.tf
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# AgentCore Gateway support (task T4). Terraform's AgentCore coverage lags, so the
|
||||||
|
# Gateway + target themselves are created by scripts/agentcore_setup.sh via
|
||||||
|
# `aws bedrock-agentcore-control`. What IS declarative here is the IAM role the
|
||||||
|
# Gateway assumes to invoke the RAG tool Lambda -- kept in Terraform so it is
|
||||||
|
# tagged, auditable, and torn down with the rest of the stack.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
resource "aws_iam_role" "agentcore_gateway" {
|
||||||
|
name = "${local.name}-gateway-role"
|
||||||
|
assume_role_policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
Effect = "Allow"
|
||||||
|
Principal = { Service = "bedrock-agentcore.amazonaws.com" }
|
||||||
|
Action = "sts:AssumeRole"
|
||||||
|
Condition = {
|
||||||
|
StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.current.account_id }
|
||||||
|
}
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
tags = local.cloud_tag # the Gateway is a cloud-zone component
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role_policy" "agentcore_gateway" {
|
||||||
|
name = "${local.name}-gateway-policy"
|
||||||
|
role = aws_iam_role.agentcore_gateway.id
|
||||||
|
policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
# The Gateway only needs to invoke the one RAG tool Lambda.
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["lambda:InvokeFunction"]
|
||||||
|
Resource = [aws_lambda_function.rag.arn]
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
data "aws_caller_identity" "current" {}
|
||||||
|
|
||||||
|
output "gateway_role_arn" { value = aws_iam_role.agentcore_gateway.arn }
|
||||||
@@ -1,9 +1,10 @@
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Hosting for the Presidio detector only. Fusion is a shared SaaS instance and is
|
# Hosting for the Presidio detector only. Fusion is a shared SaaS instance and is
|
||||||
# NOT deployed here. Because Fusion SaaS calls the detector over the internet, the
|
# NOT deployed here. The detector is PRIVATE (task T3): its port 5001 is reachable
|
||||||
# analyzer port is public in this demo (lock it down / add auth for anything real;
|
# only from the /tokenize Lambda's security group, not the internet. The only
|
||||||
# better: fold detection behind the /tokenize endpoint so only that is public --
|
# public surface is the API Gateway (/tokenize + /restore). The task keeps a public
|
||||||
# see CLAUDE.md task T3).
|
# IP purely so Fargate can pull the image from ECR (there's no NAT); inbound is
|
||||||
|
# still SG-locked, so nothing is reachable from outside the VPC.
|
||||||
#
|
#
|
||||||
# For a laptop-only rehearsal you can skip this file and run Presidio via docker.
|
# For a laptop-only rehearsal you can skip this file and run Presidio via docker.
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -17,21 +18,42 @@ data "aws_subnets" "default" {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Route tables of the default VPC -- needed to attach the DynamoDB gateway endpoint
|
||||||
|
# so the (now VPC-attached) tokenize Lambda can still reach the vault/customers tables.
|
||||||
|
data "aws_route_tables" "default" {
|
||||||
|
vpc_id = data.aws_vpc.default.id
|
||||||
|
}
|
||||||
|
|
||||||
resource "aws_ecs_cluster" "this" {
|
resource "aws_ecs_cluster" "this" {
|
||||||
name = "${local.name}-cluster"
|
name = "${local.name}-cluster"
|
||||||
tags = local.onprem_tag
|
tags = local.onprem_tag
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# SG the tokenize Lambda runs in. No ingress; egress open so it can reach Presidio
|
||||||
|
# (5001) and the DynamoDB endpoint. Presidio trusts this SG (below).
|
||||||
|
resource "aws_security_group" "tokenize_lambda" {
|
||||||
|
name = "${local.name}-tokenize-lambda-sg"
|
||||||
|
description = "tokenize Lambda ENIs"
|
||||||
|
vpc_id = data.aws_vpc.default.id
|
||||||
|
egress {
|
||||||
|
from_port = 0
|
||||||
|
to_port = 0
|
||||||
|
protocol = "-1"
|
||||||
|
cidr_blocks = ["0.0.0.0/0"]
|
||||||
|
}
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
resource "aws_security_group" "svc" {
|
resource "aws_security_group" "svc" {
|
||||||
name = "${local.name}-svc-sg"
|
name = "${local.name}-svc-sg"
|
||||||
description = "Demo services SG"
|
description = "Demo services SG"
|
||||||
vpc_id = data.aws_vpc.default.id
|
vpc_id = data.aws_vpc.default.id
|
||||||
ingress {
|
ingress {
|
||||||
description = "Presidio analyzer (public for SaaS Fusion demo - restrict for real use)"
|
description = "Presidio analyzer -- only the tokenize Lambda may call it"
|
||||||
from_port = 5001
|
from_port = 5001
|
||||||
to_port = 5001
|
to_port = 5001
|
||||||
protocol = "tcp"
|
protocol = "tcp"
|
||||||
cidr_blocks = ["0.0.0.0/0"]
|
security_groups = [aws_security_group.tokenize_lambda.id]
|
||||||
}
|
}
|
||||||
egress {
|
egress {
|
||||||
from_port = 0
|
from_port = 0
|
||||||
@@ -42,6 +64,15 @@ resource "aws_security_group" "svc" {
|
|||||||
tags = local.onprem_tag
|
tags = local.onprem_tag
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Gateway endpoint so VPC-attached Lambdas reach DynamoDB without a NAT / public route.
|
||||||
|
resource "aws_vpc_endpoint" "dynamodb" {
|
||||||
|
vpc_id = data.aws_vpc.default.id
|
||||||
|
service_name = "com.amazonaws.${var.region}.dynamodb"
|
||||||
|
vpc_endpoint_type = "Gateway"
|
||||||
|
route_table_ids = data.aws_route_tables.default.ids
|
||||||
|
tags = merge(local.onprem_tag, { Name = "${local.name}-dynamodb" })
|
||||||
|
}
|
||||||
|
|
||||||
resource "aws_iam_role" "ecs_exec" {
|
resource "aws_iam_role" "ecs_exec" {
|
||||||
name = "${local.name}-ecs-exec"
|
name = "${local.name}-ecs-exec"
|
||||||
assume_role_policy = jsonencode({
|
assume_role_policy = jsonencode({
|
||||||
@@ -64,17 +95,37 @@ resource "aws_ecs_task_definition" "presidio" {
|
|||||||
requires_compatibilities = ["FARGATE"]
|
requires_compatibilities = ["FARGATE"]
|
||||||
network_mode = "awsvpc"
|
network_mode = "awsvpc"
|
||||||
cpu = 512
|
cpu = 512
|
||||||
memory = 1024
|
# zh_core_web_lg (~600MB) plus presidio/spacy needs headroom; 1GB OOMs on load.
|
||||||
execution_role_arn = aws_iam_role.ecs_exec.arn
|
memory = 2048
|
||||||
|
execution_role_arn = aws_iam_role.ecs_exec.arn
|
||||||
|
# Built natively on Apple Silicon (podman) -> run on Fargate Graviton.
|
||||||
|
runtime_platform {
|
||||||
|
cpu_architecture = "ARM64"
|
||||||
|
operating_system_family = "LINUX"
|
||||||
|
}
|
||||||
container_definitions = jsonencode([{
|
container_definitions = jsonencode([{
|
||||||
name = "presidio"
|
name = "presidio"
|
||||||
image = var.presidio_image_uri
|
image = var.presidio_image_uri
|
||||||
essential = true
|
essential = true
|
||||||
portMappings = [{ containerPort = 5001 }]
|
portMappings = [{ containerPort = 5001 }]
|
||||||
|
logConfiguration = {
|
||||||
|
logDriver = "awslogs"
|
||||||
|
options = {
|
||||||
|
"awslogs-group" = aws_cloudwatch_log_group.presidio.name
|
||||||
|
"awslogs-region" = var.region
|
||||||
|
"awslogs-stream-prefix" = "presidio"
|
||||||
|
}
|
||||||
|
}
|
||||||
}])
|
}])
|
||||||
tags = local.onprem_tag
|
tags = local.onprem_tag
|
||||||
}
|
}
|
||||||
|
|
||||||
|
resource "aws_cloudwatch_log_group" "presidio" {
|
||||||
|
name = "/ecs/${local.name}-presidio"
|
||||||
|
retention_in_days = 7
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
resource "aws_ecs_service" "presidio" {
|
resource "aws_ecs_service" "presidio" {
|
||||||
name = "${local.name}-presidio"
|
name = "${local.name}-presidio"
|
||||||
cluster = aws_ecs_cluster.this.id
|
cluster = aws_ecs_cluster.this.id
|
||||||
|
|||||||
296
terraform/gateway.tf
Normal file
296
terraform/gateway.tf
Normal file
@@ -0,0 +1,296 @@
|
|||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Public HTTPS endpoints that Fusion SaaS calls: POST /tokenize (T1) and
|
||||||
|
# POST /restore (T2), both on one API Gateway HTTP API. Fusion is shared SaaS, so
|
||||||
|
# it can only reach PUBLIC HTTPS with auth -- each handler checks a shared secret
|
||||||
|
# (tokenize_api_key, sent as x-api-key / bearer). These live in the "on-prem" zone
|
||||||
|
# (tags): they own the reversible token<->PII map, which must never leave it.
|
||||||
|
# (API Gateway, not a Lambda Function URL: this org's SCP blocks unauthenticated
|
||||||
|
# Function URLs.)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
data "archive_file" "tokenize" {
|
||||||
|
type = "zip"
|
||||||
|
source_dir = "${path.module}/../gateway_api/tokenize"
|
||||||
|
output_path = "${path.module}/tokenize_lambda.zip"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role" "tokenize" {
|
||||||
|
name = "${local.name}-tokenize-role"
|
||||||
|
assume_role_policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
Effect = "Allow"
|
||||||
|
Principal = { Service = "lambda.amazonaws.com" }
|
||||||
|
Action = "sts:AssumeRole"
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role_policy" "tokenize" {
|
||||||
|
name = "${local.name}-tokenize-policy"
|
||||||
|
role = aws_iam_role.tokenize.id
|
||||||
|
policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [
|
||||||
|
{
|
||||||
|
# Mint into the vault; scan customers for best-effort PERSON -> customer_id.
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["dynamodb:PutItem"]
|
||||||
|
Resource = [aws_dynamodb_table.vault.arn]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["dynamodb:Scan"]
|
||||||
|
Resource = [aws_dynamodb_table.customers.arn]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
|
||||||
|
Resource = "arn:aws:logs:*:*:*"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_function" "tokenize" {
|
||||||
|
function_name = "${local.name}-tokenize"
|
||||||
|
role = aws_iam_role.tokenize.arn
|
||||||
|
runtime = "python3.12"
|
||||||
|
handler = "handler.lambda_handler"
|
||||||
|
filename = data.archive_file.tokenize.output_path
|
||||||
|
source_code_hash = data.archive_file.tokenize.output_base64sha256
|
||||||
|
timeout = 15
|
||||||
|
environment {
|
||||||
|
variables = {
|
||||||
|
VAULT_TABLE = aws_dynamodb_table.vault.name
|
||||||
|
CUSTOMERS_TABLE = aws_dynamodb_table.customers.name
|
||||||
|
PRESIDIO_URL = var.presidio_url
|
||||||
|
TOKENIZE_API_KEY = var.tokenize_api_key
|
||||||
|
VAULT_TTL_SECONDS = "3600"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# In-VPC so it can reach the now-private Presidio over 5001. DynamoDB is reached
|
||||||
|
# via the gateway endpoint (see ecs.tf); no NAT needed.
|
||||||
|
vpc_config {
|
||||||
|
subnet_ids = data.aws_subnets.default.ids
|
||||||
|
security_group_ids = [aws_security_group.tokenize_lambda.id]
|
||||||
|
}
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
# VPC-attached Lambdas need ENI management permissions.
|
||||||
|
resource "aws_iam_role_policy_attachment" "tokenize_vpc" {
|
||||||
|
role = aws_iam_role.tokenize.name
|
||||||
|
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Public HTTPS front door. We use an API Gateway HTTP API rather than a Lambda
|
||||||
|
# Function URL because this org's SCP blocks unauthenticated (auth_type=NONE)
|
||||||
|
# Function URLs. The API is public; auth is the shared secret checked in-handler
|
||||||
|
# (Fusion sends it as x-api-key / bearer). T2 adds a POST /restore route here.
|
||||||
|
resource "aws_apigatewayv2_api" "gateway" {
|
||||||
|
name = "${local.name}-gateway"
|
||||||
|
protocol_type = "HTTP"
|
||||||
|
# The static UI (CloudFront origin) calls /demo cross-origin -> allow CORS.
|
||||||
|
cors_configuration {
|
||||||
|
allow_origins = ["*"]
|
||||||
|
allow_methods = ["POST", "OPTIONS"]
|
||||||
|
allow_headers = ["content-type", "x-api-key"]
|
||||||
|
}
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_integration" "tokenize" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
integration_type = "AWS_PROXY"
|
||||||
|
integration_uri = aws_lambda_function.tokenize.invoke_arn
|
||||||
|
payload_format_version = "2.0"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_route" "tokenize" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
route_key = "POST /tokenize"
|
||||||
|
target = "integrations/${aws_apigatewayv2_integration.tokenize.id}"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_stage" "default" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
name = "$default"
|
||||||
|
auto_deploy = true
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_permission" "tokenize_apigw" {
|
||||||
|
statement_id = "AllowApiGatewayInvoke"
|
||||||
|
action = "lambda:InvokeFunction"
|
||||||
|
function_name = aws_lambda_function.tokenize.function_name
|
||||||
|
principal = "apigateway.amazonaws.com"
|
||||||
|
source_arn = "${aws_apigatewayv2_api.gateway.execution_arn}/*/*"
|
||||||
|
}
|
||||||
|
|
||||||
|
########################################
|
||||||
|
# T2: /restore -- egress re-identification (vault lookup by session_id)
|
||||||
|
########################################
|
||||||
|
data "archive_file" "restore" {
|
||||||
|
type = "zip"
|
||||||
|
source_dir = "${path.module}/../gateway_api/restore"
|
||||||
|
output_path = "${path.module}/restore_lambda.zip"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role" "restore" {
|
||||||
|
name = "${local.name}-restore-role"
|
||||||
|
assume_role_policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
Effect = "Allow"
|
||||||
|
Principal = { Service = "lambda.amazonaws.com" }
|
||||||
|
Action = "sts:AssumeRole"
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role_policy" "restore" {
|
||||||
|
name = "${local.name}-restore-policy"
|
||||||
|
role = aws_iam_role.restore.id
|
||||||
|
policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [
|
||||||
|
{
|
||||||
|
# Read-only: look up this session's tokens to re-attach identity.
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["dynamodb:Scan"]
|
||||||
|
Resource = [aws_dynamodb_table.vault.arn]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
|
||||||
|
Resource = "arn:aws:logs:*:*:*"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_function" "restore" {
|
||||||
|
function_name = "${local.name}-restore"
|
||||||
|
role = aws_iam_role.restore.arn
|
||||||
|
runtime = "python3.12"
|
||||||
|
handler = "handler.lambda_handler"
|
||||||
|
filename = data.archive_file.restore.output_path
|
||||||
|
source_code_hash = data.archive_file.restore.output_base64sha256
|
||||||
|
timeout = 15
|
||||||
|
environment {
|
||||||
|
variables = {
|
||||||
|
VAULT_TABLE = aws_dynamodb_table.vault.name
|
||||||
|
TOKENIZE_API_KEY = var.tokenize_api_key
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_integration" "restore" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
integration_type = "AWS_PROXY"
|
||||||
|
integration_uri = aws_lambda_function.restore.invoke_arn
|
||||||
|
payload_format_version = "2.0"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_route" "restore" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
route_key = "POST /restore"
|
||||||
|
target = "integrations/${aws_apigatewayv2_integration.restore.id}"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_permission" "restore_apigw" {
|
||||||
|
statement_id = "AllowApiGatewayInvoke"
|
||||||
|
action = "lambda:InvokeFunction"
|
||||||
|
function_name = aws_lambda_function.restore.function_name
|
||||||
|
principal = "apigateway.amazonaws.com"
|
||||||
|
source_arn = "${aws_apigatewayv2_api.gateway.execution_arn}/*/*"
|
||||||
|
}
|
||||||
|
|
||||||
|
########################################
|
||||||
|
# T6: /demo -- Fusion-less orchestrator the static UI calls (tokenize->agent->restore)
|
||||||
|
########################################
|
||||||
|
data "archive_file" "orchestrator" {
|
||||||
|
type = "zip"
|
||||||
|
source_dir = "${path.module}/../gateway_api/orchestrator"
|
||||||
|
output_path = "${path.module}/orchestrator_lambda.zip"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role" "orchestrator" {
|
||||||
|
name = "${local.name}-orchestrator-role"
|
||||||
|
assume_role_policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
Effect = "Allow"
|
||||||
|
Principal = { Service = "lambda.amazonaws.com" }
|
||||||
|
Action = "sts:AssumeRole"
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_iam_role_policy" "orchestrator" {
|
||||||
|
name = "${local.name}-orchestrator-policy"
|
||||||
|
role = aws_iam_role.orchestrator.id
|
||||||
|
policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [
|
||||||
|
{
|
||||||
|
# Invoke the AgentCore Runtime. Scoped to the configured runtime ARN when set.
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["bedrock-agentcore:InvokeAgentRuntime"]
|
||||||
|
Resource = var.agent_runtime_arn != "" ? [var.agent_runtime_arn, "${var.agent_runtime_arn}/*"] : ["*"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Effect = "Allow"
|
||||||
|
Action = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
|
||||||
|
Resource = "arn:aws:logs:*:*:*"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_function" "orchestrator" {
|
||||||
|
function_name = "${local.name}-orchestrator"
|
||||||
|
role = aws_iam_role.orchestrator.arn
|
||||||
|
runtime = "python3.12"
|
||||||
|
handler = "handler.lambda_handler"
|
||||||
|
filename = data.archive_file.orchestrator.output_path
|
||||||
|
source_code_hash = data.archive_file.orchestrator.output_base64sha256
|
||||||
|
timeout = 120
|
||||||
|
environment {
|
||||||
|
variables = {
|
||||||
|
REGION = var.region
|
||||||
|
TOKENIZE_URL = "${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/tokenize"
|
||||||
|
RESTORE_URL = "${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/restore"
|
||||||
|
TOKENIZE_API_KEY = var.tokenize_api_key
|
||||||
|
AGENT_RUNTIME_ARN = var.agent_runtime_arn
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_integration" "orchestrator" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
integration_type = "AWS_PROXY"
|
||||||
|
integration_uri = aws_lambda_function.orchestrator.invoke_arn
|
||||||
|
payload_format_version = "2.0"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_apigatewayv2_route" "orchestrator" {
|
||||||
|
api_id = aws_apigatewayv2_api.gateway.id
|
||||||
|
route_key = "POST /demo"
|
||||||
|
target = "integrations/${aws_apigatewayv2_integration.orchestrator.id}"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_lambda_permission" "orchestrator_apigw" {
|
||||||
|
statement_id = "AllowApiGatewayInvoke"
|
||||||
|
action = "lambda:InvokeFunction"
|
||||||
|
function_name = aws_lambda_function.orchestrator.function_name
|
||||||
|
principal = "apigateway.amazonaws.com"
|
||||||
|
source_arn = "${aws_apigatewayv2_api.gateway.execution_arn}/*/*"
|
||||||
|
}
|
||||||
|
|
||||||
|
output "tokenize_url" { value = "${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/tokenize" }
|
||||||
|
output "restore_url" { value = "${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/restore" }
|
||||||
|
output "demo_url" { value = "${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/demo" }
|
||||||
@@ -8,6 +8,14 @@ terraform {
|
|||||||
|
|
||||||
provider "aws" {
|
provider "aws" {
|
||||||
region = var.region
|
region = var.region
|
||||||
|
|
||||||
|
# Every taggable resource created by this stack carries this tag, on top of any
|
||||||
|
# per-resource tags (e.g. the on-prem/cloud Zone tags).
|
||||||
|
default_tags {
|
||||||
|
tags = {
|
||||||
|
Owner = "conan hncb demo"
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
locals {
|
locals {
|
||||||
|
|||||||
85
terraform/ui.tf
Normal file
85
terraform/ui.tf
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T6: host the advisor UI on S3 + CloudFront. The bucket is private; CloudFront
|
||||||
|
# reaches it via Origin Access Control. index.html is static and committed; the
|
||||||
|
# live endpoint is injected via a generated config.js so we never bake an
|
||||||
|
# ephemeral URL into the repo.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
resource "aws_s3_bucket" "ui" {
|
||||||
|
bucket_prefix = "${local.name}-ui-"
|
||||||
|
force_destroy = true
|
||||||
|
tags = local.onprem_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_cloudfront_origin_access_control" "ui" {
|
||||||
|
name = "${local.name}-ui-oac"
|
||||||
|
origin_access_control_origin_type = "s3"
|
||||||
|
signing_behavior = "always"
|
||||||
|
signing_protocol = "sigv4"
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_cloudfront_distribution" "ui" {
|
||||||
|
enabled = true
|
||||||
|
default_root_object = "index.html"
|
||||||
|
comment = "${local.name} advisor UI"
|
||||||
|
|
||||||
|
origin {
|
||||||
|
domain_name = aws_s3_bucket.ui.bucket_regional_domain_name
|
||||||
|
origin_id = "ui-s3"
|
||||||
|
origin_access_control_id = aws_cloudfront_origin_access_control.ui.id
|
||||||
|
}
|
||||||
|
|
||||||
|
default_cache_behavior {
|
||||||
|
allowed_methods = ["GET", "HEAD"]
|
||||||
|
cached_methods = ["GET", "HEAD"]
|
||||||
|
target_origin_id = "ui-s3"
|
||||||
|
viewer_protocol_policy = "redirect-to-https"
|
||||||
|
forwarded_values {
|
||||||
|
query_string = false
|
||||||
|
cookies { forward = "none" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
restrictions {
|
||||||
|
geo_restriction { restriction_type = "none" }
|
||||||
|
}
|
||||||
|
|
||||||
|
viewer_certificate {
|
||||||
|
cloudfront_default_certificate = true
|
||||||
|
}
|
||||||
|
|
||||||
|
tags = local.cloud_tag
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_s3_bucket_policy" "ui" {
|
||||||
|
bucket = aws_s3_bucket.ui.id
|
||||||
|
policy = jsonencode({
|
||||||
|
Version = "2012-10-17"
|
||||||
|
Statement = [{
|
||||||
|
Effect = "Allow"
|
||||||
|
Principal = { Service = "cloudfront.amazonaws.com" }
|
||||||
|
Action = "s3:GetObject"
|
||||||
|
Resource = "${aws_s3_bucket.ui.arn}/*"
|
||||||
|
Condition = { StringEquals = { "AWS:SourceArn" = aws_cloudfront_distribution.ui.arn } }
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "aws_s3_object" "index" {
|
||||||
|
bucket = aws_s3_bucket.ui.id
|
||||||
|
key = "index.html"
|
||||||
|
source = "${path.module}/../ui/index.html"
|
||||||
|
etag = filemd5("${path.module}/../ui/index.html")
|
||||||
|
content_type = "text/html"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Injected config: the demo orchestrator endpoint the UI calls.
|
||||||
|
resource "aws_s3_object" "config" {
|
||||||
|
bucket = aws_s3_bucket.ui.id
|
||||||
|
key = "config.js"
|
||||||
|
content = "window.DEMO_ENDPOINT = \"${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/demo\";\n"
|
||||||
|
content_type = "application/javascript"
|
||||||
|
etag = md5("${trimsuffix(aws_apigatewayv2_stage.default.invoke_url, "/")}/demo")
|
||||||
|
}
|
||||||
|
|
||||||
|
output "ui_url" { value = "https://${aws_cloudfront_distribution.ui.domain_name}" }
|
||||||
@@ -23,3 +23,30 @@ variable "presidio_image_uri" {
|
|||||||
type = string
|
type = string
|
||||||
default = "REPLACE_ME_PRESIDIO_IMAGE_URI"
|
default = "REPLACE_ME_PRESIDIO_IMAGE_URI"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Where the /tokenize Lambda reaches the Presidio detector. After deploy this is
|
||||||
|
# the Presidio Fargate task's public endpoint (http://<public-ip>:5001). Kept a
|
||||||
|
# variable so a laptop rehearsal can point at a local/ngrok detector.
|
||||||
|
variable "presidio_url" {
|
||||||
|
description = "Base URL of the Presidio detector /analyze service."
|
||||||
|
type = string
|
||||||
|
default = "http://localhost:5001"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Shared secret Fusion SaaS presents to the public endpoints (bearer / x-api-key).
|
||||||
|
# NEVER commit a real value -- pass via TF_VAR_tokenize_api_key or a .tfvars file
|
||||||
|
# that is gitignored. Empty default leaves the endpoint open (dev only).
|
||||||
|
variable "tokenize_api_key" {
|
||||||
|
description = "Shared secret required on /tokenize (and /restore) requests."
|
||||||
|
type = string
|
||||||
|
default = ""
|
||||||
|
sensitive = true
|
||||||
|
}
|
||||||
|
|
||||||
|
# AgentCore Runtime ARN (created out-of-band by `agentcore launch`, see T5). The
|
||||||
|
# demo orchestrator (T6) invokes it. Empty -> orchestrator skips the agent step.
|
||||||
|
variable "agent_runtime_arn" {
|
||||||
|
description = "Bedrock AgentCore Runtime ARN the demo orchestrator invokes."
|
||||||
|
type = string
|
||||||
|
default = ""
|
||||||
|
}
|
||||||
|
|||||||
@@ -40,19 +40,26 @@
|
|||||||
<div class="card cloud">
|
<div class="card cloud">
|
||||||
<h2>What the cloud actually saw <span class="tag">tokenized</span></h2>
|
<h2>What the cloud actually saw <span class="tag">tokenized</span></h2>
|
||||||
<pre id="deid">—</pre>
|
<pre id="deid">—</pre>
|
||||||
|
<h2 style="margin-top:12px">Agent talking points <span class="tag">tokenized</span></h2>
|
||||||
|
<pre id="agent">—</pre>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<!-- Injected at deploy time (S3) with the live demo orchestrator URL. -->
|
||||||
|
<script src="config.js"></script>
|
||||||
<script>
|
<script>
|
||||||
// TODO: point this at your Fusion gateway endpoint after deploy.
|
// Fusion is shared SaaS; for a self-contained dry run the UI calls the demo
|
||||||
const GATEWAY_URL = "http://REPLACE_ME_FUSION_HOST:8080/advisor";
|
// orchestrator (/demo), which runs tokenize -> agent -> restore server-side.
|
||||||
|
// In production, point this at the Fusion gateway entrypoint instead.
|
||||||
|
const GATEWAY_URL = window.DEMO_ENDPOINT || "http://REPLACE_ME_FUSION_HOST:8080/advisor";
|
||||||
|
|
||||||
async function run() {
|
async function run() {
|
||||||
const btn = document.getElementById("go");
|
const btn = document.getElementById("go");
|
||||||
btn.disabled = true;
|
btn.disabled = true;
|
||||||
document.getElementById("final").textContent = "…thinking…";
|
document.getElementById("final").textContent = "…thinking…";
|
||||||
document.getElementById("deid").textContent = "…";
|
document.getElementById("deid").textContent = "…";
|
||||||
|
document.getElementById("agent").textContent = "…";
|
||||||
try {
|
try {
|
||||||
const res = await fetch(GATEWAY_URL, {
|
const res = await fetch(GATEWAY_URL, {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
@@ -60,9 +67,10 @@
|
|||||||
body: JSON.stringify({ query: document.getElementById("q").value }),
|
body: JSON.stringify({ query: document.getElementById("q").value }),
|
||||||
});
|
});
|
||||||
const data = await res.json();
|
const data = await res.json();
|
||||||
// Fusion is expected to return { final, deidentified_prompt }.
|
// Orchestrator (or Fusion) returns { final, deidentified_prompt, agent_tokenized }.
|
||||||
document.getElementById("final").textContent = data.final || JSON.stringify(data, null, 2);
|
document.getElementById("final").textContent = data.final || JSON.stringify(data, null, 2);
|
||||||
document.getElementById("deid").textContent = data.deidentified_prompt || "(gateway did not echo the de-identified prompt)";
|
document.getElementById("deid").textContent = data.deidentified_prompt || "(no de-identified prompt echoed)";
|
||||||
|
document.getElementById("agent").textContent = data.agent_tokenized || "(no agent answer)";
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
document.getElementById("final").textContent = "Error: " + e.message;
|
document.getElementById("final").textContent = "Error: " + e.message;
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
Reference in New Issue
Block a user