Conan Scott
78d67a2469
Build the AWS side of the HNCB demo end to end (region ap-southeast-1): - T1 /tokenize + T2 /restore: Lambdas behind a public API Gateway (shared-secret auth), Presidio detection, random per-request tokens, DynamoDB vault; overlap resolution so a ROC ID stays TW_ROC_ID. - T3: Presidio made private (SG-locked to the tokenize Lambda in-VPC; DynamoDB gateway endpoint); only /tokenize + /restore are public. - T4: RAG Lambda registered as an MCP tool on an AgentCore Gateway (AWS_IAM/SigV4); agentcore_setup.sh + a SigV4 MCP invoke test. - T5: Strands agent deployed to AgentCore Runtime; SigV4 gateway auth, apac inference profile, pinned deps. - T6: advisor UI on S3+CloudFront with a Fusion-less demo orchestrator (/demo) chaining tokenize -> runtime -> restore. - T7: README runbook + trace check; teardown deletes gateway/runtime/memory/ECR. Verified live: the cloud AgentCore/Bedrock trace shows only tokens, never the real name. Secrets stay in gitignored local.auto.tfvars. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
120 lines
6.6 KiB
Markdown
120 lines
6.6 KiB
Markdown
# hncb-fusion-deid-demo
|
|
|
|
Demo of the HNCB use case: a **reversible PII de-identification round trip**
|
|
through **Axway Amplify AI Gateway (Fusion)**, with the reasoning done by an
|
|
**Amazon Bedrock AgentCore** agent. The advisor types a query containing a real
|
|
name; the cloud only ever sees a token; the real identity is restored before the
|
|
answer is shown.
|
|
|
|
> **Working with Claude Code? Read `CLAUDE.md` first** — it has the context,
|
|
> commands, and task backlog.
|
|
|
|
> **Status: unrun scaffold.** Authored, not executed. Review everything, pin
|
|
> dependency versions, and adjust AgentCore specifics to the current CLI.
|
|
> Demo-grade, not production-grade.
|
|
|
|
## Fusion is shared SaaS
|
|
Fusion is **not deployed by this repo** — it's a shared Amplify AI Gateway SaaS
|
|
instance, configured in its console (`fusion/POLICY_SETUP.md`). Because it's SaaS,
|
|
anything it calls must be a **public HTTPS endpoint with auth**. So the AWS side's
|
|
job is to expose two endpoints Fusion calls — **`/tokenize`** (ingress) and
|
|
**`/restore`** (egress) — plus host the detector, vault, RAG tool, and agent.
|
|
Those two endpoints are the main remaining build (tasks T1/T2 in `CLAUDE.md`).
|
|
|
|
## What it demonstrates
|
|
Raw PII (a Chinese name + a Taiwan ROC ID) is detected on ingress, replaced with a
|
|
**random, format-safe token**, and only the tokenized prompt goes to the cloud
|
|
agent. The agent tool-calls back "on-prem" with the token, gets a de-identified
|
|
evidence package, and writes talking points. Fusion restores the identity on the
|
|
way out. **The money shot:** show the Bedrock request / AgentCore trace live — the
|
|
cloud only ever saw `CUST_000123`, never `王小明`.
|
|
|
|
## Trust zones are logical
|
|
Everything is one AWS account. The "on-prem" zone is tag-labelled resources
|
|
(`Zone = on-prem-VPC-A`) standing in for HNCB's branch data centre. Proves
|
|
data-flow behaviour, not physical residency — say so on camera.
|
|
|
|
## Component → service map (their 8 steps)
|
|
| Step | Component | Service in this repo |
|
|
|---|---|---|
|
|
| 1, 8 | Advisor UI | `ui/index.html` on S3+CloudFront (or local) |
|
|
| 2, 3-route, 8 | **Fusion AI Gateway** (the product) | **shared SaaS** — configured via `fusion/POLICY_SETUP.md` (not deployed) |
|
|
| 2 (ingress), 8 (egress) | `/tokenize` + `/restore` endpoints Fusion calls | **TODO** `gateway_api/` (tasks T1/T2) |
|
|
| 2 | PII **detector** (typed findings, not redaction) | Presidio on Fargate — `presidio/` |
|
|
| 2, 4, 8 | Token **vault** (reversible map) | DynamoDB — `terraform/main.tf` |
|
|
| 3, 7 | Cloud **agent** + model | AgentCore Runtime + Bedrock — `agent/` |
|
|
| 4 | Tool bridge (Lambda → MCP tool) | AgentCore Gateway — `scripts/agentcore_setup.sh` |
|
|
| 4-6 | On-prem **RAG tool** + data | Lambda + DynamoDB — `lambda_rag/`, `seed/` |
|
|
| all | Observability | AgentCore Observability + CloudWatch |
|
|
|
|
## Repo layout
|
|
```
|
|
CLAUDE.md start here if using Claude Code (context + task backlog)
|
|
terraform/ DynamoDB (vault + customers), RAG Lambda, IAM, Presidio hosting
|
|
lambda_rag/ RAG tool: token resolve -> de-identified evidence package
|
|
gateway_api/ TODO: /tokenize + /restore endpoints Fusion SaaS calls (T1/T2)
|
|
agent/ Strands agent for AgentCore Runtime + tool schema
|
|
presidio/ PII detector service (typed findings) + Dockerfile
|
|
seed/ fake customer (Wang Xiaoming) + seed script
|
|
ui/ advisor UI (restored-vs-tokenized split view)
|
|
scripts/ deploy.sh, agentcore_setup.sh, teardown.sh
|
|
fusion/ POLICY_SETUP.md (SaaS console config — the manual part)
|
|
```
|
|
|
|
## Prerequisites
|
|
`aws-cli` (creds + region — this demo runs in `ap-southeast-1`), `terraform >= 1.5`,
|
|
`podman` or `docker` (Presidio image), `python3` + a venv with `boto3`,
|
|
`bedrock-agentcore-starter-toolkit` (`pip install`, provides `agentcore`), Bedrock
|
|
access for the **apac** Claude 3.5 Sonnet v2 *inference profile*
|
|
(`apac.anthropic.claude-3-5-sonnet-20241022-v2:0` — the raw id is not on-demand in
|
|
this region), and — for the production path — the shared **Fusion SaaS** instance.
|
|
|
|
## Deploy
|
|
The live stack was brought up in this order (authoritative runbook: the task notes
|
|
in `CLAUDE.md`, which record the exact sequencing and gotchas):
|
|
1. `terraform -chdir=terraform apply` core infra (DynamoDB, RAG + tokenize/restore/
|
|
orchestrator Lambdas, API Gateway, Presidio on Fargate, UI on S3+CloudFront).
|
|
Presidio needs its image in ECR and `presidio_url` set to the task's **private** IP.
|
|
2. Seed the synthetic customer + the demo vault token (`seed/seed.py`).
|
|
3. `scripts/agentcore_setup.sh` → MCP Gateway + RAG tool target.
|
|
4. `agentcore configure` + `agentcore launch` → Runtime ARN; then grant the runtime
|
|
execution role `bedrock-agentcore:InvokeGateway` (command in `agentcore_setup.sh`).
|
|
5. Put the Runtime ARN in `terraform/local.auto.tfvars` (`agent_runtime_arn`) and
|
|
re-apply so `/demo` can invoke it.
|
|
|
|
## Demo script (Fusion-less dry run — maps to the 8 steps)
|
|
```bash
|
|
cd terraform && DEMO=$(terraform output -raw demo_url) && UI=$(terraform output -raw ui_url); cd ..
|
|
|
|
# 1-2, 4-8: advisor query -> tokenize -> agent (on tokens) -> restore, in one call:
|
|
curl -s -X POST "$DEMO" -H 'content-type: application/json' \
|
|
-d '{"query":"請幫我整理王小明最近三個月的理財往來，並給我下次拜訪話術。"}' | python3 -m json.tool
|
|
# -> deidentified_prompt: "...CUST_xxxxxx..." (what left for the cloud)
|
|
# agent_tokenized: talking points, tokens only
|
|
# final: "（客戶：王小明）..." (identity restored on-prem)
|
|
|
|
# 3. money shot — the cloud runtime trace only ever shows the token:
|
|
LG=/aws/bedrock-agentcore/runtimes/<runtime-id>-DEFAULT
|
|
aws logs filter-log-events --log-group-name "$LG" --filter-pattern '"王小明"' --query 'length(events)' # 0
|
|
aws logs filter-log-events --log-group-name "$LG" --filter-pattern '"CUST_"' --query 'length(events)' # >0
|
|
|
|
# 8 (visual): open the split-view UI and submit the same query
|
|
echo "$UI"
|
|
```
|
|
|
|
## Teardown
|
|
```bash
|
|
bash scripts/teardown.sh # deletes Gateway/Runtime/Memory + `terraform destroy` + ECR repos
|
|
```
|
|
Leftover-but-free after teardown: the hand-made default VPC, the
|
|
`AmazonBedrockAgentCoreSDKRuntime-*` role, and a CodeBuild project (delete by hand
|
|
if you want it spotless).
|
|
|
|
## Honest caveats
|
|
- **zh-TW detection is demo-narrow** — tuned to the scripted entities, not
|
|
production recall. That remains the real-engagement risk.
|
|
- **Per-request randomization** lives in the `/tokenize` mint step; confirm it
|
|
satisfies HNCB's "different each time" requirement.
|
|
- **The agentic token-resolution loop** (agent tool call → on-prem RAG resolves the
|
|
token) is custom orchestration by design — where Fusion's depth is the argument.
|