Files

Conan Scott 78d67a2469 Implement reversible PII de-identification round trip (T1–T7)

Build the AWS side of the HNCB demo end to end (region ap-southeast-1):

- T1 /tokenize + T2 /restore: Lambdas behind a public API Gateway (shared-secret
  auth), Presidio detection, random per-request tokens, DynamoDB vault; overlap
  resolution so a ROC ID stays TW_ROC_ID.
- T3: Presidio made private (SG-locked to the tokenize Lambda in-VPC; DynamoDB
  gateway endpoint); only /tokenize + /restore are public.
- T4: RAG Lambda registered as an MCP tool on an AgentCore Gateway (AWS_IAM/SigV4);
  agentcore_setup.sh + a SigV4 MCP invoke test.
- T5: Strands agent deployed to AgentCore Runtime; SigV4 gateway auth, apac
  inference profile, pinned deps.
- T6: advisor UI on S3+CloudFront with a Fusion-less demo orchestrator (/demo)
  chaining tokenize -> runtime -> restore.
- T7: README runbook + trace check; teardown deletes gateway/runtime/memory/ECR.

Verified live: the cloud AgentCore/Bedrock trace shows only tokens, never the
real name. Secrets stay in gitignored local.auto.tfvars.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-01 17:10:58 +10:00

3.3 KiB

Raw Blame History

Fusion (shared SaaS) policy setup (steps 2, 3-route, 8)

Fusion is a shared SaaS instance — configured in the Amplify AI Gateway console, not deployed by this repo. Because it's SaaS it cannot reach private VPC resources or use local AWS creds, so it does not call Presidio or DynamoDB directly. Instead it calls two public HTTPS endpoints this repo exposes (built in gateway_api/, tasks T1/T2), which do the detection, minting, vault writes, and restore on the AWS side. Fusion owns the orchestration and routing.

Endpoints Fusion calls

TOKENIZE_URL = https://<api-id>.execute-api.<region>.amazonaws.com/tokenize (ingress: detect + mint + vault-write + splice). Get the live value with terraform -chdir=terraform output -raw tokenize_url.
RESTORE_URL = https://<...>/restore (egress: vault lookup + re-attach identity — T2)
AGENT_RUNTIME_ARN (or its HTTPS invoke endpoint) = printed by scripts/agentcore_setup.sh
Auth (T1, live): the endpoint is a public API Gateway HTTP API; it requires a shared secret in the x-api-key header (a Authorization: Bearer <secret> header also works). Configure Fusion's outbound request to send it. The secret is the Terraform tokenize_api_key var (kept in gitignored terraform/local.auto.tfvars, never committed) — hand it to the Fusion console operator out of band.
- Note: a Lambda Function URL was the first choice, but this account's SCP blocks unauthenticated Function URLs, so the public front door is API Gateway instead.

Ingress policy (advisor request → cloud)

Authenticate the advisor; apply the RBAC / business-purpose check.
Tokenize: POST { query } to TOKENIZE_URL. Receive { deidentified_prompt, session_id }. (The endpoint runs detect → mint → vault-write → splice; detection returns typed findings, never a redacted blob.)
Route: invoke the agent with { "prompt": deidentified_prompt }.
Trace: log the tokenized payload only — never the raw query.

Egress policy (cloud response → advisor)

Receive the agent's de-identified result.
Restore: POST { session_id, text } to RESTORE_URL; receive { final }.
Return { "final": ..., "deidentified_prompt": ... } so the UI shows the split view.

Vault item shape (DynamoDB, written by /tokenize)

One row per detected entity. value is the original PII (so /restore can put it back); a resolvable PERSON also gets a customer_id so the RAG tool can turn the token into a de-identified evidence package.

{ "token": "CUST_863651", "type": "PERSON",    "value": "王小明",
  "session_id": "<conv id>", "expires_at": <epoch+ttl>, "customer_id": "cust-0001" }
{ "token": "TW_683250",   "type": "TW_ROC_ID", "value": "A123456789",
  "session_id": "<conv id>", "expires_at": <epoch+ttl> }

(The seed's --with-demo-token writes a different type=CUSTOMER, value=cust-0001 row; that's only a standalone RAG test aid, not what /tokenize mints.)

Why this split

Keeping detection, minting, and the vault behind /tokenize and /restore means the only things exposed to the SaaS gateway are two authenticated HTTPS endpoints — no AWS creds or private resources leave the account, and Fusion stays a pure orchestration/routing layer. That is also the cleanest story on camera: the gateway owns the flow; the cloud only ever sees tokens.

3.3 KiB Raw Blame History