Implement reversible PII de-identification round trip (T1–T7)

Build the AWS side of the HNCB demo end to end (region ap-southeast-1):

- T1 /tokenize + T2 /restore: Lambdas behind a public API Gateway (shared-secret
  auth), Presidio detection, random per-request tokens, DynamoDB vault; overlap
  resolution so a ROC ID stays TW_ROC_ID.
- T3: Presidio made private (SG-locked to the tokenize Lambda in-VPC; DynamoDB
  gateway endpoint); only /tokenize + /restore are public.
- T4: RAG Lambda registered as an MCP tool on an AgentCore Gateway (AWS_IAM/SigV4);
  agentcore_setup.sh + a SigV4 MCP invoke test.
- T5: Strands agent deployed to AgentCore Runtime; SigV4 gateway auth, apac
  inference profile, pinned deps.
- T6: advisor UI on S3+CloudFront with a Fusion-less demo orchestrator (/demo)
  chaining tokenize -> runtime -> restore.
- T7: README runbook + trace check; teardown deletes gateway/runtime/memory/ECR.

Verified live: the cloud AgentCore/Bedrock trace shows only tokens, never the
real name. Secrets stay in gitignored local.auto.tfvars.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-01 17:10:58 +10:00
parent 6b2a051be3
commit 78d67a2469
21 changed files with 1439 additions and 107 deletions

View File

@@ -8,10 +8,18 @@ directly. Instead it calls two **public HTTPS endpoints** this repo exposes
writes, and restore on the AWS side. Fusion owns the orchestration and routing.
## Endpoints Fusion calls
- `TOKENIZE_URL` = `https://<...>/tokenize` (ingress: detect + mint + vault-write + splice)
- `RESTORE_URL` = `https://<...>/restore` (egress: vault lookup + re-attach identity)
- `TOKENIZE_URL` = `https://<api-id>.execute-api.<region>.amazonaws.com/tokenize`
(ingress: detect + mint + vault-write + splice). Get the live value with
`terraform -chdir=terraform output -raw tokenize_url`.
- `RESTORE_URL` = `https://<...>/restore` (egress: vault lookup + re-attach identity — T2)
- `AGENT_RUNTIME_ARN` (or its HTTPS invoke endpoint) = printed by `scripts/agentcore_setup.sh`
- Secure all three with an API key / OAuth from the Fusion outbound config.
- **Auth (T1, live):** the endpoint is a public API Gateway HTTP API; it requires a
shared secret in the **`x-api-key`** header (a `Authorization: Bearer <secret>`
header also works). Configure Fusion's outbound request to send it. The secret is
the Terraform `tokenize_api_key` var (kept in gitignored `terraform/local.auto.tfvars`,
never committed) — hand it to the Fusion console operator out of band.
- *Note:* a Lambda Function URL was the first choice, but this account's SCP blocks
unauthenticated Function URLs, so the public front door is API Gateway instead.
## Ingress policy (advisor request → cloud)
1. **Authenticate** the advisor; apply the RBAC / business-purpose check.
@@ -27,10 +35,17 @@ writes, and restore on the AWS side. Fusion owns the orchestration and routing.
3. Return `{ "final": ..., "deidentified_prompt": ... }` so the UI shows the split view.
## Vault item shape (DynamoDB, written by /tokenize)
One row per detected entity. `value` is the original PII (so `/restore` can put it
back); a resolvable PERSON also gets a `customer_id` so the RAG tool can turn the
token into a de-identified evidence package.
```
{ "token": "CUST_000123", "type": "CUSTOMER", "value": "cust-0001",
{ "token": "CUST_863651", "type": "PERSON", "value": "王小明",
"session_id": "<conv id>", "expires_at": <epoch+ttl>, "customer_id": "cust-0001" }
{ "token": "TW_683250", "type": "TW_ROC_ID", "value": "A123456789",
"session_id": "<conv id>", "expires_at": <epoch+ttl> }
```
(The seed's `--with-demo-token` writes a different `type=CUSTOMER, value=cust-0001`
row; that's only a standalone RAG test aid, not what `/tokenize` mints.)
## Why this split
Keeping detection, minting, and the vault behind `/tokenize` and `/restore` means