diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..94efe45 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,192 @@ +# Architecture & data flow + +Reversible PII de-identification round trip for HNCB. An advisor query with a real +name is tokenized before it leaves for the cloud; the AgentCore agent reasons on +tokens only; identity is restored on-prem before the advisor sees the answer. + +Everything runs in **one AWS account** (`ap-southeast-1`); the trust zones are +**logical**, represented by tags (`Zone=on-prem-VPC-A` vs `Zone=cloud-VPC-B`) — the +demo proves data-flow behaviour, not physical residency. + +> **Core invariant:** cloud-zone components (AgentCore Runtime, Gateway, Bedrock) +> never receive the real name. Proven live — for one session the token `CUST_*` +> appeared in 13 CloudWatch trace events while `王小明` / `A123456789` appeared in 0. + +## Component reference + +| # | Component | Repo | AWS resource | Zone | Sees real PII? | +|---|---|---|---|---|---| +| UI | Advisor split-view UI | `ui/` | S3 + CloudFront | cloud (host) | shows restored answer to the advisor | +| entry | Orchestration | `fusion/` (prod) · `gateway_api/orchestrator/` (dry run) | Fusion SaaS · `/demo` Lambda | — | ✅ owns the flow | +| 2 | `/tokenize` | `gateway_api/tokenize/` | Lambda (in-VPC) + API Gateway | on-prem | ✅ mints the map | +| 2 | PII detector | `presidio/` | Fargate (private, SG-locked `:5001`) | on-prem | typed findings only | +| 2,4,8 | Token vault | `terraform/main.tf` | DynamoDB `…-vault` | on-prem | ✅ reversible map | +| 4-6 | Customer records | `seed/` | DynamoDB `…-customers` | on-prem | ✅ never leave zone | +| 3,7 | Cloud agent + model | `agent/` | AgentCore Runtime + Bedrock (apac Claude 3.5 Sonnet v2) | **cloud** | ❌ tokens only | +| 4 | Tool bridge | `scripts/agentcore_setup.sh` | AgentCore Gateway (MCP, AWS_IAM) | **cloud** | ❌ tokens only | +| 4-6 | RAG tool | `lambda_rag/` | Lambda | on-prem | ✅ resolves token, returns none | +| 8 | `/restore` | `gateway_api/restore/` | Lambda + API Gateway | on-prem | ✅ re-attaches identity | + +## Flow diagram + +```mermaid +flowchart TB + advisor(["Advisor (browser)"]) + ui["Advisor UI
S3 + CloudFront
(split view)"] + + subgraph entry["Entry / orchestration"] + fusion["Fusion SaaS (prod)
— or —
/demo Lambda (Fusion-less dry run)"] + end + + subgraph cloudzone["CLOUD zone (Zone=cloud-VPC-B) — sees TOKENS only"] + runtime["AgentCore Runtime
Strands agent + Bedrock
apac Claude 3.5 Sonnet v2"] + gateway["AgentCore Gateway
MCP · AWS_IAM / SigV4"] + end + + subgraph onprem["ON-PREM zone (Zone=on-prem-VPC-A) — holds the reversible map + PII"] + tokenize["/tokenize Lambda (in-VPC)
detect → mint → vault → splice"] + presidio["Presidio detector
Fargate (PRIVATE, SG-locked :5001)"] + vault[("DynamoDB VAULT
token <-> PII (+customer_id)")] + customers[("DynamoDB CUSTOMERS
raw records — never leave")] + rag["RAG tool Lambda
token -> evidence (no PII out)"] + restore["/restore Lambda
re-attach identity"] + end + + advisor -- "1 query: 王小明 + A123456789" --> ui + ui -- "POST {query}" --> fusion + fusion -- "2 {query}" --> tokenize + tokenize -- "POST /analyze" --> presidio + presidio -- "typed findings" --> tokenize + tokenize -- "write {token,type,value,session}" --> vault + tokenize -- "3 deidentified_prompt (CUST_*)" --> runtime + runtime -- "4 tools/call get_customer_activity_summary(CUST_*)" --> gateway + gateway -- "5 invoke (SigV4)" --> rag + rag -- "6 resolve token" --> vault + rag -- "read record" --> customers + rag -- "7 evidence package (token-keyed, no PII)" --> runtime + runtime -- "talking points (tokens only)" --> fusion + fusion -- "8 {session_id, text}" --> restore + restore -- "lookup session tokens" --> vault + restore -- "final: （客戶：王小明）…" --> fusion + fusion -- "{final, deidentified_prompt, agent_tokenized}" --> ui + ui -- "split view: restored vs tokenized" --> advisor +``` + +## Sequence (the 8-step round trip) + +```mermaid +sequenceDiagram + autonumber + actor A as Advisor + participant UI as UI (S3/CloudFront) + participant F as Fusion or demo + participant TK as tokenize (on-prem) + participant PR as Presidio (private) + participant V as Vault (DynamoDB) + participant RT as AgentCore Runtime + Bedrock (cloud) + participant GW as AgentCore Gateway (cloud) + participant RG as RAG Lambda (on-prem) + participant RS as restore (on-prem) + + A->>UI: query with 王小明 + A123456789 + UI->>F: POST {query} + F->>TK: {query} + TK->>PR: POST /analyze + PR-->>TK: typed findings + TK->>V: write token to PII map (session) + TK-->>F: deidentified_prompt (CUST_*) + F->>RT: {prompt: CUST_*} + Note over RT,GW: cloud sees TOKENS only + RT->>GW: tools/call get_customer_activity_summary(CUST_*) + GW->>RG: invoke (SigV4) + RG->>V: resolve token to customer_id + RG-->>RT: de-identified evidence package + RT-->>F: talking points (tokens only) + F->>RS: {session_id, text} + RS->>V: lookup session tokens + RS-->>F: final (王小明 restored) + F-->>UI: {final, deidentified_prompt, agent_tokenized} + UI-->>A: split view (restored vs tokenized) +``` + +## ASCII fallback + +``` + ┌──────────────────────────────────────────────┐ + Advisor │ "請問客戶 王小明 (A123456789) 最近三個月…" │ ← REAL PII + (browser) └──────────────────────────────────────────────┘ + │ │ + │ 1 │ + ▼ ▼ + ┌─────────────────┐ ┌───────────────────────────┐ + │ Advisor UI │ │ Entry / orchestration │ + │ S3 + CloudFront │──POST─────▶│ • Fusion SaaS (prod) │ stands in for Fusion + │ (split view) │ {query} │ • /demo Lambda (dry run) │ only, in the demo + └─────────────────┘◀───────────│ tokenize→agent→restore │ + ▲ {final, └───────────┬───────────────┘ + │ deidentified, │ + │ agent_tokenized} │ 2 {query} +════════╪════════════════════════════════════╪════════ TRUST BOUNDARY (public HTTPS + x-api-key) + ON-PREM│ (Zone=on-prem-VPC-A) ▼ + │ ┌───────────────────────┐ findings ┌────────────────────┐ + │ │ /tokenize Lambda │────────────▶│ Presidio detector │ + │ │ (in VPC) │◀────────────│ Fargate (PRIVATE, │ + │ │ detect→mint→vault→splice│ │ SG-locked :5001) │ + │ └───────┬───────────────┘ └────────────────────┘ + │ │ write {token,type,value,session} + │ ▼ + │ ┌───────────────────────┐ (VPC gateway endpoint) + │ │ DynamoDB VAULT │◀────────────────┐ + │ │ token ⇄ PII (+cust_id) │ │ + │ └───────────────────────┘ │ + │ deidentified_prompt = "…CUST_317499…" ← TOKENS ONLY │ +════════╪═════════════════════════════════│═════════════════════════════════╪══════════════ + CLOUD │ (Zone=cloud-VPC-B) │ 3 │ resolve token + │ ▼ │ + │ ┌────────────────────────┐ │ + │ │ AgentCore RUNTIME │ │ + │ │ Strands agent + Bedrock│ │ + │ │ (apac Claude 3.5 Sonnet)│ │ + │ └───────────┬────────────┘ │ + │ │ 4 tools/call (MCP, SigV4) │ + │ ▼ get_customer_activity_summary │ + │ ┌────────────────────────┐ │ + │ │ AgentCore GATEWAY (MCP) │ │ + │ │ AWS_IAM auth │ │ + │ └───────────┬────────────┘ │ + │ ← trace shows ONLY tokens │ 5 invoke │ +════════╪══(王小明: 0 hits, CUST_: 13)═══│═════════════════════════════════════╪══════════════ + ON-PREM│ ▼ │ + │ ┌────────────────────────┐ 6 reads │ + │ │ RAG tool Lambda │─────────────────────┘ + │ │ token→customer_id→ │ ┌────────────────────┐ + │ │ de-identified evidence │─────▶│ DynamoDB CUSTOMERS │ ← raw PII + │ │ (raw record stays here) │ read │ (never leaves zone)│ stays put + │ └───────────┬────────────┘ └────────────────────┘ + │ │ 7 evidence package (token-keyed, no PII) + │ ▼ → agent writes talking points (tokens only) + │ ┌────────────────────────┐ + │ 8 {session_id, │ /restore Lambda │ + └───────────────────▶│ vault lookup by session │ prepend 王小明 + swap inline tokens + final = "（客戶：王小明）…" │ re-attach identity │ + ← PII restored on-prem └────────────────────────┘ +``` + +## Auth & network per hop + +| Hop | Transport | Auth | +|---|---|---| +| UI → entry | public HTTPS | `/demo` (dry run) or Fusion SaaS (prod) | +| entry → `/tokenize`, `/restore` | public API Gateway (HTTPS) | shared secret (`x-api-key` / bearer, checked in-handler) | +| `/tokenize` → Presidio | private (in-VPC) | Presidio SG allows only the tokenize Lambda's SG | +| `/tokenize`,`/restore`,RAG → DynamoDB | AWS API | IAM (tokenize reaches it via a VPC gateway endpoint) | +| Runtime → Gateway → RAG | MCP over HTTPS | SigV4 (`AWS_IAM`); runtime role has `bedrock-agentcore:InvokeGateway`, gateway role has `lambda:InvokeFunction` | + +Notes: +- The **UI is cloud-hosted** but only ever displays the *restored* answer to the + authorized advisor; the cloud *reasoning* path (Runtime/Gateway/Bedrock) is the + part that never sees PII. +- **Fusion is shared SaaS** and is never built or hosted here. The `/demo` + orchestrator only stands in for Fusion's orchestration in the Fusion-less dry run. +- Tokens are **random per request** (`CUST_` for a PERSON, `TW_` for a + Taiwan ROC ID); the reversible map lives only in the on-prem vault.