cscott/hncb-fusion-deid-demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
edcb4567ef1aee0f6fd49d1b5d80ddf389945be4
hncb-fusion-deid-demo/docs/architecture.md
Conan Scott afea012d32 docs: add architecture.md (flow + component diagrams) 
Mermaid flowchart + sequence diagram and an ASCII fallback for the reversible
de-identification round trip, plus component/auth/network reference tables and
the trust-zone boundaries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 17:20:36 +10:00

193 lines
13 KiB
Markdown
Raw Blame History

# Architecture & data flow
Reversible PII de-identification round trip for HNCB. An advisor query with a real
name is tokenized before it leaves for the cloud; the AgentCore agent reasons on
tokens only; identity is restored on-prem before the advisor sees the answer.
Everything runs in **one AWS account** (`ap-southeast-1`); the trust zones are
**logical**, represented by tags (`Zone=on-prem-VPC-A` vs `Zone=cloud-VPC-B`) — the
demo proves data-flow behaviour, not physical residency.
> **Core invariant:** cloud-zone components (AgentCore Runtime, Gateway, Bedrock)
> never receive the real name. Proven live — for one session the token `CUST_*`
> appeared in 13 CloudWatch trace events while `王小明` / `A123456789` appeared in 0.
## Component reference
| # | Component | Repo | AWS resource | Zone | Sees real PII? |
|---|---|---|---|---|---|
| UI | Advisor split-view UI | `ui/` | S3 + CloudFront | cloud (host) | shows restored answer to the advisor |
| entry | Orchestration | `fusion/` (prod) · `gateway_api/orchestrator/` (dry run) | Fusion SaaS · `/demo` Lambda | — | ✅ owns the flow |
| 2 | `/tokenize` | `gateway_api/tokenize/` | Lambda (in-VPC) + API Gateway | on-prem | ✅ mints the map |
| 2 | PII detector | `presidio/` | Fargate (private, SG-locked `:5001`) | on-prem | typed findings only |
| 2,4,8 | Token vault | `terraform/main.tf` | DynamoDB `…-vault` | on-prem | ✅ reversible map |
| 4-6 | Customer records | `seed/` | DynamoDB `…-customers` | on-prem | ✅ never leave zone |
| 3,7 | Cloud agent + model | `agent/` | AgentCore Runtime + Bedrock (apac Claude 3.5 Sonnet v2) | **cloud** | ❌ tokens only |
| 4 | Tool bridge | `scripts/agentcore_setup.sh` | AgentCore Gateway (MCP, AWS_IAM) | **cloud** | ❌ tokens only |
| 4-6 | RAG tool | `lambda_rag/` | Lambda | on-prem | ✅ resolves token, returns none |
| 8 | `/restore` | `gateway_api/restore/` | Lambda + API Gateway | on-prem | ✅ re-attaches identity |
## Flow diagram
```mermaid
flowchart TB
advisor(["Advisor (browser)"])
ui["Advisor UI<br/>S3 + CloudFront<br/>(split view)"]
subgraph entry["Entry / orchestration"]
fusion["Fusion SaaS (prod)<br/>— or —<br/>/demo Lambda (Fusion-less dry run)"]
end
subgraph cloudzone["CLOUD zone (Zone=cloud-VPC-B) — sees TOKENS only"]
runtime["AgentCore Runtime<br/>Strands agent + Bedrock<br/>apac Claude 3.5 Sonnet v2"]
gateway["AgentCore Gateway<br/>MCP · AWS_IAM / SigV4"]
end
subgraph onprem["ON-PREM zone (Zone=on-prem-VPC-A) — holds the reversible map + PII"]
tokenize["/tokenize Lambda (in-VPC)<br/>detect → mint → vault → splice"]
presidio["Presidio detector<br/>Fargate (PRIVATE, SG-locked :5001)"]
vault[("DynamoDB VAULT<br/>token &lt;-&gt; PII (+customer_id)")]
customers[("DynamoDB CUSTOMERS<br/>raw records — never leave")]
rag["RAG tool Lambda<br/>token -> evidence (no PII out)"]
restore["/restore Lambda<br/>re-attach identity"]
end
advisor -- "1 query: 王小明 + A123456789" --> ui
ui -- "POST {query}" --> fusion
fusion -- "2 {query}" --> tokenize
tokenize -- "POST /analyze" --> presidio
presidio -- "typed findings" --> tokenize
tokenize -- "write {token,type,value,session}" --> vault
tokenize -- "3 deidentified_prompt (CUST_*)" --> runtime
runtime -- "4 tools/call get_customer_activity_summary(CUST_*)" --> gateway
gateway -- "5 invoke (SigV4)" --> rag
rag -- "6 resolve token" --> vault
rag -- "read record" --> customers
rag -- "7 evidence package (token-keyed, no PII)" --> runtime
runtime -- "talking points (tokens only)" --> fusion
fusion -- "8 {session_id, text}" --> restore
restore -- "lookup session tokens" --> vault
restore -- "final: （客戶：王小明）…" --> fusion
fusion -- "{final, deidentified_prompt, agent_tokenized}" --> ui
ui -- "split view: restored vs tokenized" --> advisor
```
## Sequence (the 8-step round trip)
```mermaid
sequenceDiagram
autonumber
actor A as Advisor
participant UI as UI (S3/CloudFront)
participant F as Fusion or demo
participant TK as tokenize (on-prem)
participant PR as Presidio (private)
participant V as Vault (DynamoDB)
participant RT as AgentCore Runtime + Bedrock (cloud)
participant GW as AgentCore Gateway (cloud)
participant RG as RAG Lambda (on-prem)
participant RS as restore (on-prem)
A->>UI: query with 王小明 + A123456789
UI->>F: POST {query}
F->>TK: {query}
TK->>PR: POST /analyze
PR-->>TK: typed findings
TK->>V: write token to PII map (session)
TK-->>F: deidentified_prompt (CUST_*)
F->>RT: {prompt: CUST_*}
Note over RT,GW: cloud sees TOKENS only
RT->>GW: tools/call get_customer_activity_summary(CUST_*)
GW->>RG: invoke (SigV4)
RG->>V: resolve token to customer_id
RG-->>RT: de-identified evidence package
RT-->>F: talking points (tokens only)
F->>RS: {session_id, text}
RS->>V: lookup session tokens
RS-->>F: final (王小明 restored)
F-->>UI: {final, deidentified_prompt, agent_tokenized}
UI-->>A: split view (restored vs tokenized)
```
## ASCII fallback
```
┌──────────────────────────────────────────────┐
Advisor │ "請問客戶 王小明 (A123456789) 最近三個月…" │ ← REAL PII
(browser) └──────────────────────────────────────────────┘
│ │
│ 1 │
▼ ▼
┌─────────────────┐ ┌───────────────────────────┐
│ Advisor UI │ │ Entry / orchestration │
│ S3 + CloudFront │──POST─────▶│ • Fusion SaaS (prod) │ stands in for Fusion
│ (split view) │ {query} │ • /demo Lambda (dry run) │ only, in the demo
└─────────────────┘◀───────────│ tokenize→agent→restore │
▲ {final, └───────────┬───────────────┘
│ deidentified, │
│ agent_tokenized} │ 2 {query}
════════╪════════════════════════════════════╪════════ TRUST BOUNDARY (public HTTPS + x-api-key)
ON-PREM│ (Zone=on-prem-VPC-A) ▼
│ ┌───────────────────────┐ findings ┌────────────────────┐
│ │ /tokenize Lambda │────────────▶│ Presidio detector │
│ │ (in VPC) │◀────────────│ Fargate (PRIVATE, │
│ │ detect→mint→vault→splice│ │ SG-locked :5001) │
│ └───────┬───────────────┘ └────────────────────┘
│ │ write {token,type,value,session}
│ ▼
│ ┌───────────────────────┐ (VPC gateway endpoint)
│ │ DynamoDB VAULT │◀────────────────┐
│ │ token ⇄ PII (+cust_id) │ │
│ └───────────────────────┘ │
│ deidentified_prompt = "…CUST_317499…" ← TOKENS ONLY │
════════╪═════════════════════════════════│═════════════════════════════════╪══════════════
CLOUD │ (Zone=cloud-VPC-B) │ 3 │ resolve token
│ ▼ │
│ ┌────────────────────────┐ │
│ │ AgentCore RUNTIME │ │
│ │ Strands agent + Bedrock│ │
│ │ (apac Claude 3.5 Sonnet)│ │
│ └───────────┬────────────┘ │
│ │ 4 tools/call (MCP, SigV4) │
│ ▼ get_customer_activity_summary │
│ ┌────────────────────────┐ │
│ │ AgentCore GATEWAY (MCP) │ │
│ │ AWS_IAM auth │ │
│ └───────────┬────────────┘ │
│ ← trace shows ONLY tokens │ 5 invoke │
════════╪══(王小明: 0 hits, CUST_: 13)═══│═════════════════════════════════════╪══════════════
ON-PREM│ ▼ │
│ ┌────────────────────────┐ 6 reads │
│ │ RAG tool Lambda │─────────────────────┘
│ │ token→customer_id→ │ ┌────────────────────┐
│ │ de-identified evidence │─────▶│ DynamoDB CUSTOMERS │ ← raw PII
│ │ (raw record stays here) │ read │ (never leaves zone)│ stays put
│ └───────────┬────────────┘ └────────────────────┘
│ │ 7 evidence package (token-keyed, no PII)
│ ▼ → agent writes talking points (tokens only)
│ ┌────────────────────────┐
│ 8 {session_id, │ /restore Lambda │
└───────────────────▶│ vault lookup by session │ prepend 王小明 + swap inline tokens
final = "（客戶：王小明）…" │ re-attach identity │
← PII restored on-prem └────────────────────────┘
```
## Auth & network per hop
| Hop | Transport | Auth |
|---|---|---|
| UI → entry | public HTTPS | `/demo` (dry run) or Fusion SaaS (prod) |
| entry → `/tokenize`, `/restore` | public API Gateway (HTTPS) | shared secret (`x-api-key` / bearer, checked in-handler) |
| `/tokenize` → Presidio | private (in-VPC) | Presidio SG allows only the tokenize Lambda's SG |
| `/tokenize`,`/restore`,RAG → DynamoDB | AWS API | IAM (tokenize reaches it via a VPC gateway endpoint) |
| Runtime → Gateway → RAG | MCP over HTTPS | SigV4 (`AWS_IAM`); runtime role has `bedrock-agentcore:InvokeGateway`, gateway role has `lambda:InvokeFunction` |
Notes:
- The **UI is cloud-hosted** but only ever displays the *restored* answer to the
authorized advisor; the cloud *reasoning* path (Runtime/Gateway/Bedrock) is the
part that never sees PII.
- **Fusion is shared SaaS** and is never built or hosted here. The `/demo`
orchestrator only stands in for Fusion's orchestration in the Fusion-less dry run.
- Tokens are **random per request** (`CUST_<rand>` for a PERSON, `TW_<rand>` for a
Taiwan ROC ID); the reversible map lives only in the on-prem vault.
Reference in New Issue View Git Blame Copy Permalink