From 25a005e7408da719c6177a3f5ae38b240f7b38b5 Mon Sep 17 00:00:00 2001 From: Conan Scott Date: Thu, 16 Jul 2026 23:54:27 +0000 Subject: [PATCH] Add T8 task: tokenize credit card + mask email + rotation proof --- .claude/tasks/T8-multi-entity-tokenization.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 .claude/tasks/T8-multi-entity-tokenization.md diff --git a/.claude/tasks/T8-multi-entity-tokenization.md b/.claude/tasks/T8-multi-entity-tokenization.md new file mode 100644 index 0000000..2f2fde0 --- /dev/null +++ b/.claude/tasks/T8-multi-entity-tokenization.md @@ -0,0 +1,82 @@ +# T8 — Multi-entity treatment: tokenize Credit Card + mask Email (partner feedback) + +Status: ⬜ TODO + +## Why +Partner watched the demo video and gave two pieces of feedback: +1. Token **rotation** wasn't hammered home enough (highlighting the text on screen + wasn't sufficient — make it undeniable). +2. They want more than the name de-identified — fields matching their "Azure + Processing Content" table (ID/CC/Name tokenized, Email/Mobile masked, etc.). + +Name (`PERSON`) and the Taiwan ROC ID (`TW_ROC_ID`) are already tokenized (T1/T2). +This task adds **Credit Card (tokenize)** and **Email (mask)** and makes rotation +visible. Mobile masking and Address/Amount generalization are explicitly OUT of +scope here (leave for a later T9 if they ask). + +## Entity → treatment policy +Introduce a small policy map in the tokenize handler and branch per finding: + +| Entity | Treatment | Token / mask form | Notes | +|---|---|---|---| +| `PERSON` | tokenize (rotate + vault) | `tkn_NAME_` | already works; just adopt the `tkn__` naming | +| `TW_ROC_ID` | tokenize (rotate + vault) | `tkn_ID_` | already works; adopt naming | +| `CREDIT_CARD` | tokenize (rotate + vault) | `tkn_CC_` | enable Presidio built-in `CreditCardRecognizer` (has Luhn) for `zh` | +| `EMAIL_ADDRESS` | **mask** (one-way, partial) | e.g. `m***.w***@gmail.com` | enable built-in `EmailRecognizer`; **no vault row, not restored** | + +Adopt `tkn__` naming so the on-screen output matches the partner's table +one-to-one. + +## Where +- `presidio/app.py` — ensure `CREDIT_CARD` and `EMAIL_ADDRESS` recognizers are active + for the `zh` analyzer (they're built-in; confirm they load alongside the zh model + and the custom ROC-ID recognizer; add them to supported entities if needed). +- `gateway_api/tokenize/handler.py` — read the treatment policy; for `tokenize` + entities keep the current mint→vault→splice; for `mask` entities apply a + deterministic partial mask (keep the first char of each local-part segment + the + full domain for email), splice, and **do not** write a vault row. +- `gateway_api/restore/handler.py` — no change needed for masked fields (they're + never restored). Confirm restore still only touches tokenized `tkn_*` values. +- `gateway_api/orchestrator/` (`/demo`) + `ui/index.html` — add the rotation proof + (below). +- `seed/customers.json` — optional: add `credit_card` / `email` fields for realism; + not required since detection reads from the prompt text, not the DB. + +## Critical gotchas (do NOT "fix" these) +1. **Masked fields must NOT rotate.** Masking is deterministic by design — the same + email masks to the same string every run. Only tokenized fields (NAME/ID/CC) + rotate. Do not add randomness to the mask. If the partner "notices" the email not + changing, that is correct behaviour, not a bug. +2. **Email is one-way.** The advisor sees it masked in the final answer, never + restored. If product later wants the real email back, that is + tokenization-with-a-masked-looking-surrogate (goes through the vault) — a + different path. Don't silently switch masking to reversible. + +## Rotation proof (the actual ask from feedback #1) +Make rotation undeniable on camera: run the **identical prompt twice** and show the +tokenized fields differ across runs (`tkn_ID_8F72…` vs `tkn_ID_Xq93…`) while the +**final restored answer is byte-identical** and the masked email is identical. + +- Easiest implementation: a `?runs=2` mode (or a second call) on `/demo` that returns + both tokenized prompts plus the single restored answer. +- Surface it in `ui/index.html` as a "same input → different tokens → same answer" + split. Optionally show the two vault entries: different tokens, same real value. +- This is far harder to miss than highlighted text. + +## Acceptance criteria +- A prompt containing name + ROC ID + credit card + email produces **three rotating + `tkn_*` tokens** (NAME/ID/CC) and **one stable partial-masked email**. +- Running the same prompt twice yields **different** NAME/ID/CC tokens but an + **identical** masked email and an **identical** restored final answer. +- The vault contains rows only for the tokenized fields — never the email. +- On-screen token names match the partner's table (`tkn_ID_…`, `tkn_CC_…`, `tkn_NAME_…`). + +## Demo prompt (partner's) +> "Analyze customer Wang Xiaoming (ID No. A123456789, Credit Card 4567-1234-5678-9012, +> Mobile 0912-345-678, Email ming.wang@gmail.com, Address: No. 3, Songren Rd., Xinyi +> Dist., Taipei) — most recent transaction of NTD 58,600 — and provide a visit summary +> and service recommendations." + +(Mobile, Address, Amount are out of scope for T8. Note internally: this prompt is +contrived — a real agent would have done the research to build it — but we're giving +the partner the on-screen show they asked for.)