Add T8 task: tokenize credit card + mask email + rotation proof

This commit is contained in:
2026-07-16 23:54:27 +00:00
parent 0d4d4dcc67
commit 25a005e740

View File

@@ -0,0 +1,82 @@
# T8 — Multi-entity treatment: tokenize Credit Card + mask Email (partner feedback)
Status: ⬜ TODO
## Why
Partner watched the demo video and gave two pieces of feedback:
1. Token **rotation** wasn't hammered home enough (highlighting the text on screen
wasn't sufficient — make it undeniable).
2. They want more than the name de-identified — fields matching their "Azure
Processing Content" table (ID/CC/Name tokenized, Email/Mobile masked, etc.).
Name (`PERSON`) and the Taiwan ROC ID (`TW_ROC_ID`) are already tokenized (T1/T2).
This task adds **Credit Card (tokenize)** and **Email (mask)** and makes rotation
visible. Mobile masking and Address/Amount generalization are explicitly OUT of
scope here (leave for a later T9 if they ask).
## Entity → treatment policy
Introduce a small policy map in the tokenize handler and branch per finding:
| Entity | Treatment | Token / mask form | Notes |
|---|---|---|---|
| `PERSON` | tokenize (rotate + vault) | `tkn_NAME_<rand>` | already works; just adopt the `tkn_<TYPE>_` naming |
| `TW_ROC_ID` | tokenize (rotate + vault) | `tkn_ID_<rand>` | already works; adopt naming |
| `CREDIT_CARD` | tokenize (rotate + vault) | `tkn_CC_<rand>` | enable Presidio built-in `CreditCardRecognizer` (has Luhn) for `zh` |
| `EMAIL_ADDRESS` | **mask** (one-way, partial) | e.g. `m***.w***@gmail.com` | enable built-in `EmailRecognizer`; **no vault row, not restored** |
Adopt `tkn_<TYPE>_<rand>` naming so the on-screen output matches the partner's table
one-to-one.
## Where
- `presidio/app.py` — ensure `CREDIT_CARD` and `EMAIL_ADDRESS` recognizers are active
for the `zh` analyzer (they're built-in; confirm they load alongside the zh model
and the custom ROC-ID recognizer; add them to supported entities if needed).
- `gateway_api/tokenize/handler.py` — read the treatment policy; for `tokenize`
entities keep the current mint→vault→splice; for `mask` entities apply a
deterministic partial mask (keep the first char of each local-part segment + the
full domain for email), splice, and **do not** write a vault row.
- `gateway_api/restore/handler.py` — no change needed for masked fields (they're
never restored). Confirm restore still only touches tokenized `tkn_*` values.
- `gateway_api/orchestrator/` (`/demo`) + `ui/index.html` — add the rotation proof
(below).
- `seed/customers.json` — optional: add `credit_card` / `email` fields for realism;
not required since detection reads from the prompt text, not the DB.
## Critical gotchas (do NOT "fix" these)
1. **Masked fields must NOT rotate.** Masking is deterministic by design — the same
email masks to the same string every run. Only tokenized fields (NAME/ID/CC)
rotate. Do not add randomness to the mask. If the partner "notices" the email not
changing, that is correct behaviour, not a bug.
2. **Email is one-way.** The advisor sees it masked in the final answer, never
restored. If product later wants the real email back, that is
tokenization-with-a-masked-looking-surrogate (goes through the vault) — a
different path. Don't silently switch masking to reversible.
## Rotation proof (the actual ask from feedback #1)
Make rotation undeniable on camera: run the **identical prompt twice** and show the
tokenized fields differ across runs (`tkn_ID_8F72…` vs `tkn_ID_Xq93…`) while the
**final restored answer is byte-identical** and the masked email is identical.
- Easiest implementation: a `?runs=2` mode (or a second call) on `/demo` that returns
both tokenized prompts plus the single restored answer.
- Surface it in `ui/index.html` as a "same input → different tokens → same answer"
split. Optionally show the two vault entries: different tokens, same real value.
- This is far harder to miss than highlighted text.
## Acceptance criteria
- A prompt containing name + ROC ID + credit card + email produces **three rotating
`tkn_*` tokens** (NAME/ID/CC) and **one stable partial-masked email**.
- Running the same prompt twice yields **different** NAME/ID/CC tokens but an
**identical** masked email and an **identical** restored final answer.
- The vault contains rows only for the tokenized fields — never the email.
- On-screen token names match the partner's table (`tkn_ID_…`, `tkn_CC_…`, `tkn_NAME_…`).
## Demo prompt (partner's)
> "Analyze customer Wang Xiaoming (ID No. A123456789, Credit Card 4567-1234-5678-9012,
> Mobile 0912-345-678, Email ming.wang@gmail.com, Address: No. 3, Songren Rd., Xinyi
> Dist., Taipei) — most recent transaction of NTD 58,600 — and provide a visit summary
> and service recommendations."
(Mobile, Address, Amount are out of scope for T8. Note internally: this prompt is
contrived — a real agent would have done the research to build it — but we're giving
the partner the on-screen show they asked for.)