maskflow

# MaskFlow

**Stop Indian PII from ever reaching an LLM.**

Aadhaar, PAN, GSTIN, UPI, IFSC, ABHA, Indian names and addresses — detected and replaced with
reversible, typed placeholders before a prompt leaves your process, restored in the response.
28 entity types, checksum-validated where a public checksum exists, MIT-licensed, runs entirely
on your own infrastructure.

[![CI](https://github.com/maskflow/maskflow/actions/workflows/ci.yml/badge.svg)](https://github.com/maskflow/maskflow/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/maskflow-sdk)](https://pypi.org/project/maskflow-sdk/)
[![npm](https://img.shields.io/npm/v/%40maskflow%2Fdetection)](https://www.npmjs.com/package/@maskflow/detection)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](pyproject.toml)

<p align="center">
  <img src=".github/assets/demo.svg" alt="Terminal demo: pip install maskflow-sdk, then mask() replaces an Aadhaar number and email with &lt;AADHAAR_1&gt; and &lt;EMAIL_1&gt; before an LLM call, and unmask() restores the originals in the response" width="720">
</p>

## Why

India's DPDP Act sets a compliance deadline of **13 May 2027**, with penalties of up to
**₹250 crore** for a breach where the required safeguards weren't in place. Every prompt sent to
an LLM provider is a potential data-sharing event — and general-purpose PII tools weren't built to
recognize Aadhaar, PAN, GSTIN, UPI VPAs, IFSC codes, ABHA health IDs, or Indian names and addresses
reliably. [Presidio](https://github.com/microsoft/presidio) already owns generic PII and is more
mature everywhere else; MaskFlow exists specifically to close that gap, with accuracy that's
measured and published, not asserted. See [MaskFlow vs. alternatives](#maskflow-vs-alternatives).

## Quickstart

```bash
pip install maskflow-sdk
python -m spacy download en_core_web_sm
```

```python
from maskflow import mask, unmask

result = mask("My Aadhaar is 2346 8907 6543 and you can reach me at alice@example.com.")
result.masked_text
# "My Aadhaar is <AADHAAR_1> and you can reach me at <EMAIL_1>."
unmask(result.masked_text, result.mapping)  # original text, restored
```

For a one-line wrapper around your actual LLM call, or session-scoped masking across a multi-turn
agent (same value → same token for as long as the session is open), see
[`packages/maskflow-sdk/README.md`](packages/maskflow-sdk/README.md).

## How it works

1. **Tier-0 excision first.** Deterministic regex/checksum matches (Aadhaar, PAN, GSTIN, email,
   credit card, ...) are found and locked in *before* the NER pass ever runs — spaCy parses each
   document at most once, only over what tier-0 didn't already claim.
2. **Every match is a `Span`.** Start/end offsets, entity type, confidence, which recognizer
   produced it, whether a checksum validated it, and a human-readable explanation trail. Run
   `maskflow explain "<text>"` (from `maskflow-cli`) to see that trail for any input, span by span
   — including near-misses that fell just below threshold and what config change would catch them.
3. **Deterministic resolution on overlaps.** Below-threshold spans are dropped; among what's left,
   a checksum-validated span always beats an overlapping unvalidated one, then higher confidence,
   then longer span, then earliest start wins — greedy, non-overlapping placement.
4. **Placeholders are typed, stable, and collision-proof.** `<AADHAAR_1>`, `<EMAIL_1>`, ... — the
   same value gets the same token within a session, and if the input text already contains
   something that looks like a placeholder, a nonce suffix (`<AADHAAR_1_a4f9>`) is used instead so
   a real placeholder is never ambiguous with attacker-controlled input.
5. **Recognizers are pluggable.** `maskflow-pack-intl` and `maskflow-pack-india` are just two
   `"maskflow.recognizers"` entry-point plugins sharing one memoised analysis context — write and
   register your own the same way. See [`docs/custom-recognizers.md`](docs/custom-recognizers.md).

## Protecting your own logs

Regex/checksum-based recognizers can also scrub your application's own `logging` calls — not just
text passed through `mask()` — closing the gap where a raw value gets logged before it's ever
masked:

```python
from maskflow_core import install_pii_filter

install_pii_filter()  # attaches to the root logger, once, at startup
```

Opt-in only; importing `maskflow_core` never touches global logging state on its own. It doesn't
cover NER-only entity types (bare names/addresses) or `exc_info` tracebacks — see
[`docs/logging.md`](docs/logging.md) for the exact boundary.

## Auditing what already reached a provider

Going forward, `mask()` keeps PII out of your prompts. But the DPDP audit asks a backward-looking
question first: *what has this system already sent to a third-party LLM?* `maskflow scan` answers
it. It reads your historical LLM traffic — a JSONL/CSV export, a recursive directory, an S3
archive, a Postgres table, or the Langfuse / Helicone / LangSmith API — streams it through the
same detection with bounded memory (parallel, resumable), and writes **one self-contained HTML
report**: a single headline number, breakdowns by entity type / provider / model / time, a
severity ranking with a plain-English "why this matters" per row, **masked excerpts only** (never
a raw value), and a DPDP Rule 6 mapping appendix. Also `--format json|csv`. Runs entirely locally
— the API sources only *read* from your own account, nothing is transmitted.

```bash
pipx install maskflow-cli   # or: docker run --rm -v "$PWD:/work" ghcr.io/maskflow/cli
maskflow scan jsonl requests.jsonl --field 'messages[].content' --deep -o exposure.html
```

Also ships as a standalone binary (mac/linux/windows, no Python — pattern pass only) and a
[GitHub Action](packaging/scan-action/) that can fail a build over a PII-exposure threshold. A
runnable 60-record synthetic example is in
[`packages/maskflow-cli/examples/`](packages/maskflow-cli/examples/); full reference,
including the Rule 6 mapping, in [`docs/scan.md`](docs/scan.md).

## Gateway: no code change at all

`maskflow-gateway` is a drop-in OpenAI/Anthropic-compatible proxy. Point your existing client's
base URL at it and PII is masked before every request reaches the provider and restored in the
response — **streaming included** (a `<PERSON_NAME_1>` split across SSE chunks is stitched back
together; fuzz-tested at every byte boundary). Tool-call arguments are walked as JSON; multi-turn
token identity is kept in Redis (AES-256-GCM at rest).

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="sk-...")  # your real key, passed through
```

```bash
pip install "maskflow-gateway[redis]"   # or: docker run -p 8000:8000 ghcr.io/maskflow/gateway
```

Full reference in [`packages/maskflow-gateway/README.md`](packages/maskflow-gateway/README.md) and
[`docs/gateway.md`](docs/gateway.md).

## LiteLLM: a guardrail on your existing proxy

Already running a [LiteLLM](https://github.com/BerriAI/litellm) proxy? `maskflow-litellm` is a
custom guardrail — no separate service. It masks PII (Indian identifiers included) before a
request leaves the proxy and restores it in the response, streaming and tool calls included.

```bash
pip install maskflow-litellm
```

```yaml
guardrails:
  - guardrail_name: maskflow
    litellm_params:
      guardrail: maskflow_litellm.MaskflowGuardrail
      mode: [pre_call, post_call]
```

Full reference in [`packages/maskflow-litellm/README.md`](packages/maskflow-litellm/README.md) and
[`docs/litellm-guardrail.md`](docs/litellm-guardrail.md).

## LangChain: a one-line import swap

`maskflow-langchain` is a drop-in for
[`langchain-experimental`](https://github.com/langchain-ai/langchain-experimental)'s Presidio
anonymizer — same `.anonymize()` / `.deanonymize()` / `.deanonymizer_mapping` — so an existing
chain migrates by changing one import. The deanonymizer is a streaming-aware `Runnable` (a
placeholder split across streamed chunks is stitched back), and there's an optional leak-guard
callback that fails a call closed if PII reaches the model.

```bash
pip install maskflow-langchain
```

```python
# from langchain_experimental.data_anonymizer import PresidioReversibleAnonymizer
from maskflow_langchain import MaskflowReversibleAnonymizer as PresidioReversibleAnonymizer
```

Full reference in [`packages/maskflow-langchain/README.md`](packages/maskflow-langchain/README.md)
and [`docs/langchain.md`](docs/langchain.md).

## LlamaIndex: keep PII out of RAG

`maskflow-llamaindex` gives a LlamaIndex RAG pipeline two components and an unmask helper.
`MaskflowNodePostprocessor` is a drop-in for `llama_index.core.postprocessor.PIINodePostprocessor`
(same `__pii_node_info__` contract) that masks retrieved context before the synthesizer, with no
LLM call. `MaskflowIngestionTransform` masks node text at ingestion so raw PII never reaches the
vector store. `unmask_response()` / `MaskflowQueryEngine` restore the real values in the answer.

```bash
pip install maskflow-llamaindex
```

```python
from maskflow_llamaindex import MaskflowNodePostprocessor, unmask_response

engine = index.as_query_engine(node_postprocessors=[MaskflowNodePostprocessor()])
response = engine.query("What is Ramesh's PAN?")
answer = unmask_response(str(response), response.source_nodes)
```

Full reference in [`packages/maskflow-llamaindex/README.md`](packages/maskflow-llamaindex/README.md)
and [`docs/llamaindex.md`](docs/llamaindex.md).

## MCP: a masking proxy for agent tool calls

`maskflow-mcp` is a [Model Context Protocol](https://modelcontextprotocol.io) proxy. Put it in
front of any MCP server and PII in outbound `tools/call` arguments is masked before it reaches the
tool, results are unmasked on the way back, and placeholders stay consistent for the whole agent
run. Agent tooling is where PII leakage is least examined; this is a drop-in shi