What it is
Redaktion is an automated detect-and-remove engine for sensitive documents. It finds personal and corporate information across 45 configurable categories, burns opaque boxes at exact coordinates with true content removal, strips document metadata, and produces a forensically inert artifact — then verifies its own work before calling the job done.
It runs as a set of workers behind a queue. You hand it a document and a job notification; it hands back a redacted artifact, a structured summary, and a per-box audit trail that never contains the sensitive text itself.
| Property | Value |
|---|---|
| Integration surface | Message queue + object storage + a job record |
| Pipelines | Native PDF · OCR/image · Office documents |
| Languages | English · Arabic (full right-to-left) · Hindi/Devanagari |
| Categories | 45 rule keys, individually toggleable per tenant |
| Verification | Unconditional post-redaction leak check on every job |
| Audit | One row per box; keyed one-way digest, never the text |
How you integrate
Redaktion is a queue-driven worker, not a REST endpoint. That is deliberate: redaction is a long, bursty, retry-heavy workload, and a durable queue plus a job record is a better contract for it than an HTTP request that can time out halfway through a 300-page scan.
Four steps, and the shape does not change with volume:
| Step | What you do |
|---|---|
| 1 · Encrypt & store | Envelope-encrypt the source file and put it in object storage. |
| 2 · Record the job | Insert a document row with status pending — this record, not the message, is the source of truth. |
| 3 · Notify | Publish a small job message to the queue for the right pipeline. |
| 4 · Consume | Read the result from the job record, or subscribe to result and progress events. |
Workers read document_id from the message and then load the authoritative record. That means a duplicate or out-of-order message is harmless, and the retry budget lives in one place instead of being split between your queue and ours.
Queues
Every queue is durable and dead-lettered — a matching dlq- queue receives anything that cannot be processed, so a poison message is quarantined rather than looping forever. Queues must be declared with the dead-letter arguments the workers expect; a mismatched declaration is rejected outright rather than silently accepted.
A PDF whose text layer is too thin is re-routed to the OCR pipeline automatically, and a very large scan to the extra-large lane — without consuming retry budget, because a handoff is not a failure.
Job contract
What you publish:
{
"type": "redaction",
"document_id": "0f1e2d3c-…",
"case_id": null,
"client_id": "acme",
"file_name": "contract.pdf",
"file_path": "tenant/<uuid>/contract.pdf",
"file_extension": "pdf",
"redaction_type": "pdf",
"timestamp": "2026-08-03T09:12:44Z",
"storage_backend": "minio"
}
| Field | Required | Notes |
|---|---|---|
type | yes | Literal "redaction". Messages without it are dropped. |
document_id | yes | The only field the PDF and OCR workers strictly need — everything else is read from the record. |
file_name, file_path | office path | Storage object key and display name. |
case_id, client_id | no | Carried through onto audit rows. |
file_extension | no | Used to re-route a misfiled document to the right pipeline. |
redaction_type | no | pdf · ocr · production. |
config | optional mode | A per-job snapshot of enabled rules and settings — see Configuration. |
Workers consume with a prefetch of one, so a single worker takes one document at a time and a slow job never blocks a queue behind it.
Result & progress events
Rather than polling the record, you can subscribe. Both event types are versioned and field-locked across every worker implementation.
Result
{
"type": "redaction_result",
"timestamp": "2026-08-03T09:14:01Z",
"document_id": "0f1e2d3c-…",
"client_id": "acme",
"attempt": 1,
"outcome": "completed",
"status": "completed",
"redacted_file_path": "tenant/<uuid>/contract_redacted.pdf",
"redacted_file_size": 481233,
"redaction_summary": "{…}",
"redaction_has_warnings": false,
"redaction_vision_qa_verdict": "pass",
"redaction_quality_verdict": "ok",
"redaction_quality_issues": [],
"redaction_rules_version_applied": 7,
"audit_rows": [ … ]
}
| Field | Meaning |
|---|---|
outcome | completed · failed · kick_back — the disposition your consumer acts on. |
attempt | Attempt-fenced. Apply results conditionally on this, and a late result from a superseded attempt can never overwrite a fresh one. |
status | The document status — see Statuses. |
redaction_vision_qa_verdict | pass · low_confidence · fail. |
redaction_quality_verdict | ok · warn · block — input legibility, not output correctness. |
audit_rows | One entry per drawn box — see Output guarantees. |
Progress
{ "type": "redaction_progress", "timestamp": "…", "document_id": "…",
"attempt": 1, "progress_page": 7, "processing_stage": "detect" }
Stages are named and ordered, so a progress bar is a lookup rather than a guess: preflight → detect → apply_and_verify → upload_and_complete on the PDF path, and ocr_preflight → ocr_pipeline → ocr_upload_and_complete on the OCR path. Heartbeat-only events omit the page and stage.
Workers also expose /health and Prometheus /metrics for your own monitoring.
What it detects
Forty-five rule keys, each individually enabled or disabled per tenant.
| Group | Rule keys |
|---|---|
| Identity & personal | individual_names date_of_birth photo signature biometric_identifiers family_registry_numbers |
| Contact & digital | email_address phone_number addresses_all online_identifiers password |
| Government & official IDs | national_ids passport_numbers license_numbers tax_id work_authorization vehicle_ids property_ids education_ids employee_ids insurance_numbers |
| Financial | bank_account_numbers bank_routing_codes credit_card monetary_amount financial_statement |
| Special category | health_information criminal_record ethnicity religion |
| Organisations | organization_names |
| Corporate entity | entity_legal_name entity_trade_name entity_parent_company entity_subsidiary entity_registration_number entity_license_number entity_tax_number entity_incorporation_number entity_commercial_registration entity_registered_address entity_office_address entity_email entity_phone entity_website |
How toggles behave
| Case | Behaviour |
|---|---|
| Known key, enabled | Redacted. |
| Known key, disabled | Not redacted — surfaced as a suggestion so a reviewer can see what was deliberately left. |
| Unrecognised type | Redacted anyway. An unknown category is never silently dropped — over-redaction is recoverable, a leak is not. |
| Empty rule set | Everything is kept. A failed rules load can never cause silent under-redaction. |
| Faces & signatures | Geometric detections bypass the toggle entirely, so a mis-set rule cannot expose a face. |
Detected values are also normalised through an alias map, so a model that reports DOB, iban or Driver License lands on the right canonical key rather than being lost.
Regional ID packs
Generic PII tools do badly on identity documents outside the US. Redaktion ships country packs where the pattern is paired with the checksum that proves it — and a value that fails its checksum is rejected rather than redacted, so look-alike numbers don't produce noise.
| Profile | Covers |
|---|---|
| India | Aadhaar and VID (checksum-validated), PAN, voter ID, driving licence, GSTIN (checksum-validated), IFSC, PIN code, phone |
| UAE | Emirates ID (checksum-validated), TRN, unified number, visa file number, trade licence, driving licence, PO box, Arabic address and road components, phone |
| Gulf | Saudi national ID (checksum-validated), Qatar QID, Bahrain CPR, Oman civil ID, Kuwait civil ID |
| United States | SSN (reserved ranges excluded), ITIN, EIN, passport, ABA routing (checksum-validated), Medicare beneficiary ID |
| United Kingdom | National Insurance number, NHS number (checksum-validated), UTR, passport, postcode |
| Australia | TFN, Medicare, ABN (all checksum-validated), passport |
| Universal | Email, credit card and masked PAN (Luhn), IBAN (mod-97), SWIFT/BIC, IPv4, Bitcoin and Ethereum wallets, monetary amounts, passport MRZ lines |
Machine-readable zones are always removed — an MRZ encodes every field on the page, so leaving it visible leaks the document you just redacted.
A further set of label-anchored patterns catches values that only a nearby label identifies. Those redact the value and not the label, so a form stays readable as a form.
Detection stack
Five layers, run in a fixed order, merged once with a defined precedence — deterministic matches beat statistical ones, and statistical beat generative.
| # | Layer | What it contributes |
|---|---|---|
| 1 | Regex + checksum | Deterministic identifiers. Full confidence when a checksum validates, high when the format alone matches. |
| 2 | Named-entity recognition | Names, organisations and places that no pattern can enumerate. Runs as a separate service. |
| 3 | Stoplist | Removes legal and procedural boilerplate that looks like a name but isn't. |
| 4 | Language model | Adjudicates the ambiguous remainder with document context. |
| 5 | Vision pass | Finds what is not in the text layer at all — faces, signatures, stamps, photos. |
The recognition layer runs remotely and is fail-soft: if it is unavailable, detection degrades to the deterministic layers and a circuit breaker stops the worker hammering a service that is down. A downed component slows detection; it does not take redaction offline.
Document-type skills
Fixed-layout identity documents get dedicated handling — Aadhaar, Emirates ID, and any ICAO-standard passport — that knows where the face, signature, number and MRZ sit rather than searching for them. A skill only takes over when it recognises the document confidently; otherwise the generic pipeline runs. Every skill failure path falls through rather than failing the job.
The verification pass
This is the part that separates Redaktion from a tool that draws rectangles.
After boxes are applied, the engine re-reads its own output — OCRs the redacted file and runs the same detection pipeline over what it can still see. Anything found there is a leak by definition. Because it reuses the same rules, its false-positive behaviour is identical to the primary pass, rather than a second, noisier detector second-guessing the first.
| Property | Behaviour |
|---|---|
| Always on | Unconditional on every job. There is no flag to turn it off. |
| Dual engine | A second pass in sparse-text mode catches text the primary pass reads past. |
| Bounded | Page and time caps, checked before each page, so verification cannot run away on a huge document. |
| Context aware | Amounts and routing codes deliberately retained on receipts aren't reported as leaks. |
On the PDF path there is a further chain: a fast text-layer check, then an OCR check, then any residual is re-boxed and re-verified, with page rasterisation as the last resort. A vision QA pass then asks a model a deliberately narrow question — we drew N boxes, do you see N boxes, and is anything still legible? — because that is far more reliable than asking a model to find all PII from scratch.
Geometry is audited too: a job where boxes were drawn but landed over the wrong tokens is flagged, which is a failure mode a simple count check would pass.
Statuses
| Status | Meaning | Artifact |
|---|---|---|
pending | Recorded, awaiting a worker. | — |
processing | Atomically claimed by one worker — the claim is a compare-and-set, so two workers cannot take the same document. | — |
reprocessing | Re-running after a rules change. | previous |
completed | Clean terminal success. | yes |
needs_review | Pipeline finished but verification flagged something. The artifact exists and can be downloaded, but a human must review it before it is shared. | yes |
failed | Permanent failure, or retry budget exhausted. | no |
not_supported | Input type not handled. | no |
needs_review is its own status
It used to be a warning flag on a completed document. In practice people trust a green badge and ignore flags — so uncertain redactions shipped. Making it a distinct terminal status forces the review step instead of hoping for it. Gate your download or share action on the status, not on a warnings boolean.
Escalation is triggered by concrete signals, not a vague score: uncovered PII or low confidence from vision QA, partial OCR coverage, a degraded OCR engine, too few boxes for the entities found, boxes landing in the wrong place, a photo-mode document, or a low-quality scan where nothing was found at all.
Configuration
Tenant settings, applied per job.
| Setting | Default | Effect |
|---|---|---|
| Country profiles | IN, AE | Which regional ID packs run. Universal patterns are always on. |
| Form policy | values_only | values_only keeps "Name:" and redacts the value; labels_and_values removes both; full_document redacts the whole surface. |
| Vision QA pages | 50 | Cap on pages sent for visual verification. |
| Vision QA confidence | medium | Below this, a page is downgraded to low-confidence rather than passed. |
| Minimum DPI | 300 (hard floor 150) | Below the floor a scan is flagged — optionally blocked. |
| Blur / skew thresholds | tunable | Legibility gates on the input, with warn and block behaviour separated. |
| Role policy | built-in per doc type | Who is redacted vs kept by role — e.g. the subject of a receipt vs the issuing merchant. Fail-closed: an undeterminable role is redacted. |
| Rules version | — | Stamped onto every result so you can prove which rule set produced an artifact. |
Rule toggles take effect on the next job — the enabled set is re-read before each document, with no worker restart. A transient failure to load keeps the last good set rather than clearing it.
Supported inputs
| Pipeline | Accepts | Produces |
|---|---|---|
| Native PDF | PDFs with a text layer, Markdown, embedded images | …_redacted.pdf |
| OCR / image | Scanned PDFs, JPG, PNG, TIFF, BMP, GIF, WebP, HEIC/HEIF | …_redacted.pdf or …_redacted.png |
| Office | DOCX, DOC, XLSX, XLS, PPTX, PPT, RTF, CSV, HTML, ODT, ODS, TXT | …_redacted.txt |
The office path does not preserve formatting — it returns redacted text, not a redacted DOCX. If you need the original layout, convert to PDF before submitting.
Assurance is not uniform across pipelines. The full stack — document skills, vision verification and the post-redaction leak check — runs on the PDF and OCR paths. The office path runs pattern and model detection with recognition support. For regulated material, route through PDF.
Arabic is handled properly rather than nominally: right-to-left reordering, Arabic-Indic and Devanagari digit normalisation before matching — while the original characters are what get located on the page — plus Arabic address and road patterns.
Output guarantees
Removal is real
On PDFs the engine applies true redaction annotations that delete the underlying content, then scrubs metadata, embedded attachments and page remnants, and saves with full garbage collection. The result cannot be un-redacted by copying a layer, selecting invisible text, or reading document properties. On images the pixels are overwritten, not covered.
Box geometry is padded for serifs and shrunk slightly so bars don't bleed into adjacent rows, overlapping boxes are collapsed, and a form label is never drawn over.
Fail-closed
A page the pipeline cannot process confidently ships fully blacked out and the document is flagged. The safe failure is a page you re-run, not a page that leaks.
The audit trail never holds the data
One row per drawn box:
| Field | Value |
|---|---|
page_index, bbox | Where the box was drawn. |
rule_key, pattern, validator | What matched, and what proved it. |
detector | Which layer found it — regex, recognition, model or vision. |
confidence | Normalised score. |
text_hash, text_length | A keyed one-way digest and a length. Never the text. |
The digest is keyed per document. That preserves duplicate analytics inside one file while making it impossible to correlate "this identifier appears in documents A, B and C" across your corpus — which an unkeyed hash would allow to anyone holding the table. Audit writes are best-effort by design: a failing audit insert never fails the redaction that produced it.
Encryption
Sources are read through an AES-256-GCM envelope and the redacted artifact is written back to the same backend, re-encrypted. The summary attached to each result reports counts and categories with fingerprints rather than examples, so a summary is safe to log.
Failure & retries
Failures are typed, not generic — you can route on them. Hard failures include model timeouts and rate limits, OCR timeouts, empty text extraction, storage read/write failures, verification failures, unsupported input, files that are too large, and input quality below threshold. Warnings — partial chunk failures, suspicious zero-entity results, low-confidence vision QA, partial OCR coverage, misplaced boxes — surface on the result rather than failing the job, and escalate to review where they matter.
| Mechanism | Behaviour |
|---|---|
| Retry budget | Bounded per document, held in the job record so it can't be double-counted. |
| Transient failure | Returned to pending, attempt incremented. |
| Permanent failure | Terminal — no retry, because the input won't change between attempts. |
| Pipeline handoff | Does not consume retry budget. |
| Poison message | Acknowledged and dropped rather than looping. |
| Heartbeat | Long jobs heartbeat on a dedicated connection, so a slow document is never mistaken for a dead worker. |
| Queue outage | Workers fall back to polling the job record and keep draining. |
Your data
Source documents stay encrypted at rest under a key you control, and redacted artifacts are written back encrypted. Removal is real, so a redacted file cannot be reversed into the original. The audit trail records where and why each redaction happened without ever storing the sensitive text.
Sensitive content is necessarily processed in memory to be detected, and detection uses model services to adjudicate ambiguous spans. Working files are swept on a short timer. Processing runs on EU/Swiss-resident infrastructure. Retention, subprocessors and erasure are set out in the privacy policy.
Availability & access
Redaktion is a SeKondBrain Labs product. The engine runs in production inside our legal platform today. As a standalone service it is at early access: we integrate it with your pipeline and provision the queues, storage and credentials with you, rather than offering an open self-serve endpoint. It is licensed, not open source.
Where it runs
| Model | What it means |
|---|---|
| Multi-tenant cloud | The default. We operate the workers, queues and storage; your organisation gets its own isolated tenant and endpoints. |
| Your own cloud | For enterprise — and the usual choice here. Deployed into your cloud account so documents never leave your boundary, which is normally the point for the material people redact. |
What you get when you're set up
| Unique endpoints | Your own queue addresses and credentials for the pipelines you use. |
| An encryption key you control | Sources are encrypted with it before they reach us, and artifacts are written back under it. |
| Your rule configuration | Which of the 45 categories are enabled, your country profiles, form policy and quality thresholds. |
| Setup guide | Provisioning covers queue declaration, the job record, and consuming results — the contracts documented above. |
Email hello@sekondbrain.ai with the subject “Access for Redaktion” — tell us the document types, languages and volumes you need to redact.