How Ghostchars decides what to strip
One pass over the text, one decision per character, and a reason you can read for every decision. No model, no heuristics, no score.
Four passes
Cleaning runs in a fixed order. Only the third pass looks at individual characters; the first two exist so that the third one sees a predictable string.
Normalise (optional)
If you asked for it, the text is normalised to NFC or NFKC first. This is what defeats normalisation-form steganography, where a message is encoded in the choice between "é" as one codepoint and "e" plus a combining accent. NFKC goes further and folds styled mathematical letters, ligatures and fullwidth forms — which is lossy, so it is off by default.
Fold line endings and whitespace runs (optional)
CRLF and CR become LF; a spaced em dash becomes a spaced hyphen; trailing spaces and tabs are cut from each line; runs of spaces inside a line collapse to one. These are pattern rewrites over the whole string rather than per-character decisions, and each is its own option because each changes text you can see.
Classify every character
Every character is looked up against the engine's tables in a fixed order, and the first match wins. The result is a category, the option that governs it, and what it should be replaced with — an empty string for "delete", a plain space for an exotic space, an ASCII letter for a lookalike. Ordinary characters match nothing and are returned untouched.
Rebuild, with context
The output is assembled character by character. A character whose governing option is off is copied through and reported as found-but-not-targeted. A character whose option is on is removed or replaced. Throughout, the engine remembers the last character it actually KEPT — not the last character it saw — because that is the base a joiner or a variation selector attaches to, and glue does not advance the base.
The decision order
The categories are tested in this order, and the first match decides. The order matters: a zero width space is checked before the generic format-control rule, so it is reported as a zero-width character rather than as an anonymous "Cf".
| # | By category | Governed by |
|---|---|---|
| 1 | Zero-width and invisible | Default pass (always on) |
| 2 | Bidirectional overrides | Default pass (always on) |
| 3 | Bidirectional marks | Bidi marks --bidi |
| 4 | Joiners, selectors and fillers | Aggressive -a |
| 5 | Any other format control | Default pass (always on) |
| 6 | Private-use characters | Private-use characters --pua |
| 7 | Exotic spaces | Spaces -s |
| 8 | Line separators | Line endings -n |
| 9 | Control characters | Control characters --controls |
| 10 | Typographic punctuation | Punctuation --punct |
| 11 | Lookalike letters | Lookalike letters --confusables |
Two exemptions sit inside this order. Arabic, Syriac and Kaithi have format controls that are ordinary orthography — the Arabic number signs, the Kaithi number sign — and those are matched before the generic format-control rule and left alone. Tab, line feed and carriage return are excluded from the control-character group, because deleting them would join lines together.
Why context decides
Four families of character are invisible AND load-bearing: joiners, variation selectors, script-specific fillers, and bidirectional marks. The same codepoint can be contraband in one position and the correct spelling in another, so the engine looks at the neighbours before it decides. This is the part most cleaners get wrong, in both directions: they either leave the hidden ones or they break the real ones.
The family emoji
A family emoji is three or four people welded together by zero width joiners. Remove them and one emoji becomes three. Ghostchars keeps a joiner whose neighbours on both sides are emoji bases, and removes the identical codepoint when it sits between two Latin letters — where it joins nothing and hides well.
The Persian word
Persian spells its continuous and plural forms with a zero width non-joiner: it is not decoration, it is the spelling. Ghostchars keeps a joiner or non-joiner when the characters on both sides belong to the same joining script — Arabic, Persian, Devanagari, Malayalam — and removes it everywhere else. The same rule protects Khmer vowels, Mongolian selectors and Hangul jamo fillers next to their own scripts.
The Scotland flag
A subdivision flag is a black flag followed by tag characters spelling a region code and a cancel tag. Those same tag characters, free-floating, are the ASCII-smuggling attack — invisible instructions that a language model will read. Ghostchars parses the whole sequence: tag characters inside a complete flag are kept, and every tag character outside one is removed.
The Hebrew sentence
Left-to-right and right-to-left marks are how mixed-direction text keeps its punctuation on the correct side. Ghostchars keeps a bidi mark whose neighbour is a right-to-left character and removes one adrift in an English paragraph, where its only function is to carry a bit. The five override and embedding controls get no such benefit of the doubt: they can reorder text you did not write, so they always go.
Every one of these decisions is reported. A kept character is listed as "kept — load-bearing" with the option that would remove it anyway, so nothing is decided silently and nothing is hidden from you.
Verified against eleven published schemes
The command-line tool ships an adversarial test that implements eleven documented text-hiding techniques as encoder/decoder pairs. For each one it embeds a secret, cleans the text, and checks two things: that the decoder can no longer recover the secret, and that the visible cover text survives byte-for-byte identical.
| Scheme | Defeated with |
|---|---|
| Zero-width binary (ZWSP / ZWNJ / ZWJ) | default pass |
| Word-joiner and invisible-maths binary | default pass |
| Tag-block ASCII smuggling | default pass |
| Variation-selector byte smuggling | default pass |
| Bidi mark binary (LRM / RLM) | default pass |
| SNOW trailing whitespace | Trailing whitespace --trailing |
| Cyrillic homoglyph binary | Lookalike letters --confusables |
| NFC / NFD normalisation-form binary | Normalize (NFC) --nfc |
| NBSP / space homoglyph binary | Spaces -s |
| Single / double space binary | Collapse spaces --collapse |
| Curly / straight apostrophe binary | Punctuation --punct |
Five of the eleven fall to the default pass alone. The other six need an option, because the carrier is a character you can see — a space, an apostrophe, a lookalike letter — and removing those without being asked would change the text.
The same engine in two places
The browser engine is a port of the Python command-line tool, not a re-interpretation of it.
A generator runs the Python implementation over a fixture corpus — every listed codepoint, every context case, all eleven steganography payloads, realistic mixed prose — for fourteen combinations of options, and records the cleaned output and every finding. The TypeScript engine is tested against that recording. Any drift between the two fails the build, so "the website does what the CLI does" is a test result rather than a promise.
That matters for a practical reason: you can check a file in a pre-commit hook with the CLI and get exactly the result the website would have given you.
What this cannot do
Everything above is about characters. These four things are not about characters, and no amount of cleaning touches them.
- Statistical watermarks — Schemes such as SynthID-Text and the Kirchenbauer "green list" method bias which WORDS a model picks. The signal lives in word choice across a whole passage, not in any character. Only rewriting the text changes it, and this tool does not rewrite text.
- Stylometric detection — Detectors such as GPTZero score how predictable each word is and how much that predictability varies. Removing a zero width space does not move that score by a single point. The prose-habits report measures the editable habits behind it — sentence-length variation, tell words, stock transitions — and still refuses to output a probability.
- Format-level hiding — White-on-white text, hidden runs, HTML comments and PDF metadata hide OUTSIDE the text. The document tool reports them — hidden runs, tracked changes, comments, editing-session ids, authoring metadata — but deliberately does not delete them, because deleting them changes the document and that should be your decision.
- Sub-one-bit-per-emoji channels — A joiner chain between emoji and a variation selector after one are kept by default because that is how emoji work. A determined sender can still hide a small amount of data there. Turn on Aggressive if you need them gone, and accept that your emoji will come apart.
Ghostchars reports no probability, no score and no verdict about who wrote a text. It tells you which characters are in your string, where they are, and what happens to each one.
Questions
Why not just remove every invisible character?
Because several of them are load-bearing. Removing every zero width joiner turns a family emoji into three separate people and misspells Persian words. Removing every bidi mark breaks mixed Hebrew and English layout. A cleaner that does this quietly is worse than no cleaner, because you cannot see what it took.
Is the browser version the same as the command-line tool?
Yes, and it is tested to be. Golden fixtures generated from the Python implementation pin the TypeScript engine to byte-identical output for every fixture and every combination of options; the build fails on any difference.
Can it tell me whether AI wrote a text?
No, and it will never claim to. Hidden characters tell you a text passed through some piece of software. They do not tell you which software, and they do not tell you who wrote the words. Any tool that turns a character count into a confidence percentage is inventing the number.
Does my text leave my browser?
No. There is no server-side processing and nothing to upload to: the engine is JavaScript running on your device, and the site is a set of static files. You can disconnect from the network and it still works.