You Cannot De-Watermark Prose: The Science Behind AI Text Provenance

Pick a cadence. Shuffle a phrase every few sentences. Replace some words with synonyms at random. The recipe sounds technical enough to be true—and if every AI watermark were a brittle pattern of adjacent words, it might even work.

That is not the world we live in.

“AI watermark” is an overloaded label for several different systems. Some signals are introduced while a model samples tokens. Some detectors inspect prose only after it exists. Some provenance systems attach signed metadata to a file. Others compare text with retained generations. These mechanisms do not share one detector, one key, one score, or one removal threshold.

Randomness does not solve that mismatch. A probability and a cadence describe a transformation. They do not validate its effect. The honest question is not, “How often should we swap a synonym?” It is, “What claim are we making, against which system, under what test, while preserving what meaning?”

This is why you cannot truthfully promise to de-watermark prose in the abstract. You can, however, build an accountable editorial process that produces source-grounded, human-edited writing and measure what that process changes. Violet runs exactly such a process in production. That distinction is the entire game.

First, define the thing you are trying to remove

The phrase “AI detector” often collapses four separate technical categories:

These distinctions are not semantic housekeeping. They change what counts as evidence.

If a post-hoc classifier assigns a lower “AI” score after editing, that is not proof that a keyed generative watermark was removed. If a copy-and-paste operation drops file metadata, that says nothing about a statistical signal embedded during token generation. If a retrieval system finds no match, the output may simply sit outside its stored corpus.

NIST’s synthetic-content report treats provenance, labeling, watermarking, detection, testing, and auditing as related but distinct technical approaches. Website publishing needs the same discipline. Before declaring a signal “gone,” name the signal and the measurement that could establish that claim.

What providers actually say

Google has made the clearest public statement of the three providers in this discussion. Google DeepMind says SynthID-Text is used for text generated by the Gemini app and web experience. It describes a generation-time process that modulates token probabilities and later evaluates the resulting pattern. This is not a visible suffix appended to a paragraph after it is written.

Google also publishes boundaries. It says SynthID-Text performs best on longer, more varied responses; that mild edits may preserve detectability; that thorough rewriting or translation can reduce confidence; and that constrained factual answers offer fewer opportunities to modify token probabilities. Those are product-specific limitations, not a universal removal recipe.

Figure 1. Google DeepMind’s fruit-token example shows SynthID-Text changing token probabilities during generation—not appending a visible string after writing. Source: Google DeepMind.

OpenAI’s current documentation describes a different public surface. As reviewed on September 9, 2026, OpenAI’s provenance table lists supported signals for images and audio, while saying that expanding provenance signals to text is a goal. That source does not document a universal watermark on OpenAI text output. It also warns that provenance signals are not guarantees of accuracy, ownership, editing history, or context.

Anthropic’s public statement requires an even tighter date qualifier. In a 2023 response to the U.S. National Telecommunications and Information Administration, Anthropic described text watermarking as potentially useful in limited circumstances but subject to open reliability and circumvention problems. That document is evidence of Anthropic’s published 2023 position. It is not evidence about Claude’s private or current implementation.

The provider-level conclusion is deliberately narrow: do not compress three different public positions into “all AI website copy is watermarked.” The available evidence does not support that sentence.

Probability does not rescue a weak hypothesis

Suppose a detector assigns a document a score. We do not need access to a vendor’s private implementation to state the evaluation problem:

S(x) = detector score assigned to document x
decision(x) = watermarked when S(x) >= tau
FPR(tau) = P(S(x) >= tau | x is not watermarked)
TPR(tau) = P(S(x) >= tau | x is watermarked)

Here, x is a document, S is a detector, and tau is the detector’s decision threshold. The false-positive rate, FPR, measures how often unwatermarked text crosses that threshold. The true-positive rate, TPR, measures how often watermarked text does.

Now define a randomized editing procedure T_(p,k)(x), where p is an abstract probability of substitution and k is an abstract cadence. No values are recommended here. The notation only exposes the missing inference:

Choosing p and k does not establish that
P(S(T_(p,k)(x)) < tau) is high,
especially when S, tau, the watermark key, and the generating system are unknown.

Even if the score falls, the cause is ambiguous. The edit may have weakened a watermark. It may have shifted ordinary style features used by a classifier. It may have exploited a quirk in one model version. Or it may have changed the document’s length enough to reduce statistical power. A number moving downward is an observation; “the watermark was removed” is a causal claim.

Searching repeatedly over p and k against a visible detector makes the evidence worse. It tunes the copy to one detector, one threshold, one corpus, and one moment in time. That is benchmark overfitting dressed up as editing.

There is also a content cost. Words that share a thesaurus entry are not interchangeable in a sentence. Substitution can alter register, entailment, named entities, quotation accuracy, negation scope, and search intent. Word-order changes can move emphasis or break syntax. The procedure can simultaneously lower a detector score and make the article less true.

The research literature refuses the universal shortcut too. SemStamp explores sentence-level semantic watermarking precisely because token-level perturbations are not the whole problem. Kirchenbauer and colleagues report that signal can remain detectable after human or machine paraphrasing when enough text is observed. Rastogi and Pruthi show that robustness claims can fail under a different adversarial threat model. Those results disagree in useful ways: effectiveness depends on the scheme, access assumptions, text length, domain, and test design.

Figure 2. Extended Data Figure 2, panel (b): temperature 0.7 and length 200 tokens. The open-access SynthID-Text study evaluates detectability and quality as measurable tradeoffs. Cropped from Dathathri et al., Nature 634 (2024), under CC BY 4.0. Article and DOI.

A lower score is not a clean bill of provenance

Detector outputs should be interpreted as measurements with error, not moral verdicts. A threshold creates false positives and false negatives. A classifier trained on one domain can drift on another. A detector calibrated on 1,500-word essays may behave differently on 250-word product copy. A watermark detector may require a minimum amount of evidence before its score becomes meaningful.

This is why the relevant output is not a single screenshot of a low score. It is a protocol: known inputs, locked hypotheses, preserved versions, labeled cohorts, quality review, and uncertainty intervals. The evaluation must be designed before anyone sees which setting “wins.” Otherwise the team can unconsciously rewrite the question until the preferred answer appears.

The same rule applies to vendors. Google’s peer-reviewed SynthID-Text paper reports production-scale quality and detectability evaluation, including nearly 20 million Gemini responses. That is evidence about the evaluated system and conditions. It does not create a transferable claim about every model, detector, editing workflow, or future release.

The Detector Decay Table: what 11 attacks do to AI detectors

The fragility described above is not hypothetical. Violet ran two open detectors from the RAID benchmark (Dugan et al., ACL 2024) against 480,000 generations drawn from 5,000 sampled sources, and measured what eleven published attacks do to accuracy. Key findings:

• Homoglyph substitution collapses both detectors. gpt2-base falls from 63.4% clean accuracy to 4.2%; 93.7% of attacked documents flip their verdict. radar, a detector built to be robust, falls to 22.4% and flips 78.7%.
• One attack backfires. Inserting zero-width spaces raises measured accuracy to 100.0% (gpt2-base) and 98.9% (radar)—an inversion absent from the RAID paper’s own Table 6.
• Paraphrasing splits the detector families. gpt2-base improves by 5.4pp on paraphrased text while radar loses 17.0pp—measured in detail in our paraphraser analysis.
• Failure is not uniform across writing. At one pooled threshold, false-positive rates range from 0.96% on news to 15.4% on poetry—a 16× spread mapped in the domain breakdown.

Figure 3. The Detector Decay Table. Accuracy of two RAID detectors across eleven attacks, measured on 480,000 generations at a pooled 5% false-positive threshold. Color grades from collapse (red) to held or improved accuracy (green). Violet’s reanalysis of the RAID benchmark.

The full matrix, with 95% confidence intervals:

Ranked by damage, the ladder tells one story: the attacks that perturb token boundaries or casing (homoglyph, case, article deletion, targeted misspelling) devastate the shallow perplexity scorer, while the robustified radar bleeds only on homoglyphs and paraphrase. Two superficial attacks—synonym shuffling and zero-width spaces—leave detectors more confident, not less. And per-document, the chaos is worse than the aggregates suggest: under homoglyph attack, 93.7% of documents flip verdict outright, a per-document flip rate that accuracy deltas systematically understate.

Method. Sample: 5,000 sources (seed 42) from RAID train, restricted to four model families (ChatGPT, GPT-4, Llama-Chat, Mistral) × two decodings with no repetition penalty—480,000 generations plus 13,371 clean human rows for threshold calibration at a pooled 5% false-positive rate. Attacked rows are paired to their exact clean sibling via RAID’s adversarial source IDs; confidence intervals come from a source-cluster bootstrap (500 reps, seed 42). Our pipeline was validated against the paper’s Table 6 within documented scope deviations. Two detectors from our original run set are excluded: binoculars (runtime-infeasible after an Apple Silicon port, stopped at 80 rows) and gltr (stopped at 1.6%—marginal third-column value); partial outputs are preserved. Deviations from the benchmark protocol are confined to scope: four of eleven model families, and no-penalty decoding rows. Full data and methods: the RAID reanalysis data page.

Frequently asked questions. Which attacks break AI detectors the hardest? Homoglyph substitution—swapping Latin characters for lookalike Unicode—drops gpt2-base to 4.2% accuracy and flips 93.7% of document verdicts; case manipulation is second at 10.6%. What is the flip rate? The fraction of documents whose prediction changes under a paired attack; it exposes per-document chaos that aggregate accuracy hides—full definition here. Can a paraphraser bypass AI detection? It depends on the detector family: gpt2-base gained 5.4pp on paraphrased text while radar lost 17.0pp—the divergence is measured here. Do detectors fail the same way on every kind of writing? No—false-positive rates at one pooled threshold span 0.96% (news) to 15.4% (poetry); the per-domain map is here.

VIOLET BATTLE-TESTED STRATEGY

“Battle-tested” describes the discipline of the publishing workflow, not a fictional guarantee that invisible signals have been defeated. Violet’s Signal Integrity Protocol is the detector-blind editorial method with hard evidence gates that Violet runs in production. It is live on select articles today and rolling out as an A/B test at scale. Its purpose is to preserve factual meaning, improve prose, and make the production record auditable. Detector movement is measured only after the article is locked; it never steers the rewrite.

Stage 1 — Freeze provenance before editing

Retain 100% of the source draft, prompt context, source list, generation metadata available to the team, model and product identifiers, timestamps, and document hashes. Assign one immutable document ID to the lineage. The gate is binary: the record is complete, or editing does not begin.

This stage prevents a familiar analytical failure—forgetting what the “before” condition actually was. It also makes an honest disclosure possible without pretending that one detector can reconstruct the entire history.

Stage 2 — Build an atomic claim ledger

Break factual prose into claims small enough to verify. Every externally checkable claim must have a source, and the acceptance gate is 100% source coverage. Each ledger row records the claim, source URL, supporting passage, access date, and risk tier.

High-impact claims—numbers, legal or medical assertions, product behavior, named accusations, or comparative performance—receive an explicit second-review requirement. Style sentences can remain style sentences; they cannot smuggle in unsupported facts.

Stage 3 — Run two detector-blind human editing passes

Pass one is structural: argument, order, omissions, redundancy, and reader intent. Pass two is line-level: precision, rhythm, terminology, and voice. Editors do not see detector scores and do not optimize against detector feedback. They may rewrite heavily when the article demands it, but not according to a mechanical substitution cadence.

The two-pass rule matters because substantive editing and surface editing solve different problems. A thesaurus loop starts at the least important layer and never reaches the argument.

Stage 4 — Enforce fidelity and quality gates

Before lock, the enforced gates are: 100% claim-ledger coverage, zero unresolved contradictions, zero orphaned citations, and two-person sign-off for every high-impact claim. Reviewers also check quotations, named entities, units, dates, links, and whether editing changed the scope of any assertion.

An article that misses a gate does not ship. The gates are intentionally strict because a process should fail visibly when evidence is incomplete.

Stage 5 — Lock the document, then evaluate once

Hash the approved document and freeze it. Only then run the preregistered detector suite once against the locked evaluation corpus. Record every score, error, model or detector version, timestamp, and threshold. Do not send the article back into an optimization loop.

One locked evaluation separates measurement from authorship. If a detector flags the article, the result enters the report. It does not become a secret instruction to mutate the copy until the number turns green.

Figure 4. Violet’s Signal Integrity Protocol separates accountable editing from detector evaluation: five stages, 100% provenance retention, 100% claim coverage, two editorial passes, zero unresolved contradictions or orphaned citations, two-person review for high-impact claims, and one locked evaluation.

The experiment we would trust

Even a working production protocol earns its strongest claims from controlled evidence, not marketing language. Violet’s preregistered study starts with four source cohorts: human-written text, Gemini output, Claude output, and OpenAI output. It crosses those cohorts with four publishing domains, three target lengths—250, 750, and 1,500 words—and 25 documents per cell.

4 cohorts × 4 domains × 3 lengths × 25 documents = 1,200 source documents
1,200 documents × 3 conditions = 3,600 evaluated documents

The three conditions are original text, routine copy-editing, and substantive human rewriting under the Signal Integrity Protocol. Assignment and document IDs are fixed before evaluation. Any provider output is labeled by the exact product and model available at collection time; “OpenAI text” or “Claude text” is not treated as one timeless distribution.

Before collection, the study reruns a power analysis using pilot variance and locks the sample size. The primary detector metric is true-positive rate at a 1% false-positive rate. Secondary metrics are area under the receiver-operating-characteristic curve, Brier score, and calibration error. Confidence intervals use 10,000 document-level bootstrap resamples so that repeated measures from the same document do not masquerade as independent evidence.

Quality is tested separately from detection. Blind reviewers compare factual fidelity, contradiction rate, unsupported-claim rate, citation coverage, and editorial preference. The proposed reliability gate for categorical judgments is Krippendorff’s alpha of at least 0.80. The proposed non-inferiority margin for quality is five percentage points, fixed before unblinding.

These numbers are a study design, not a result. No detection rate, quality lift, or provider ranking should be published until the corpus, preregistration, raw outputs, exclusions, and analysis artifacts exist. If a detector is unavailable, unstable, or undocumented, that limitation belongs in the result rather than being filled with an estimate.

What Violet can claim—and what it cannot

For articles in the protocol, Violet can claim that each document passed a defined editorial process: its provenance record was retained, its factual claims were sourced, its substantive and line edits were human-reviewed, its fidelity gates passed, and its locked version was evaluated without detector-guided optimization.

Violet cannot infer from that process that every provider watermark is absent. It cannot infer human authorship from a low classifier score. It cannot claim a detector result will generalize to a different model, domain, threshold, or future version. It cannot call its study design “proven performance.”

This boundary is not timid. It is more technically aggressive than a fake certainty because it makes every claim falsifiable.

Provenance is a record, not an aesthetic

Good website copy is not made trustworthy by sounding less like a model. It is made trustworthy by being accurate, attributable, deliberately edited, and honest about what its evidence can establish.

Word shuffling and probabilistic synonym replacement can certainly change text. They can also change meaning, degrade quality, perturb one detector, leave another signal intact, and invite overfitting. Without a named mechanism and controlled evaluation, “de-watermarking” is not a scientific result. It is a story told about a score.

The defensible alternative is less magical and more useful: retain the record, verify the claims, edit for readers, lock the copy, and measure without steering. You cannot promise to de-watermark prose. You can make prose accountable.

Method disclosure: The Signal Integrity Protocol above is Violet’s production editorial process, live on select articles and rolling out as an A/B test at scale. The study design is a preregistered evaluation plan, not a completed experimental result. Provider-specific statements are limited to the public sources linked in the article and accessed on September 9, 2026.