Google's SynthID marks Gemini's output so it can be identified later, and the mechanism underneath it is simpler than the coverage suggests. This tutorial builds that class of text watermark from scratch in Python, on GPT-2, and then detects it from text alone at p = 3.07e-56. Intermediate level, about 45 minutes.
What you'll build
By the end of this tutorial you will run this:
text scored green rate z p verdict plain 199 50 25.1% 0.04 4.84e-01 no watermark watermarked 199 146 73.4% 15.76 3.07e-56 WATERMARKED wrong key 199 49 24.6% -0.12 5.49e-01 no watermarkThree pieces of text. One was written by a model with a watermark switched on, one by the same model with it switched off. The detector reads only the text, never the token ids the model emitted, and separates them with a p-value of 3.07e-56.
The third row is the control, and it is the row that makes the other two mean anything. It scores the same watermarked text with the wrong secret key, and the signal vanishes. Whatever the detector is locking onto, it lives in the key.
By the end you will have a watermarker that biases token choices with a secret key, and a detector that pulls the watermark back out of plain text. You will also have measured answers to the two questions that decide whether any of this is usable in your system: how much text detection needs, and how much editing it survives.
This is an intermediate tutorial. It assumes you write Python and have loaded a model with
Hugging Face transformers before. It does not assume you know what a logits processor is.
Budget about 45 minutes, most of which is a 548 MB model download.
Verified against Python 3.13.9, transformers 5.15.0, torch 2.13.0+cpu, and numpy 2.5.2 on Windows, on 2026-08-17.
One thing to be clear about before you start, because a lot of writing on this topic blurs
it. You are going to build the green-list watermark from Kirchenbauer et al. (2023).
That is not the algorithm Google ships. SynthID-Text uses tournament sampling, a
different mechanism from a different lineage. The green list is the right thing to build
first because it is the mechanism you can implement, detect, and break in an afternoon, and
because it is what Hugging Face transformers ships natively so you can check your work
against a reference. The closing section explains how the real thing differs.
Prerequisites
You need Python 3.13 and about 1.5 GB of free disk for the model and the wheels. There is no API key and no paid service anywhere in this tutorial. GPT-2 is MIT licensed and downloads without authentication.
Create the project and a virtual environment:
mkdir synthid-labcd synthid-labpython -m venv .venvActivate it. On macOS or Linux:
source .venv/bin/activateOn Windows PowerShell:
.venv\Scripts\Activate.ps1If that is refused on a fresh Windows machine, that is the execution policy talking.
Nothing to do with this project. Set-ExecutionPolicy -Scope Process RemoteSigned clears
it for the current shell only.
Every command in this tutorial runs from inside synthid-lab with this environment active.
If you come back to it in a new terminal, activate it again before anything else.
Create requirements.txt:
transformers==5.15.0torch==2.13.0numpy==2.5.2Install:
pip install -r requirements.txtNow verify the install before writing any code. This is the step that catches a broken
environment while it is still cheap to fix. Create check_env.py:
"""Confirm the pinned stack is the one actually active."""import sysimport numpyimport torchimport transformersprint(sys.version.split()[0], torch.__version__, transformers.__version__, numpy.__version__)Run it:
python check_env.pyExpected output:
3.13.9 2.13.0+cpu 5.15.0 2.5.2Your Python patch number may differ; anything on 3.13 is fine. Torch also prints a build
suffix that is not part of the pin: +cpu here, +cu128 or similar on a CUDA machine.
Either is fine. The transformers, torch and numpy versions themselves should match. If they
do not, you have a stale environment active.
You do not need scipy. The one statistic this tutorial computes needs a normal tail
probability, and math.erfc in the standard library gives it in a single line with better
precision far out in the tail than subtracting a CDF from one.
A note on output for the rest of the tutorial. Every expected-output block below shows
standard output only. On first run transformers also writes a model-loading progress bar
to standard error, and huggingface_hub prints a notice about unauthenticated requests.
Both are harmless and neither is shown below.
How Kirchenbauer green-list watermarking works
A language model does not choose the next word. It produces a score for every token in its vocabulary at once, all 50,257 of them for GPT-2, and a sampler turns those scores into one choice. Watermarking lives entirely in the gap between those two stages.
The whole technique rests on one fact about samplers: they usually have real freedom. When the model is deciding how to end "my favourite tropical fruits are mango and", several continuations are close to equally good. Nudging that choice costs almost nothing in quality, because you are picking between options the model already considered acceptable.
So: before the sampler runs, split the vocabulary in two. Call one part the green list and the other the red list. Add a small constant to the score of every green token. Do that at every step and the finished text will contain more green tokens than chance predicts.
The split cannot be fixed. A fixed split would be a word blacklist, and anyone could recover it by reading enough output. That secrecy is the whole mechanism, and it is a recurring theme in why provenance signals matter for AI engineers. So the split is derived fresh at every step from two things: a secret key, and the token that was just generated. Without the key the resulting pattern is noise. With the key it reproduces perfectly, and that is what makes detection possible.
flowchart LR
K["Secret key"] --> S["Seed = key x previous token id"]
P["Previous token"] --> S
S --> V["Shuffle all 50257 token ids"]
V --> G["Green list<br/>first 12564 ids"]
V --> R["Red list<br/>other 37693 ids"]
G --> B["Add delta to their scores"]
R --> N["Leave their scores alone"]
B --> M["Sampler picks one token"]
N --> M
style K fill:#C2185B,color:#FFFFFF
style P fill:#95A5A6,color:#FFFFFF
style S fill:#7B68EE,color:#FFFFFF
style V fill:#4A90E2,color:#FFFFFF
style G fill:#6BCF7F,color:#2C2C2A
style R fill:#E74C3C,color:#FFFFFF
style B fill:#6BCF7F,color:#2C2C2A
style N fill:#FFA07A,color:#2C2C2A
style M fill:#4A90E2,color:#FFFFFF
The diagram covers one generation step. Both knobs carry their names from the literature, so keep the names:
- gamma is the fraction of the vocabulary that is green. This tutorial uses 0.25.
- delta is the constant added to green scores. This tutorial uses 2.0.
Detection reverses the same procedure. Take a candidate text, tokenise it, and for each token ask: given the token before it, and given the key, was this token green? Count the hits. If gamma is 0.25 then unwatermarked text should score about 25% green. Watermarked text scores far higher, and a one-proportion z-test turns that gap into a p-value.
Step 1: Inspect GPT-2's next-token probability distribution
Goal. Print the model's next-token distribution at a single position and see how much of the probability mass is genuinely contested.
Why this step. Every claim later in this tutorial depends on the sampler having room to be nudged. If the model were confident about every token, adding a small bias would either change nothing or wreck the text. Before building anything, confirm the room exists and get a feel for how much.
Create explore.py:
"""Show how much freedom GPT-2 has at a single generation step."""import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerMODEL_ID = "openai-community/gpt2"tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32)model.eval()prompt = "My favourite tropical fruits are mango and"inputs = tokenizer(prompt, return_tensors="pt")with torch.no_grad(): logits = model(**inputs).logitsnext_token_logits = logits[0, -1, :]probs = torch.softmax(next_token_logits, dim=-1)top = torch.topk(probs, k=10)print(f"vocab size: {model.config.vocab_size}")print(f"prompt: {prompt!r}")print("top 10 continuations:")for prob, token_id in zip(top.values, top.indices): piece = tokenizer.decode([token_id]) print(f" {token_id.item():>5} {prob.item():6.2%} {piece!r}")print(f"top-10 probability mass: {top.values.sum().item():.2%}")Run it from inside synthid-lab:
python explore.pyExpected output. The first run downloads 548 MB and takes a few minutes.
vocab size: 50257prompt: 'My favourite tropical fruits are mango and'top 10 continuations: 49364 11.38% ' mango' 45540 10.88% ' pineapple' 442 3.63% ' ch' 474 3.01% ' j' 279 2.96% ' p' 25996 2.10% ' banana' 613 1.85% ' pe' 269 1.71% ' c' 25286 1.65% ' pear' 28738 1.60% ' lime'top-10 probability mass: 40.77%What just happened. The top two candidates sit at 11.38% and 10.88%. A tiny nudge decides between them, and no reader could tell you which one the model "meant". Look at the last line too: the whole top ten accounts for only 40.77% of the probability mass, leaving nearly 60% spread across the other 50,247 tokens. All of that is unused freedom, and it is the raw material the watermark gets built from.
Vocab size 50,257, confirmed. That number matters in step 2.
Step 2: Derive a green list from a secret key
Goal. Write the function that turns a secret key plus the previous token into a reproducible list of green token ids.
Why this step. This function is the watermark. Everything after it is plumbing. Be explicit about the three properties it has to satisfy, because a break in any one silently destroys the scheme and no error will tell you: the same key and the same previous token must always give the same list, a different key must give an unrelated list, and the list must look like a random quarter of the vocabulary to anyone without the key.
Create watermark.py:
"""A green-list text watermark, built from scratch."""import torch# The millionth prime. Any large odd number works; this one is the default# that transformers' own WatermarkingConfig uses, so our green lists and# the library's green lists come out identical.DEFAULT_SECRET_KEY = 15485863class GreenListWatermarker: """Splits the vocabulary into a green list and a red list at every step. The split is a pure function of (secret key, previous token id), so the generator and the detector derive the same lists without exchanging anything except the key. """ def __init__( self, vocab_size: int, secret_key: int = DEFAULT_SECRET_KEY, gamma: float = 0.25, delta: float = 2.0, ) -> None: self.vocab_size = vocab_size self.secret_key = secret_key self.gamma = gamma self.delta = delta self.green_size = int(gamma * vocab_size) def green_ids(self, prev_token_id: int) -> torch.Tensor: """Token ids on the green list, given the token that came before.""" rng = torch.Generator(device="cpu") rng.manual_seed((self.secret_key * int(prev_token_id)) % (2**64 - 1)) permutation = torch.randperm(self.vocab_size, generator=rng) return permutation[: self.green_size] def is_green(self, prev_token_id: int, token_id: int) -> bool: """True if token_id is on the green list seeded by prev_token_id.""" green = self.green_ids(prev_token_id) return bool((green == int(token_id)).any())The seeded torch.Generator carries the whole scheme. A fresh generator gets created and
seeded on every call, so the shuffle depends on the seed and nothing else: no global random
state, no call ordering. Without that property the detector could not reproduce these lists
later, out of order, from text alone.
Now check all three. Create try_greenlist.py:
"""Check that the green list is deterministic, key-dependent, and balanced."""from transformers import AutoTokenizerfrom watermark import GreenListWatermarkertokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")vocab_size = tokenizer.vocab_sizealice = GreenListWatermarker(vocab_size, secret_key=15485863)mallory = GreenListWatermarker(vocab_size, secret_key=99999989)prev_token_id = tokenizer.encode(" mango")[0]print(f"vocab size: {vocab_size}")print(f"green size: {alice.green_size} (gamma = {alice.gamma})")print(f"prev token: {prev_token_id} {tokenizer.decode([prev_token_id])!r}")first_call = alice.green_ids(prev_token_id)second_call = alice.green_ids(prev_token_id)print(f"same key, same prev token -> identical: " f"{bool((first_call == second_call).all())}")other_key = mallory.green_ids(prev_token_id)overlap = len(set(first_call.tolist()) & set(other_key.tolist()))print(f"overlap with a wrong key: {overlap} of {alice.green_size} " f"({overlap / alice.green_size:.1%})")for word in [" bananas", " papaya", " airplanes"]: token_id = tokenizer.encode(word)[0] verdict = "green" if alice.is_green(prev_token_id, token_id) else "red" print(f" {word!r:>12} -> {verdict}")This bites people who swap the model out later, so note it now: try_greenlist.py sizes
the watermarker from tokenizer.vocab_size, while steps 3 and 4 will size it from
model.config.vocab_size. For GPT-2 both are 50,257, so the two halves derive the same
permutations. They are separate objects and can disagree for other checkpoints. If they ever
do, the generator and the detector build different green lists and detection silently
returns noise. Step 8 prints both numbers to prove the equality holds here.
Run it:
python try_greenlist.pyExpected output:
vocab size: 50257green size: 12564 (gamma = 0.25)prev token: 49364 ' mango'same key, same prev token -> identical: Trueoverlap with a wrong key: 3127 of 12564 (24.9%) ' bananas' -> red ' papaya' -> green ' airplanes' -> redWhat just happened. All three hold. The same key twice gives an identical list. A different key overlaps by 24.9%, and two independent random quarters of a 50,257-token vocabulary would overlap by 25%, so the wrong key has told you nothing at all. No partial credit, no warmer-colder.
The last three lines make the same point from a different angle. " papaya" is green, " bananas" is red, and both are perfectly good things to say after " mango". The split ignores meaning completely. It has no idea what a mango is, and the invisibility comes free from that: an arbitrary quarter of the vocabulary leaves no stylistic fingerprint for a reader to catch.
watermark.py now holds one class with three methods. The next step adds a second class to
the same file.
Step 3: Wrap the green list in a LogitsProcessor
Goal. Turn the green list into something model.generate() will call at every step.
Why this step. You cannot bias generation from outside the loop. transformers exposes
exactly one hook for this: a LogitsProcessor, an object whose __call__ receives the
scores for the whole vocabulary and returns modified scores. Implementing that interface is
the difference between a green-list function and a working watermarker.
The contract is narrow and unforgiving. Your __call__ takes (input_ids, scores) and
must return a tensor of the same shape and dtype it was given. scores has shape
(batch_size, vocab_size).
Replace the single import torch line under the module docstring in watermark.py, so
that the import block reads:
import torchfrom transformers import LogitsProcessorThen append this class to the end of watermark.py. The assembled file appears in full at
the end of this tutorial if you want to check yours against it:
class GreenListLogitsProcessor(LogitsProcessor): """Adds delta to the logit of every green-list token, every step.""" def __init__(self, watermarker: GreenListWatermarker) -> None: self.watermarker = watermarker def __call__( self, input_ids: torch.LongTensor, scores: torch.FloatTensor ) -> torch.FloatTensor: biased = scores.clone() for row in range(input_ids.shape[0]): prev_token_id = input_ids[row, -1].item() green = self.watermarker.green_ids(prev_token_id) biased[row, green.to(scores.device)] += self.watermarker.delta return biasedscores.clone() earns its line: it avoids mutating a tensor the caller still owns.
green.to(scores.device) earns its line for a worse reason. Build that index on CPU and
assume, and the processor runs perfectly on your laptop and dies the first time it lands on
a GPU box. The troubleshooting section quotes the exact error you get when it does.
Create try_bias.py to see the effect on one real distribution:
"""Show what the bias does to a single step's scores."""import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom watermark import GreenListLogitsProcessor, GreenListWatermarkerMODEL_ID = "openai-community/gpt2"tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32)model.eval()prompt = "My favourite tropical fruits are mango and"inputs = tokenizer(prompt, return_tensors="pt")with torch.no_grad(): scores = model(**inputs).logits[:, -1, :]watermarker = GreenListWatermarker(model.config.vocab_size)processor = GreenListLogitsProcessor(watermarker)biased = processor(inputs["input_ids"], scores)print(f"shape in {tuple(scores.shape)} -> shape out {tuple(biased.shape)}")before = torch.softmax(scores[0], dim=-1)after = torch.softmax(biased[0], dim=-1)prev_token_id = inputs["input_ids"][0, -1].item()print(f"{'token':>22} {'list':>5} {'before':>7} {'after':>7}")for token_id in torch.topk(before, k=8).indices: colour = "green" if watermarker.is_green(prev_token_id, token_id) else "red" piece = tokenizer.decode([token_id]) print(f"{piece!r:>22} {colour:>5} {before[token_id]:6.2%} " f"{after[token_id]:6.2%}")Run it:
python try_bias.pyExpected output:
shape in (1, 50257) -> shape out (1, 50257) token list before after ' mango' red 11.38% 5.13% ' pineapple' red 10.88% 4.91% ' ch' red 3.63% 1.64% ' j' red 3.01% 1.36% ' p' red 2.96% 1.34% ' banana' green 2.10% 7.00% ' pe' red 1.85% 0.84% ' c' red 1.71% 0.77%What just happened. The shape survived the round trip, which was the whole contract. Now look at the size of the effect: " banana" went from 2.10% to 7.00% and jumped from sixth place to first, while every red token roughly halved.
Notice how much bigger that is than the "2 to 3 percent nudge" this technique usually gets described with. Adding 2.0 to a logit multiplies the token's odds by e squared, about 7.4 times, before renormalisation. Green-list watermarking is distortionary by construction. It changes the distribution the model samples from, and you are paying that quality cost in exchange for detectability.
Turn delta down and you buy the quality back at the exact cost of detectability. There is no dial position that gives you both. Google went to a sampling-based design specifically to escape that dial. The closing section covers how.
Step 4: Generate watermarked and unwatermarked text
Goal. Produce two continuations from the same prompt and the same random seed, one with the processor attached and one without, and save both to disk.
Why this step. You need a matched pair to have anything to detect. Fixing the seed across both runs means any difference between the two texts is caused by the watermark and nothing else. Saving to disk matters more than it looks: it forces the detector in the next step to work from text, the way a real detector must, rather than from token ids it was handed by the generator.
Two things in the code below will be new. generate() wants a LogitsProcessorList, not a
bare processor: it is an ordered list, and yours gets appended to the ones generate()
builds internally from your other keyword arguments. Where yours lands in that order turns
out to matter, and the troubleshooting section has the measurement. Passing
logits_processor=None is also legal, and that is how the plain run is produced.
Create run_generate.py:
"""Generate one watermarked and one plain continuation from the same seed."""import textwrapfrom pathlib import Pathimport torchfrom transformers import ( AutoModelForCausalLM, AutoTokenizer, LogitsProcessorList, set_seed,)from watermark import GreenListLogitsProcessor, GreenListWatermarkerMODEL_ID = "openai-community/gpt2"PROMPT = "The old lighthouse keeper had a rule about storms:"MAX_NEW_TOKENS = 200SEED = 7tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)tokenizer.pad_token_id = tokenizer.eos_token_idmodel = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32)model.eval()watermarker = GreenListWatermarker(model.config.vocab_size)processors = LogitsProcessorList([GreenListLogitsProcessor(watermarker)])inputs = tokenizer(PROMPT, return_tensors="pt")prompt_len = inputs["input_ids"].shape[1]def continuation(logits_processor) -> str: set_seed(SEED) with torch.no_grad(): output = model.generate( **inputs, do_sample=True, temperature=1.0, top_k=50, max_new_tokens=MAX_NEW_TOKENS, pad_token_id=tokenizer.eos_token_id, logits_processor=logits_processor, ) return tokenizer.decode(output[0, prompt_len:], skip_special_tokens=True)plain = continuation(None)watermarked = continuation(processors)out_dir = Path("output")out_dir.mkdir(exist_ok=True)(out_dir / "plain.txt").write_text(plain, encoding="utf-8")(out_dir / "watermarked.txt").write_text(watermarked, encoding="utf-8")print(f"prompt: {PROMPT!r}")print(f"prompt tokens stripped: {prompt_len}")print()print("--- plain, first 220 chars ---")print(textwrap.fill(plain[:220], width=76))print()print("--- watermarked, first 220 chars ---")print(textwrap.fill(watermarked[:220], width=76))print()print(f"wrote {out_dir / 'plain.txt'} and {out_dir / 'watermarked.txt'}")The textwrap.fill calls only keep the terminal preview readable. The files on disk hold
the unwrapped text, and the detector reads the files.
Note output[0, prompt_len:]. The model returns the prompt followed by the continuation,
and the prompt was not watermarked. Leaving it in would dilute every statistic in the rest
of the tutorial with ten tokens of noise.
Run it:
python run_generate.pyExpected output. Two runs of 200 tokens each on CPU takes roughly a minute.
prompt: 'The old lighthouse keeper had a rule about storms:'prompt tokens stripped: 10--- plain, first 220 chars --- only when they got too close to the shore, would they go away. He livedwith two little children. In 1959, he spent 25 years as a local policecaptain. During his eight years on the force he was often called a "dirty l--- watermarked, first 220 chars --- only when they struck it, and it never happened again. She had something tosay of the little green ship's recent woes, and those she saw, though, mightbe the latest to come: fire. The current star was burning. The darwrote output\plain.txt and output\watermarked.txtOn macOS and Linux the last line will show output/plain.txt with forward slashes.
What just happened. Both texts are fluent English and both are plausible continuations of the prompt. They share the first three words, " only when they", then diverge at the fourth. That fourth position is where the bias first got large enough to change the sampler's pick. Nothing about the watermarked version announces itself. Read on its own it runs a little more repetitive, and that is the step 3 distortion surfacing in prose, but no human could point at a specific word and call it.
You now have output/plain.txt and output/watermarked.txt on disk, 200 generated tokens
each, and no record of which token ids produced them.
Step 5: Detect the watermark with a z-test
Goal. Recover the watermark from text alone and turn the green-token count into a p-value.
Why this step. A green-token count means nothing until you know how many tokens produced it. 60% green in a twelve-token tweet is unremarkable; 60% green across 500 tokens is not something chance produces. The z-test converts a count into a statement about how surprised you should be. Bring a bare green rate into a dispute and you have an argument. Bring 3.07e-56 and you have a measurement someone has to attack on its own terms.
Under the null hypothesis, meaning text written with no knowledge of the green lists, each
scored token is green with probability gamma, independently. So across T scored tokens the
green count has mean gamma * T and variance T * gamma * (1 - gamma), and
z = (green_count - gamma * T) / sqrt(T * gamma * (1 - gamma))Create detect.py:
"""Detect the green-list watermark with a one-proportion z-test."""import mathfrom dataclasses import dataclassfrom watermark import GreenListWatermarker@dataclassclass Detection: scored: int green: int green_rate: float z: float p_value: float def verdict(self, threshold: float = 4.0) -> str: return "WATERMARKED" if self.z > threshold else "no watermark"def detect(token_ids: list[int], watermarker: GreenListWatermarker) -> Detection: """Score every token that has a predecessor inside the candidate text.""" scored = 0 green = 0 for prev_token_id, token_id in zip(token_ids, token_ids[1:]): scored += 1 if watermarker.is_green(prev_token_id, token_id): green += 1 if scored == 0: raise ValueError("need at least 2 tokens to score a watermark") gamma = watermarker.gamma expected = gamma * scored z = (green - expected) / math.sqrt(scored * gamma * (1.0 - gamma)) p_value = 0.5 * math.erfc(z / math.sqrt(2.0)) return Detection(scored, green, green / scored, z, p_value)The zip(token_ids, token_ids[1:]) is where the off-by-one lives. The first token of the
text has no predecessor inside the text, so it cannot be scored. A 200-token text yields
199 scored tokens, not 200. Getting this wrong shifts z by a few percent, which matters at
short lengths.
Steps 5 through 8 all read the files in output/, so step 4 has to have run first. And note
that the detector re-encodes text that was decoded from token ids. That round trip is not
guaranteed in general to return the same token count; for this text it does, which is why
200 tokens out gives 199 scored.
The threshold: float = 4.0 default is not arbitrary either. A z above 4 is the
conventional cut-off in this literature, and the reading after the run below says what false
positive rate it buys you.
Create run_detect.py:
"""Run the detector over both saved texts, using only the text itself."""from pathlib import Pathfrom transformers import AutoTokenizerfrom detect import detectfrom watermark import GreenListWatermarkertokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")watermarker = GreenListWatermarker(tokenizer.vocab_size)print(f"{'text':>14} {'scored':>6} {'green':>5} {'rate':>6} " f"{'z':>7} {'p':>10} verdict")for name in ["plain", "watermarked"]: text = Path("output", f"{name}.txt").read_text(encoding="utf-8") token_ids = tokenizer.encode(text) result = detect(token_ids, watermarker) print(f"{name:>14} {result.scored:>6} {result.green:>5} " f"{result.green_rate:>6.1%} {result.z:>7.2f} " f"{result.p_value:>10.2e} {result.verdict()}")wrong_key = GreenListWatermarker(tokenizer.vocab_size, secret_key=99999989)text = Path("output", "watermarked.txt").read_text(encoding="utf-8")result = detect(tokenizer.encode(text), wrong_key)print(f"{'wrong key':>14} {result.scored:>6} {result.green:>5} " f"{result.green_rate:>6.1%} {result.z:>7.2f} " f"{result.p_value:>10.2e} {result.verdict()}")Run it:
python run_detect.pyExpected output:
text scored green rate z p verdict plain 199 50 25.1% 0.04 4.84e-01 no watermark watermarked 199 146 73.4% 15.76 3.07e-56 WATERMARKED wrong key 199 49 24.6% -0.12 5.49e-01 no watermarkWhat just happened. The unwatermarked text scored 25.1% green against an expected 25.0%, a z of 0.04. Real measurements do not come much closer to zero signal than that. The watermarked text scored 73.4%, a z of 15.76. For context, the conventional threshold is z above 4, which corresponds to a false positive rate around 3 in 100,000.
Row three is why I bothered running it at all. Swap the key 15485863 for 99999989 and the same string comes back at 24.6% green, a z of -0.12, which is chance. Nothing about the text changed between rows two and three. Prose that was merely machine-flavoured or repetitive would have lit row three up as well, and it came back at noise. The whole signal is sitting inside 15485863.
Step 6: Find the minimum text length for watermark detection
Goal. Find the shortest prefix of the watermarked text that still clears the detection threshold.
Why this step. "Watermarking does not work on short text" is repeated everywhere, usually attached to a specific number. Those numbers are mostly unsourced. You now have a working detector, so you can stop taking anyone's word for it. The threshold for your own configuration is the only number that actually governs your system.
Create run_sweep.py:
"""Find the shortest text this watermark can actually be detected in."""from pathlib import Pathfrom transformers import AutoTokenizerfrom detect import detectfrom watermark import GreenListWatermarkerTHRESHOLD = 4.0LENGTHS = [10, 20, 30, 40, 50, 75, 100, 150, 200]tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")watermarker = GreenListWatermarker(tokenizer.vocab_size)text = Path("output", "watermarked.txt").read_text(encoding="utf-8")all_token_ids = tokenizer.encode(text)print(f"{'length':>6} {'scored':>6} {'green':>5} {'rate':>6} {'z':>7} " f"{'p':>10} verdict")crossing = Nonefor length in LENGTHS: if length > len(all_token_ids): break result = detect(all_token_ids[:length], watermarker) if crossing is None and result.z > THRESHOLD: crossing = length print(f"{length:>6} {result.scored:>6} {result.green:>5} " f"{result.green_rate:>6.1%} {result.z:>7.2f} " f"{result.p_value:>10.2e} {result.verdict()}")print()print(f"first length in the sweep to clear z > {THRESHOLD}: {crossing} tokens")Run it:
python run_sweep.pyExpected output:
length scored green rate z p verdict 10 9 7 77.8% 3.66 1.28e-04 no watermark 20 19 15 78.9% 5.43 2.81e-08 WATERMARKED 30 29 24 82.8% 7.18 3.41e-13 WATERMARKED 40 39 31 79.5% 7.86 1.95e-15 WATERMARKED 50 49 40 81.6% 9.16 2.71e-20 WATERMARKED 75 74 53 71.6% 9.26 1.00e-20 WATERMARKED 100 99 69 69.7% 10.27 4.78e-25 WATERMARKED 150 149 110 73.8% 13.76 2.10e-43 WATERMARKED 200 199 146 73.4% 15.76 3.07e-56 WATERMARKEDThen, after a blank line, the same run prints its conclusion:
first length in the sweep to clear z > 4.0: 20 tokensWhat just happened. At delta 2.0 and gamma 0.25, this watermark becomes detectable somewhere between 10 and 20 tokens, which is roughly fifteen words at GPT-2's average token length. The "200 words minimum" figure that circulates online is off by more than an order of magnitude, appears in no primary source I could find, and should be discarded.
The ten-token row teaches the more important lesson. Its green rate is 77.8%, higher than the 200-token row's 73.4%, and yet it is correctly reported as undetected. Rate is not evidence. With only nine scored tokens, seven greens is well within what chance produces, so z comes in at 3.66 and the null hypothesis survives.
Do not port the number 20 anywhere. It is one sample from one text, and the crossing point moves with both. It also describes this configuration only. I have not measured the curve at delta 1.0, so I am not going to tell you where it lands.
If you want that number, copy the project first, change the delta default in the copy's
watermark.py, then re-run run_generate.py and run_sweep.py there. Both, in that order.
The sweep reads text from disk, so it will happily report the old numbers until the text is
regenerated.
Do not make that change in place and carry on. Steps 7 and 8 both read
output/watermarked.txt, and their numbers only reproduce for the delta 2.0 text. Step 8's
assertions would still pass, because both implementations would agree on whatever text is
on disk. Nothing would tell you the ground had moved. That is the same silent failure step 2
warned about, and it is worth having met it twice.
Step 7: Test how much editing the watermark survives
Goal. Break it. Replace a growing fraction of the tokens at random and watch the z-score fall.
Why this step. Somebody pastes your model's output into a document, cuts a third of it, and reworks a paragraph. That is the normal case, not the adversarial one, and you need to know what survives it. This is also the attack surface. The same curve tells an adversary exactly how much work removal costs.
One deliberate simplification below: this script edits token ids directly and never decodes back to text, which is the opposite of what step 4 insisted on. That is so the edit fraction can be controlled exactly. A real rewrite would be messier and would change the token count as well.
Create run_robustness.py:
"""Edit the watermarked text and watch the evidence decay."""import randomfrom pathlib import Pathfrom transformers import AutoTokenizerfrom detect import detectfrom watermark import GreenListWatermarkerFRACTIONS = [0.0, 0.05, 0.10, 0.20, 0.30, 0.50]SEED = 7tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")watermarker = GreenListWatermarker(tokenizer.vocab_size)text = Path("output", "watermarked.txt").read_text(encoding="utf-8")original = tokenizer.encode(text)print(f"{'edited':>7} {'green':>5} {'rate':>6} {'z':>7} " f"{'p':>10} verdict")for fraction in FRACTIONS: rng = random.Random(SEED) edited = list(original) positions = rng.sample(range(len(edited)), int(fraction * len(edited))) for position in positions: edited[position] = rng.randrange(tokenizer.vocab_size) result = detect(edited, watermarker) print(f"{fraction:>7.0%} {result.green:>5} {result.green_rate:>6.1%} " f"{result.z:>7.2f} {result.p_value:>10.2e} {result.verdict()}")Run it:
python run_robustness.pyExpected output:
edited green rate z p verdict 0% 146 73.4% 15.76 3.07e-56 WATERMARKED 5% 135 67.8% 13.96 1.44e-44 WATERMARKED 10% 129 64.8% 12.97 8.60e-39 WATERMARKED 20% 111 55.8% 10.03 5.79e-24 WATERMARKED 30% 98 49.2% 7.90 1.41e-15 WATERMARKED 50% 66 33.2% 2.66 3.90e-03 no watermarkWhat just happened. The watermark survives light editing comfortably. Replacing every tenth token still leaves z at 12.97. Even a 30% replacement, a heavy rewrite by any standard, leaves z at 7.90 and well clear of the threshold. Detection fails only at 50%, and by then half the tokens are random vocabulary items and the text is no longer readable English.
Each replacement damages the count twice over. The replaced token is unlikely to be green, and it then becomes the seed for the next token's green list, turning that token's score into a coin flip as well. So the green rate falls faster than the edit fraction does.
A graceful decay curve sounds like good news and mostly is not. Yes, the watermark shrugs off ordinary copy-editing. But a smooth curve also means an attacker never needs a clever exploit. They just grind. Jovanović et al. (2024) put a number on the grinding: under $50 of API queries was enough to both spoof and scrub schemes in this family, at over 80% average success. That attack targets green-list watermarks: the thing you just built.
Step 8: Validate against transformers' WatermarkDetector
Goal. Find out whether you actually got it right. transformers ships the same
algorithm, so make the two green counts agree to the token.
Why this step. Everything so far only proves your code agrees with itself. Your
generator and your detector share the same green_ids function, so a bug in it would
cancel out and still produce a beautiful z-score. The only way to rule that out is to check
against an independent implementation of the same algorithm.
transformers ships one. WatermarkLogitsProcessor implements this exact scheme, and under
seeding_scheme="lefthash" it seeds with hashing_key * previous_token_id. That is
precisely what GreenListWatermarker.green_ids does, and it is no coincidence: it is why
step 2 took the millionth prime as its default key. If both implementations are right,
their green counts have to agree to the token.
Create run_builtin.py:
"""Cross-check our detector against the one shipped in transformers."""from pathlib import Pathimport torchfrom transformers import ( AutoModelForCausalLM, AutoTokenizer, WatermarkDetector, WatermarkingConfig,)from detect import detectfrom watermark import GreenListWatermarkerMODEL_ID = "openai-community/gpt2"tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32)config = WatermarkingConfig( greenlist_ratio=0.25, bias=2.0, hashing_key=15485863, seeding_scheme="lefthash", context_width=1,)library_detector = WatermarkDetector( model_config=model.config, device="cpu", watermarking_config=config)ours = GreenListWatermarker(model.config.vocab_size)print(f"tokenizer.vocab_size={tokenizer.vocab_size} " f"model.config.vocab_size={model.config.vocab_size}")print(f"{'text':>14} {'ours: green/T':>15} {'theirs: green/T':>15} " f"{'ours z':>7} {'theirs z':>8}")for name in ["plain", "watermarked"]: text = Path("output", f"{name}.txt").read_text(encoding="utf-8") token_ids = tokenizer.encode(text) mine = detect(token_ids, ours) batch = torch.tensor([token_ids], dtype=torch.long) theirs = library_detector(batch, z_threshold=4.0, return_dict=True) their_green = int(theirs.num_green_tokens[0]) their_scored = int(theirs.num_tokens_scored[0]) their_z = float(theirs.z_score[0]) print(f"{name:>14} {f'{mine.green}/{mine.scored}':>15} " f"{f'{their_green}/{their_scored}':>15} " f"{mine.z:>7.2f} {their_z:>8.2f}") assert mine.green == their_green, "green counts disagree" assert mine.scored == their_scored, "scored-token counts disagree"print()print("green counts and z-scores agree: our implementation is bit-compatible")print("with transformers' WatermarkLogitsProcessor under seeding_scheme='lefthash'")Run it:
python run_builtin.pyExpected output:
tokenizer.vocab_size=50257 model.config.vocab_size=50257 text ours: green/T theirs: green/T ours z theirs z plain 50/199 50/199 0.04 0.04 watermarked 146/199 146/199 15.76 15.76green counts and z-scores agree: our implementation is bit-compatiblewith transformers' WatermarkLogitsProcessor under seeding_scheme='lefthash'What just happened. Both assertions passed. Two independently written implementations counted 146 green tokens out of 199 and computed the same z to two decimals. Your green lists are the library's green lists, so the 15.76 from step 5 was measuring the watermark. A shared bug would have surfaced here as a mismatch.
The first printed line is part of the check, not debug noise: it confirms
tokenizer.vocab_size and model.config.vocab_size really are the same 50,257 that both
halves have been assuming. And although the script only instantiates WatermarkDetector,
that class builds a WatermarkLogitsProcessor internally and derives its green lists from
it, so the comparison reaches the generator side too. Both implementations call their own
equivalent of green_ids, and the counts match.
The library spells the parameters differently, so map them once: greenlist_ratio is gamma,
bias is delta, hashing_key is your secret key, and context_width=1 means the green
list is seeded from one preceding token, matching the single prev_token_id your
green_ids takes.
If you extend this comparison, watch ignore_repeated_ngrams on WatermarkDetector. It
defaults to False, and only that default matches the detector you wrote. Set it to True
and it counts each distinct bigram once, and the two implementations will legitimately
disagree.
Troubleshooting LogitsProcessor and detection errors
These are the failures you are most likely to hit, with the actual strings.
ValueError: are passed to the logits processor
Reproduced on the pinned stack. You will see
ValueError: Make sure that all the required parameters: ['self', 'input_ids', 'scores'] for <class 'type'> are passed to the logits processor.
Cause. You passed the class instead of an instance:
LogitsProcessorList([GreenListLogitsProcessor]). LogitsProcessorList inspects each
entry's __call__ signature, and on an unbound class it sees three parameters including
self, decides your processor needs extra keyword arguments, and refuses. The <class 'type'> in the message is the tell.
Fix. Construct it: LogitsProcessorList([GreenListLogitsProcessor(watermarker)]).
IndexError: Dimension out of range
Reproduced on the pinned stack.
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)Cause. Your __call__ returned a tensor with the wrong rank, usually by returning
scores[0] instead of the full (batch_size, vocab_size) tensor. Generation then indexes
a dimension that no longer exists.
Fix. Always return a tensor of the same shape you were given. Assert it while
developing: assert out.shape == scores.shape.
RuntimeError: probability tensor contains inf, nan or element < 0
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0Cause. The processor wrote a non-finite value into scores, and sampling then failed
on the resulting probabilities. The usual source is bad index arithmetic writing over the
whole row, or adding to a value that was already negative infinity in a way that produces
nan. Widely reported against torch.multinomial; not reproduced here, because the
implementation in this tutorial does not hit it.
Fix. Check torch.isfinite(biased).all() inside __call__ while developing, and
confirm your index tensor holds token ids rather than positions.
RuntimeError: Expected all tensors to be on the same device
On a GPU box you will see
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Cause. The green-list index tensor was built on CPU while scores lives on the GPU.
This tutorial was verified on CPU only, so this is cited from the upstream issue trackers
rather than reproduced here.
Fix. This is why GreenListLogitsProcessor.__call__ writes
green.to(scores.device) rather than green. Build device-dependent tensors inside
__call__, never in __init__.
ValueError: need at least 2 tokens to score a watermark
ValueError: need at least 2 tokens to score a watermarkCause. Your own guard in detect.py, raised on a text of fewer than two tokens. The
first token has no predecessor, so there is nothing to score.
Fix. Nothing to fix. The guard is doing its job, so handle it at the call site instead of lowering it. A one-token detection result would be meaningless.
LogitsProcessor runs before temperature and top-k, not after
This one produces no error at all. Dangerous for exactly that reason. I only went looking
for it because a temperature sweep returned numbers I could not explain, and the
explanation turned out to be ordering rather than statistics. It is natural to assume that
by the time your processor runs, top_k=50 has
already restricted the candidates and temperature has already scaled the scores. On
transformers 5.15.0, neither is true. Custom processors are appended to the end of the
merged list, but the temperature and top-k warpers run after that list, so your __call__
receives the model's raw logits, untouched.
Measured on the pinned stack: with top_k=50, a probe processor still saw all 50,257
scores finite, and at temperature=0.5 it still saw the model's raw maximum logit
unchanged. If the sampling stack itself is unfamiliar, how inference and serving actually
transform logits covers the surrounding
machinery.
This bites twice. Your delta lands before temperature scaling, so temperature divides it downstream, and delta 2.0 at temperature 0.5 behaves like a 4.0 in probability space. Your bias can also promote a token that top-k would otherwise have cut, because top-k has not run yet. Change the temperature and you must re-measure. Delta will not mean what it meant before.
Warnings you can ignore
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
is informational. GPT-2 is not gated and no token is needed.
On Windows you may also get a UserWarning from huggingface_hub saying its cache system
uses symlinks and your machine does not support them. Downloads still work, they just use
more disk. Set HF_HUB_DISABLE_SYMLINKS_WARNING=1 to silence it.
How the watermarker and detector fit together
You have built two halves that never communicate except through a shared secret.
flowchart TD
KEY["Secret key<br/>15485863"]
subgraph GEN["Generation - run_generate.py"]
P["Prompt"] --> M["GPT-2 forward pass"]
M --> L["Raw logits<br/>50257 scores"]
L --> W["GreenListLogitsProcessor<br/>adds delta to green ids"]
W --> SM["Temperature, top-k, sampler"]
SM --> T["Watermarked text on disk"]
end
subgraph DET["Detection - run_detect.py"]
C["Candidate text"] --> TK["Tokenizer"]
TK --> CNT["Count green tokens<br/>over each adjacent pair"]
CNT --> Z["z-score and p-value"]
Z --> VD{"z above 4?"}
end
KEY -.-> W
KEY -.-> CNT
T -.-> C
style KEY fill:#C2185B,color:#FFFFFF
style P fill:#95A5A6,color:#FFFFFF
style M fill:#4A90E2,color:#FFFFFF
style L fill:#4A90E2,color:#FFFFFF
style W fill:#6BCF7F,color:#2C2C2A
style SM fill:#7B68EE,color:#FFFFFF
style T fill:#FFD93D,color:#2C2C2A
style C fill:#FFD93D,color:#2C2C2A
style TK fill:#98D8C8,color:#2C2C2A
style CNT fill:#6BCF7F,color:#2C2C2A
style Z fill:#7B68EE,color:#FFFFFF
style VD fill:#FFA07A,color:#2C2C2A
Generation needs logits, so it runs inside your serving stack, in the same process as the model. Detection needs a string, so it runs anywhere a string goes: a background job, a support ticket queue, somebody's laptop. Two things pass between the halves. The key, which nobody outside them may hold, and the text, which goes out into the world and comes back edited. Step 4 made you write to disk precisely so the detector could only ever see the second one.
Whoever holds 15485863 can detect. Nobody else can, and that is not an implementation gap someone closes in the next release. Publishing the key publishes the recipe for stripping and forging the mark in the same breath, so a provider who wants detection to keep working has to keep it secret. What you end up with is not public verification. Google can tell you whether a string came out of Gemini. You, holding the same string and no key, cannot, and nothing you built in the last 45 minutes changes that.
The complete watermarker and detector code
Final file tree, excluding the virtual environment:
synthid-lab/├── requirements.txt├── check_env.py├── watermark.py├── detect.py├── explore.py├── try_greenlist.py├── try_bias.py├── run_generate.py├── run_detect.py├── run_sweep.py├── run_robustness.py├── run_builtin.py└── output/ ├── plain.txt └── watermarked.txtEvery file except watermark.py was shown complete in the step that created it.
watermark.py was built across steps 2 and 3, so here it is in full:
"""A green-list text watermark, built from scratch."""import torchfrom transformers import LogitsProcessor# The millionth prime. Any large odd number works; this one is the default# that transformers' own WatermarkingConfig uses, so our green lists and# the library's green lists come out identical.DEFAULT_SECRET_KEY = 15485863class GreenListWatermarker: """Splits the vocabulary into a green list and a red list at every step. The split is a pure function of (secret key, previous token id), so the generator and the detector derive the same lists without exchanging anything except the key. """ def __init__( self, vocab_size: int, secret_key: int = DEFAULT_SECRET_KEY, gamma: float = 0.25, delta: float = 2.0, ) -> None: self.vocab_size = vocab_size self.secret_key = secret_key self.gamma = gamma self.delta = delta self.green_size = int(gamma * vocab_size) def green_ids(self, prev_token_id: int) -> torch.Tensor: """Token ids on the green list, given the token that came before.""" rng = torch.Generator(device="cpu") rng.manual_seed((self.secret_key * int(prev_token_id)) % (2**64 - 1)) permutation = torch.randperm(self.vocab_size, generator=rng) return permutation[: self.green_size] def is_green(self, prev_token_id: int, token_id: int) -> bool: """True if token_id is on the green list seeded by prev_token_id.""" green = self.green_ids(prev_token_id) return bool((green == int(token_id)).any())class GreenListLogitsProcessor(LogitsProcessor): """Adds delta to the logit of every green-list token, every step.""" def __init__(self, watermarker: GreenListWatermarker) -> None: self.watermarker = watermarker def __call__( self, input_ids: torch.LongTensor, scores: torch.FloatTensor ) -> torch.FloatTensor: biased = scores.clone() for row in range(input_ids.shape[0]): prev_token_id = input_ids[row, -1].item() green = self.watermarker.green_ids(prev_token_id) biased[row, green.to(scores.device)] += self.watermarker.delta return biasedHow SynthID's tournament sampling differs from a green list
You have built the green-list watermark. Google's SynthID-Text is a different mechanism. The two schemes share a goal and almost nothing else.
The green list modifies the logits. It adds delta before the softmax, which changes the distribution the model samples from. That is why step 3 showed " banana" tripling in probability, and it is why the technique is called distortionary: the watermarked model is not the same model any more.
SynthID-Text leaves the logits alone and modifies the sampling instead. It draws multiple candidate tokens from the model's own unmodified distribution, then runs a single-elimination tournament between them, where a keyed pseudorandom function decides each match. The winner is emitted. The candidates all came from the true distribution, so the scheme can be configured to be non-distortionary: it preserves the model's output distribution in expectation. The green list cannot do that at any delta.
I went into this expecting tournament sampling to be something you could bolt onto what you just built. You cannot, and the reason is worth understanding. Once you accept that the mark lives in the choice between candidates instead of in the scores, the clean z-test goes with it. There is no closed form as tidy as the one you wrote, and Google's strongest detector is a trained Bayesian model. What you buy for that is a nearly free watermark: Gemma 7B-IT generated at 15.527 ms per token unwatermarked and 15.615 ms with 30 tournament layers, a 0.57% increase.
You do not get out of paying, you just pay somewhere else. The green list bills you in text
quality up front, and hands you a detector that fits in one function with one call to
math.erfc. SynthID bills you 0.57% of generation latency, and hands you a Bayesian model
somebody has to train, version alongside the generator, and recalibrate whenever the
sampling config moves. Only one of those bills shows up on a model card.
Your own numbers contradict two things people repeat about SynthID constantly:
- "SynthID needs about 200 words." I went looking for this in the Nature paper, DeepMind's blog and Anthropic's write-up. It is in none of them. Your own sweep crossed at 20 tokens; Kirchenbauer et al. report detection from 25. The 400-token figure people may be half-remembering is Nature's evaluation setting at a 1% false positive rate. An evaluation setting is not a minimum.
- "Watermarking does not change quality." True for a non-distortionary scheme. Not true for the one in this tutorial, as step 3 measured directly.
This stopped being purely academic recently. EU AI Act Article 50(2), which requires providers of generative systems to mark outputs in a machine-readable format, became applicable on 2 August 2026 - an obligation that gets considerably harder when the model is running on a device you cannot reach. Anthropic announced in August 2026 that Claude models released on or after that same date will carry a watermark it describes as a version of the SynthID-Text approach.
Watermarking marks the text itself. The complementary approach is to record where the text came from, which is the subject of signed provenance logs in Python. The two solve different halves of the same problem, and a system that needs to answer "where did this come from" in court will want both.
Where to go next
If you want to keep going, selfhash is the cheapest next thing and the attack is the most
interesting.
- Switch the seeding scheme to
selfhash. The version you built seeds only from the previous token, so an attacker who learns the mapping for common tokens learns a lot.selfhashfolds the candidate token into the seed as well, which resists that. ReadWatermarkLogitsProcessor._score_rejection_samplingintransformersand implement it inGreenListWatermarker, then confirm your z-scores still match the library's withseeding_scheme="selfhash". - Map the delta-versus-quality frontier. In a copy of the project, as in step 6, re-run step 4 across delta values from 0.5 to 5.0, recording the z-score from step 5 and the model's perplexity on the generated text. You will get a curve, not a number. Picking a point on it is a product call, and you should make it with the perplexity column in front of you.
- Reproduce the watermark-stealing attack. This is the one worth your weekend, and it matters beyond attribution: reliable detection is one practical defence against synthetic text quietly reentering training corpora. Jovanović et al. describe querying a watermarked model repeatedly until you can approximate its green lists without ever holding the key. You are in an unusually good position to try it, because you have both a watermarked model and its ground truth sitting on disk. Recover the lists, then measure two things the paper cannot tell you about your own configuration: how many queries it took, and how far the recovered lists have to drift before your step 5 detector stops believing them. Most attack write-ups report only the first number. The second is the one that decides whether the attack matters to you.
References
Papers
- Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023). A Watermark for Large Language Models. ICML 2023. arXiv:2301.10226. https://arxiv.org/abs/2301.10226
- Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., et al. (2024). Scalable watermarking for identifying large language model outputs. Nature, 634, 818-823. https://doi.org/10.1038/s41586-024-08025-4
- Jovanović, N., Staab, R., & Vechev, M. (2024). Watermark Stealing in Large Language Models. ICML 2024. arXiv:2402.19361. https://arxiv.org/abs/2402.19361
Documentation
- Hugging Face Transformers 5.15.0, Generation features (watermarking with
WatermarkingConfigandWatermarkDetector). https://huggingface.co/docs/transformers/main/en/generation_features - Hugging Face Transformers 5.15.0, Utilities for Generation (
LogitsProcessor,LogitsProcessorList). https://huggingface.co/docs/transformers/main/en/internal/generation_utils - Hugging Face Transformers, v5 Migration Guide. https://github.com/huggingface/transformers/blob/main/MIGRATION_GUIDE_V5.md
- Hugging Face, openai-community/gpt2 model card (124M parameters, MIT). https://huggingface.co/openai-community/gpt2
- Python 3.13, math - Mathematical functions (
math.erfc). https://docs.python.org/3/library/math.html
Code
- Kirchenbauer, J., et al. lm-watermarking - the reference implementation of the green-list scheme. https://github.com/jwkirchenbauer/lm-watermarking
- Google DeepMind. synthid-text (Apache-2.0). https://github.com/google-deepmind/synthid-text
Industry and regulation
- Google DeepMind. (2024). Watermarking AI-generated text and video with SynthID. https://deepmind.google/blog/watermarking-ai-generated-text-and-video-with-synthid/
- Anthropic. (2026, August 14). How Claude's text watermarking works. https://www.anthropic.com/news/claude-text-watermark
- EU Artificial Intelligence Act, Article 50 - Transparency obligations. https://artificialintelligenceact.eu/article/50/
Related Articles
- How to Rerank Retrieval Results with a Cross-Encoder
- BM25 vs Dense Retrieval: Measure It on Your Own Corpus
- Superpowers Plugin for Claude Code: Install and Verify
Natural Language Processing Nlp
