How to Check if a Chat Template Supports a System Prompt

What you'll build

Seeing your system prompt in the rendered output does not prove a chat template supports it. Move a system prompt from Qwen3 to Gemma 3 and it is still in the rendered prompt, word for word, but Gemma 3's template pastes it into the first user turn instead of giving it a turn of its own, and nothing raises. In this intermediate tutorial you build a Python probe, templatestudio/anatomy.py, that inspects a Hugging Face chat template for Qwen3, Llama 3 and Gemma 3 without reading its Jinja. It is for engineers who already call apply_chat_template.

This is the table you print at the end, one row per model family:

family  system    developer  tools  generation promptqwen3   own_turn  dropped    True   '<|im_start|>assistant\n'llama3  own_turn  own_turn   False  '<|start_header_id|>assistant<|end_header_id|>\n\n'gemma3  folded    rejected   False  '<start_of_turn>model\n'gemma4  own_turn  own_turn   True   '<|turn>model\n'

• system is where a system message lands: in its own turn, folded into the first user turn, or dropped.
• developer is the same question for a developer message, plus rejected when the template raises on it.
• tools says whether the template's source mentions tools at all.
• generation prompt is the exact string the template appends to cue the model's turn.

The four families give three different answers for a developer message, and Gemma 3 folds the system message. Once the tokenizers are cached, you get all of it in one run of about 20 seconds on a laptop CPU.

By the end you will have templatestudio/anatomy.py, a module that tells you, before you swap models, how a Hugging Face tokenizer's chat template treats your messages. You give it the model family's turn markers (the special strings that open and close each turn, such as <|im_start|> and <|im_end|>), and it reports:

• the exact generation prompt the template appends;
• whether a system message gets its own turn, is folded into the user turn, or is dropped;
• how the template treats any other role, such as developer;
• what add_generation_prompt does to a half-written assistant turn.

It gets every answer except the tools hint by rendering small test conversations, never by reading the template's Jinja.

Verified against Python 3.12.2, transformers==5.22.1, tokenizers==0.23.3, huggingface-hub==2.4.0, jinja2==3.1.6 on 2026-11-10. The Gemma 2 row in the troubleshooting table was checked by rendering its template string, fetched from the Hub's public API; the gated Gemma 2 tokenizer itself was not loaded.

It is Part 4 of a series and continues the project from Part 3, How to Render a Chat Template with Jinja Like Transformers. It takes about 25 minutes, needs no API key and no account, and downloads about 73 MB of tokenizer files, or 89 MB if Qwen3 is not cached from Part 1.

Prerequisites: transformers 5.22.1, four Hub tokenizers and the Part 3 project

You need the project from Parts 1 to 3: the chat-templates-part1/ folder with its virtual environment and the templatestudio/ package. This part uses only two modules from it, templatestudio/messages.py and templatestudio/render.py, both as Part 2 left them. It adds one module, templatestudio/anatomy.py, a data file, and twelve scripts, and it adds one function to render.py in step 10. If you do not have the project, build it in three moves:

• Follow Part 1's Prerequisites to create the folder and the environment.
• Create the templatestudio/ folder with an empty __init__.py in it.
• Copy templatestudio/messages.py and templatestudio/render.py from Part 2's complete code.

Go to the project folder, activate the environment, and turn off the advisory that Transformers prints when PyTorch is not installed. Run the cd from the folder that contains chat-templates-part1/:

cd chat-templates-part1source .venv/bin/activateexport TRANSFORMERS_NO_ADVISORY_WARNINGS=1

On Windows, activate with .venv\Scripts\activate in PowerShell or source .venv/Scripts/activate in Git Bash, and set the variable in PowerShell with $env:TRANSFORMERS_NO_ADVISORY_WARNINGS = "1". Every script in this part runs from the project root.

The versions are unchanged from Parts 1 to 3. Transformers 5.22.2 (2026-11-06) hardens the Jinja sandbox and is not adopted yet, so that the whole series stays on one pin. If you are starting fresh, install them inside the activated environment:

pip install "transformers==5.22.1" "tokenizers==0.23.3" \    "huggingface-hub==2.4.0" "jinja2==3.1.6"

In PowerShell, continue the line with a backtick instead of a backslash:

pip install "transformers==5.22.1" "tokenizers==0.23.3" `    "huggingface-hub==2.4.0" "jinja2==3.1.6"

The series needs Python 3.11 or later, because messages.py imports typing.NotRequired. It was verified on 3.12.2, so use 3.12 if you can. Check the install and the two modules before you go on:

python -c "import transformers as t, jinja2 as j; print(t.__version__, j.__version__)"python -c "from templatestudio.render import render; print(render.__name__)"python -c "from templatestudio.messages import Conversation; print('messages ok')"

Expected output:

5.22.1 3.1.6rendermessages ok

Download the Qwen3, Llama 3, Gemma 3 and Gemma 4 tokenizers without a token

The official Llama 3 and Gemma 3 repositories on the Hugging Face Hub are gated. Loading them without an access token fails with OSError: You are trying to access a gated repo. This part uses two ungated copies instead, plus Gemma 4, which is ungated and downloads anonymously like Qwen3. Their chat templates are character-for-character identical to the official ones, which you can confirm without a token: the Hub's model API (https://huggingface.co/api/models/<repo>) shows each repository's template, gated or not.

The licence terms still apply to the copies: the Meta Llama 3 Community License and the Gemma Terms of Use. Gemma 4 is released under Apache 2.0, as its model card states.

Create families.py in the project root. Every script imports the model IDs from here, so you change a model in one place. Gemma 4 has its own constant because it only joins in step 6:

# families.py# Three families, three wire formats. The Llama 3 and Gemma 3 repos are ungated# copies whose chat templates match the official, gated repos exactly.FAMILIES: dict[str, str] = {    "qwen3": "Qwen/Qwen3-0.6B",    "llama3": "NousResearch/Meta-Llama-3-8B-Instruct",    "gemma3": "unsloth/gemma-3-1b-it",}# Gemma 4 is ungated. It joins in step 6, where its turn markers matter.GEMMA4 = "google/gemma-4-E2B-it"

Then create fetch_models.py, which downloads all four tokenizers, Gemma 4 included, and prints the length of each chat template:

# fetch_models.pyfrom families import FAMILIES, GEMMA4from templatestudio.render import load_tokenizerfor model_id in [*FAMILIES.values(), GEMMA4]:    tokenizer = load_tokenizer(model_id)    print(f"{model_id:<38} template: {len(tokenizer.chat_template)} chars")

Run it:

python fetch_models.py

Expected output:

Qwen/Qwen3-0.6B                        template: 4168 charsNousResearch/Meta-Llama-3-8B-Instruct  template: 389 charsunsloth/gemma-3-1b-it                  template: 1532 charsgoogle/gemma-4-E2B-it                  template: 18567 chars

The first run downloads the tokenizer files: about 9 MB for Llama 3, 33 MB for Gemma 3 and 31 MB for Gemma 4, plus 16 MB for Qwen3 if Part 1 did not already cache it. That first run can also print two warnings, and both are harmless. One starts Warning: You are sending unauthenticated requests to the HF Hub., and on Windows the other starts UserWarning: `huggingface_hub` cache-system uses symlinks by default. Later runs read the local cache and print only the four lines. The template lengths are the fingerprint: if one differs, you have a different template, and every later output will differ too.

How to inspect a Hugging Face chat template: the four-part skeleton and differential probing

Part 3 built the faithful environment: the Jinja settings and variables that turn a template into the same string Transformers produces. This part reads what that string contains. It calls apply_chat_template directly, through render, because a probe should test the exact path your application uses, and Part 3 proved that its faithful environment and apply_chat_template give the same string anyway. The Chat Templates Handbook, the book this series accompanies, splits every chat template into the same four parts, in order, and calls them the four-part skeleton:

1. Preamble: what is written once, before any message, usually a beginning-of-sequence (BOS) token.
2. System handling: what happens to a system message. It can get its own turn, be folded into the first user turn, or be dropped.
3. Message loop: how each user and assistant message becomes a turn, with its markers and role name.
4. Generation prompt: the assistant header the template appends when add_generation_prompt=True, so that the model knows its turn has started.

You can find the four parts by reading a template's Jinja, and step 1 does that once. But templates run to hundreds or thousands of characters, and what matters is the string that reaches the model. So the rest of this part uses the book's second idea, differential probing: render two conversations that differ in exactly one way, compare the two outputs, and the difference shows you exactly one behaviour.

grid-rows: 1
grid-gap: 40
skeleton: "The four-part skeleton (what a render contains)" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 12
  p1: "1. Preamble\nBOS token, once" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
  p2: "2. System handling\nown turn, folded or dropped" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  p3: "3. Message loop\nmarkers + role name per turn" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  p4: "4. Generation prompt\nappended by add_generation_prompt" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
}
probe: "Differential probing (how you read it)" {
  label.near: top-center
  a: "Conversation A" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
  b: "Conversation B\nA with one change" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
  t: "apply_chat_template\nthe model's own template" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  d: "Difference between the renders\n= one behaviour" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
  a -> t: "1. render"
  b -> t: "1. render"
  t -> d: "2. compare"
}

The left side is what you look for, and the right side is how you find it. Each probe in this part changes one thing in the conversation or in one argument (the generation prompt, the system message, the role) and reads one of the four parts off the difference. The template stays a black box, which is the point: the generation-prompt probe in step 3 works the same on a 389-character template and an 18,567-character one. The turn-boundary probes need one input from you, the family's turn markers, as step 6 shows.

Step 1: Read the four parts in Llama 3's chat template source

Goal: print Llama 3's template in a readable shape and find the four parts in it.

Why this step: Llama 3's template is the shortest of the four, 389 characters, so it is the one place where reading the source is quick. Reading it once gives you a picture of where each part lives in Jinja, which makes the probe results in later steps easy to place. The template is stored on one line, and its strings contain real newlines. So the script prints each newline as \n and starts a new line before each {% tag and each +. The template itself is not modified.

# show_llama3_source.pyimport refrom families import FAMILIESfrom templatestudio.render import load_tokenizersource = load_tokenizer(FAMILIES["llama3"]).chat_templateprint(f"{len(source)} chars. Newlines shown as \\n, a line break before {{% and +:")readable = source.replace("\n", "\\n").replace("{%", "\n{%")print(re.sub(r" ?\+ ", "\n    + ", readable).strip())

Run it:

python show_llama3_source.py

Expected output:

389 chars. Newlines shown as \n, a line break before {% and +:{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>'    + message['role']    + '<|end_header_id|>\n\n'    + message['content'] | trim    + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token    + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}

What just happened: you can now point at all four parts. The diagram puts each part next to the lines that write it.

grid-rows: 4
grid-columns: 2
grid-gap: 10
h1: "1. Preamble" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
c1: "bos_token + content, only when loop.index0 == 0" {style: {fill: "#FFFFFF"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
h2: "2. System handling" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
c2: "no special case: system goes through the loop like any role" {style: {fill: "#FFFFFF"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
h3: "3. Message loop" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
c3: "header + role + header end + content | trim + <|eot_id|>" {style: {fill: "#FFFFFF"; stroke: "#2C6FB0"; font-color: "#2C2C2A"}}
h4: "4. Generation prompt" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
c4: "if add_generation_prompt: the assistant header" {style: {fill: "#FFFFFF"; stroke: "#5A4BC4"; font-color: "#2C2C2A"}}

Three details in that source decide the output. The preamble is written inside the loop, on the first message, so a template can mix parts that the skeleton keeps separate. There is no if message['role'] == 'system' anywhere, which means any role string becomes a header: remember that for step 8. And nothing writes a newline after <|eot_id|>, so Llama 3 turns run straight into each other, as the next step's render shows.

Step 2: Compare the four parts across Qwen3, Llama 3 and Gemma 3 renders

Goal: render one test conversation with all three templates and read the four parts off the output.

Why this step: Qwen3's template is 4,168 characters and Gemma 3's is 1,532. Reading them the way you read Llama 3's would take a while, and you would still be guessing what they print. The render is the ground truth. The test conversation uses marker strings, SYS_MARKER, USER_MARKER and ASST_MARKER, that no template contains, so you can find each message in the output with no doubt about which text came from where. Steps 3 to 6 reuse it, so it goes in the new module.

Create templatestudio/anatomy.py:

# templatestudio/anatomy.pyfrom __future__ import annotationsfrom templatestudio.messages import Conversation# Each marker is a string no real template contains, so find() locates it exactly.PROBE: Conversation = [    {"role": "system", "content": "SYS_MARKER"},    {"role": "user", "content": "USER_MARKER"},    {"role": "assistant", "content": "ASST_MARKER"},]

Then create show_skeleton.py, which uses Part 1's visible_whitespace to show every newline:

# show_skeleton.pyimport sysfrom families import FAMILIESfrom templatestudio.anatomy import PROBEfrom templatestudio.render import load_tokenizer, render, visible_whitespacesys.stdout.reconfigure(encoding="utf-8")  # visible_whitespace prints non-ASCIIfor name, model_id in FAMILIES.items():    tokenizer = load_tokenizer(model_id)    rendered = render(tokenizer, PROBE, add_generation_prompt=True)    print(f"=== {name} ({len(rendered)} chars)")    print(visible_whitespace(rendered))

Run it:

python show_skeleton.py

Expected output:

=== qwen3 (164 chars)<|im_start|>system\nSYS_MARKER<|im_end|>\n<|im_start|>user\nUSER_MARKER<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nASST_MARKER<|im_end|>\n<|im_start|>assistant\n=== llama3 (259 chars)<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nSYS_MARKER<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nUSER_MARKER<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nASST_MARKER<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n=== gemma3 (129 chars)<bos><start_of_turn>user\nSYS_MARKER\n\nUSER_MARKER<end_of_turn>\n<start_of_turn>model\nASST_MARKER<end_of_turn>\n<start_of_turn>model\n

What just happened: you have the same conversation in three wire formats. Read across each row of the diagram to compare one part across the three families.

grid-rows: 5
grid-columns: 4
grid-gap: 8
x: "" {style: {opacity: 0}}
q: "qwen3" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
l: "llama3" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
g: "gemma3" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
r1: "1. Preamble" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
q1: "(none)" {style: {fill: "#FFFFFF"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
l1: "<|begin_of_text|>" {style: {fill: "#FFFFFF"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
g1: "<bos>" {style: {fill: "#FFFFFF"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
r2: "2. System handling" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
q2: "own turn" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
l2: "own turn" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
g2: "folded into the user turn" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
r3: "3. Message loop" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
q3: "<|im_start|>role\\n ... <|im_end|>\\n" {style: {fill: "#FFFFFF"; stroke: "#2C6FB0"; font-color: "#2C2C2A"}}
l3: "header tags ... <|eot_id|>, no newline" {style: {fill: "#FFFFFF"; stroke: "#2C6FB0"; font-color: "#2C2C2A"}}
g3: "<start_of_turn>, role 'model'" {style: {fill: "#FFFFFF"; stroke: "#2C6FB0"; font-color: "#2C2C2A"}}
r4: "4. Generation prompt" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
q4: "<|im_start|>assistant\\n" {style: {fill: "#FFFFFF"; stroke: "#5A4BC4"; font-color: "#2C2C2A"}}
l4: "assistant header + \\n\\n" {style: {fill: "#FFFFFF"; stroke: "#5A4BC4"; font-color: "#2C2C2A"}}
g4: "<start_of_turn>model\\n" {style: {fill: "#FFFFFF"; stroke: "#5A4BC4"; font-color: "#2C2C2A"}}

The red cell is the one that changes how your policy reaches the model. Gemma 3 has no system turn: its template puts the system text at the front of the first user turn, followed by a blank line. Your instructions are still in the prompt, but they now arrive in the user's turn, as if the user had typed them.

Gemma 3 also calls the assistant model on the wire, so code that searches the output for assistant breaks on it. Llama 3 has no newline between turns, as step 1 predicted, so a hand-built Llama 3 prompt that adds one is not the format the model was trained on. That is Part 1's silent format penalty again. And Qwen3 writes an empty <think> block into the last assistant turn even though the conversation carries no reasoning; Part 6 covers why.

Step 3: Find a template's generation prompt by diffing two renders

Goal: get the exact generation prompt string of each template, without reading any source.

Why this step: this is the first real differential probe. add_generation_prompt=True and add_generation_prompt=False give two renders of the same conversation, and the only difference between them is the generation prompt. Code that finds where the model's answer starts, or strips the assistant header from a stream, depends on those exact bytes, so take them from what the template writes rather than typing them in from a model card.

Replace the imports at the top of templatestudio/anatomy.py with every import the module needs up to step 9. dataclass, re and TemplateError stay unused until steps 4, 5 and 8, which is harmless, and it leaves only one import change for later, in step 10:

# templatestudio/anatomy.py (replace the imports at the top)from __future__ import annotationsimport refrom dataclasses import dataclassfrom jinja2.exceptions import TemplateErrorfrom transformers import PreTrainedTokenizerBasefrom templatestudio.messages import Conversationfrom templatestudio.render import render

Then add two functions at the end of the file. suffix_diff walks two strings from the front while they agree and returns what is left of the longer one:

# templatestudio/anatomy.py (add at the end)def suffix_diff(shorter: str, longer: str) -> str:    """Return the tail of `longer` after the prefix it shares with `shorter`."""    i = 0    while i < len(shorter) and i < len(longer) and shorter[i] == longer[i]:        i += 1    return longer[i:]def generation_prompt(tokenizer: PreTrainedTokenizerBase) -> str:    """Return exactly what add_generation_prompt=True appends to the probe."""    without = render(tokenizer, PROBE, add_generation_prompt=False)    with_prompt = render(tokenizer, PROBE, add_generation_prompt=True)    return suffix_diff(without, with_prompt)

Create show_gen_prompt.py:

# show_gen_prompt.pyfrom families import FAMILIESfrom templatestudio.anatomy import generation_promptfrom templatestudio.render import load_tokenizerfor name, model_id in FAMILIES.items():    tokenizer = load_tokenizer(model_id)    print(f"{name:<7} {generation_prompt(tokenizer)!r}")

Run it:

python show_gen_prompt.py

Expected output:

qwen3   '<|im_start|>assistant\n'llama3  '<|start_header_id|>assistant<|end_header_id|>\n\n'gemma3  '<start_of_turn>model\n'

What just happened: the diagram shows the probe on Gemma 3. The two renders share everything up to the end of the last turn, and what the longer one adds is the generation prompt.

direction: down
off: "add_generation_prompt=False\n<bos> ... ASST_MARKER<end_of_turn>\\n" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
on: "add_generation_prompt=True\n<bos> ... ASST_MARKER<end_of_turn>\\n<start_of_turn>model\\n" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
diff: "suffix_diff: '<start_of_turn>model\\n'\nthe generation prompt" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
off -> diff: "1. shared prefix"
on -> diff: "2. what is left"

!r prints each string with its newlines escaped, so you can count them: Llama 3's prompt ends with two newlines, the other two with one. That difference matters downstream: code that strips the assistant header from model output with one hard-coded \n leaves a stray newline at the start of every Llama 3 answer, and code that uses the probed value does not. The probe makes no assumption about the template: it would find the generation prompt of a family it has never seen, which is not true of the probe in step 5.

Step 4: Check if a chat template supports a system prompt, and why a folded one still passes

Goal: collect the probes into one report object, and check the most obvious system-message test on all three families.

Why this step: a report you can print, store and compare is easier to use than separate function calls. The first question most people ask on a model swap is "does my system message survive?", so the report answers it directly: has_system_role is True when the system text appears anywhere in the render. One more field comes along only because it is cheap: whether the template's source mentions tools, a first hint of tool support that Part 5 replaces with a real probe. Running it on Gemma 3 shows what the survival question misses.

Add the report at the end of templatestudio/anatomy.py. It uses the dataclass decorator that step 3 imported:

# templatestudio/anatomy.py (add at the end)@dataclassclass TemplateReport:    has_system_role: bool  # does the system text appear in the render at all?    generation_prompt: str  # exactly what add_generation_prompt appends    declares_tools: bool  # does the word "tools" appear in the template source?    sample_render: str  # the probe render, for reading the skeleton by eyedef describe_template(tokenizer: PreTrainedTokenizerBase) -> TemplateReport:    """Report a template's behaviour by rendering the probe, not by reading it."""    rendered = render(tokenizer, PROBE, add_generation_prompt=True)    source = tokenizer.chat_template or ""    return TemplateReport(        has_system_role="SYS_MARKER" in rendered,        generation_prompt=generation_prompt(tokenizer),        declares_tools="tools" in source,        sample_render=rendered,    )

Create show_report.py:

# show_report.pyfrom families import FAMILIESfrom templatestudio.anatomy import describe_templatefrom templatestudio.render import load_tokenizerprint(f"{'family':<7} {'has_system_role':<16} declares_tools")for name, model_id in FAMILIES.items():    report = describe_template(load_tokenizer(model_id))    print(f"{name:<7} {report.has_system_role!s:<16} {report.declares_tools}")

Run it:

python show_report.py

Expected output:

family  has_system_role  declares_toolsqwen3   True             Truellama3  True             Falsegemma3  True             False

What just happened: has_system_role is True for Gemma 3, the model that folds the system message. A model-swap check written as assert describe_template(tokenizer).has_system_role passes on all three, so it would have approved the switch to Gemma 3 without a warning.

grid-rows: 2
grid-columns: 3
grid-gap: 12
o: "Own turn\nQwen3, Llama 3" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
f: "Folded\nGemma 3" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
d: "Dropped\nno family here" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
ot: "has_system_role = True" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
ft: "has_system_role = True" {style: {fill: "#FFFFFF"; stroke: "#B03A2E"; font-color: "#2C2C2A"}}
dt: "has_system_role = False" {style: {fill: "#FFFFFF"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}

The question has three answers and the flag has two values. Folding keeps the text, so "folded" and "own turn" both print True, and only "dropped" prints False. The next step asks the question that separates the first two.

declares_tools is only a substring test: it would also be True for a template that mentions tools in a comment. Read it as a hint until Part 5 probes tool rendering for real.

Step 5: Tell a system message's own turn from a folded one with a turn boundary

Goal: report own_turn, folded or dropped for the system message.

Why this step: position alone cannot separate the three families, because on all three the system text comes first. What separates them is a turn boundary in the text between the two markers. In the step 2 renders, Qwen3 has <|im_end|> and <|im_start|> between SYS_MARKER and USER_MARKER, Llama 3 has <|eot_id|> and a header, and Gemma 3 has only a blank line.

Add the boundary pattern and two functions at the end of templatestudio/anatomy.py. classify_marker works on any marker in any render that also contains USER_MARKER, so steps 7 and 8 can reuse it. system_handling applies it to the probe:

# templatestudio/anatomy.py (add at the end)# Default turn markers: those of Qwen3, Llama 3 and Gemma 3. Two markers with one# of these between them sit in different turns.TURN_BOUNDARY = re.compile(    r"<\|im_start\|>|<\|im_end\|>"    r"|<\|start_header_id\|>|<\|end_header_id\|>|<\|eot_id\|>"    r"|<start_of_turn>|<end_of_turn>")def classify_marker(    rendered: str,    marker: str,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Say where `marker` landed: 'own_turn', 'folded' or 'dropped'."""    marker_at = rendered.find(marker)    user_at = rendered.find("USER_MARKER")    if marker_at == -1:        return "dropped"    if user_at == -1:        return "own_turn"    lo, hi = sorted((marker_at, user_at))    if boundary.search(rendered[lo:hi]):        return "own_turn"    return "folded"def system_handling(    tokenizer: PreTrainedTokenizerBase,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Report whether a system message gets its own turn, is folded, or dropped."""    rendered = render(tokenizer, PROBE, add_generation_prompt=False)    return classify_marker(rendered, "SYS_MARKER", boundary)

Create show_system.py:

# show_system.pyfrom families import FAMILIESfrom templatestudio.anatomy import system_handlingfrom templatestudio.render import load_tokenizerfor name, model_id in FAMILIES.items():    print(f"{name:<7} {system_handling(load_tokenizer(model_id))}")

Run it:

python show_system.py

Expected output:

qwen3   own_turnllama3  own_turngemma3  folded

What just happened: classify_marker reads only the text between the two markers, and the diagram shows that text for each family.

grid-rows: 3
grid-columns: 2
grid-gap: 10
q: "qwen3: SYS_MARKER<|im_end|>\\n<|im_start|>user\\nUSER_MARKER" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
qv: "boundary found -> own_turn" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
l: "llama3: SYS_MARKER<|eot_id|><|start_header_id|>user ... USER_MARKER" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
lv: "boundary found -> own_turn" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
g: "gemma3: SYS_MARKER\\n\\nUSER_MARKER" {style: {fill: "#FFFFFF"; stroke: "#B03A2E"; font-color: "#2C2C2A"}}
gv: "no boundary -> folded" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}

This is the check to run before a model swap. folded is not a failure on its own, but it means your system prompt now competes with user text from inside the user's turn. You have to decide what to do about that: accept it, move the policy into your own first user message on purpose, or pick a different model.

Two branches need a word each. If USER_MARKER itself is missing, the template dropped the user message, so there is no user turn for the marker to be folded into, and classify_marker reports own_turn. If the probed marker is missing, the answer is dropped, the third outcome. None of these families drops a system message, but a template that skips roles it does not handle would remove SYS_MARKER entirely, and find would return -1.

Step 6: Find a new model family's turn markers and fix a wrong "folded" result

Goal: see the probe misreport Gemma 4's system message, and fix it by passing Gemma 4's own turn markers.

Why this step: step 3's probe works on any template, but system_handling depends on TURN_BOUNDARY, a list of markers from three families. A family with other markers breaks it. Gemma 4, which Google released on 2026-04-02, is the example. It has a real system turn, written with markers called <|turn> and <turn|> that the default list does not contain. With no known marker between SYS_MARKER and USER_MARKER, the probe concludes that they share a turn.

Add one helper at the end of templatestudio/anatomy.py. It builds a boundary pattern from the markers you pass and nothing else, escaped so that | and < match literally:

# templatestudio/anatomy.py (add at the end)def turn_boundary(*markers: str) -> re.Pattern[str]:    """Build a boundary pattern from one family's own turn markers, matched literally."""    return re.compile("|".join(re.escape(marker) for marker in markers))

Create show_unknown_markers.py:

# show_unknown_markers.pyimport sysfrom families import GEMMA4from templatestudio.anatomy import PROBE, system_handling, turn_boundaryfrom templatestudio.render import load_tokenizer, render, visible_whitespacesys.stdout.reconfigure(encoding="utf-8")  # visible_whitespace prints non-ASCIIgemma4 = load_tokenizer(GEMMA4)print(visible_whitespace(render(gemma4, PROBE, add_generation_prompt=False)))print("default boundary:", system_handling(gemma4))gemma4_markers = turn_boundary("<|turn>", "<turn|>")print("gemma4 markers:  ", system_handling(gemma4, gemma4_markers))

Run it:

python show_unknown_markers.py

Expected output:

<bos><|turn>system\nSYS_MARKER<turn|>\n<|turn>user\nUSER_MARKER<turn|>\n<|turn>model\nASST_MARKER<turn|>\ndefault boundary: foldedgemma4 markers:   own_turn

What just happened: the render shows the default answer is wrong. <turn|> closes the system turn and <|turn>user opens the next one, so Gemma 4 gives the system message its own turn. With Gemma 4's markers the probe agrees.

grid-rows: 2
grid-columns: 2
grid-gap: 10
t1: "default boundary: SYS_MARKER<turn|>\\n<|turn>user\\nUSER_MARKER\nno known marker" {style: {fill: "#FFFFFF"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
v1: "folded (false alarm)" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
t2: "turn_boundary('<|turn>', '<turn|>'): the same text\n<turn|> and <|turn> found" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
v2: "own_turn (correct)" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}

This error is a false alarm: you investigate and find the system turn is fine. To find a new family's markers, read them off two things you already have. Run step 3's generation_prompt on Gemma 4's tokenizer and it returns '<|turn>model\n', which starts with the turn-opening marker. The render above shows the marker that closes each turn, <turn|>.

Step 7: Catch a false "own turn" when a system prompt quotes another family's marker

Goal: see the default boundary report own_turn on a model that folds, and fix it with that family's own markers.

Why this step: the opposite failure is worse, because it reports a safety you do not have. A folded system prompt can contain one of the default markers as plain text, for example an instruction about output format, or a prompt copied from a ChatML-based model that mentions <|im_start|>. The marker then sits between SYS_MARKER and USER_MARKER, and the default boundary takes it for a turn break. You need no new function: classify_marker from step 5 takes any rendered string, and turn_boundary from step 6 builds the fix.

Create show_quoted_marker.py:

# show_quoted_marker.pyimport sysfrom families import FAMILIESfrom templatestudio.anatomy import classify_marker, turn_boundaryfrom templatestudio.render import load_tokenizer, render, visible_whitespacesys.stdout.reconfigure(encoding="utf-8")  # visible_whitespace prints non-ASCIIgemma3 = load_tokenizer(FAMILIES["gemma3"])quoting = [    {"role": "system", "content": "SYS_MARKER Never print <|im_start|>."},    {"role": "user", "content": "USER_MARKER"},]rendered = render(gemma3, quoting, add_generation_prompt=False)print(visible_whitespace(rendered))print("default boundary:", classify_marker(rendered, "SYS_MARKER"))gemma3_markers = turn_boundary("<start_of_turn>", "<end_of_turn>")print("gemma3 markers:  ", classify_marker(rendered, "SYS_MARKER", gemma3_markers))

Run it:

python show_quoted_marker.py

Expected output:

<bos><start_of_turn>user\nSYS_MARKER·Never·print·<|im_start|>.\n\nUSER_MARKER<end_of_turn>\ndefault boundary: own_turngemma3 markers:   folded

What just happened: in the render, · marks a space, as visible_whitespace showed in Part 1. The default boundary says own_turn for a system message that is plainly folded into the user turn. Gemma 3's own markers give the right answer, folded, because Gemma 3's template never writes <|im_start|>, so a pattern built from Gemma 3's markers treats it as ordinary text.

grid-rows: 1
grid-gap: 30
a: "Default boundary (three families)" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 10
  a1: "matches <|im_start|> in the system text" {style: {fill: "#FFFFFF"; stroke: "#B03A2E"; font-color: "#2C2C2A"}}
  a2: "own_turn (wrong, and looks safe)" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
}
b: "Gemma 3 markers only" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 10
  b1: "<|im_start|> is ordinary text" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
  b2: "folded (correct)" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
}

The rule that comes out of steps 6 and 7: probe a model with its own family's markers only. TURN_BOUNDARY stays the default because it is convenient for a quick look across three families, but the check you rely on before a model swap should pass turn_boundary(...) with the deployed family's markers. Step 11 does that for every row.

Step 8: Check how a chat template handles a developer role message

Goal: ask the same question as step 5 for any role, and add a fourth answer for templates that raise.

Why this step: harnesses (the agent frameworks and SDKs that build the message list for you) now send developer messages, and the input contract from Part 2 accepts the role. Accepting it says nothing about what each template does with it. Step 1 showed that Llama 3 turns any role string into a header, and Part 2 found that Qwen3 drops roles it does not know. A third behaviour is possible too: a template can call raise_exception, which throws jinja2.exceptions.TemplateError. The probe has to catch that and report it as an answer.

Add role_handling at the end of templatestudio/anatomy.py. It catches TemplateError, which step 3 imported. It builds a two-message probe with the role first and reuses classify_marker. Part 2's Message type in messages.py types role as a fixed list of names, and here it must accept any string, so that line tells the type checker to allow it:

# templatestudio/anatomy.py (add at the end)def role_handling(    tokenizer: PreTrainedTokenizerBase,    role: str,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Probe any role: 'own_turn', 'folded', 'dropped', or 'rejected' on a raise."""    probe: Conversation = [        {"role": role, "content": "ROLE_MARKER"},  # type: ignore[typeddict-item]        {"role": "user", "content": "USER_MARKER"},    ]    try:        rendered = render(tokenizer, probe, add_generation_prompt=False)    except TemplateError:        return "rejected"    return classify_marker(rendered, "ROLE_MARKER", boundary)

Create show_roles.py:

# show_roles.pyfrom families import FAMILIESfrom templatestudio.anatomy import role_handlingfrom templatestudio.render import load_tokenizerfor name, model_id in FAMILIES.items():    tokenizer = load_tokenizer(model_id)    print(f"{name:<7} developer: {role_handling(tokenizer, 'developer')}")

Run it:

python show_roles.py

Expected output:

qwen3   developer: droppedllama3  developer: own_turngemma3  developer: rejected

What just happened: each family gives a different answer for the same role. The diagram shows the order in which role_handling decides.

grid-rows: 3
grid-columns: 2
grid-gap: 22
q1: "1. Did render raise TemplateError?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
a1: "yes: rejected (gemma3)" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
q2: "2. Is ROLE_MARKER missing?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
a2: "yes: dropped (qwen3)" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
q3: "3. Boundary before USER_MARKER?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
a3: "yes: own_turn (llama3)\nno: folded" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
q1 -> q2
q2 -> q3

Each answer means something different for your application:

• Qwen3 drops it with no error. A developer instruction sent to Qwen3 never reaches the model.
• Llama 3 gives it its own turn with a developer header. That is a header the model was not trained on, so the probe proves only that the text arrives in a separate turn. Whether the model gives that turn any authority is a property of the weights, which a template probe cannot read.
• Gemma 3 rejects it with Conversation roles must alternate user/assistant/user/assistant/.... The next step checks what that error is really about.

Step 9: Check whether a template rejects a role or a message position

Goal: find out whether Gemma 3 rejects the developer role itself, or only a developer message in first place.

Why this step: role_handling always puts the role first, because that is where harnesses put system and developer messages. But a template that raises can be enforcing an order rule rather than a list of allowed roles, and then the answer changes with position. Gemma 3's error message is the hint: it complains about alternation and says nothing about which roles are allowed. Two renders that differ only in where the developer message sits settle it.

Create show_position.py. It catches TemplateError itself, so both renders print:

# show_position.pyfrom jinja2.exceptions import TemplateErrorfrom families import FAMILIESfrom templatestudio.render import load_tokenizer, rendergemma3 = load_tokenizer(FAMILIES["gemma3"])first = [{"role": "developer", "content": "D"}, {"role": "user", "content": "U"}]second = [{"role": "user", "content": "U"}, {"role": "developer", "content": "D"}]for label, conversation in [("first", first), ("second", second)]:    print(f"developer {label}:")    try:        print(repr(render(gemma3, conversation, add_generation_prompt=False)))    except TemplateError as error:        print(f"TemplateError: {error}")

Run it:

python show_position.py

Expected output:

developer first:TemplateError: Conversation roles must alternate user/assistant/user/assistant/...developer second:'<bos><start_of_turn>user\nU<end_of_turn>\n<start_of_turn>developer\nD<end_of_turn>\n'

What just happened: the same developer message is rejected in first place and rendered as its own developer turn in second place.

grid-rows: 2
grid-columns: 2
grid-gap: 12
f: "developer first, user second" {style: {fill: "#FFFFFF"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
fv: "TemplateError: roles must alternate" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
s: "user first, developer second" {style: {fill: "#FFFFFF"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
sv: "<start_of_turn>developer turn" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}

Gemma 3's template checks positions: after an optional leading system message, every even position must be user and every odd position must not be. The two probes show the effect; the template source, which you can print the way step 1 printed Llama 3's, contains the check itself. A developer message at position 0 breaks that rule, and at position 1 it passes. rejected from role_handling therefore means "rejected in first place". For a template that raises, probe the position you will actually send.

Step 10: Probe add_generation_prompt and continue_final_message for a prefilled assistant turn

Goal: add a render that leaves the last assistant turn open, then compare it with the render that closes it.

Why this step: the generation prompt has a second mode, called prefill. You write the start of the answer yourself, for example an opening [ so that the model answers with a JSON list, and the model continues that same turn. Transformers supports it with continue_final_message=True: the last message is rendered without its closing marker, and no generation prompt follows.

If you prefill but leave add_generation_prompt=True, the template closes your turn and opens a new one. Nothing raises. The model just starts a fresh answer instead of finishing yours, and a differential probe shows exactly what that mistake adds.

Part 2 made render.py the only module that calls apply_chat_template, so the new render goes there. The step has two parts: first the render, then the probe that uses it. Add the render at the end of templatestudio/render.py. It needs no new imports, because Part 2's render.py already imports PreTrainedTokenizerBase and Conversation:

# templatestudio/render.py (add at the end)def render_continuation(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,) -> str:    """Render with the final assistant turn left open, so the model continues it."""    if not conversation or conversation[-1]["role"] != "assistant":        raise ValueError("render_continuation needs an assistant message last")    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=False,        continue_final_message=True,    )

Replace the last import line of templatestudio/anatomy.py, so it imports both renders. The other imports stay as step 3 left them:

# templatestudio/anatomy.py (replace the line `from templatestudio.render import render`)from templatestudio.render import render, render_continuation

Add a prefilled conversation and the probe at the end of templatestudio/anatomy.py:

# templatestudio/anatomy.py (add at the end)PREFILL: Conversation = [    {"role": "user", "content": "USER_MARKER"},    {"role": "assistant", "content": "PREFILL_MARKER"},]def prefill_closer(tokenizer: PreTrainedTokenizerBase) -> str:    """Return what add_generation_prompt=True adds after a prefilled turn."""    open_turn = render_continuation(tokenizer, PREFILL)    closed = render(tokenizer, PREFILL, add_generation_prompt=True)    return suffix_diff(open_turn, closed)

Create show_prefill.py:

# show_prefill.pyfrom families import FAMILIESfrom templatestudio.anatomy import PREFILL, prefill_closerfrom templatestudio.render import load_tokenizer, render_continuationfor name, model_id in FAMILIES.items():    tokenizer = load_tokenizer(model_id)    open_turn = render_continuation(tokenizer, PREFILL)    still_open = open_turn.endswith("PREFILL_MARKER")    print(f"{name:<7} open: {still_open!s:<5} closer: {prefill_closer(tokenizer)!r}")

Run it:

python show_prefill.py

Expected output:

qwen3   open: True  closer: '<|im_end|>\n<|im_start|>assistant\n'llama3  open: True  closer: '<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'gemma3  open: True  closer: '<end_of_turn>\n<start_of_turn>model\n'

What just happened: open: True means render_continuation ends exactly on your prefill text on all three families, with no closing marker after it. The closer column is what the mistake adds: each family's end-of-turn marker, then the generation prompt from step 3.

direction: down
open: "render_continuation\n... <|im_start|>assistant\\n ... PREFILL_MARKER" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
closed: "render(add_generation_prompt=True)\n... PREFILL_MARKER<|im_end|>\\n<|im_start|>assistant\\n" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
closer: "prefill_closer (Qwen3)\nend-of-turn marker + generation prompt" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
open -> closer: "1. shared prefix"
closed -> closer: "2. what is left"

The model continues from the last token it sees. After the open render that is PREFILL_MARKER, so it continues your text. After the closed render it is a brand-new assistant header, so it starts a fresh answer that may ignore your prefill completely.

render_continuation has two guards:

• Passing continue_final_message=True and add_generation_prompt=True together raises a ValueError in Transformers before anything renders. render_continuation sets the second flag to False, so you cannot hit it through this module.
• render_continuation also refuses a conversation whose last message is not an assistant message. Transformers would render one without complaint and leave a user turn open, and the model would then continue the user's text instead of yours.

The flag also accepts a field name instead of True, which prefills a reasoning field instead of content, as the Transformers v5.22.1 chat templating docs describe. That form belongs to reasoning templates, and Part 6 uses it.

Step 11: Build a model-swap checklist table for four model families

Goal: print one table that answers the four model-swap questions for every family, Gemma 4 included.

Why this step: each probe so far answered one question for three families. Before a model swap you want all the answers side by side, for the model you run now and the one you are moving to. Separate assertions fail one at a time, while a table changes as one diff you can review after each upgrade. Gemma 4 joins here, and every family is classified with its own markers, as steps 6 and 7 recommend.

Create compare_families.py:

# compare_families.pyfrom families import FAMILIES, GEMMA4from templatestudio.anatomy import (    describe_template,    role_handling,    system_handling,    turn_boundary,)from templatestudio.render import load_tokenizerMODELS = {**FAMILIES, "gemma4": GEMMA4}# Each family is classified with its own turn markers only (steps 6 and 7).BOUNDARIES = {    "qwen3": turn_boundary("<|im_start|>", "<|im_end|>"),    "llama3": turn_boundary("<|start_header_id|>", "<|end_header_id|>", "<|eot_id|>"),    "gemma3": turn_boundary("<start_of_turn>", "<end_of_turn>"),    "gemma4": turn_boundary("<|turn>", "<turn|>"),}print(f"{'family':<7} {'system':<9} {'developer':<10} {'tools':<6} generation prompt")for name, model_id in MODELS.items():    tokenizer = load_tokenizer(model_id)    boundary = BOUNDARIES[name]    report = describe_template(tokenizer)    system = system_handling(tokenizer, boundary)    developer = role_handling(tokenizer, "developer", boundary)    tools = str(report.declares_tools)    prompt = repr(report.generation_prompt)    print(f"{name:<7} {system:<9} {developer:<10} {tools:<6} {prompt}")

Run it:

python compare_families.py

Expected output:

family  system    developer  tools  generation promptqwen3   own_turn  dropped    True   '<|im_start|>assistant\n'llama3  own_turn  own_turn   False  '<|start_header_id|>assistant<|end_header_id|>\n\n'gemma3  folded    rejected   False  '<start_of_turn>model\n'gemma4  own_turn  own_turn   True   '<|turn>model\n'

What just happened: this is the table from the top of the page, and every cell came from a render. Gemma 4's row also shows how much a family can change between versions. Gemma 3 folds the system message and rejects developer in first place. Gemma 4 gives the system message its own turn and maps a leading developer message into that system turn. The Gemma 2 template rejected a leading system message outright, with System role not supported. Pin the template with the model version, and rerun this table on every upgrade.

Fix "Conversation roles must alternate" and other chat template probe errors

Chat template probe architecture: anatomy.py, render.py and apply_chat_template

The diagram shows the module you built, with the real names from your code. Every probe goes through render.py, and render.py is still the only module that calls apply_chat_template. Each probe changes one input, either the conversation or one argument, and reads one of the four parts off the difference.

direction: down
scripts: "Scripts" {
  label.near: top-center
  grid-rows: 1
  grid-gap: 16
  fam: "families.py\nFAMILIES" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
  cmp: "compare_families.py\nthe model-swap table" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
}
anatomy: "templatestudio/anatomy.py" {
  label.near: top-center
  grid-rows: 2
  grid-columns: 3
  grid-gap: 12
  gp: "generation_prompt\nsuffix_diff (step 3)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
  dt: "describe_template\nTemplateReport (step 4)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
  sh: "system_handling\nclassify_marker (step 5)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
  bw: "TURN_BOUNDARY\nturn_boundary (steps 6-7)" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  rh: "role_handling (steps 8-9)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
  pc: "prefill_closer\nPREFILL (step 10)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
}
render: "templatestudio/render.py\nrender · render_continuation · load_tokenizer" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
hf: "tokenizer.apply_chat_template\nthe model's own template" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
scripts -> anatomy: "1. tokenizer + boundary"
anatomy -> render: "2. PROBE / PREFILL, two settings"
render -> hf: "3. tokenize=False"

TURN_BOUNDARY and the patterns you build with turn_boundary are the only parts of the module that know any family's markers, which is why they are the part you choose per family. No box reads the template's source except describe_template, for the tools hint. Everything else comes from renders, so the same module works on a template you have never opened.

Complete code: a chat template probe in Python

The first line of each step's code block, such as # templatestudio/anatomy.py, is a location label. The listings below leave it out, and the file works with or without it.

The project after this part, with the files it adds or changes marked. The files from Parts 1 and 3 are listed for readers who kept the whole project; this part does not need them:

chat-templates-part1/├── .venv/├── families.py               (new)├── fetch_models.py           (new)├── show_llama3_source.py     (new)├── show_skeleton.py          (new)├── show_gen_prompt.py        (new)├── show_report.py            (new)├── show_system.py            (new)├── show_unknown_markers.py   (new)├── show_quoted_marker.py     (new)├── show_roles.py             (new)├── show_position.py          (new)├── show_prefill.py           (new)├── compare_families.py       (new)├── naive.py                  (unchanged, Part 1)├── show_wire.py ... no_gen_prompt.py   (unchanged, Part 1's scripts)├── drift.py, tool_turns.py   (unchanged, Part 2)├── show_drop.py ... show_safe_render.py (unchanged, Part 2's scripts)├── readable_chatml.jinja     (unchanged, Part 3)├── order_tools.py, reasoning_turns.py   (unchanged, Part 3)├── show_vanilla.py ... prove_faithful.py (unchanged, Part 3's ten scripts)└── templatestudio/    ├── __init__.py    ├── messages.py           (unchanged, from Part 2)    ├── render.py             (render_continuation added)    ├── jinja_env.py          (unchanged, from Part 3)    └── anatomy.py            (new)

templatestudio/anatomy.py in full:

from __future__ import annotationsimport refrom dataclasses import dataclassfrom jinja2.exceptions import TemplateErrorfrom transformers import PreTrainedTokenizerBasefrom templatestudio.messages import Conversationfrom templatestudio.render import render, render_continuation# Each marker is a string no real template contains, so find() locates it exactly.PROBE: Conversation = [    {"role": "system", "content": "SYS_MARKER"},    {"role": "user", "content": "USER_MARKER"},    {"role": "assistant", "content": "ASST_MARKER"},]def suffix_diff(shorter: str, longer: str) -> str:    """Return the tail of `longer` after the prefix it shares with `shorter`."""    i = 0    while i < len(shorter) and i < len(longer) and shorter[i] == longer[i]:        i += 1    return longer[i:]def generation_prompt(tokenizer: PreTrainedTokenizerBase) -> str:    """Return exactly what add_generation_prompt=True appends to the probe."""    without = render(tokenizer, PROBE, add_generation_prompt=False)    with_prompt = render(tokenizer, PROBE, add_generation_prompt=True)    return suffix_diff(without, with_prompt)@dataclassclass TemplateReport:    has_system_role: bool  # does the system text appear in the render at all?    generation_prompt: str  # exactly what add_generation_prompt appends    declares_tools: bool  # does the word "tools" appear in the template source?    sample_render: str  # the probe render, for reading the skeleton by eyedef describe_template(tokenizer: PreTrainedTokenizerBase) -> TemplateReport:    """Report a template's behaviour by rendering the probe, not by reading it."""    rendered = render(tokenizer, PROBE, add_generation_prompt=True)    source = tokenizer.chat_template or ""    return TemplateReport(        has_system_role="SYS_MARKER" in rendered,        generation_prompt=generation_prompt(tokenizer),        declares_tools="tools" in source,        sample_render=rendered,    )# Default turn markers: those of Qwen3, Llama 3 and Gemma 3. Two markers with one# of these between them sit in different turns.TURN_BOUNDARY = re.compile(    r"<\|im_start\|>|<\|im_end\|>"    r"|<\|start_header_id\|>|<\|end_header_id\|>|<\|eot_id\|>"    r"|<start_of_turn>|<end_of_turn>")def classify_marker(    rendered: str,    marker: str,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Say where `marker` landed: 'own_turn', 'folded' or 'dropped'."""    marker_at = rendered.find(marker)    user_at = rendered.find("USER_MARKER")    if marker_at == -1:        return "dropped"    if user_at == -1:        return "own_turn"    lo, hi = sorted((marker_at, user_at))    if boundary.search(rendered[lo:hi]):        return "own_turn"    return "folded"def system_handling(    tokenizer: PreTrainedTokenizerBase,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Report whether a system message gets its own turn, is folded, or dropped."""    rendered = render(tokenizer, PROBE, add_generation_prompt=False)    return classify_marker(rendered, "SYS_MARKER", boundary)def turn_boundary(*markers: str) -> re.Pattern[str]:    """Build a boundary pattern from one family's own turn markers, matched literally."""    return re.compile("|".join(re.escape(marker) for marker in markers))def role_handling(    tokenizer: PreTrainedTokenizerBase,    role: str,    boundary: re.Pattern[str] = TURN_BOUNDARY,) -> str:    """Probe any role: 'own_turn', 'folded', 'dropped', or 'rejected' on a raise."""    probe: Conversation = [        {"role": role, "content": "ROLE_MARKER"},  # type: ignore[typeddict-item]        {"role": "user", "content": "USER_MARKER"},    ]    try:        rendered = render(tokenizer, probe, add_generation_prompt=False)    except TemplateError:        return "rejected"    return classify_marker(rendered, "ROLE_MARKER", boundary)PREFILL: Conversation = [    {"role": "user", "content": "USER_MARKER"},    {"role": "assistant", "content": "PREFILL_MARKER"},]def prefill_closer(tokenizer: PreTrainedTokenizerBase) -> str:    """Return what add_generation_prompt=True adds after a prefilled turn."""    open_turn = render_continuation(tokenizer, PREFILL)    closed = render(tokenizer, PREFILL, add_generation_prompt=True)    return suffix_diff(open_turn, closed)

templatestudio/render.py in full, with Part 2's functions unchanged and render_continuation at the end:

from __future__ import annotationsfrom transformers import AutoTokenizer, PreTrainedTokenizerBasefrom templatestudio.messages import Conversation, assert_validdef load_tokenizer(model_id: str) -> PreTrainedTokenizerBase:    """Load a tokenizer together with the chat template it ships with."""    return AutoTokenizer.from_pretrained(model_id)def render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> str:    """Render a conversation into the model's wire format, using its own template."""    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=add_generation_prompt,    )def visible_whitespace(text: str) -> str:    """Show spaces, tabs and newlines so two renders can be compared by eye."""    return (        text.replace("\t", "[TAB]")        .replace(" ", "·")        .replace("\n", "\\n\n")    )def first_divergence(a: str | list[int], b: str | list[int]) -> int | None:    """Return the first index where a and b differ, or None if they are equal.    Works on strings (characters) and on lists of token IDs alike.    """    for index, (item_a, item_b) in enumerate(zip(a, b)):        if item_a != item_b:            return index    if len(a) != len(b):        return min(len(a), len(b))    return Nonedef render_ids(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> list[int]:    """Render a conversation straight to the token IDs the model receives."""    encoded = tokenizer.apply_chat_template(        conversation,        tokenize=True,        add_generation_prompt=add_generation_prompt,    )    return encoded["input_ids"]def safe_render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> str:    """Check the input contract, then render. Use this in production."""    assert_valid(conversation)    return render(        tokenizer,        conversation,        add_generation_prompt=add_generation_prompt,    )def render_continuation(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,) -> str:    """Render with the final assistant turn left open, so the model continues it."""    if not conversation or conversation[-1]["role"] != "assistant":        raise ValueError("render_continuation needs an assistant message last")    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=False,        continue_final_message=True,    )

families.py, fetch_models.py and the eleven step scripts, one per step, are exactly as shown in the Prerequisites and in steps 1 to 11; each is complete as printed there. For the earlier parts' files, see Part 3's complete code.

Next steps: turn the probe into a model-swap gate

• Turn the table into a CI gate. Store compare_families.py's output for the models you serve, rerun it on every dependency or model upgrade, and fail the build when a row changes. A template that changes under you becomes a reviewable diff instead of a silent behaviour change. Part 11 grows this into golden-token tests.
• Probe the assistant role's wire name. Add a column that finds the last turn-opening marker before ASST_MARKER in sample_render and reads the role name written after it, so the table shows assistant for Qwen3 and Llama 3 and model for Gemma. Do not just take the text right before ASST_MARKER: on Qwen3 that is the empty <think> block. Any code that matches role names in rendered output needs that value.
• Decide what folded means for you. Write the model-swap assertion against system_handling, not has_system_role, and choose per model whether own_turn to folded fails the deploy or moves your policy into the first user message on purpose.

This tutorial is the runnable slice of chapter 4 of The Chat Templates Handbook, which adds a deeper reading of the three templates and why role names are token sequences the model learned, not labels it interprets. The chapter's code is the v1.2/ch04-anatomy tag of Template Studio. This tutorial differs from the tag in four ways:

• it makes PROBE, suffix_diff and TURN_BOUNDARY public;
• it splits the shared decision into classify_marker;
• it adds turn_boundary, render_continuation and prefill_closer;
• it runs on real tokenizers, where the tag's tests use hand-written stand-in templates.

Part 5 takes the message loop further with tool calling: rendering the tools variable into a model's expected format and reading tool calls back out.

References

• Hugging Face. Chat templates, Transformers v5.22.1 documentation, including continue_final_message.
• Hugging Face. apply_chat_template source and render_jinja_template source, Transformers v5.22.1.
• Hugging Face. Transformers v5.22.1 release, the pinned version, and the v5.22.2 release, noted but not adopted.
• Hugging Face. Hub model API, used to compare the ungated templates with the official ones without a token, and the hf auth login command.
• PyPI. tokenizers 0.23.3, huggingface-hub 2.4.0, transformers 5.22.1 and Jinja2 3.1.6.
• Qwen. Qwen/Qwen3-0.6B model card.
• Meta. meta-llama/Meta-Llama-3-8B-Instruct (gated) and the ungated copy NousResearch/Meta-Llama-3-8B-Instruct.
• Meta. Meta Llama 3 prompt format.
• Google. google/gemma-3-1b-it (gated) and the ungated copy unsloth/gemma-3-1b-it.
• Google. google/gemma-4-E2B-it (Apache 2.0) and google/gemma-2-2b-it model cards. Google. Gemma 4: Byte for byte, the most capable open models, 2026-04-02.
• Meta. Meta Llama 3 Community License.
• Google. Gemma formatting and system instructions and Gemma Terms of Use.
• Python. re.escape and typing.NotRequired, Python 3.12.
• Pallets. Jinja TemplateError, Jinja 3.1.
• Kumar, R. The Chat Templates Handbook, Edition 1.2, chapter 4. Book site: chattemplates.ranjankumar.in.
• Kumar, R. Chat Templates: The Last Unowned Layer in Your LLM Stack.
• Series: Part 1, Part 2, Part 3.

Related Articles

Llms

• How to Validate Chat Messages Before apply_chat_template
• How to Render a Chat Template with Jinja Like Transformers
• How to See What apply_chat_template Sends Your Model
• How to Pass Tools to apply_chat_template and Parse the Calls