How to See What apply_chat_template Sends Your Model

What you'll build

You will build a small Python module around apply_chat_template and Qwen3's own chat template. It renders a conversation the way the model was trained to read it. It shows every newline and space in that string. And it points to the first character where a hand-built prompt stops matching it. Here is what it prints when it compares a prompt typed by hand (by hand) with the model's own render (template). Each line shows the end of one string, and <END> marks where that string stops:

lengths: 291 by hand, 292 templatefirst divergence at character 291by hand:  \n<|im_start|>assistant<END>template: \n<|im_start|>assistant\n<END>

By the end you will be able to take a chat prompt your code builds by hand and find where it goes wrong. You will get the exact character and the exact token where it stops matching the model's own template render of the same conversation. The tool is templatestudio/render.py, a module of five small functions you write yourself.

This is an intermediate tutorial. You should be comfortable with Python and virtual environments, and you should have loaded something from the Hugging Face Hub before. It takes about 20 minutes, or about 30 with the optional model run.

The output above comes from a ChatML prompt written from memory. One newline is missing, at the very last character, and nothing raises. In the optional last step you run the model on three prompts. The f-string breaks the model's output badly. With this one-newline miss, the model emits one stray token and then recovers, on this run, with a different answer. That is why you catch the miss with a diff instead of waiting for the model to show it.

Verified against Python 3.12.2, transformers==5.22.1, tokenizers==0.23.3, huggingface-hub==2.4.0, jinja2==3.1.6, and torch==2.15.0+cpu for step 8, on 2026-11-04.

Prerequisites

You need Python 3.12 (this tutorial was verified on 3.12.2; Transformers 5.22.1 itself supports 3.11 to 3.14), and nothing that costs money. Steps 1 to 7 download only Qwen3-0.6B's tokenizer files, a few megabytes in total, and need no GPU, no API key, no account, and no PyTorch. Step 8 is optional: it downloads the model weights (about 1.5 GB) and runs them on CPU.

Check which Python python is before you create the environment:

python --version

It should print Python 3.12.x. If it prints another version, use python3.12 on Linux and macOS, or py -3.12 on Windows, in place of python in the next command. Then create a project folder and a virtual environment:

mkdir chat-templates-part1cd chat-templates-part1python -m venv .venvsource .venv/bin/activate

On Windows, activate with .venv\Scripts\activate in PowerShell or source .venv/Scripts/activate in Git Bash; those are the two Windows shells this tutorial covers. If PowerShell says that running scripts is disabled, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned once and activate again (the troubleshooting table at the end has the full error). Once the environment is active, python means the environment's Python. The commands below are bash; the two places where PowerShell differs are spelled out next to them.

Install exact versions:

pip install "transformers==5.22.1" "tokenizers==0.23.3" \    "huggingface-hub==2.4.0" "jinja2==3.1.6"

In PowerShell, put the install on one line instead, because PowerShell does not use \ to continue a line: pip install "transformers==5.22.1" "tokenizers==0.23.3" "huggingface-hub==2.4.0" "jinja2==3.1.6".

jinja2 is on that line on purpose, because apply_chat_template cannot run without it and Transformers 5.22.1 lists it only as an optional extra, transformers[chat-template]. Most people never notice this, since PyTorch installs jinja2 as one of its own dependencies. In a tokenizer-only environment like this one, nothing installs it for you.

Without PyTorch, every import transformers prints a one-line advisory that models are unavailable. That is true and expected, since steps 1 to 7 use only the tokenizer. Turn the advisory off with TRANSFORMERS_NO_ADVISORY_WARNINGS so your output matches the page. The setting lasts only for the current shell, so set it again in every new terminal you use for this tutorial:

export TRANSFORMERS_NO_ADVISORY_WARNINGS=1

In PowerShell, use $env:TRANSFORMERS_NO_ADVISORY_WARNINGS = "1" instead.

Check the install before you start:

python -c "import transformers as t, jinja2 as j; print(t.__version__, j.__version__)"

Expected output:

5.22.1 3.1.6

Three warnings can appear on stderr during this tutorial. None of them needs action:

• [transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. appears above your output if TRANSFORMERS_NO_ADVISORY_WARNINGS is not set in the current shell.
• Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. appears on the first download. You do not need a token, because Qwen3-0.6B is public and not gated.
• On Windows without Developer Mode, the first download prints a two-paragraph warning that starts `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in followed by your cache path. The cache still works; it only uses more disk, as the cache documentation explains.

Last, create the package folder the module will live in, with an empty __init__.py that marks it as a regular Python package. Every script in this tutorial runs from the project root, next to this folder:

mkdir templatestudiopython -c "open('templatestudio/__init__.py', 'w').close()"

How a list of messages becomes one token stream

Your code holds a conversation as a list of dictionaries, each with a role and a content. Call that the message model; it is the same list whether you run Qwen3, Llama, or Gemma. The model never receives that list. What it receives is one string, which becomes one sequence of token IDs in the exact shape the model saw during training. Call that the wire format. Every model family has its own.

The chat template is the program that turns the first into the second. It is a short Jinja program written by whoever trained the model, it ships with the tokenizer, and tokenizer.apply_chat_template runs it. The Chat Templates Handbook calls this relationship the render contract: your message list on one side, the model's training format on the other, and the template as the only thing that is allowed to translate between them.

direction: down
messages: "Message model\nlist of {role, content} dicts\nsame for every model" {
  style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}
}
template: "Chat template (Jinja)\nshipped with the tokenizer\nwritten by the model's trainer" {
  style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}
}
wire: "Wire format\none string, then one list of token IDs\ndifferent for every model" {
  style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}
}
fstring: "Hand-built f-string\nskips the template" {
  style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}
}
messages -> template: "1. apply_chat_template"
template -> wire: "2. rendered string -> token IDs"
messages -> fstring: "the shortcut"
fstring -> wire: "a string the model\nwas never trained on" {style: {stroke: "#E74C3C"; stroke-dash: 4}}

This tutorial is about the red path. The string it produces is valid, the model accepts it, and nothing raises an error. The model reads a format it was never trained on and does worse, while every log line looks healthy. The book names this the silent format penalty. A try/except cannot catch it. Only a comparison of strings can, and that comparison is what the module you build here does.

Step 1: Build a chat prompt by hand, two ways

Goal. Write the two prompts people build without a template: an f-string, and ChatML typed from memory.

Why this step. The correct format is easier to understand once you have two incorrect ones to compare it with, and these two fail in different ways. The f-string is the obvious way to turn a message list into text: role, colon, text, one line per turn. A person reads it easily, but nothing in it has a special meaning to the model. The second function writes ChatML, the format Qwen3 was trained on, where each turn is wrapped as <|im_start|>role, a newline, the content, and <|im_end|>. It is what an engineer writes after reading the ChatML format notes: the right markers, typed by hand. It is far closer to correct. That is what makes it more dangerous, because it passes code review. Every later step compares these two strings against the template's.

The conversation has four turns, so the template has to handle a system message, an earlier assistant reply, and a final user question. Create naive.py in the project root:

# naive.pyCONVERSATION = [    {        "role": "system",        "content": "You are a terse support agent. Answer in one sentence.",    },    {"role": "user", "content": "Where is my order #4471?"},    {"role": "assistant", "content": "It shipped Monday and arrives Thursday."},    {"role": "user", "content": "Can I change the delivery address?"},]def build_prompt_naive(messages: list[dict[str, str]]) -> str:    """The prompt most people write first. It runs, and it is wrong."""    lines = [f"{m['role']}: {m['content']}" for m in messages]    lines.append("assistant:")    return "\n".join(lines)def build_prompt_by_hand(messages: list[dict[str, str]]) -> str:    """ChatML written from memory. Close to right, which makes it worse."""    turns = [f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>" for m in messages]    turns.append("<|im_start|>assistant")    return "\n".join(turns)if __name__ == "__main__":    print(build_prompt_naive(CONVERSATION))

Running the file prints only the f-string prompt. The by-hand prompt is too close to the real format to judge by eye, so you will measure it in step 4.

Run it.

python naive.py

Expected output.

system: You are a terse support agent. Answer in one sentence.user: Where is my order #4471?assistant: It shipped Monday and arrives Thursday.user: Can I change the delivery address?assistant:

What just happened. naive.py now holds the three things every later script imports: CONVERSATION, with exactly the role and content keys a chat template reads, and the two hand-built prompts to test against it. To the model, system: in the f-string output is not a marker. It is the word "system" and a colon, two ordinary text tokens, which is exactly how step 5 displays them.

Step 2: Render the conversation with the model's own chat template

Goal. Write load_tokenizer and render, and print Qwen3's real wire format for the conversation.

Why this step. This is the render contract in code. You stop writing the format yourself and ask the model's tokenizer to run the template its trainer wrote. render overrides two defaults of apply_chat_template. First, it asks for a string instead of token IDs, so you can read the result. Second, it adds the generation prompt, which is the model's cue to start answering. Step 7 shows what that cue contains.

The template comes down with the tokenizer files. For Qwen3-0.6B it is a 4,168-character Jinja program inside tokenizer_config.json. Newer repositories use a separate chat_template.jinja file instead, shown dashed in the diagram. apply_chat_template runs the template in a sandboxed Jinja environment, which stops template code from reaching your files or arbitrary Python objects.

direction: down
config: "Qwen3-0.6B: tokenizer_config.json\nchat_template field\n(legacy location, used here)" {
  style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}
}
jinja: "chat_template.jinja\n(current location; wins if present;\nnot in this repo)" {
  style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#2C2C2A"; stroke-dash: 4}
}
tok: "AutoTokenizer.from_pretrained\ntokenizer.chat_template = Jinja source" {
  style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}
}
render: "apply_chat_template(\n  tokenize=False,\n  add_generation_prompt=True)" {
  style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}
}
out: "wire-format string" {
  style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}
}
config -> tok: "1. download"
jinja -> tok: "newer repos" {style: {stroke-dash: 4}}
tok -> render: "2. conversation in"
render -> out: "3. sandboxed Jinja render"

Create templatestudio/render.py inside the package folder you made at the end of Prerequisites:

# templatestudio/render.pyfrom __future__ import annotationsfrom transformers import AutoTokenizer, PreTrainedTokenizerBaseMessage = dict[str, str]Conversation = list[Message]def load_tokenizer(model_id: str) -> PreTrainedTokenizerBase:    """Load a tokenizer together with the chat template it ships with."""    return AutoTokenizer.from_pretrained(model_id)def render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> str:    """Render a conversation into the model's wire format, using its own template."""    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=add_generation_prompt,    )

Now create show_wire.py in the project root:

# show_wire.pyfrom naive import CONVERSATIONfrom templatestudio.render import load_tokenizer, rendertokenizer = load_tokenizer("Qwen/Qwen3-0.6B")wire_format = render(tokenizer, CONVERSATION)print(wire_format, end="")print("--- ends with a newline:", wire_format.endswith("\n"))

Run it. The first run downloads the tokenizer files.

python show_wire.py

Expected output.

<|im_start|>systemYou are a terse support agent. Answer in one sentence.<|im_end|><|im_start|>userWhere is my order #4471?<|im_end|><|im_start|>assistantIt shipped Monday and arrives Thursday.<|im_end|><|im_start|>userCan I change the delivery address?<|im_end|><|im_start|>assistant--- ends with a newline: True

What just happened. This is ChatML, the format Qwen3 was trained on. Every turn opens with <|im_start|> and the role on its own line, and closes with <|im_end|>. The final <|im_start|>assistant is the generation prompt, the cue that says the user is done and the model's turn begins. The last line of output reports something the lines above it cannot show: the string ends with a newline after assistant. The script printed the string with end="", so the only line break after assistant is the string's own newline. On screen it looks like an ordinary line end. The next step makes it visible.

Step 3: Show hidden whitespace and newlines in a chat prompt

Goal. Add visible_whitespace and print both prompts with every space and newline marked.

Why this step. Most template bugs are in characters you cannot see: a newline that is missing, doubled, or in the wrong place, or a space after a colon. In step 2, the most important character in the whole string was the final newline, and you could not see it. To compare two renders by eye, you need a view of the string in which whitespace is as visible as text.

The function maps each invisible character to a visible mark. A newline keeps its line break, so the output still reads as lines:

grid-rows: 4
grid-columns: 2
grid-gap: 10
h1: "character" {style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#FFFFFF"; bold: true}}
h2: "shown as" {style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#FFFFFF"; bold: true}}
sp: "space" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
spv: "·  (middle dot)" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
nl: "newline" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
nlv: "\\n  then a real line break" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
tb: "tab" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
tbv: "[TAB]" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}

Add this function to the end of templatestudio/render.py. The first line of this block, and of every later block marked "add at the end", is a comment that names the file; skip that line when you paste:

# templatestudio/render.py (add at the end)def visible_whitespace(text: str) -> str:    """Show spaces, tabs and newlines so two renders can be compared by eye."""    return (        text.replace("\t", "[TAB]")        .replace(" ", "·")        .replace("\n", "\\n\n")    )

Then create show_whitespace.py:

# show_whitespace.pyimport sysfrom naive import CONVERSATION, build_prompt_naivefrom templatestudio.render import load_tokenizer, render, visible_whitespacesys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")print("=== f-string ===")print(visible_whitespace(build_prompt_naive(CONVERSATION)))print("=== template ===")print(visible_whitespace(render(tokenizer, CONVERSATION)))

The sys.stdout.reconfigure line makes Python write UTF-8 even when your terminal or a pipe uses an older encoding, which is common on Windows. If the output encoding has no · character, as with plain ASCII, the first dot printed raises UnicodeEncodeError: 'ascii' codec can't encode character '\xb7'. Every script in this tutorial that prints non-ASCII characters carries the same line.

Run it.

python show_whitespace.py

Expected output.

=== f-string ===system:·You·are·a·terse·support·agent.·Answer·in·one·sentence.\nuser:·Where·is·my·order·#4471?\nassistant:·It·shipped·Monday·and·arrives·Thursday.\nuser:·Can·I·change·the·delivery·address?\nassistant:=== template ===<|im_start|>system\nYou·are·a·terse·support·agent.·Answer·in·one·sentence.<|im_end|>\n<|im_start|>user\nWhere·is·my·order·#4471?<|im_end|>\n<|im_start|>assistant\nIt·shipped·Monday·and·arrives·Thursday.<|im_end|>\n<|im_start|>user\nCan·I·change·the·delivery·address?<|im_end|>\n<|im_start|>assistant\n

What just happened. The final \n after <|im_start|>assistant is now on the page, and the blank line after it is the line break that newline produces. The structural difference between the two prompts is visible too. The f-string puts the role and the content on the same line, separated by a colon and a space. The template puts the role alone on the marker line, starts the content on the next line, and uses no space anywhere in the structure.

That difference reaches the model. The tokenizer folds a leading space into the word after it, so : You and \nYou become different tokens, and a character you cannot see on screen changes the integers the model receives (step 5 shows them). The f-string prompt differs from the wire format from its first character, so a character-level diff of it tells you nothing you do not already know. The next step uses a harder case.

Step 4: Find the first character where a hand-built ChatML prompt goes wrong

Goal. Find the exact character where step 1's hand-built ChatML prompt diverges from the template.

Why this step. You can see that the f-string is wrong just by reading it. build_prompt_by_hand is the dangerous one, because it is almost right. A real template bug is usually this small, one character in a string of hundreds, and finding it takes a tool rather than your eyes.

grid-rows: 1
grid-gap: 30
hand: "build_prompt_by_hand ends" {
  label.near: top-center
  style: {fill: "#FDEDEC"; stroke: "#E74C3C"; font-color: "#2C2C2A"}
  grid-columns: 3
  grid-gap: 6
  a1: "<|im_end|>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  a2: "\\n<|im_start|>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  a3: "assistant  <END>" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
}
tmpl: "template ends" {
  label.near: top-center
  style: {fill: "#EAFAF1"; stroke: "#3F9E52"; font-color: "#2C2C2A"}
  grid-columns: 3
  grid-gap: 6
  b1: "<|im_end|>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  b2: "\\n<|im_start|>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  b3: "assistant\\n  <END>" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
}

The two strings match for 291 characters, and the last cell in the diagram is the only place they differ. Add first_divergence to the end of templatestudio/render.py:

# templatestudio/render.py (add at the end)def first_divergence(a: str | list[int], b: str | list[int]) -> int | None:    """Return the first index where a and b differ, or None if they are equal.    Works on strings (characters) and on lists of token IDs alike.    """    for index, (item_a, item_b) in enumerate(zip(a, b)):        if item_a != item_b:            return index    if len(a) != len(b):        return min(len(a), len(b))    return None

The final if handles the case this step is about. When one sequence is a prefix of the other, zip stops at the end of the shorter one before any pair differs. The function only compares items, so it works on a list of token IDs as well as on a string, and step 6 uses it that way. Then create diff_prompts.py:

# diff_prompts.pyimport sysfrom naive import CONVERSATION, build_prompt_by_handfrom templatestudio.render import (    first_divergence,    load_tokenizer,    render,    visible_whitespace,)sys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")by_hand = build_prompt_by_hand(CONVERSATION)wire_format = render(tokenizer, CONVERSATION)index = first_divergence(by_hand, wire_format)print("lengths:", len(by_hand), "by hand,", len(wire_format), "template")if index is None:    print("identical: the prompt matches the template exactly")    raise SystemExitprint("first divergence at character", index)start = max(index - 22, 0)print("by hand: ", visible_whitespace(by_hand[start:]) + "<END>")print("template:", visible_whitespace(wire_format[start:]).rstrip("\n") + "<END>")

The script prints each string from 22 characters before the divergence point to its end, so you can see the context around the first difference. 22 is the length of \n<|im_start|>assistant, the generation prompt plus the newline before it, so the window covers exactly the tail this step is about. start never goes below 0, so a prompt that diverges within its first 22 characters prints from its beginning. <END> marks where each string stops.

The template's tail ends in a newline, so .rstrip("\n") removes the real line break after its visible \n. That keeps <END> on the same line. The by-hand tail has no final newline to remove.

If the two prompts are identical, first_divergence returns None. The script checks for that and stops, instead of slicing with None.

To check a prompt from your own code, replace build_prompt_by_hand(CONVERSATION) with the string your code builds, and change the model ID to the model you serve. CONVERSATION must stay the list of messages that string was built from. Step 6 repeats this comparison at the token level, and the same substitution works there. Qwen3 has no BOS (beginning-of-sequence) token, so the scripts here tokenize with the defaults. If your model has one, tokenize your string the way your server does, with a matching add_special_tokens= setting. Otherwise the token diff reports a BOS difference your server never produces.

Run it.

python diff_prompts.py

Expected output.

lengths: 291 by hand, 292 templatefirst divergence at character 291by hand:  \n<|im_start|>assistant<END>template: \n<|im_start|>assistant\n<END>

What just happened. The hand-built prompt is the template's output minus one character: the newline after the generation prompt's assistant. The cause is "\n".join, which puts newlines between items and never adds one after the last item. That is an easy mistake to make in Python, and neither a code review nor a log would show it to you. first_divergence found it at index 291 of 292.

Step 5: See the token IDs apply_chat_template produces

Goal. Add render_ids to the module and look at the tokens that make up all three prompts.

Why this step. A string is still only an approximation of the input, because the model receives integers. In Qwen3's tokenizer, <|im_start|> is a special token. It is twelve characters on the screen, but to the model it is a single ID, one the model learned to treat as a turn boundary. The word system followed by a colon is two ordinary text tokens with no special meaning at all. That difference exists only at the token level, so the module needs a second render function that returns IDs instead of a string. In the token names below, Ċ is a newline and Ġ is a leading space: that is how Qwen3's byte-level tokenizer writes them, not characters in your prompt.

grid-rows: 3
grid-columns: 3
grid-gap: 10
h1: "text" {style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#FFFFFF"; bold: true}}
h2: "tokens" {style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#FFFFFF"; bold: true}}
h3: "what the model sees" {style: {fill: "#95A5A6"; stroke: "#6C7A7B"; font-color: "#FFFFFF"; bold: true}}
t1: "<|im_start|>system\\n" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
k1: "<|im_start|>  system  Ċ\n(id 151644 first)" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
m1: "one learned boundary\nthen the role" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
t2: "system: You" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
k2: "system  :  ĠYou" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
m2: "plain text that\nstarts with a word" {style: {fill: "#FFA07A"; stroke: "#D9785A"; font-color: "#2C2C2A"}}

Add render_ids to the end of templatestudio/render.py:

# templatestudio/render.py (add at the end)def render_ids(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> list[int]:    """Render a conversation straight to the token IDs the model receives."""    encoded = tokenizer.apply_chat_template(        conversation,        tokenize=True,        add_generation_prompt=add_generation_prompt,    )    return encoded["input_ids"]

In Transformers 5, apply_chat_template(tokenize=True) returns a BatchEncoding, the same dictionary-like object a normal tokenizer call returns, so the IDs are under ["input_ids"]. Then create show_tokens.py:

# show_tokens.pyimport sysfrom naive import CONVERSATION, build_prompt_by_hand, build_prompt_naivefrom templatestudio.render import load_tokenizer, render_idssys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")template_ids = render_ids(tokenizer, CONVERSATION)naive_ids = tokenizer(build_prompt_naive(CONVERSATION))["input_ids"]hand_ids = tokenizer(build_prompt_by_hand(CONVERSATION))["input_ids"]marker = "<|im_start|>"print(marker, "is", len(marker), "characters and token id",      tokenizer.convert_tokens_to_ids(marker))print("template:", len(template_ids), "tokens")print("  first 4:", tokenizer.convert_ids_to_tokens(template_ids[:4]))print("  last 5: ", tokenizer.convert_ids_to_tokens(template_ids[-5:]))print("f-string:", len(naive_ids), "tokens")print("  first 4:", tokenizer.convert_ids_to_tokens(naive_ids[:4]))print("  last 5: ", tokenizer.convert_ids_to_tokens(naive_ids[-5:]))print("by hand: ", len(hand_ids), "tokens")print("  last 5: ", tokenizer.convert_ids_to_tokens(hand_ids[-5:]))

The two hand-built prompts are tokenized with a plain tokenizer(...) call, because they are strings you built yourself. Only the template's render goes through render_ids.

Run it.

python show_tokens.py

Expected output.

<|im_start|> is 12 characters and token id 151644template: 59 tokens  first 4: ['<|im_start|>', 'system', 'Ċ', 'You']  last 5:  ['<|im_end|>', 'Ċ', '<|im_start|>', 'assistant', 'Ċ']f-string: 46 tokens  first 4: ['system', ':', 'ĠYou', 'Ġare']  last 5:  ['Ġdelivery', 'Ġaddress', '?Ċ', 'assistant', ':']by hand:  58 tokens  last 5:  ['?', '<|im_end|>', 'Ċ', '<|im_start|>', 'assistant']

What just happened. The template's wire format starts with token 151644 and ends with assistant followed by a newline token.

The f-string has no special tokens at all. Look at '?Ċ' in its last five. Nothing marks the end of the turn, so the question mark and the newline merged into one ordinary token. The by-hand prompt is one token shorter than the template's. You now have all three prompts as ID lists, so the next step can compare two of them item by item and find exactly which token is missing.

Step 6: Find the first token where two prompts differ

Goal. Run first_divergence on token IDs instead of characters.

Why this step. A character index tells you where the text differs, but not which token the model reads differently. The tokenizer merges characters into tokens: in step 5, the f-string's question mark and the newline after it became the single token '?Ċ'. To know what the model actually loses, you have to compare tokens. first_divergence already works on any two sequences, so this step needs a new script but no new function. Create diff_tokens.py:

# diff_tokens.pyimport sysfrom naive import CONVERSATION, build_prompt_by_handfrom templatestudio.render import first_divergence, load_tokenizer, render_idssys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")template_ids = render_ids(tokenizer, CONVERSATION)hand_ids = tokenizer(build_prompt_by_hand(CONVERSATION))["input_ids"]index = first_divergence(hand_ids, template_ids)print("by hand:", len(hand_ids), "tokens, template:", len(template_ids), "tokens")if index is None:    print("identical: the token IDs match the template exactly")    raise SystemExitprint("first different token at index", index)print("  template has:", tokenizer.convert_ids_to_tokens(template_ids[index:]))print("  by hand has: ", tokenizer.convert_ids_to_tokens(hand_ids[index:]))

Run it.

python diff_tokens.py

Expected output.

by hand: 58 tokens, template: 59 tokensfirst different token at index 58  template has: ['Ċ']  by hand has:  []

What just happened. The by-hand prompt matches the template for 58 tokens and then ends. The one token it is missing is the newline after assistant, which is the last thing the model reads before it starts writing. Step 4 found the same newline at character 291, but a character index could not tell you whether that newline was its own token or would have merged into assistant. Now you know: it is a separate token, and the by-hand prompt never sends it.

With this step the module is complete: load_tokenizer, render, render_ids, visible_whitespace, and first_divergence. The next step uses it to look at an argument that render has been setting for you since step 2.

Step 7: See what add_generation_prompt adds

Goal. Render the conversation with and without the generation prompt and print exactly what changes.

Why this step. add_generation_prompt defaults to False, and the Transformers chat templating guide warns that without the generation prompt a model "may get confused and do something strange, like continuing the user's message instead of replying to it." The mistake shows up in scripts that call apply_chat_template themselves, such as offline evals, batch jobs, and the eval step inside a training loop. vLLM's chat endpoint sets it to True by default, so traffic through a server is usually safe, and the bug lives in your own scripts.

grid-rows: 1
grid-gap: 30
off: "add_generation_prompt=False" {
  label.near: top-center
  style: {fill: "#FDEDEC"; stroke: "#E74C3C"; font-color: "#2C2C2A"}
  grid-columns: 1
  o1: "...address?<|im_end|>\\n" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  o2: "(nothing: whose turn is it?)" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
}
on: "add_generation_prompt=True" {
  label.near: top-center
  style: {fill: "#EAFAF1"; stroke: "#3F9E52"; font-color: "#2C2C2A"}
  grid-columns: 1
  n1: "...address?<|im_end|>\\n" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  n2: "<|im_start|>assistant\\n" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
}

Create no_gen_prompt.py:

# no_gen_prompt.pyfrom naive import CONVERSATIONfrom templatestudio.render import load_tokenizer, rendertokenizer = load_tokenizer("Qwen/Qwen3-0.6B")with_cue = render(tokenizer, CONVERSATION)without_cue = render(tokenizer, CONVERSATION, add_generation_prompt=False)print("with cue ends:   ", repr(with_cue[-30:]))print("without cue ends:", repr(without_cue[-30:]))print("difference:      ", repr(with_cue[len(without_cue):]))

Run it.

python no_gen_prompt.py

Expected output.

with cue ends:    'm_end|>\n<|im_start|>assistant\n'without cue ends: 'e delivery address?<|im_end|>\n'difference:       '<|im_start|>assistant\n'

What just happened. Without the cue, the string ends right after the user's <|im_end|>, and nothing tells the model that it is now the assistant's turn. It might write the assistant header itself, keep writing as the user, or stop, and you cannot rely on any of those. The whole difference is the 22 characters printed on the last line. Your render adds them by default because it overrides the library's False. The module did not change in this step; it has been complete since step 6.

False still has a correct use, which is why it is an argument and not a fixed behavior. When you format a finished conversation as training data, the same guide says to set add_generation_prompt=False, "because the additional tokens to prompt an assistant response aren't helpful during training." The rule for your own code: True whenever the model is about to generate, False when you are rendering a conversation that is already complete.

Step 8 (optional): Compare model output with and without the chat template

Goal. Run the model on all three prompts and compare what it does.

Why this step. Steps 1 to 7 show that the strings differ. They do not show that the difference matters to the model. This step measures it on a real model, on your CPU, with greedy decoding (do_sample=False), so the same machine prints the same result on every run.

direction: down
p1: "template prompt" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
p2: "by-hand prompt\n(missing one \\n)" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
p3: "f-string prompt" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
model: "Qwen3-0.6B\ngreedy, max 300 new tokens" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
r1: "thinks, answers in one sentence,\nstops at <|im_end|> after 161 tokens" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
r2: "first emits a stray ':\\n',\nthen recovers, stops after 170" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
r3: "answers, then writes 6 fake\n'user:' turns, never stops" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
p1 -> model
p2 -> model
p3 -> model
model -> r1
model -> r2
model -> r3

Install PyTorch into the same environment. On Linux, the default PyPI wheel pulls in CUDA libraries, so the command uses the CPU wheel index to keep the download small. On macOS and Windows the default PyPI wheel is already CPU-only, so you can drop --index-url there:

pip install "torch==2.15.0" --index-url https://download.pytorch.org/whl/cpu

One fact about Qwen3 explains part of the script. By default it reasons before answering: it writes its reasoning inside <think>...</think> and then gives the answer. The script therefore keeps only the text after </think>. Create generate.py:

# generate.pyimport sysimport textwrapfrom transformers import AutoModelForCausalLMfrom naive import CONVERSATION, build_prompt_by_hand, build_prompt_naivefrom templatestudio.render import load_tokenizer, rendersys.stdout.reconfigure(encoding="utf-8")MODEL_ID = "Qwen/Qwen3-0.6B"tokenizer = load_tokenizer(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID)end_of_turn = tokenizer.convert_tokens_to_ids("<|im_end|>")prompts = {    "template": render(tokenizer, CONVERSATION),    "by hand": build_prompt_by_hand(CONVERSATION),    "f-string": build_prompt_naive(CONVERSATION),}for name, prompt in prompts.items():    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)    output = model.generate(**inputs, max_new_tokens=300, do_sample=False)    new_ids = output[0][inputs["input_ids"].shape[1] :].tolist()    reply = tokenizer.decode(new_ids, skip_special_tokens=False)    answer = reply.split("</think>")[-1].replace("<|im_end|>", "").strip()    print(f"=== {name} ===")    print("new tokens:", len(new_ids))    print("stopped at <|im_end|>:", new_ids[-1] == end_of_turn)    print("invented 'user:' turns:", reply.count("\nuser:"))    print("first token:", repr(tokenizer.decode(new_ids[:1])))    print("closed a <think> block:", "</think>" in reply)    print(textwrap.fill("first line: " + answer.splitlines()[0], width=80))

Each prompt is tokenized with add_special_tokens=False, so the model receives exactly the string you built, with nothing added. The script prints counts and the first line of each answer instead of the full replies, because the f-string reply is 300 tokens of invented conversation. The answer line keeps only the text after </think>. When a reply has no think block, split returns the whole reply, so the script also reports whether a block was closed.

Run it. The first run downloads about 1.5 GB of weights. Generation takes a few minutes on a laptop CPU.

python generate.py

Expected output. A progress bar for Loading weights prints first. After it:

=== template ===new tokens: 161stopped at <|im_end|>: Trueinvented 'user:' turns: 0first token: '<think>'closed a <think> block: Truefirst line: You can change the delivery address, but please provide your newaddress and contact details to proceed.=== by hand ===new tokens: 170stopped at <|im_end|>: Trueinvented 'user:' turns: 0first token: ':\n'closed a <think> block: Truefirst line: You can change the delivery address, but please contact customersupport for specific instructions.=== f-string ===new tokens: 300stopped at <|im_end|>: Falseinvented 'user:' turns: 6first token: ' Please'closed a <think> block: Falsefirst line: Please check the order details for the delivery address.

On different hardware, floating-point differences can shift a long generation by a few tokens. The new tokens counts and the wording of the first line answers may therefore differ from the output above. The other lines must match exactly: all three stopped at <|im_end|> values, all three closed a <think> block values, the three first tokens, and invented 'user:' turns: 0 for the template and by-hand runs. The f-string's count of 6 can differ slightly if its 300 tokens fill differently, but it should stay well above 0. If one of those differs, the cause is your setup, not your hardware.

What just happened. With the template prompt, Qwen3 did what it was trained to do. It opened a <think> block, because Qwen3 reasons by default, then answered in one sentence as the system prompt asked and ended its turn with <|im_end|>.

With the f-string prompt, the model never entered its reasoning mode, so there was no think block. It answered in one sentence, and a quick check of the first line would pass. Then it kept going. It wrote six turns of a conversation nobody had, with user: lines it made up, until it hit the 300-token limit, because the f-string never gave it the <|im_end|> boundary it was trained to stop at. No exception occurred, and every reply had text in it. That is the silent format penalty.

The by-hand prompt shows how much one run can prove, and how much it cannot. Its first token was :\n, a colon and a newline. That is f-string syntax, which the template never produces. The model had no newline after assistant to start from, and it wrote one itself, with a colon in front. After that it opened and closed a think block like the template run did, and answered with different wording. One missing character did not break this model on this conversation. You cannot know in advance whether a given model will recover, and that is why the check belongs at the string level, before the model runs. Chapter 1 of The Chat Templates Handbook runs the larger experiment: twenty sampled trials per prompt, measuring how often each format follows the system prompt.

Fix common apply_chat_template errors

Every error string below was reproduced on the versions this tutorial pins, except two. The TypeError was reproduced on Transformers 4.57.6, the last 4.x release, because that is the only place it happens. The doubled-BOS row comes from a Llama 3.1 discussion, because Qwen3 has no BOS token to double. The PowerShell row quotes the message from the venv folder onward, because the path before it is different on every machine.

How the render module and scripts fit together

You now have two files of your own code, the scripts that call them, and one artifact you did not write: the template from the Hub.

direction: down
scripts: "scripts: show_wire · show_whitespace\ndiff_prompts · show_tokens · diff_tokens\nno_gen_prompt · generate" {
  style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}
}
naive: "naive.py\nCONVERSATION\nbuild_prompt_naive\nbuild_prompt_by_hand" {
  style: {fill: "#FFA07A"; stroke: "#D9785A"; font-color: "#2C2C2A"}
}
render: "templatestudio/render.py\nload_tokenizer · render · render_ids\nvisible_whitespace · first_divergence" {
  style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}
}
hub: "Hub: Qwen/Qwen3-0.6B\ntokenizer files + chat_template\n(+ weights for generate.py)" {
  style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}
}
scripts -> naive: "import CONVERSATION\n+ hand-built prompts"
scripts -> render: "render() / render_ids()\nfor the wire format"
render -> hub: "AutoTokenizer.from_pretrained\napply_chat_template"

render.py is the only file that calls apply_chat_template. The scripts never format a prompt themselves. They ask render or render_ids for the wire format and compare it against the two hand-built prompts in naive.py. Keep that rule as the project grows: exactly one place in a stack should call apply_chat_template. If no place does, you are building prompts by hand. If two places do, you get a template applied on top of a template.

Complete code: the chat template inspector in Python

If your files differ from the steps anywhere, these are the final versions, exactly as the steps leave them, without the # filename comment lines that label each step's code block.

chat-templates-part1/├── naive.py├── templatestudio/│   ├── __init__.py        (empty)│   └── render.py├── show_wire.py├── show_whitespace.py├── diff_prompts.py├── show_tokens.py├── diff_tokens.py├── no_gen_prompt.py└── generate.py            (optional, needs torch)

templatestudio/render.py:

from __future__ import annotationsfrom transformers import AutoTokenizer, PreTrainedTokenizerBaseMessage = dict[str, str]Conversation = list[Message]def load_tokenizer(model_id: str) -> PreTrainedTokenizerBase:    """Load a tokenizer together with the chat template it ships with."""    return AutoTokenizer.from_pretrained(model_id)def render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> str:    """Render a conversation into the model's wire format, using its own template."""    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=add_generation_prompt,    )def visible_whitespace(text: str) -> str:    """Show spaces, tabs and newlines so two renders can be compared by eye."""    return (        text.replace("\t", "[TAB]")        .replace(" ", "·")        .replace("\n", "\\n\n")    )def first_divergence(a: str | list[int], b: str | list[int]) -> int | None:    """Return the first index where a and b differ, or None if they are equal.    Works on strings (characters) and on lists of token IDs alike.    """    for index, (item_a, item_b) in enumerate(zip(a, b)):        if item_a != item_b:            return index    if len(a) != len(b):        return min(len(a), len(b))    return Nonedef render_ids(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> list[int]:    """Render a conversation straight to the token IDs the model receives."""    encoded = tokenizer.apply_chat_template(        conversation,        tokenize=True,        add_generation_prompt=add_generation_prompt,    )    return encoded["input_ids"]

naive.py:

CONVERSATION = [    {        "role": "system",        "content": "You are a terse support agent. Answer in one sentence.",    },    {"role": "user", "content": "Where is my order #4471?"},    {"role": "assistant", "content": "It shipped Monday and arrives Thursday."},    {"role": "user", "content": "Can I change the delivery address?"},]def build_prompt_naive(messages: list[dict[str, str]]) -> str:    """The prompt most people write first. It runs, and it is wrong."""    lines = [f"{m['role']}: {m['content']}" for m in messages]    lines.append("assistant:")    return "\n".join(lines)def build_prompt_by_hand(messages: list[dict[str, str]]) -> str:    """ChatML written from memory. Close to right, which makes it worse."""    turns = [f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>" for m in messages]    turns.append("<|im_start|>assistant")    return "\n".join(turns)if __name__ == "__main__":    print(build_prompt_naive(CONVERSATION))

show_wire.py (step 2):

from naive import CONVERSATIONfrom templatestudio.render import load_tokenizer, rendertokenizer = load_tokenizer("Qwen/Qwen3-0.6B")wire_format = render(tokenizer, CONVERSATION)print(wire_format, end="")print("--- ends with a newline:", wire_format.endswith("\n"))

show_whitespace.py (step 3):

import sysfrom naive import CONVERSATION, build_prompt_naivefrom templatestudio.render import load_tokenizer, render, visible_whitespacesys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")print("=== f-string ===")print(visible_whitespace(build_prompt_naive(CONVERSATION)))print("=== template ===")print(visible_whitespace(render(tokenizer, CONVERSATION)))

diff_prompts.py (step 4):

import sysfrom naive import CONVERSATION, build_prompt_by_handfrom templatestudio.render import (    first_divergence,    load_tokenizer,    render,    visible_whitespace,)sys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")by_hand = build_prompt_by_hand(CONVERSATION)wire_format = render(tokenizer, CONVERSATION)index = first_divergence(by_hand, wire_format)print("lengths:", len(by_hand), "by hand,", len(wire_format), "template")if index is None:    print("identical: the prompt matches the template exactly")    raise SystemExitprint("first divergence at character", index)start = max(index - 22, 0)print("by hand: ", visible_whitespace(by_hand[start:]) + "<END>")print("template:", visible_whitespace(wire_format[start:]).rstrip("\n") + "<END>")

show_tokens.py (step 5):

import sysfrom naive import CONVERSATION, build_prompt_by_hand, build_prompt_naivefrom templatestudio.render import load_tokenizer, render_idssys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")template_ids = render_ids(tokenizer, CONVERSATION)naive_ids = tokenizer(build_prompt_naive(CONVERSATION))["input_ids"]hand_ids = tokenizer(build_prompt_by_hand(CONVERSATION))["input_ids"]marker = "<|im_start|>"print(marker, "is", len(marker), "characters and token id",      tokenizer.convert_tokens_to_ids(marker))print("template:", len(template_ids), "tokens")print("  first 4:", tokenizer.convert_ids_to_tokens(template_ids[:4]))print("  last 5: ", tokenizer.convert_ids_to_tokens(template_ids[-5:]))print("f-string:", len(naive_ids), "tokens")print("  first 4:", tokenizer.convert_ids_to_tokens(naive_ids[:4]))print("  last 5: ", tokenizer.convert_ids_to_tokens(naive_ids[-5:]))print("by hand: ", len(hand_ids), "tokens")print("  last 5: ", tokenizer.convert_ids_to_tokens(hand_ids[-5:]))

diff_tokens.py (step 6):

import sysfrom naive import CONVERSATION, build_prompt_by_handfrom templatestudio.render import first_divergence, load_tokenizer, render_idssys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer("Qwen/Qwen3-0.6B")template_ids = render_ids(tokenizer, CONVERSATION)hand_ids = tokenizer(build_prompt_by_hand(CONVERSATION))["input_ids"]index = first_divergence(hand_ids, template_ids)print("by hand:", len(hand_ids), "tokens, template:", len(template_ids), "tokens")if index is None:    print("identical: the token IDs match the template exactly")    raise SystemExitprint("first different token at index", index)print("  template has:", tokenizer.convert_ids_to_tokens(template_ids[index:]))print("  by hand has: ", tokenizer.convert_ids_to_tokens(hand_ids[index:]))

no_gen_prompt.py (step 7):

from naive import CONVERSATIONfrom templatestudio.render import load_tokenizer, rendertokenizer = load_tokenizer("Qwen/Qwen3-0.6B")with_cue = render(tokenizer, CONVERSATION)without_cue = render(tokenizer, CONVERSATION, add_generation_prompt=False)print("with cue ends:   ", repr(with_cue[-30:]))print("without cue ends:", repr(without_cue[-30:]))print("difference:      ", repr(with_cue[len(without_cue):]))

generate.py (step 8, optional):

import sysimport textwrapfrom transformers import AutoModelForCausalLMfrom naive import CONVERSATION, build_prompt_by_hand, build_prompt_naivefrom templatestudio.render import load_tokenizer, rendersys.stdout.reconfigure(encoding="utf-8")MODEL_ID = "Qwen/Qwen3-0.6B"tokenizer = load_tokenizer(MODEL_ID)model = AutoModelForCausalLM.from_pretrained(MODEL_ID)end_of_turn = tokenizer.convert_tokens_to_ids("<|im_end|>")prompts = {    "template": render(tokenizer, CONVERSATION),    "by hand": build_prompt_by_hand(CONVERSATION),    "f-string": build_prompt_naive(CONVERSATION),}for name, prompt in prompts.items():    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)    output = model.generate(**inputs, max_new_tokens=300, do_sample=False)    new_ids = output[0][inputs["input_ids"].shape[1] :].tolist()    reply = tokenizer.decode(new_ids, skip_special_tokens=False)    answer = reply.split("</think>")[-1].replace("<|im_end|>", "").strip()    print(f"=== {name} ===")    print("new tokens:", len(new_ids))    print("stopped at <|im_end|>:", new_ids[-1] == end_of_turn)    print("invented 'user:' turns:", reply.count("\nuser:"))    print("first token:", repr(tokenizer.decode(new_ids[:1])))    print("closed a <think> block:", "</think>" in reply)    print(textwrap.fill("first line: " + answer.splitlines()[0], width=80))

Next steps: render other models and guard add_generation_prompt

• Render a second model. Change the model ID in show_wire.py to HuggingFaceTB/SmolLM3-3B, which is public and needs no token. It uses the same <|im_start|> markers, but its template writes its own system block before your system message, with a knowledge-cutoff line and today's date in it. As a result, the same conversation renders differently on different days. Its template also ships as a separate chat_template.jinja file, the newer location that Qwen3-0.6B does not use.
• Guard the generation prompt. Make render warn when add_generation_prompt=False is used on a conversation whose last message is from the user. Decide whether a warning, an exception, or silence is right for your code, and write down why.
• Read the template itself. Print tokenizer.chat_template and find the add_generation_prompt block that writes <|im_start|>assistant\n. Part 3 of this series sets up a Jinja environment that renders those templates exactly as Transformers does.

This tutorial is the runnable slice of chapter 1 of The Chat Templates Handbook. The chapter adds the twenty-trial experiment, the history of why the format lives in the tokenizer, and the production failure modes around double-templating and training/serving drift. The finished code for chapter 1 is the v1.2/ch01-the-gap tag of Template Studio. For why nobody in a typical stack owns this layer, read Chat Templates: The Last Unowned Layer in Your LLM Stack.

References

• Carrigan, M. Chat Templates: An End to the Silent Performance Killer (2023-10-03). Hugging Face. https://huggingface.co/blog/chat-templates
• Hugging Face. Chat templates (Transformers v5.22.1 documentation). https://huggingface.co/docs/transformers/v5.22.1/en/chat_templating
• Hugging Face. huggingface_hub environment variables (v2.4.0). https://huggingface.co/docs/huggingface_hub/v2.4.0/en/package_reference/environment_variables
• Hugging Face. huggingface-hub 2.4.0 on PyPI. https://pypi.org/project/huggingface-hub/2.4.0/
• Hugging Face. Logging (Transformers v5.22.1, TRANSFORMERS_NO_ADVISORY_WARNINGS). https://huggingface.co/docs/transformers/v5.22.1/en/main_classes/logging
• Hugging Face. Manage the huggingface_hub cache (symlink limitations on Windows, v2.4.0). https://huggingface.co/docs/huggingface_hub/v2.4.0/en/guides/manage-cache#limitations
• Hugging Face. PreTrainedTokenizerBase.apply_chat_template (Transformers v5.22.1 API reference). https://huggingface.co/docs/transformers/v5.22.1/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.apply_chat_template
• Hugging Face. SmolLM3-3B model card. https://huggingface.co/HuggingFaceTB/SmolLM3-3B
• Hugging Face. tokenizers 0.23.3 release. https://github.com/huggingface/tokenizers/releases/tag/v0.23.3
• Hugging Face. transformers 4.57.6 on PyPI (the last 4.x release, used for the TypeError row). https://pypi.org/project/transformers/4.57.6/
• Hugging Face. transformers 5.22.1 on PyPI. https://pypi.org/project/transformers/5.22.1/
• Hugging Face. Transformers v5 migration guide (v5.22.1). https://github.com/huggingface/transformers/blob/v5.22.1/MIGRATION_GUIDE_V5.md
• Hugging Face. Writing a chat template (Transformers v5.22.1 documentation). https://huggingface.co/docs/transformers/v5.22.1/en/chat_templating_writing
• Kumar, R. Chat Templates: The Last Unowned Layer in Your LLM Stack. ranjankumar.in. /chat-templates-the-last-unowned-layer-in-your-llm-stack
• Kumar, R. The Chat Templates Handbook, Edition 1.2, chapter 1. Book page on this site: /the-chat-templates-handbook; book site: https://chattemplates.ranjankumar.in/
• Kumar, R. Template Studio, tag v1.2/ch01-the-gap. https://github.com/ranjankumar-gh/template-studio/tree/v1.2/ch01-the-gap
• Meta Llama. Llama-3.1-8B-Instruct discussion #97 (doubled BOS token). https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/discussions/97
• Microsoft. about_Execution_Policies (PowerShell). https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_execution_policies
• OpenAI. ChatML format notes (openai-python v0.28.0, archived). https://github.com/openai/openai-python/blob/release-v0.28.0/chatml.md
• OpenAI. gpt2 model card (a model with no chat template). https://huggingface.co/openai-community/gpt2
• Pallets. Jinja2 3.1.6 on PyPI. https://pypi.org/project/Jinja2/3.1.6/
• Python Software Foundation. io.TextIOWrapper.reconfigure (Python 3.12). https://docs.python.org/3.12/library/io.html#io.TextIOWrapper.reconfigure
• Python Software Foundation. venv (Python 3.12). https://docs.python.org/3.12/library/venv.html
• PyTorch. Get Started: install locally (CPU wheel index https://download.pytorch.org/whl/cpu). https://pytorch.org/get-started/locally/
• PyTorch. torch 2.15.0 on PyPI. https://pypi.org/project/torch/2.15.0/
• Qwen Team. Qwen3-0.6B model card. https://huggingface.co/Qwen/Qwen3-0.6B
• Qwen Team. Qwen3-0.6B tokenizer_config.json (the chat template source). https://huggingface.co/Qwen/Qwen3-0.6B/blob/main/tokenizer_config.json
• vLLM. Chat completion request protocol (v0.33.0, add_generation_prompt default). https://github.com/vllm-project/vllm/blob/v0.33.0/vllm/entrypoints/openai/chat_completion/protocol.py

Related Articles

Llms

• How to Validate Chat Messages Before apply_chat_template
• How to Render a Chat Template with Jinja Like Transformers
• How to Pass Tools to apply_chat_template and Parse the Calls
• How to Pass reasoning_content Back Through a Chat Template