Guides · Series · 13 parts
Chat Templates in Practice
A chat template is the contract between your messages and the exact token stream the model was trained on. Get it wrong and nothing raises — the model just gets quietly worse.
The book
The Chat Templates Handbook
apply_chat_template and Model-Ready Prompts
How chat templates turn conversations into model-ready prompts, and how to get apply_chat_template right in production.
The bug that never raises
Most prompt bugs are loud. A wrong chat template is not. Render a conversation into the wrong format — a missing generation prompt, a stray newline, a system turn the model never saw in training — and apply_chat_template returns a string, the model returns an answer, and every log line looks healthy. The only symptom is that the model is a little worse than its benchmark, and nobody on the team can say why.
That is the gap this series is about: between the list of messages your code holds and the single token stream a specific model actually receives. The chat template sits in that gap. It is a Jinja program that ships with the weights, runs on every request, and is trusted completely by everything around it.
How the series works
Each part is a tutorial you run, not an essay you read. You start from the real template of a small open model, build one working piece — a renderer, a validator, a probe, a parser, a cross-engine diff, a golden-token test — and see the failure it catches in your own output.
Every part maps to one chapter of The Chat Templates Handbook. The tutorial teaches the core idea and the runnable slice; the chapter carries the variants, the edge cases, and the full module, and its tag in the companion repo holds the finished code.
Which part solves my problem
The series reads in order, but if something is already broken, start here.
- “The model is worse through my code than in its own demo, and nothing errors.”Part 1 →
- “A conversation renders fine and the model still answers as if a turn were missing.”Part 2 →
- “My local render and the production render differ by whitespace I cannot explain.”Part 3 →
- “I need to know what a model’s template does without reading 200 lines of Jinja.”Part 4 →
- “My agent leaks raw tool-call JSON to users instead of calling the tool.”Part 5 →
- “Multi-turn latency jumped after we turned on thinking mode.”Part 6 →
- “The vision model fails with a placeholder or image-count mismatch.”Part 7 →
- “My fine-tuned model behaves differently when served than it did in training.”Part 8 →
- “The same model gives different answers in llama.cpp and in Transformers.”Part 9 →
- “I cannot prove which token IDs my inference server actually sent the model.”Part 10 →
- “A tokenizer or template update shipped and nobody noticed the prompt changed.”Part 11 →
- “I pulled a model from the Hub and never reviewed the code in its template.”Part 12 →
- “I want one CI check that fails when anything about rendering changes.”Part 13 →
The series
0 of 13 published
The gap between messages and tokens
Render a real conversation through Qwen3’s own template, print the wire format with whitespace visible, and diff it against a hand-built prompt — the silent format penalty, made visible.
The message model and the input contract
Build a validator for the conversation rules the template assumes and never checks — roles, ordering, content shape, tool calls — and a safe render that refuses malformed input before it reaches the model.
Just enough Jinja to read a chat template
Set up a Jinja environment that matches what Transformers actually uses — sandbox, whitespace flags, injected helpers, extensions — and watch one missing dash change the tokens.
Anatomy of a chat template
Take any template apart into preamble, system handling, message loop, and generation prompt, then build a probe that reports how a template behaves by rendering near-identical conversations and diffing them.
Tool-calling templates
Render tool schemas and tool calls into the prompt the way the template expects, then write the parser for what comes back out — the half of tool calling the template does not own.
Reasoning and thinking templates
Control thinking mode, strip reasoning from history at the right checkpoint, and catch the empty think blocks that quietly invalidate the KV cache on every turn.
Multimodal templates
Render image-and-text conversations through the Processor, not the tokenizer, and check that the placeholder count matches the media count before a mismatch reaches the model.
Authoring a template for your fine-tuned model
Write a chat template that reproduces your training format, then prove it with a token-level round trip against the trainer’s input IDs — the only check that sees what a string diff misses.
The cross-engine problem
Render one conversation through more than one Jinja implementation, find the first byte where they diverge, and lint a template for the constructs that do not port.
The template behind the endpoint
Ask the inference server itself for the token IDs it produced, and diff them against your local render — the only proof of what the model actually received.
Debugging broken templates with golden-token tests
Record a template’s exact output for a fixed set of conversations and assert byte-identity on every change, so a template edit, tokenizer bump, or engine swap fails CI instead of production.
Templates as an attack surface
Fingerprint and pin a template like the code it is, scan it for backdoor shapes, and catch forged control tokens in user content before they become a fake turn.
Building a template toolkit
Compose every check from the series into one audit and one CI gate with an exit code, so a new model becomes a line of wiring instead of a new rendering path.
Parts still marked in progress are covered in full by the book today — the chapter ranges above are exact. The articles are being written against those chapters.
The companion code
Template Studio is the toolkit the book builds one module per chapter — render, validate, probe, parse, diff across engines, golden-token tests, and a CI gate. Every chapter has its own tag, so each tutorial points at the exact code it ends with.
View the companion repository on GitHub →Other guide series
GenAI Architectures
There are eight architectures and seven boundaries between them. Every boundary is a yes-or-no question about the task — not a judgment call about the design.
Agentic AI with LangGraph
An eight-part guide to the part of agent engineering nobody demos: making the thing survive a restart, a concurrent write, a bad tool call, and a Tuesday deploy.
LangGraph Agent Evals
Every layer of an agent eval stack is a measuring instrument with its own error rate — the grader included — and none of them should gate a deploy until it has been scored against known faults.
Harness Engineering
AI systems do not fail at the model layer. They fail in the code around it — and that code is a designable layer with its own architecture.
RAG Engineering
Most RAG failures happen upstream of the LLM, accumulate invisibly, and only surface as an incident. This series names each one.
Claude Code Playbook
Claude Code has four layers — context, skills, subagents, hooks — and the variable that decides which one you need is enforceability, not capability.
AI Control Plane
A fleet of agents needs a layer that is not itself an agent — observability, policy, orchestration, versioning, cost, and audit, built as infrastructure.
Agentic AI in the Data Center
In physical infrastructure, an agent’s autonomy is capped by reversibility and jurisdiction — not by how good the model is.
Codebase to Architecture Doc
A three-pass methodology for compressing any codebase into validated diagrams and decisions — and then a LangGraph pipeline that runs it for you.
Federated AI Agents
Every call that crosses from one owner to another is a seam, and a seam has five properties — contract, authority, state, failure, and accountability — that are free inside one deployment and have to be built across it.
