How to Pass Tools to apply_chat_template and Parse the Calls

What you'll build: a tool-call renderer and parser tested on four model families

Two of the four chat templates in this tutorial discard the tools you pass, without an error. Those two models never supported tools, but nothing tells your agent that when you swap one in. Only one of the four ships a parser for its own tool calls. By the end of this intermediate tutorial you will have templatestudio/tools.py, which passes tools= and tool calls through apply_chat_template and the model's own Hugging Face chat template, then parses the model's calls back out with parse_response or a fallback parser. When it cannot read a call, it raises an error instead of leaking it. The tutorial is for engineers who connect a tool-calling agent to an open-weight model through Transformers or vLLM.

This is the table you print at the end, one row per model family:

family  tools  call   parse withqwen3   True   True   your own parserllama3  False  False  none: no tool callinggemma3  False  False  none: no tool callinggemma4  True   True   response template

• tools says whether the template renders the tool schemas you pass at all.
• call says whether an assistant tool call survives the render.
• parse with says what reads the model's tool calls back out: the model's own response template, a parser you pick, or nothing, because the template has no tool calling.

Once the tokenizers are cached, one run of about 20 seconds prints the whole table.

The failure this guards against is quiet. The model calls the tool correctly but writes the call in a format your parser does not look for. The parser finds nothing, the tool never runs, and the raw call text goes to the user as if it were the answer. Users have reported it. In vLLM issue #22975, Qwen3-Coder tool calls came back in the content field with tool_calls empty, and a later comment shows the same for Qwen3-8B. The Qwen2.5-Coder tool parser README describes a model that writes its calls as fenced JSON blocks, which a parser that only looks for <tool_call> tags never sees.

On the way in, tools.py builds tool schemas and call and result messages and checks that the template keeps them. On the way out it uses the model's response template when there is one and a parser you chose when there is not. An optional last step runs Qwen3-0.6B on your CPU, so you see a real tool call go out and come back through the whole loop.

Verified against Python 3.12.2, transformers==5.22.1, tokenizers==0.23.3, huggingface-hub==2.4.0, jinja2==3.1.6, and torch==2.15.0+cpu for step 13, on 2026-11-10.

This is Part 5 of a series and continues the project from Part 4, How to Check if a Chat Template Supports a System Prompt. It takes about 30 minutes. If you finished Part 4, it downloads nothing new. It needs no API key, no account and no Hugging Face token, because every model it loads is public and ungated. The optional step 13 needs PyTorch and the Qwen3-0.6B weights, about 1.5 GB, and runs on a laptop CPU with no GPU.

Prerequisites: transformers 5.22.1, the Part 4 project and its tokenizers

You need the project from Parts 1 to 4: the chat-templates-part1/ folder with its virtual environment, the templatestudio/ package, and Part 4's families.py and fetch_models.py. This part uses templatestudio/messages.py as Part 2 left it and templatestudio/render.py as Part 4 left it. It adds one module, templatestudio/tools.py, two shared Python modules (rates.py and samples.py) and fifteen scripts, and step 2 changes two functions in render.py. If you do not have the project, build it in three moves:

• Follow Part 1's Prerequisites to create the folder and the environment.
• Create the templatestudio/ folder with an empty __init__.py. Copy templatestudio/messages.py from Part 2's complete code and templatestudio/render.py from Part 4's complete code.
• Copy families.py and fetch_models.py from Part 4's Prerequisites into the project root.

Go to the project folder, activate the environment, and turn off the advisory that Transformers prints when PyTorch is not installed. Run the cd from the folder that contains chat-templates-part1/:

cd chat-templates-part1source .venv/bin/activateexport TRANSFORMERS_NO_ADVISORY_WARNINGS=1

On Windows, activate with .venv\Scripts\activate in PowerShell or source .venv/Scripts/activate in Git Bash, and set the variable in PowerShell with $env:TRANSFORMERS_NO_ADVISORY_WARNINGS = "1". Every script in this part runs from the project root.

The versions are unchanged from Parts 1 to 4. Transformers 5.22.2 (2026-11-06) only hardens the Jinja sandbox and fixes a distributed-training bug. Nothing in tool calling changed, so the series stays on one pin. If you are starting fresh, install them inside the activated environment:

pip install "transformers==5.22.1" "tokenizers==0.23.3" \    "huggingface-hub==2.4.0" "jinja2==3.1.6"

In PowerShell, continue the line with a backtick instead of a backslash:

pip install "transformers==5.22.1" "tokenizers==0.23.3" `    "huggingface-hub==2.4.0" "jinja2==3.1.6"

The series needs Python 3.11 or later, because messages.py imports typing.NotRequired. It was verified on 3.12.2. Check the install and the cached tokenizers before you go on:

python -c "import transformers as t, jinja2 as j; print(t.__version__, j.__version__)"python fetch_models.py

Expected output:

5.22.1 3.1.6Qwen/Qwen3-0.6B                        template: 4168 charsNousResearch/Meta-Llama-3-8B-Instruct  template: 389 charsunsloth/gemma-3-1b-it                  template: 1532 charsgoogle/gemma-4-E2B-it                  template: 18567 chars

If you skipped Part 4, the first fetch_models.py run downloads about 89 MB of tokenizer files. It can print two harmless warnings: one starting Warning: You are sending unauthenticated requests to the HF Hub. and, on Windows, one starting UserWarning: `huggingface_hub` cache-system uses symlinks by default. The Llama 3 and Gemma 3 repositories are ungated copies whose templates match the official, gated ones, as Part 4 explains. Step 13 installs PyTorch when you reach it.

How tool calling works in a chat template: the render/parse asymmetry

A tool call crosses the model boundary twice. On the way in, you hand apply_chat_template a list of tool schemas through its tools argument, plus assistant messages that carry tool_calls and tool messages that carry results. The model's chat template turns all of it into that model's wire format, which is the exact text the model was trained to read. You write none of that formatting. Some docs and libraries call this function calling; in Transformers it is the same mechanism, passed as tools=.

On the way out, the model writes a tool call as plain text in whatever format it was trained on: JSON inside <tool_call> tags for Qwen3, a <|tool_call>call:name{...} line for Gemma 4, a fenced json block for Qwen2.5-Coder. apply_chat_template does nothing on this side. Something else has to find the call in that text and turn it back into a name and a dict of arguments.

The Chat Templates Handbook, the book this series accompanies, calls this the render/parse asymmetry: the template owns how tools go in, while the way calls come out has historically had no owner. It is one face of the problem the last unowned layer in your LLM stack describes. Transformers has started to close the gap. A tokenizer can now ship a response template, a parser for the model's output format that is stored in the tokenizer config next to the chat template, which tokenizer.parse_response runs. Most models on the Hub do not ship one yet.

grid-rows: 1
grid-gap: 40
in: "Way in: the chat template owns it" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 14
  fn: "Python function\nget_exchange_rate" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
  msgs: "messages + tools=\ntool_calls, tool results" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
  tpl: "apply_chat_template\nthe model's own template" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  wire: "Prompt text in the\nmodel's wire format" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
}
out: "Way out: owned only if the model ships a parser" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 14
  raw: "Raw model output\n<tool_call>{...}</tool_call>" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  rt: "Response template\nparse_response (Gemma 4)" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
  own: "Your parser\nHermesParser (Qwen3)" {style: {fill: "#FFA07A"; stroke: "#D9785A"; font-color: "#2C2C2A"}}
  call: "ParsedToolCall\nname + arguments dict" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
}

Read the left column top to bottom. The model's authors fixed every box on the way in, and your only job is to hand over correct schemas and messages. The right column has two middle boxes because only some models ship a response template. When one exists you use it. When it does not, the parser is yours to choose, and choosing the wrong one is the silent failure from the opening. The module you build in this part covers both columns, and steps 7 to 11 deal with the fork on the right.

Step 1: Turn a Python function into a tool schema with get_json_schema

Goal: create the tool, then create templatestudio/tools.py with a tool_schema function that builds the tool's JSON schema.

Why this step: a template does not read Python. It renders a JSON schema: the tool's name, a description, and a typed list of parameters. Transformers can build that schema from a function with transformers.utils.get_json_schema, which reads the type hints and a Google-style docstring. That makes the docstring part of the tool's interface. Its first line becomes the description the model reads to decide whether to call the tool, and each Args: line becomes a parameter description. A docstring in another style, or one that skips an argument, does not produce a weaker schema. It produces an exception.

direction: down
fn: "def get_exchange_rate(base: str, quote: str) -> float" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
hints: "Type hints\n-> type: string, required" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
doc: "Docstring first line\n-> description" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
args: "Args: lines\n-> parameter descriptions" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
schema: "JSON schema\nthe text the template renders" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
fn -> hints
fn -> doc: "get_json_schema reads"
fn -> args
hints -> schema
doc -> schema: "builds"
args -> schema

In the diagram, two of the three sources of the schema are prose you write by hand. That is why the docstring matters.

Create rates.py in the project root. It holds the tool the model may call and the question that should make it call the tool:

# rates.py# The tool the model may call, and the question that should make it call it.def get_exchange_rate(base: str, quote: str) -> float:    """Look up how many units of one currency one unit of another buys.    Args:        base: ISO 4217 code of the currency you hold, such as "EUR".        quote: ISO 4217 code of the currency you want, such as "JPY".    """    return 162.4  # a fixed rate, so every run prints the same numbersQUESTION = [{"role": "user", "content": "How many yen is one euro?"}]

Now create templatestudio/tools.py. The import block below is the one for the whole module, so you write it once here, and later steps use the names this step does not:

# templatestudio/tools.pyfrom __future__ import annotationsimport jsonimport refrom collections.abc import Callablefrom dataclasses import dataclassfrom typing import Protocolfrom transformers import PreTrainedTokenizerBasefrom transformers.utils import get_json_schemafrom templatestudio.messages import Conversationfrom templatestudio.render import renderdef tool_schema(fn: Callable[..., object]) -> dict:    """Build the JSON schema a template renders for fn.    It is read from fn's type hints and its Google-style docstring, so the    docstring is part of the tool's interface, not a comment.    """    return get_json_schema(fn)

Create show_schema.py:

# show_schema.pyimport jsonfrom rates import get_exchange_ratefrom templatestudio.tools import tool_schemaprint(json.dumps(tool_schema(get_exchange_rate), indent=2))

Run it:

python show_schema.py

Expected output:

{  "type": "function",  "function": {    "name": "get_exchange_rate",    "description": "Look up how many units of one currency one unit of another buys.",    "parameters": {      "type": "object",      "properties": {        "base": {          "type": "string",          "description": "ISO 4217 code of the currency you hold, such as \"EUR\"."        },        "quote": {          "type": "string",          "description": "ISO 4217 code of the currency you want, such as \"JPY\"."        }      },      "required": [        "base",        "quote"      ]    },    "return": {      "type": "number"    }  }}

Now the failure case: a docstring with no Args: section. Create show_bad_docstring.py:

# show_bad_docstring.pyimport textwrapfrom transformers.utils.chat_template_utils import DocstringParsingExceptionfrom templatestudio.tools import tool_schemadef get_rate(base: str, quote: str) -> float:    """Exchange rate between two currencies."""    return 162.4try:    tool_schema(get_rate)except DocstringParsingException as error:    print(type(error).__name__)    print(textwrap.fill(str(error), width=72))

Run it:

python show_bad_docstring.py

Expected output:

DocstringParsingExceptionCannot generate JSON schema for get_rate because the docstring has nodescription for the argument 'base'

What just happened: tool_schema turned get_exchange_rate into the schema a template renders. Both parameters are required because neither has a default, and the -> float annotation added a return key of type number. The function whose docstring skips its arguments produced no schema at all. It produced an exception, and you see it before any model does. The script wraps the message at 72 characters so it fits the page; the exception text itself is one line. Every later step passes tool_schema(get_exchange_rate) to a template.

Step 2: Pass tools to apply_chat_template and read Qwen3's tools block

Goal: add a keyword-only tools argument to render and safe_render, and render the question with the tool attached.

Why this step: every render in this series goes through render.py, so that is where tools has to enter. apply_chat_template accepts tools= as a list of schemas or plain functions, and passes the list to the template as a variable named tools. This module always passes schemas from tool_schema, which is why render types the argument as list[dict]. The template then decides where the schemas go and what instructions surround them. A keyword-only argument with a default of None keeps every call from Parts 1 to 4 working unchanged. apply_chat_template always hands the template a tools variable, set to None when you pass nothing (Part 3 traced this in the 5.22.1 source), so tools=None renders exactly what leaving it out renders.

direction: down
render: "render(tokenizer, QUESTION, tools=[schema])" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
act: "apply_chat_template(..., tools=tools)" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
prompt: "Qwen3 prompt" {
  grid-columns: 1
  grid-gap: 8
  label.near: top-center
  sys: "system turn: # Tools\n<tools> schema JSON </tools>\nhow to write <tool_call>" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  user: "user turn: the question" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
  gen: "<|im_start|>assistant" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
}
render -> act: "1. tools passes through"
act -> prompt: "2. the template places it"

The diagram shows where Qwen3 puts the tools. It creates a system turn just for them, even though the conversation has no system message.

In templatestudio/render.py, replace the render function with this version:

# templatestudio/render.py (replace render)def render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,    tools: list[dict] | None = None,) -> str:    """Render a conversation into the model's wire format, using its own template.    tools is a list of JSON tool schemas. A template that has no tools branch    ignores it without an error.    """    return tokenizer.apply_chat_template(        conversation,        tools=tools,        tokenize=False,        add_generation_prompt=add_generation_prompt,    )

Then replace safe_render in the same file, so the production path can carry tools too:

# templatestudio/render.py (replace safe_render)def safe_render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,    tools: list[dict] | None = None,) -> str:    """Check the input contract, then render. Use this in production."""    assert_valid(conversation)    return render(        tokenizer,        conversation,        add_generation_prompt=add_generation_prompt,        tools=tools,    )

Create show_tools_block.py. Two lines of Qwen3's prompt are wider than the page, so the script cuts any line over 80 characters and prints its length instead:

# show_tools_block.pyfrom families import FAMILIESfrom rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizer, renderfrom templatestudio.tools import tool_schematokenizer = load_tokenizer(FAMILIES["qwen3"])prompt = render(tokenizer, QUESTION, tools=[tool_schema(get_exchange_rate)])# Two lines of this prompt are too wide for the screen, so cut any over 80.for line in prompt.splitlines():    print(line if len(line) <= 80 else f"{line[:60]}... ({len(line)} chars)")print("schema line has the return key:", '"return": {"type": "number"}' in prompt)

Run it:

python show_tools_block.py

Expected output:

<|im_start|>system# ToolsYou may call one or more functions to assist with the user query.You are provided with function signatures within <tools></tools> XML tags:<tools>{"type": "function", "function": {"name": "get_exchange_rate... (464 chars)</tools>For each function call, return a json object with function n... (118 chars)<tool_call>{"name": <function-name>, "arguments": <args-json-object>}</tool_call><|im_end|><|im_start|>userHow many yen is one euro?<|im_end|><|im_start|>assistantschema line has the return key: True

What just happened: Qwen3's template opened a system turn, wrote the whole schema from step 1 as one 464-character JSON line between <tools> tags, and told the model to answer with JSON inside <tool_call> tags. It copies the schema whole, so even the return key reaches the model. Those instructions define the format the model will write its calls in, and step 7 parses exactly that format.

Step 3: Check whether a chat template renders tools at all

Goal: add renders_tools, a probe that tells you whether a template does anything with the tools you pass.

Why this step: the apply_chat_template docstring says it plainly: "If the template does not support function calling, this argument will have no effect." There is no error and no warning. A model swap can therefore remove the tools from every prompt while your code still passes them. Part 4's describe_template reported a declares_tools hint by searching the template source for the word tools, and that only tells you the word is there. The reliable check is Part 4's differential probing: render the same conversation with and without tools and compare.

direction: down
with: "render(QUESTION, tools=[schema])" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
without: "render(QUESTION)" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
cmp: "Are the two prompts different?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
yes: "Different: the template\nrenders tools" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
no: "Identical: tools were\nthrown away" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
with -> cmp
without -> cmp
cmp -> yes: "yes"
cmp -> no: "no"

If the two renders are identical, the tools went nowhere, and the model will never know they exist.

Add renders_tools to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)def renders_tools(    tokenizer: PreTrainedTokenizerBase, conversation: Conversation, tools: list[dict]) -> bool:    """True if passing tools changes the render at all."""    return render(tokenizer, conversation, tools=tools) != render(tokenizer, conversation)

Create show_ignored_tools.py, which runs the probe on Part 4's three families plus Gemma 4:

# show_ignored_tools.pyfrom families import FAMILIES, GEMMA4from rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizerfrom templatestudio.tools import renders_tools, tool_schematools = [tool_schema(get_exchange_rate)]for family, model_id in [*FAMILIES.items(), ("gemma4", GEMMA4)]:    tokenizer = load_tokenizer(model_id)    print(f"{family:<7} renders tools: {renders_tools(tokenizer, QUESTION, tools)}")

Run it:

python show_ignored_tools.py

Expected output:

qwen3   renders tools: Truellama3  renders tools: Falsegemma3  renders tools: Falsegemma4  renders tools: True

What just happened: Llama 3 and Gemma 3 produced the same prompt with and without the tool, so their templates threw it away. If you ran Part 4's compare_families.py, all four answers match its tools column (TemplateReport.declares_tools), and this time they come from what each template does with the tools. Gemma 4 renders tools in a format of its own. You never have to read that format in this part, because step 10 lets Gemma 4's response template do the parsing. renders_tools is the first column of the final table.

Step 4: Render a tool call and a tool result with apply_chat_template

Goal: add tool_call_message and tool_result_message, render the four-message tool round trip on Qwen3, and see where Qwen3 puts a tool result.

Why this step: after the model calls a tool and the tool answers, both events go back into the conversation so the model can write its final answer. They have a fixed shape that Part 2's input contract already checks. The call is an assistant message whose tool_calls field is a list, even for one call, because templates loop over it. That message needs no content. Part 2's validate_conversation allows exactly that case, and its _validate_tool_calls checks the shape of each call, so assert_valid covers what the builders produce. The result is a tool message whose content is a string. Turn a number or a dict into a string first, or the template converts it in some way the model was never trained on. With builder functions, you never write either shape by hand.

grid-rows: 1
grid-gap: 18
u: "1. user\n\"How many yen\nis one euro?\"" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
c: "2. assistant\ntool_calls: [\n get_exchange_rate\n EUR -> JPY ]" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
r: "3. tool\ncontent: \"162.4\"" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
a: "4. assistant\nthe final answer\n(the model writes it)" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}

You build messages 2 and 3. The template renders all of them, and message 4 is what the model writes after the generation prompt.

Add the two builders to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)def tool_call_message(name: str, arguments: dict) -> dict:    """An assistant turn that calls one tool. tool_calls is always a list."""    function = {"name": name, "arguments": arguments}    call = {"type": "function", "function": function}    return {"role": "assistant", "tool_calls": [call]}def tool_result_message(name: str, content: str) -> dict:    """A tool turn that carries one result back. content must be a string."""    if not isinstance(content, str):        kind = type(content).__name__        raise TypeError(f"tool result content must be a str, got {kind}")    return {"role": "tool", "name": name, "content": content}

Create show_round_trip.py. It renders through safe_render, so Part 2's input contract checks the new message shapes first. It prints the prompt from the user turn on, skipping the tools block you saw in step 2, and because the prompt already ends with a newline, end="" stops print from adding a second one. Part 2's tool_turns.py wrote a round trip like this by hand to test the validator. Here the builders write it, so its shape cannot drift:

# show_round_trip.pyfrom families import FAMILIESfrom rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizer, safe_renderfrom templatestudio.tools import tool_call_message, tool_result_message, tool_schemaROUND_TRIP = [    *QUESTION,    tool_call_message("get_exchange_rate", {"base": "EUR", "quote": "JPY"}),    tool_result_message("get_exchange_rate", "162.4"),]tokenizer = load_tokenizer(FAMILIES["qwen3"])prompt = safe_render(tokenizer, ROUND_TRIP, tools=[tool_schema(get_exchange_rate)])print(prompt[prompt.index("<|im_start|>user"):], end="")

Run it:

python show_round_trip.py

Expected output:

<|im_start|>userHow many yen is one euro?<|im_end|><|im_start|>assistant<tool_call>{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}</tool_call><|im_end|><|im_start|>user<tool_response>162.4</tool_response><|im_end|><|im_start|>assistant

What just happened: the assistant call came out as JSON between <tool_call> tags, the same format step 2's instructions asked the model to write. The model sees its own past calls in the form it produces them. The tool result is less obvious. Qwen3 has no tool turn, so it wraps the result in <tool_response> tags inside a user turn. The prompt ends on the generation prompt, which makes the model's next turn the final answer.

Step 5: Check whether a template keeps an assistant tool call

Goal: add call_survives, a probe that tells you whether an assistant tool call reaches the rendered prompt.

Why this step: step 3 showed that a template can ignore tools. The same templates can also ignore tool_calls, and that is worse. If the template drops the call from history, the next prompt shows the model a tool result with no call before it, or an assistant turn with nothing in it. Nothing raises in either case. The probe renders a call to a tool name and checks whether the name appears in the output. It passes no tools=, which is what makes the check sound: with no tools block in the prompt, the name can only get there through the assistant call.

Add call_survives to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)def call_survives(tokenizer: PreTrainedTokenizerBase, name: str) -> bool:    """True if an assistant tool call reaches the rendered prompt."""    conversation = [        {"role": "user", "content": "Use the tool."},        tool_call_message(name, {}),    ]    return name in render(tokenizer, conversation, add_generation_prompt=False)

Create show_dropped_call.py:

# show_dropped_call.pyfrom families import FAMILIES, GEMMA4from templatestudio.render import load_tokenizerfrom templatestudio.tools import call_survivesfor family, model_id in [*FAMILIES.items(), ("gemma4", GEMMA4)]:    tokenizer = load_tokenizer(model_id)    survives = call_survives(tokenizer, "get_exchange_rate")    print(f"{family:<7} call survives: {survives}")

Run it:

python show_dropped_call.py

Expected output:

qwen3   call survives: Truellama3  call survives: Falsegemma3  call survives: Falsegemma4  call survives: True

What just happened: Llama 3 and Gemma 3 dropped the call. Llama 3's template only renders content, so the assistant turn comes out empty: an assistant header followed directly by <|eot_id|>. This Llama 3 is the 8B Instruct model Part 4 used, and Llama 3.1 added tool calling to its template. call_survives is the second column of the final table. A False in either column means the model cannot use tools through this template at all.

Step 6: Normalize tool-call arguments from a JSON string to a dict

Goal: add normalize_arguments, and make tool_call_message use it, so arguments always reaches the template as a dict.

Why this step: in the Transformers message format, arguments is a dict. Many models write it as a JSON string on the wire, and OpenAI-style chat history carries it as a string too, so history you load from an API or a log often has string arguments. Templates disagree on what to do with a string. Qwen3's template has an explicit branch for it, {%- if tool_call.arguments is string %}, and copies the string verbatim, so the same call can render two ways. Gemma 4's template refuses the string outright.

Create show_string_arguments.py. It builds the assistant message by hand, so nothing converts the string before the template sees it. It prints the second-to-last line of each render, because that is the line holding the call; the last line is </tool_call><|im_end|>:

# show_string_arguments.pyimport textwrapfrom jinja2.exceptions import TemplateErrorfrom families import FAMILIES, GEMMA4from rates import QUESTIONfrom templatestudio.render import load_tokenizer, renderdef call_with(arguments: dict | str) -> list[dict]:    """The question plus an assistant call, built by hand around arguments."""    function = {"name": "get_exchange_rate", "arguments": arguments}    return [*QUESTION, {"role": "assistant", "tool_calls": [{"function": function}]}]as_dict = call_with({"base": "EUR", "quote": "JPY"})as_string = call_with('{"base":"EUR","quote":"JPY"}')qwen3 = load_tokenizer(FAMILIES["qwen3"])for label, conversation in [("dict:  ", as_dict), ("string:", as_string)]:    rendered = render(qwen3, conversation, add_generation_prompt=False)    print("qwen3", label, rendered.splitlines()[-2])gemma4 = load_tokenizer(GEMMA4)try:    render(gemma4, as_string, add_generation_prompt=False)except TemplateError as error:    print("gemma4 string:", type(error).__name__)    print(textwrap.fill(str(error), width=72))

Run it:

python show_string_arguments.py

Expected output:

qwen3 dict:   {"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}qwen3 string: {"name": "get_exchange_rate", "arguments": {"base":"EUR","quote":"JPY"}}gemma4 string: TemplateErrorchat_template: tool_calls[].function.arguments must be a JSON object(mapping), not a string. Deserialize arguments before passing to thetemplate.

Look at the spaces. The dict goes through the template's tojson filter, which writes "base": "EUR" with a space after each colon and comma. The string is copied as it came, with no spaces. That is a different token sequence for the same call. The fix belongs in the one place calls are built. Add normalize_arguments to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)def normalize_arguments(arguments: object) -> dict:    """Return tool-call arguments as a dict, decoding a JSON string if needed."""    if isinstance(arguments, dict):        return arguments    if isinstance(arguments, str):        decoded = json.loads(arguments)        if isinstance(decoded, dict):            return decoded        raise ValueError(f"arguments did not decode to a JSON object: {arguments!r}")    kind = type(arguments).__name__    raise ValueError(f"arguments must be a dict or a JSON string, got {kind}")

Then, in the same file, replace tool_call_message with a version that accepts either form and always stores a dict. It calls normalize_arguments, which sits further down the file. Python looks the name up when tool_call_message runs, not when it is defined, so leave the order as it is:

# templatestudio/tools.py (replace tool_call_message)def tool_call_message(name: str, arguments: dict | str) -> dict:    """An assistant turn that calls one tool. tool_calls is always a list."""    function = {"name": name, "arguments": normalize_arguments(arguments)}    call = {"type": "function", "function": function}    return {"role": "assistant", "tool_calls": [call]}

Create show_normalized.py. It builds the same call from a string and from a dict, then renders the string version on Gemma 4, the template that refused it a moment ago:

# show_normalized.pyfrom families import GEMMA4from rates import QUESTIONfrom templatestudio.render import load_tokenizer, renderfrom templatestudio.tools import tool_call_messageas_string = tool_call_message("get_exchange_rate", '{"base":"EUR","quote":"JPY"}')as_dict = tool_call_message("get_exchange_rate", {"base": "EUR", "quote": "JPY"})print("same message:", as_string == as_dict)gemma4 = load_tokenizer(GEMMA4)rendered = render(gemma4, [*QUESTION, as_string], add_generation_prompt=False)call_line = rendered.split("<tool_call|>")[0].splitlines()[-1]print(call_line + "<tool_call|>")

Run it:

python show_normalized.py

Expected output:

same message: True<|tool_call>call:get_exchange_rate{base:<|"|>EUR<|"|>,quote:<|"|>JPY<|"|>}<tool_call|>

What just happened: tool_call_message now decodes a JSON string before the template sees it. The string and the dict build the same message, and Gemma 4 renders it as its own <|tool_call> line instead of raising. The script prints only that line, because Gemma 4 writes the start of the next tool-response turn directly after it. normalize_arguments raises on anything that is not a JSON object, since a call with arguments like [1, 2] cannot be run anyway. The parsers in the next steps use the same function on the way out.

Step 7: Parse Qwen3's tool calls with a Hermes-format parser

Goal: add ParsedToolCall, a ToolCallParser interface, and HermesParser, which reads calls written as JSON inside <tool_call> tags.

Why this step: Qwen3 ships no response template, so reading its calls is your job. Step 2 showed the format its template asks for: one JSON object with name and arguments between <tool_call> and </tool_call>. That format is called Hermes, after the Nous Hermes models that popularized it, and vLLM's matching parser is --tool-call-parser hermes. A parser for it needs a regular expression to find each tagged block, json.loads to decode it, and two guards. It skips a block that is not valid JSON, and it skips JSON that names no tool. A small Protocol gives every parser the same parse(text) method, so you can swap one for another when the model changes.

direction: down
raw: "Raw model output text" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
find: "1. regex finds each\n<tool_call>{...}</tool_call>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
bad: "not JSON\n-> skip" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
noname: "no name key\n-> skip" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
ok: "name + arguments\n-> normalize_arguments" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
out: "list[ParsedToolCall]" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
raw -> find
find -> bad: "2. json.loads"
find -> noname
find -> ok
ok -> out: "3. collect"

Only the green path produces a call. Its arguments go through the same normalize_arguments as step 6, so a model that writes arguments as a string still gives you a dict.

Add the parser code to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)@dataclassclass ParsedToolCall:    name: str    arguments: dictclass ToolCallParser(Protocol):    """Reads tool calls out of one model family's raw output text."""    def parse(self, text: str) -> list[ParsedToolCall]: ...class HermesParser:    """Calls written as JSON inside <tool_call> tags, the format Qwen3 uses."""    pattern = re.compile(r"<tool_call>\s*(\{.*?\})\s*</tool_call>", re.DOTALL)    def parse(self, text: str) -> list[ParsedToolCall]:        calls: list[ParsedToolCall] = []        for match in self.pattern.finditer(text):            try:                payload = json.loads(match.group(1))            except json.JSONDecodeError:                continue  # text that only looks like a call            if isinstance(payload, dict) and isinstance(payload.get("name"), str):                arguments = normalize_arguments(payload.get("arguments", {}))                calls.append(ParsedToolCall(payload["name"], arguments))        return calls

The parsers need model output to read. Put four sample outputs, one per format, in samples.py in the project root. Each is the text a model of that family writes after its generation prompt. Step 13 has Qwen3 write a real one:

# samples.py# Raw text in each family's tool-call format, as a model writes it after its# generation prompt. Step 13 has Qwen3 write a real call instead.QWEN3_CALL = (    "<tool_call>\n"    '{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}\n'    "</tool_call>")FENCED_CALL = (    "```json\n"    '{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}\n'    "```")GEMMA4_CALL = (    '<|tool_call>call:get_exchange_rate{base:<|"|>EUR<|"|>,quote:<|"|>JPY<|"|>}'    "<tool_call|>")PLAIN_ANSWER = "One euro buys about 162.4 yen."

Create show_hermes.py:

# show_hermes.pyfrom samples import PLAIN_ANSWER, QWEN3_CALLfrom templatestudio.tools import HermesParserparser = HermesParser()print(parser.parse(QWEN3_CALL))print(parser.parse(PLAIN_ANSWER))

Run it:

python show_hermes.py

Expected output:

[ParsedToolCall(name='get_exchange_rate', arguments={'base': 'EUR', 'quote': 'JPY'})][]

What just happened: HermesParser found the one call in the Qwen3 sample and returned its name and a dict of arguments, ready to pass to get_exchange_rate(**arguments). On a plain answer it returned an empty list, which is how a parser says "no call, show this text to the user". That empty list is correct here. In the next step the same empty list is the bug.

Step 8: Reproduce a leaked tool call from a parser that does not match the model

Goal: run HermesParser on a call written in another format, see it return nothing, and add FencedJSONParser, the parser that matches that format.

Why this step: this is the opening failure, reproduced with one sample string. A model fine-tuned on a different format, such as Qwen2.5-Coder, writes the same call as a fenced json block with no <tool_call> tags. FENCED_CALL follows the shape the Qwen2.5-Coder parser README quotes. It was written for this tutorial, not generated by that model. HermesParser looks only for the tags, so it returns [], and the caller treats the call text as a plain answer and shows it to the user. Each parser is correct for its own format and wrong for every other, so you have to choose the parser per model family.

grid-rows: 1
grid-gap: 36
wrong: "Parser does not match the model" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 12
  t1: "```json {\"name\": ...} ```" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  p1: "HermesParser\nlooks for <tool_call>" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  r1: "[] -> call text shown\nto the user" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
}
right: "Parser matches the model" {
  label.near: top-center
  grid-columns: 1
  grid-gap: 12
  t2: "```json {\"name\": ...} ```" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
  p2: "FencedJSONParser\nlooks for ```json" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
  r2: "ParsedToolCall\n-> the tool runs" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
}

The input on both sides is the same text. Only the parser changes, and the parser decides whether the tool runs or the user sees raw JSON.

Add the second parser to the end of templatestudio/tools.py. FencedJSONParser inherits all of HermesParser's decoding and replaces only the pattern:

# templatestudio/tools.py (add at the end)class FencedJSONParser(HermesParser):    """Calls written as a fenced json code block, as Qwen2.5-Coder does."""    pattern = re.compile(r"```json\s*(\{.*?\})\s*```", re.DOTALL)

Create show_leak.py:

# show_leak.pyfrom samples import FENCED_CALLfrom templatestudio.tools import FencedJSONParser, HermesParserprint("HermesParser:", HermesParser().parse(FENCED_CALL))for call in FencedJSONParser().parse(FENCED_CALL):    print("FencedJSONParser:", call.name, call.arguments)

Run it:

python show_leak.py

Expected output:

HermesParser: []FencedJSONParser: get_exchange_rate {'base': 'EUR', 'quote': 'JPY'}

What just happened: the wrong parser returned [] on a perfectly good call, and the matching parser read the same text. Nothing in the wrong parser's result tells you a call was missed, because [] is also what it returns for a plain answer, as step 7 showed. You now have a parser for each of the two text formats, and step 11 names one of them as the fallback for each model family.

Step 9: Detect a tool call your parser missed

Goal: add looks_like_tool_call, a check that flags output that looks like a tool call when the parser found none.

Why this step: the right parser fixes the leak only until the model changes. When someone swaps the model, the parser you chose can be wrong again, and it will still return [] without complaint. A leak detector turns that future leak into an error. It does not parse anything. It only notices that the text contains the shape of a call, so "no calls" plus "looks like a call" can be treated as a failure instead of an answer.

grid-rows: 2
grid-columns: 3
grid-gap: 14
h1: "parser found\na call" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
h2: "no call, text\nlooks like one" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
h3: "no call, plain\nanswer" {style: {fill: "#95A5A6"; stroke: "#6F7F80"; font-color: "#2C2C2A"}}
a1: "run the tool" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
a2: "a leak: raise" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
a3: "show the answer" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}

A parser alone cannot see the middle column, because its output there is the same [] as in the right column.

Add the detector to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)def looks_like_tool_call(text: str) -> bool:    """A cheap leak detector: the text contains a quoted name and arguments key."""    return '"name"' in text and '"arguments"' in text

Create show_leak_check.py, which runs HermesParser and the detector on three outputs:

# show_leak_check.pyfrom samples import FENCED_CALL, PLAIN_ANSWER, QWEN3_CALLfrom templatestudio.tools import HermesParser, looks_like_tool_callparser = HermesParser()outputs = {"fenced": FENCED_CALL, "qwen3": QWEN3_CALL, "plain": PLAIN_ANSWER}for label, text in outputs.items():    calls = parser.parse(text)    looks = looks_like_tool_call(text)    print(f"{label:<7} calls: {len(calls)}  looks like a call: {looks}")

Run it:

python show_leak_check.py

Expected output:

fenced  calls: 0  looks like a call: Trueqwen3   calls: 1  looks like a call: Trueplain   calls: 0  looks like a call: False

What just happened: the three rows are the three cells of the diagram. The fenced output is the leak: zero calls, but it looks like one. The Qwen3 output parsed, and the plain answer neither parsed nor looked like a call. looks_like_tool_call is deliberately crude. It only checks for the two JSON keys that every Hermes-style and fenced call contains, so it can flag an answer that quotes JSON. For a tool-calling agent, a false alarm you can see is better than a leak you cannot. Step 11 raises an error on the middle case. Gemma 4's format has no quoted keys at all, which is one reason step 11 does not use your parsers for it.

Step 10: Parse tool calls with a model's response template and parse_response

Goal: parse Gemma 4's tool call with the response template it ships, and see what parse_response does on a model that ships none.

Why this step: a response template is a parser written against one model's output format and shipped with its tokenizer. It is stored in tokenizer_config.json and loaded onto the tokenizer as tokenizer.response_template. tokenizer.parse_response(text, prefix=...) runs it and returns a message dict in the same shape you pass to apply_chat_template: role, plus content or tool_calls. For models whose authors shipped one, the output side now has an owner, as the input side does. Two details matter. prefix= is required and must be the prompt that came before the generation, because a template can start the assistant turn itself and the parser has to see that start. What the parser needs from it is how the template opens the assistant turn, so the scripts below pass a render of QUESTION without tools; in a real loop, pass the exact prompt you generated from, as step 13 does. And on a tokenizer without a response template, parse_response raises instead of returning nothing.

direction: down
q: "tokenizer.response_template is None?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
g4: "Gemma 4: a dict\nparse_response(text, prefix=prompt)\n-> {role, tool_calls}" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
q3: "Qwen3: None\nparse_response(...)\n-> AttributeError" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
q -> g4: "no"
q -> q3: "yes"

parse_tool_calls in step 11 asks the question at the top first. It reads the attribute rather than calling parse_response and waiting for the error.

Create show_response_template.py. It prints with pprint at a width of 72, so the nested dicts fit the page:

# show_response_template.pyfrom pprint import pprintfrom families import FAMILIES, GEMMA4from rates import QUESTIONfrom samples import GEMMA4_CALL, PLAIN_ANSWER, QWEN3_CALLfrom templatestudio.render import load_tokenizer, rendergemma4 = load_tokenizer(GEMMA4)prompt = render(gemma4, QUESTION)pprint(gemma4.parse_response(GEMMA4_CALL, prefix=prompt), width=72)pprint(gemma4.parse_response(PLAIN_ANSWER, prefix=prompt), width=72)qwen3 = load_tokenizer(FAMILIES["qwen3"])print("qwen3 response_template:", qwen3.response_template)try:    qwen3.parse_response(QWEN3_CALL, prefix=render(qwen3, QUESTION))except AttributeError as error:    print("AttributeError:")    print(error)

Run it:

python show_response_template.py

Expected output:

{'role': 'assistant', 'tool_calls': [{'function': {'arguments': {'base': 'EUR',                                            'quote': 'JPY'},                              'name': 'get_exchange_rate'},                 'type': 'function'}]}{'content': 'One euro buys about 162.4 yen.', 'role': 'assistant'}qwen3 response_template: NoneAttributeError:This tokenizer does not have a `response_template` for parsing chat responses!

What just happened: Gemma 4's response template read its own <|tool_call>call:get_exchange_rate{...} format, which uses <|"|> as its quote marker, and returned a standard tool_calls list with the arguments already a dict. On a plain answer it returned content and no tool_calls key at all, so code that reads the result must use .get("tool_calls", []), never ["tool_calls"]. Qwen3's response_template is None, and calling parse_response on it raised AttributeError. Transformers is adding output parsers model by model, and so far Gemma 4 has one and Qwen3 does not. Step 11 uses both results.

Step 11: Parse tool calls with the response template first and a fallback parser

Goal: add parse_tool_calls, which uses the model's response template when it has one and the parser you chose when it does not, and raises when a call would otherwise leak.

Why this step: application code should not care which side of the asymmetry a model is on. It needs one function that takes the raw output and returns calls or raises. The order is a decision. The model's own response template comes first, because it ships with the tokenizer and was written against this model's format, while your parser is a guess about that format. Your parser is the fallback. You name it explicitly for each model family, so a model swap forces you to choose a new one. The function reads tokenizer.response_template instead of calling parse_response and catching what it raises, because parse_response raises AttributeError when there is no template but also raises ValueError on a missing prefix or on malformed JSON in a call. Catching exceptions would put "this model has no template" and "this output is broken" on the same path. Reading the attribute keeps them apart: only the first case falls back to your parser, and broken output still raises.

direction: down
start: "parse_tool_calls(tokenizer, text, prefix=, fallback=)" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
has: "1. response_template set?" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
yes: "parse_response\nerrors propagate" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
fb: "fallback.parse(text)" {style: {fill: "#FFA07A"; stroke: "#D9785A"; font-color: "#2C2C2A"}}
nofb: "no fallback:\nValueError" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
calls: "list[ParsedToolCall]\n(empty for plain text)" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
leak: "2. no calls but\nlooks_like_tool_call?\n-> UnparsedToolCall" {style: {fill: "#E74C3C"; stroke: "#B03A2E"; font-color: "#FFFFFF"}}
start -> has
has -> yes: "yes"
has -> fb: "no, fallback given"
has -> nofb: "no, fallback=None"
yes -> calls
fb -> calls
fb -> leak: "[] and looks like a call"

The red boxes raise. Every other path ends in a list of calls, which is empty only when the text is a plain answer. Errors from parse_response itself pass straight through.

Add the error class and parse_tool_calls to the end of templatestudio/tools.py:

# templatestudio/tools.py (add at the end)class UnparsedToolCall(ValueError):    """Raised when output looks like a tool call but the parser found none."""def parse_tool_calls(    tokenizer: PreTrainedTokenizerBase,    text: str,    *,    prefix: str,    fallback: ToolCallParser | None = None,) -> list[ParsedToolCall]:    """Read tool calls with the model's own response template, if it ships one.    Otherwise use the fallback parser you chose for this model family, and    raise instead of returning nothing when the text still looks like a call.    """    if tokenizer.response_template is not None:        message = tokenizer.parse_response(text, prefix=prefix)        return [            ParsedToolCall(                call["function"]["name"],                normalize_arguments(call["function"]["arguments"]),            )            for call in message.get("tool_calls", [])        ]    if fallback is None:        model = tokenizer.name_or_path        raise ValueError(f"{model} ships no response template; pass a fallback parser")    calls = fallback.parse(text)    if not calls and looks_like_tool_call(text):        parser = type(fallback).__name__        raise UnparsedToolCall(f"{parser} found no call in text that looks like one")    return calls

Create show_parse_tool_calls.py, which runs the four paths:

# show_parse_tool_calls.pyfrom families import FAMILIES, GEMMA4from rates import QUESTIONfrom samples import FENCED_CALL, GEMMA4_CALL, QWEN3_CALLfrom templatestudio.render import load_tokenizer, renderfrom templatestudio.tools import HermesParser, UnparsedToolCall, parse_tool_callsgemma4 = load_tokenizer(GEMMA4)qwen3 = load_tokenizer(FAMILIES["qwen3"])gemma4_prompt = render(gemma4, QUESTION)qwen3_prompt = render(qwen3, QUESTION)hermes = HermesParser()for call in parse_tool_calls(gemma4, GEMMA4_CALL, prefix=gemma4_prompt):    print("gemma4:", call.name, call.arguments)for call in parse_tool_calls(qwen3, QWEN3_CALL, prefix=qwen3_prompt, fallback=hermes):    print("qwen3: ", call.name, call.arguments)try:    parse_tool_calls(qwen3, FENCED_CALL, prefix=qwen3_prompt, fallback=hermes)except UnparsedToolCall as error:    print("fenced:", type(error).__name__, "-", error)try:    parse_tool_calls(qwen3, QWEN3_CALL, prefix=qwen3_prompt)except ValueError as error:    print("no fallback:", error)

Run it:

python show_parse_tool_calls.py

Expected output:

gemma4: get_exchange_rate {'base': 'EUR', 'quote': 'JPY'}qwen3:  get_exchange_rate {'base': 'EUR', 'quote': 'JPY'}fenced: UnparsedToolCall - HermesParser found no call in text that looks like oneno fallback: Qwen/Qwen3-0.6B ships no response template; pass a fallback parser

What just happened: one function gave the same ParsedToolCall for two families with different wire formats, using Gemma 4's response template and your HermesParser for Qwen3. The fenced-JSON leak from step 8 now raises UnparsedToolCall instead of reaching a user. A Qwen3 call with no fallback raises before it parses anything, so a forgotten parser choice fails on the first request. Gemma 4 needed no fallback, and when a future Qwen release ships a response template, the same call site starts using it without a code change. The response-template path has no leak check, though. A malformed call region raises ValueError from parse_response, but output in some other format comes back as plain content. That path trusts the model's authors to describe the model's own output.

Step 12: Check which models support tool calling through their chat template

Goal: combine the probes into one table that tells you, per model family, whether tool calling works through its template and what parses its output.

Why this step: every check in this part answers a question you should ask before you swap a model behind a tool-calling agent. Are the tools rendered? Does a call survive in history? Who parses the output? Checks you have to run by hand, once per model, get skipped. A script that prints one row per family gives you an answer you can read in a code review or compare between two runs. The parse with column checks response_template before the two probes, because a response template is the model authors' own statement of the output format, so it decides the answer whenever it exists.

Create compare_tool_families.py:

# compare_tool_families.pyfrom families import FAMILIES, GEMMA4from rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizerfrom templatestudio.tools import call_survives, renders_tools, tool_schematools = [tool_schema(get_exchange_rate)]print(f"{'family':<7} {'tools':<6} {'call':<6} parse with")for family, model_id in [*FAMILIES.items(), ("gemma4", GEMMA4)]:    tokenizer = load_tokenizer(model_id)    tools_ok = renders_tools(tokenizer, QUESTION, tools)    call_ok = call_survives(tokenizer, "get_exchange_rate")    if tokenizer.response_template is not None:        parser = "response template"    elif tools_ok and call_ok:        parser = "your own parser"    else:        parser = "none: no tool calling"    print(f"{family:<7} {tools_ok!s:<6} {call_ok!s:<6} {parser}")

Run it:

python compare_tool_families.py

Expected output:

family  tools  call   parse withqwen3   True   True   your own parserllama3  False  False  none: no tool callinggemma3  False  False  none: no tool callinggemma4  True   True   response template

What just happened: the script printed the table from the top of the page. Qwen3 works and needs a parser you choose, which today is HermesParser. Gemma 4 works and parses itself. Llama 3 and Gemma 3 have no tool calling in their templates, so an agent swapped onto them keeps running and never calls a tool. Those two models never claimed tool support, and Llama 3.1, one minor version later, does render tools. The table does not measure how well a model chooses tools. Step 13 shows one real call.

Step 13 (optional): Run a real Qwen3 tool call through the whole loop

Goal: have Qwen3-0.6B write a real tool call on your CPU, parse it, run the tool, send the result back, and get the final answer.

Why this step: every parser above read a sample string. Here the model writes the call, so you see that the format step 2's instructions asked for is the format the model actually produces, and that parse_tool_calls with HermesParser reads it. The step also replays step 8's leak on real output, as a simulated model swap. The same reply goes through FencedJSONParser, the parser a team would still have configured from a fenced-JSON model, and step 11's leak check has to catch it. Then the script runs the full loop from step 4 with real output: render with tools, generate, parse, call the function, append the call and the result, render again, generate the answer. It needs PyTorch and about 1.5 GB of model weights, and the two generations take about two minutes on a laptop CPU.

Inside the activated environment, install PyTorch from its CPU wheel index, the same pin Part 1 used for its optional step:

pip install "torch==2.15.0" --index-url https://download.pytorch.org/whl/cpu

Qwen3 writes its reasoning in a <think>...</think> block before it answers or calls a tool, so the script prints and answers from the text after </think>; Part 6 is about that block. The script also sets standard output to UTF-8, as Part 1's generate.py did, because a Windows console may not be UTF-8 and the reply can contain non-ASCII text.

Create generate_call.py. It decodes greedily, so a run repeats exactly on one machine. Your checkpoint is the parsed: line, because the wording of the answer: line can differ on other hardware. registry maps each tool name the model may call to the Python function that runs it:

# generate_call.pyimport sysfrom transformers import AutoModelForCausalLMfrom families import FAMILIESfrom rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizer, safe_renderfrom templatestudio.tools import (    FencedJSONParser,    HermesParser,    UnparsedToolCall,    parse_tool_calls,    tool_call_message,    tool_result_message,    tool_schema,)sys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer(FAMILIES["qwen3"])model = AutoModelForCausalLM.from_pretrained(FAMILIES["qwen3"])tools = [tool_schema(get_exchange_rate)]registry = {"get_exchange_rate": get_exchange_rate}def generate(prompt: str) -> str:    """Greedy-decode one assistant turn and return its raw text."""    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)    output = model.generate(**inputs, max_new_tokens=600, do_sample=False)    new_ids = output[0][inputs["input_ids"].shape[1] :]    return tokenizer.decode(new_ids, skip_special_tokens=True)conversation = list(QUESTION)prompt = safe_render(tokenizer, conversation, tools=tools)reply = generate(prompt)calls = parse_tool_calls(tokenizer, reply, prefix=prompt, fallback=HermesParser())print(reply.split("</think>")[-1].strip())try:    parse_tool_calls(tokenizer, reply, prefix=prompt, fallback=FencedJSONParser())except UnparsedToolCall as error:    print("wrong parser:", error)for call in calls:    print("parsed:", call.name, call.arguments)    result = registry[call.name](**call.arguments)    conversation.append(tool_call_message(call.name, call.arguments))    conversation.append(tool_result_message(call.name, str(result)))prompt = safe_render(tokenizer, conversation, tools=tools)answer = generate(prompt).split("</think>")[-1].strip()print("answer:", answer)

Run it:

python generate_call.py

Expected output:

<tool_call>{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}</tool_call>wrong parser: FencedJSONParser found no call in text that looks like oneparsed: get_exchange_rate {'base': 'EUR', 'quote': 'JPY'}answer: One euro is approximately 162.4 yen.

The first run downloads the weights, and every run prints a Loading weights progress bar on standard error before this output.

What just happened: Qwen3 thought first, inside a <think> block the script cuts off. Then it wrote exactly the <tool_call> format from step 2, character for character the QWEN3_CALL sample from step 7. parse_tool_calls sent it to HermesParser because Qwen3 has no response template. With FencedJSONParser as the fallback instead, as if nobody had changed the parser after a model swap, the same real reply raised UnparsedToolCall. That is the leak from step 8, caught on model output rather than on a sample. The script ran get_exchange_rate, built the call and the result with the step 4 builders, and rendered the round trip through safe_render, and the model answered from the tool's 162.4. Apart from the model and its templates, every piece of that tool-calling turn is code you wrote in this part.

Fix "has no description for the argument" and other tool-calling errors

Cannot generate JSON schema because the docstring has no description for the argument

get_json_schema raises DocstringParsingException when a tool function's Args: section skips an argument. Add one name: description line per argument. This is the error from step 1.

AttributeError: This tokenizer does not have a response_template for parsing chat responses

tokenizer.parse_response only works for a model that ships a response template, and Qwen3 does not. Call parse_tool_calls with a fallback parser instead, as in step 11.

TemplateError: tool_calls[].function.arguments must be a JSON object (mapping), not a string

Gemma 4's template raises this when arguments reaches it as a JSON string. Build every call with tool_call_message, which runs normalize_arguments first, as in step 6.

The table lists every error this tutorial raises, with its cause and fix.

How tool-call rendering and parsing fit together in Transformers

tools.py is the only new module, and it sits on top of the two you already had. The diagram names each piece of code you wrote and shows which side of the asymmetry it serves.

direction: down
loop: "generate_call.py: your agent loop\nregistry: name -> get_exchange_rate (rates.py)" {style: {fill: "#98D8C8"; stroke: "#5FA898"; font-color: "#2C2C2A"}}
t1: "tools.py, way in\ntool_schema, tool_call_message,\ntool_result_message, normalize_arguments" {style: {fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF"}}
t3: "tools.py, way out\nparse_tool_calls -> HermesParser,\nFencedJSONParser, looks_like_tool_call" {style: {fill: "#FFA07A"; stroke: "#D9785A"; font-color: "#2C2C2A"}}
t2: "tools.py, probes\nrenders_tools, call_survives" {style: {fill: "#7B68EE"; stroke: "#5A4BC4"; font-color: "#FFFFFF"}}
rend: "render.py\nrender / safe_render (tools=)" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
act: "apply_chat_template\nthe model's chat template" {style: {fill: "#6BCF7F"; stroke: "#3F9E52"; font-color: "#2C2C2A"}}
pr: "parse_response\nthe model's response template" {style: {fill: "#FFD93D"; stroke: "#C9A800"; font-color: "#2C2C2A"}}
loop -> t1: "1. schema + messages"
t1 -> rend: "2. conversation, tools"
t2 -> rend: "probe renders"
rend -> act: "3. apply_chat_template"
loop -> t3: "4. raw model output"
t3 -> pr: "5. if response_template is set"

Follow the numbers. The blue box builds what goes into render.py and then apply_chat_template. The purple probes call the same render to test a template. The orange box reads the model's output: it hands the output to parse_response only when the model ships a response template, and uses your parsers when it does not. render.py still depends on messages.py and nothing else, and tools.py depends on render.py, never the reverse.

Complete code: render and parse tool calls in Python

The project after this part, with the files this part added or changed marked:

chat-templates-part1/├── templatestudio/│   ├── __init__.py│   ├── messages.py          (unchanged since Part 2)│   ├── render.py            (render and safe_render take tools=)│   ├── anatomy.py           (only if you built Part 4; unused here)│   └── tools.py             (new)├── families.py              (Part 4)├── fetch_models.py          (Part 4)├── rates.py                 (new)├── samples.py               (new)├── show_schema.py├── show_bad_docstring.py├── show_tools_block.py├── show_ignored_tools.py├── show_round_trip.py├── show_dropped_call.py├── show_string_arguments.py├── show_normalized.py├── show_hermes.py├── show_leak.py├── show_leak_check.py├── show_response_template.py├── show_parse_tool_calls.py├── compare_tool_families.py└── generate_call.py         (optional, needs torch)

If you followed the whole series, your folder also holds the files from Parts 1 to 4, such as naive.py, tool_turns.py, templatestudio/jinja_env.py and Part 4's twelve scripts. They still run, but nothing in this part imports them, so the tree above leaves them out. Only tools.py, render.py, rates.py and samples.py carry state between steps, so the scripts never need resyncing.

templatestudio/tools.py:

# templatestudio/tools.pyfrom __future__ import annotationsimport jsonimport refrom collections.abc import Callablefrom dataclasses import dataclassfrom typing import Protocolfrom transformers import PreTrainedTokenizerBasefrom transformers.utils import get_json_schemafrom templatestudio.messages import Conversationfrom templatestudio.render import renderdef tool_schema(fn: Callable[..., object]) -> dict:    """Build the JSON schema a template renders for fn.    It is read from fn's type hints and its Google-style docstring, so the    docstring is part of the tool's interface, not a comment.    """    return get_json_schema(fn)def renders_tools(    tokenizer: PreTrainedTokenizerBase, conversation: Conversation, tools: list[dict]) -> bool:    """True if passing tools changes the render at all."""    return render(tokenizer, conversation, tools=tools) != render(tokenizer, conversation)def tool_call_message(name: str, arguments: dict | str) -> dict:    """An assistant turn that calls one tool. tool_calls is always a list."""    function = {"name": name, "arguments": normalize_arguments(arguments)}    call = {"type": "function", "function": function}    return {"role": "assistant", "tool_calls": [call]}def tool_result_message(name: str, content: str) -> dict:    """A tool turn that carries one result back. content must be a string."""    if not isinstance(content, str):        kind = type(content).__name__        raise TypeError(f"tool result content must be a str, got {kind}")    return {"role": "tool", "name": name, "content": content}def call_survives(tokenizer: PreTrainedTokenizerBase, name: str) -> bool:    """True if an assistant tool call reaches the rendered prompt."""    conversation = [        {"role": "user", "content": "Use the tool."},        tool_call_message(name, {}),    ]    return name in render(tokenizer, conversation, add_generation_prompt=False)def normalize_arguments(arguments: object) -> dict:    """Return tool-call arguments as a dict, decoding a JSON string if needed."""    if isinstance(arguments, dict):        return arguments    if isinstance(arguments, str):        decoded = json.loads(arguments)        if isinstance(decoded, dict):            return decoded        raise ValueError(f"arguments did not decode to a JSON object: {arguments!r}")    kind = type(arguments).__name__    raise ValueError(f"arguments must be a dict or a JSON string, got {kind}")@dataclassclass ParsedToolCall:    name: str    arguments: dictclass ToolCallParser(Protocol):    """Reads tool calls out of one model family's raw output text."""    def parse(self, text: str) -> list[ParsedToolCall]: ...class HermesParser:    """Calls written as JSON inside <tool_call> tags, the format Qwen3 uses."""    pattern = re.compile(r"<tool_call>\s*(\{.*?\})\s*</tool_call>", re.DOTALL)    def parse(self, text: str) -> list[ParsedToolCall]:        calls: list[ParsedToolCall] = []        for match in self.pattern.finditer(text):            try:                payload = json.loads(match.group(1))            except json.JSONDecodeError:                continue  # text that only looks like a call            if isinstance(payload, dict) and isinstance(payload.get("name"), str):                arguments = normalize_arguments(payload.get("arguments", {}))                calls.append(ParsedToolCall(payload["name"], arguments))        return callsclass FencedJSONParser(HermesParser):    """Calls written as a fenced json code block, as Qwen2.5-Coder does."""    pattern = re.compile(r"```json\s*(\{.*?\})\s*```", re.DOTALL)def looks_like_tool_call(text: str) -> bool:    """A cheap leak detector: the text contains a quoted name and arguments key."""    return '"name"' in text and '"arguments"' in textclass UnparsedToolCall(ValueError):    """Raised when output looks like a tool call but the parser found none."""def parse_tool_calls(    tokenizer: PreTrainedTokenizerBase,    text: str,    *,    prefix: str,    fallback: ToolCallParser | None = None,) -> list[ParsedToolCall]:    """Read tool calls with the model's own response template, if it ships one.    Otherwise use the fallback parser you chose for this model family, and    raise instead of returning nothing when the text still looks like a call.    """    if tokenizer.response_template is not None:        message = tokenizer.parse_response(text, prefix=prefix)        return [            ParsedToolCall(                call["function"]["name"],                normalize_arguments(call["function"]["arguments"]),            )            for call in message.get("tool_calls", [])        ]    if fallback is None:        model = tokenizer.name_or_path        raise ValueError(f"{model} ships no response template; pass a fallback parser")    calls = fallback.parse(text)    if not calls and looks_like_tool_call(text):        parser = type(fallback).__name__        raise UnparsedToolCall(f"{parser} found no call in text that looks like one")    return calls

templatestudio/render.py, complete after this part:

# templatestudio/render.pyfrom __future__ import annotationsfrom transformers import AutoTokenizer, PreTrainedTokenizerBasefrom templatestudio.messages import Conversation, assert_validdef load_tokenizer(model_id: str) -> PreTrainedTokenizerBase:    """Load a tokenizer together with the chat template it ships with."""    return AutoTokenizer.from_pretrained(model_id)def render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,    tools: list[dict] | None = None,) -> str:    """Render a conversation into the model's wire format, using its own template.    tools is a list of JSON tool schemas. A template that has no tools branch    ignores it without an error.    """    return tokenizer.apply_chat_template(        conversation,        tools=tools,        tokenize=False,        add_generation_prompt=add_generation_prompt,    )def visible_whitespace(text: str) -> str:    """Show spaces, tabs and newlines so two renders can be compared by eye."""    return (        text.replace("\t", "[TAB]")        .replace(" ", "·")        .replace("\n", "\\n\n")    )def first_divergence(a: str | list[int], b: str | list[int]) -> int | None:    """Return the first index where a and b differ, or None if they are equal.    Works on strings (characters) and on lists of token IDs alike.    """    for index, (item_a, item_b) in enumerate(zip(a, b)):        if item_a != item_b:            return index    if len(a) != len(b):        return min(len(a), len(b))    return Nonedef render_ids(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,) -> list[int]:    """Render a conversation straight to the token IDs the model receives."""    encoded = tokenizer.apply_chat_template(        conversation,        tokenize=True,        add_generation_prompt=add_generation_prompt,    )    return encoded["input_ids"]def safe_render(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,    *,    add_generation_prompt: bool = True,    tools: list[dict] | None = None,) -> str:    """Check the input contract, then render. Use this in production."""    assert_valid(conversation)    return render(        tokenizer,        conversation,        add_generation_prompt=add_generation_prompt,        tools=tools,    )def render_continuation(    tokenizer: PreTrainedTokenizerBase,    conversation: Conversation,) -> str:    """Render with the final assistant turn left open, so the model continues it."""    if not conversation or conversation[-1]["role"] != "assistant":        raise ValueError("render_continuation needs an assistant message last")    return tokenizer.apply_chat_template(        conversation,        tokenize=False,        add_generation_prompt=False,        continue_final_message=True,    )

rates.py:

# rates.py# The tool the model may call, and the question that should make it call it.def get_exchange_rate(base: str, quote: str) -> float:    """Look up how many units of one currency one unit of another buys.    Args:        base: ISO 4217 code of the currency you hold, such as "EUR".        quote: ISO 4217 code of the currency you want, such as "JPY".    """    return 162.4  # a fixed rate, so every run prints the same numbersQUESTION = [{"role": "user", "content": "How many yen is one euro?"}]

samples.py:

# samples.py# Raw text in each family's tool-call format, as a model writes it after its# generation prompt. Step 13 has Qwen3 write a real call instead.QWEN3_CALL = (    "<tool_call>\n"    '{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}\n'    "</tool_call>")FENCED_CALL = (    "```json\n"    '{"name": "get_exchange_rate", "arguments": {"base": "EUR", "quote": "JPY"}}\n'    "```")GEMMA4_CALL = (    '<|tool_call>call:get_exchange_rate{base:<|"|>EUR<|"|>,quote:<|"|>JPY<|"|>}'    "<tool_call|>")PLAIN_ANSWER = "One euro buys about 162.4 yen."

compare_tool_families.py, which prints the table at the top of the page:

# compare_tool_families.pyfrom families import FAMILIES, GEMMA4from rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizerfrom templatestudio.tools import call_survives, renders_tools, tool_schematools = [tool_schema(get_exchange_rate)]print(f"{'family':<7} {'tools':<6} {'call':<6} parse with")for family, model_id in [*FAMILIES.items(), ("gemma4", GEMMA4)]:    tokenizer = load_tokenizer(model_id)    tools_ok = renders_tools(tokenizer, QUESTION, tools)    call_ok = call_survives(tokenizer, "get_exchange_rate")    if tokenizer.response_template is not None:        parser = "response template"    elif tools_ok and call_ok:        parser = "your own parser"    else:        parser = "none: no tool calling"    print(f"{family:<7} {tools_ok!s:<6} {call_ok!s:<6} {parser}")

generate_call.py, the optional end-to-end run:

# generate_call.pyimport sysfrom transformers import AutoModelForCausalLMfrom families import FAMILIESfrom rates import QUESTION, get_exchange_ratefrom templatestudio.render import load_tokenizer, safe_renderfrom templatestudio.tools import (    FencedJSONParser,    HermesParser,    UnparsedToolCall,    parse_tool_calls,    tool_call_message,    tool_result_message,    tool_schema,)sys.stdout.reconfigure(encoding="utf-8")tokenizer = load_tokenizer(FAMILIES["qwen3"])model = AutoModelForCausalLM.from_pretrained(FAMILIES["qwen3"])tools = [tool_schema(get_exchange_rate)]registry = {"get_exchange_rate": get_exchange_rate}def generate(prompt: str) -> str:    """Greedy-decode one assistant turn and return its raw text."""    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)    output = model.generate(**inputs, max_new_tokens=600, do_sample=False)    new_ids = output[0][inputs["input_ids"].shape[1] :]    return tokenizer.decode(new_ids, skip_special_tokens=True)conversation = list(QUESTION)prompt = safe_render(tokenizer, conversation, tools=tools)reply = generate(prompt)calls = parse_tool_calls(tokenizer, reply, prefix=prompt, fallback=HermesParser())print(reply.split("</think>")[-1].strip())try:    parse_tool_calls(tokenizer, reply, prefix=prompt, fallback=FencedJSONParser())except UnparsedToolCall as error:    print("wrong parser:", error)for call in calls:    print("parsed:", call.name, call.arguments)    result = registry[call.name](**call.arguments)    conversation.append(tool_call_message(call.name, call.arguments))    conversation.append(tool_result_message(call.name, str(result)))prompt = safe_render(tokenizer, conversation, tools=tools)answer = generate(prompt).split("</think>")[-1].strip()print("answer:", answer)

templatestudio/messages.py is unchanged from Part 2's complete code. The other thirteen scripts are complete as shown in their steps. All fifteen, by step: show_schema.py and show_bad_docstring.py in step 1, show_tools_block.py in step 2, show_ignored_tools.py in step 3, show_round_trip.py in step 4, show_dropped_call.py in step 5, show_string_arguments.py and show_normalized.py in step 6, show_hermes.py in step 7, show_leak.py in step 8, show_leak_check.py in step 9, show_response_template.py in step 10, show_parse_tool_calls.py in step 11, compare_tool_families.py in step 12 and generate_call.py in step 13.

Next steps: extend tool-call parsing to Llama 3.1, result pairing and MCP

• Add a parser for Llama 3.1's format. Load unsloth/Llama-3.1-8B-Instruct and render a call. You will see a bare JSON object with "parameters" where Qwen3 writes "arguments". Write a LlamaJSONParser for it and add the family to compare_tool_families.py. Note that its template raises This model only supports single tool-calls at once! on two calls in one turn.
• Check that every tool result answers a call. Agent loops truncate history and retry calls, and most templates render an orphaned result without complaint. Qwen3's prompt carries no call ID, so result order is the only pairing signal the model gets. Write a check that walks the conversation and reports any result with no open call before it, and any call left unanswered.
• Pass tool results from an MCP server. A Model Context Protocol tool returns a CallToolResult, which is a list of content blocks plus optional structuredContent, not a string. Convert it to the string tool_result_message requires, keeping text blocks in order and marking images instead of dropping them.

This tutorial is the runnable slice of chapter 5 of The Chat Templates Handbook. The chapter goes further into the agent loop (call IDs, parallel calls, tool_choice, MCP results), into structured outputs as the alternative to parsing, and into why the output format never had a standard owner. The chapter's code is the v1.2/ch05-tool-calling tag of Template Studio. This tutorial differs from the tag in four ways:

• it uses get_exchange_rate where the tag uses get_temperature;
• it adds renders_tools and call_survives, which probe a template instead of trusting it;
• it adds parse_tool_calls and looks_like_tool_call, so the model's response template comes first and a leaked call raises;
• it leaves validate_tool_pairing, parallel_tool_call_message and mcp_result_to_tool_message to the chapter and to the extensions above.

Part 6 turns to reasoning templates: the enable_thinking switch, the <think> block that Qwen3 wrote and step 13 cut off, and which past reasoning a template keeps in history.

References

• Hugging Face. Tool use, Transformers v5.22.1 documentation: the tools argument, Google-style docstrings and the tool_calls message format.
• Hugging Face. Response parsing, Transformers v5.22.1 documentation: parse_response, prefix and response templates.
• Hugging Face. get_json_schema source and apply_chat_template and parse_response source, Transformers v5.22.1.
• Hugging Face. Transformers v5.22.1 release, the pinned version, and the v5.22.2 release, noted but not adopted.
• PyPI. tokenizers 0.23.3, huggingface-hub 2.4.0, transformers 5.22.1, Jinja2 3.1.6 and torch 2.15.0.
• PyTorch. Get Started: install locally, for the CPU wheel index.
• Qwen. Qwen/Qwen3-0.6B model card, and Function calling in the Qwen documentation (accessed 2026-11-10), which recommends the Hermes format for Qwen3.
• Hugging Face. Text generation, Transformers v5.22.1 documentation, for AutoModelForCausalLM and generate in step 13.
• Google. google/gemma-4-E2B-it model card and Function calling with Gemma 4.
• NousResearch. NousResearch/Meta-Llama-3-8B-Instruct, an ungated copy of Meta's meta-llama/Meta-Llama-3-8B-Instruct (gated). Unsloth. unsloth/Llama-3.1-8B-Instruct, an ungated copy of Meta's meta-llama/Llama-3.1-8B-Instruct (gated), for its tool-calling template and the parallel-call error.
• Unsloth. unsloth/gemma-3-1b-it, an ungated copy of Google's google/gemma-3-1b-it (gated).
• NousResearch. Hermes-Function-Calling, the <tool_call> JSON format the Hermes models introduced.
• vLLM. Tool calling documentation, v0.33.0: --enable-auto-tool-choice, --tool-call-parser and the hf parser, and the tool parser registry in v0.33.0.
• vLLM. Issue #22975: tool calls returned in content.
• hanXen. vllm-qwen2.5-coder-tool-parser: Qwen2.5-Coder's fenced-JSON tool calls, and Qwen. Qwen/Qwen2.5-Coder-7B-Instruct model card.
• NVIDIA Developer Forums. vLLM returns 400 error for tool_choice auto, 2026.
• Model Context Protocol. Specification 2025-11-25: Tools, for CallToolResult and structuredContent.
• Python. typing.Protocol, typing.NotRequired, collections.abc.Callable, dataclasses, re, json, functools.partial, textwrap and pprint, Python 3.12.
• Pallets. Jinja TemplateError, Jinja 3.1.
• Kumar, R. The Chat Templates Handbook, Edition 1.2, chapter 5. Book site: chattemplates.ranjankumar.in.
• Series: Part 1, Part 2, Part 3, Part 4.

Related Articles

Llms

• How to Validate Chat Messages Before apply_chat_template
• How to Render a Chat Template with Jinja Like Transformers
• How to See What apply_chat_template Sends Your Model
• How to Pass reasoning_content Back Through a Chat Template