Appendix A: ChatML Syntax Reference
Complete Markup Specification and Role Semantics
This appendix serves as a comprehensive technical reference for the ChatML format — the markup language that structures communication between conversational roles in large language model (LLM) systems.
It documents the syntax, role definitions, token semantics, and usage rules that enable structured, reproducible dialogue across the ChatML ecosystem.
Drawing examples from the Project Support Bot, this reference consolidates all constructs, conventions, and best practices used throughout the previous chapters.
ChatML, LLMs, Prompt Engineering, LangChain, LlamaIndex
Appendix A: ChatML Syntax Reference
A.1 Introduction: The Purpose of ChatML Syntax
ChatML (Chat Markup Language) defines a universal, machine-readable format for structuring conversations between multiple roles —
such as system, user, assistant, and tool.
Its goals are:
- Structure — Explicit role tagging and message segmentation
- Hierarchy — Ordered conversational flow
- Reproducibility — Same input yields same output
- Interoperability — Language-agnostic format for pipelines, templates, and tools
Every message in a ChatML transcript is enclosed within start and end markers that identify its role and content boundaries.
A.2 Core Message Structure
A ChatML message has the following canonical structure:
<|im_start|>role
content
<|im_end|>Example
<|im_start|>user
List open issues for Sprint 5.
<|im_end|>Each block corresponds to a single logical message in the conversation sequence.
A.3 Syntax Markers
| Marker | Meaning | Usage Example |
|---|---|---|
<|im_start|> |
Start of a message block | <|im_start|>user |
<|im_end|> |
End of a message block | ...<|im_end|> |
<|system|> |
Optional system role alias (legacy form) | <|system|>Initialize environment |
<|tool_call|> |
Tool invocation section (optional metadata marker) | <|tool_call|>fetch_data(...) |
<|metadata|> |
For storing structured metadata | <|metadata|>{"id":"1234"} |
<|im_sep|> |
Optional separator between messages for streaming | <|im_sep|> |
Note: Only
<|im_start|>and<|im_end|>are mandatory.
All other markers are extensions for specialized contexts.
A.4 Role Semantics
Each role in ChatML has specific behavioral semantics that guide the model or system in processing conversation flow.
| Role | Description | Example |
|---|---|---|
| system | Establishes context, policy, and global instructions | “You are a project management assistant.” |
| user | Initiates requests, provides input | “Summarize Sprint 5 progress.” |
| assistant | Generates responses, reasoning, or summaries | “Sprint 5 completed with velocity 42.” |
| tool | Represents function execution, external API calls, or actions | fetch_jira_tickets(sprint="5") |
| memory (optional) | Stores persistent state or retrieved context | “Previous sprint summary loaded.” |
| critic (optional) | Evaluates assistant output for quality assurance | “Response exceeds max word count.” |
| system_monitor (optional) | Captures runtime metrics and observability logs | “Latency: 1.4s |
Tip: Roles should be deterministic and non-overlapping. Each message block must belong to exactly one role.
A.5 Example: Multi-Role Conversation
<|im_start|>system
You are a project support assistant that helps manage sprints.
<|im_end|>
<|im_start|>user
Generate sprint summary for Sprint 5.
<|im_end|>
<|im_start|>assistant
Fetching sprint data...
<|im_end|>
<|im_start|>tool
fetch_jira_tickets(sprint="Sprint 5")
<|im_end|>
<|im_start|>tool
{"total_tickets": 18, "open": 3, "closed": 15}
<|im_end|>
<|im_start|>assistant
Sprint 5 completed with 15 issues closed and velocity of 42 points.
<|im_end|>This example shows the canonical ChatML dialogue loop: 
A.6 Message Ordering and Hierarchy Rules
ChatML enforces a hierarchical flow:
- system message appears once at the start (sets policy)
- user message introduces intent
- assistant performs reasoning or decides on next action
- tool executes computation or retrieval
- assistant summarizes tool output or answers query
A.6 Closing Summary
ChatML’s syntax offers clarity, reproducibility, and hierarchy in LLM communication.
By using simple but strict markers and role semantics, it enables structured reasoning pipelines, multi-agent orchestration, and full auditability.
| Concept | Description |
|---|---|
| Markers | Define start/end of structured message blocks |
| Roles | Represent conversational agents or systems |
| Metadata | Capture execution context |
| Validation | Enforces reproducibility and correctness |
| Replay | Enables memory persistence and auditing |
In essence, ChatML is not merely a markup language — it is the contract of structured intelligence, ensuring that every LLM conversation remains interpretable, testable, and reproducible across systems and time.