Ranjan Kumar

1. Introduction

Prompt engineering is the practice of designing and refining the text (prompt) that we pass to a Generative AI (GenAI) model. The prompt acts as an instruction or query, and the model generates responses based on it. Prompts can be questions, statements, or detailed instructions.

Prompt engineering serves three purposes:

1. Enhancing output quality – refining how the model responds.
2. Evaluating model behavior – testing the output against requirements.
3. Ensuring safety – reducing harmful or biased responses.

There is no single “perfect” prompt. Instead, prompt design is an iterative process involving optimization and experimentation.

Figure 1: A basic example of the prompt

2. Controlling Model Output by Adjusting Model Parameters

The behavior of large language models (LLMs) can be fine-tuned using parameters such as temperature, top_p, and top_k. For these to take effect, do_sample=True must be set, allowing the model to sample tokens instead of always choosing the most likely one.

• Temperature controls randomness.
  • temperature=0: deterministic output (always the same response).
  • Higher values → more diverse responses.
  • Example: 0.2 = focused, coherent; 0.8 = more creative.
• Top_p (nucleus sampling) restricts token choices to the smallest set whose cumulative probability ≥ p.
  • top_p=1: consider all tokens.
  • Lower values → more focused output.
• Top_k limits the selection to the k most likely tokens.

By tuning these, one can strike a balance between deterministic/focused and creative/diverse outputs.

3. Instruction-Based Prompting

Instruction-based prompting is one of the most fundamental and widely used approaches in working with large language models (LLMs). It involves providing the model with explicit, structured, and unambiguous instructions that guide how the response should be generated.

At its core, an instruction-based prompt consists of two essential components:

1. Instruction – what the model is supposed to do (e.g., “Summarize the text in one sentence.”).
2. Data – the input on which the instruction operates (e.g., the paragraph to be summarized).

A simple example:

Prompt

Instruction: Summarize the following text in one sentence.  
Data: Artificial Intelligence is revolutionizing industries such as healthcare, finance, and education by automating tasks and enabling data-driven decision-making.  

Output

AI is transforming industries by automating tasks and enabling smarter decisions.  

The following diagram depicts a basic instruction prompt. Please note the instructions and data in the prompt.

Figure 2: Instruction Prompt

3.1 Adding Output Indicators

Sometimes instructions alone are not enough. To make the response more constrained and predictable, we can add output indicators – predefined answer formats or expected categories.

For example:

Prompt

Instruction: Classify the sentiment of the following review.  
Data: “The product is amazing and works perfectly.”  
Output options: Positive | Negative  

Output

Positive  

The following diagram depicts the instruction prompt with an output indicator.

Figure 3: Instruction prompt with output indicators

3.2 Task-Specific Prompt Formats

Different NLP tasks require slightly different instruction structures. For example:

• Summarization: “Summarize the following paragraph in 2–3 sentences.”
• Classification: “Classify the following text as spam or not spam.”
• Named Entity Recognition (NER): “Extract all names of organizations mentioned in the following text and list them as a JSON array.”

These formats not only help the model but also make evaluation easier for humans.

The following diagram illustrates example formats for summarization, classification, and named-entity recognition.

Figure 4: Prompt format for summarization, classification, and NER task

3.3 Prompting Techniques for Better Results

Instruction-based prompting can be improved using several best practices:

• Specificity
  Be as precise as possible. Instead of “Explain photosynthesis”, say “Explain photosynthesis in 3 simple steps, suitable for a 10-year-old student.”
• Hallucination Control
  LLMs sometimes generate plausible but false answers (hallucinations). To mitigate this, include safeguards:
  If you are not sure about the answer, respond with: "I don't know."
• Order Matters (Primacy and Recency Effects)
  LLMs pay more attention to the beginning and end of a prompt. Place the most important instructions either at the start or the end to maximize compliance.

3.4 Beyond the Basics: Modular Prompt Components

While instruction, data, and output indicators are the core building blocks, prompts can be made more modular and adaptive by incorporating additional components such as:

• Personalization – “You are a financial advisor…”
• Instruction – “Summarize, compare, explain…”
• Context – background information that helps the model.
• Format – JSON, table, bullet points, plain text.
• Audience – “Explain as if to a high school student.”
• Tone – formal, casual, persuasive, neutral.
• Data – the actual input for the model to process.

This modularity makes prompts flexible. You can add or remove components depending on the use case, and observe how the output changes.

The following is an example that uses the above prompt components. This example demonstrates the modular nature of prompting. We can experiment by adding or removing components to see the effect.

Figure 5: Example of a prompt showing the use of the various components.

3.5 Example: Modular Prompt

Prompt

Personalization: You are a career coach.  
Instruction: Provide advice on writing a resume.  
Context: The user is applying for a software engineering role at a mid-sized startup.  
Format: List your advice as 5 bullet points.  
Audience: Entry-level job seeker with 1 year of internship experience.  
Tone: Encouraging and practical.  
Data: “How should I improve my resume?”  

Output

1. Highlight your technical skills (Python, JavaScript, databases).  
2. Add details about your internship projects—focus on impact.  
3. Tailor your resume to the startup by showing adaptability and problem-solving.  
4. Keep the format clean, concise, and one page.  
5. Add a short section for personal projects or open-source contributions.  

This demonstrates how instruction-based prompting becomes powerful and adaptable when multiple components are layered together.

4. In-Context Learning – Providing examples

Large Language Models (LLMs) do not “learn” in the traditional sense during inference. Instead, they adapt to patterns given in the prompt itself. This ability to condition their behavior on a few examples provided at runtime is called In-Context Learning (ICL).

4.1 The Idea Behind ICL

By showing the model examples of the task and the desired outputs, we “teach” it on the fly. The model does not change its weights; rather, it uses the examples as a temporary pattern guide to align its responses with the given format.

This makes ICL especially powerful when:

• We don’t want to fine-tune the model.
• Training data for fine-tuning is small or unavailable.
• We want flexibility to change tasks quickly.

4.2 Types of In-Context Learning

1. Zero-shot prompting

• No examples are provided, only instructions.
• Works best when the task is common or well-aligned with the model’s pretraining.
• Example:

Instruction: Translate the following English sentence into French.  
Data: "How are you?"  

Output: “Comment ça va ?”

2. One-shot prompting

• A single example is given to demonstrate the expected behavior.
• Useful when the task requires clarity in format or style.
• Example:

User: Translate the following English sentence into French.  
Example Input: "Good morning" → Example Output: "Bonjour"  
Task Input: "How are you?"  

Output: “Comment ça va ?”

3. Few-shot prompting

• Multiple examples are given before the actual task.
• Works well when tasks are ambiguous or domain-specific.
• Example:

Task: Classify the sentiment of the following reviews as Positive or Negative.  

Review: "I love this phone, the battery lasts long." → Positive  
Review: "The screen cracked within a week." → Negative  
Review: "Excellent sound quality and fast processor." → Positive  

Now classify: "The camera is blurry and disappointing."  

Output: Negative

The following diagram illustrates the examples of in-context learning.

Figure 6: Examples of in-context learning

4.3 Importance of Role Differentiation

When writing few-shot prompts, clearly distinguishing roles (e.g., User: and Assistant: or Q: and A:) helps the model mimic the structure consistently. Without role markers, the model may drift into producing unstructured responses.

For example:

User: What is 2 + 2?  
Assistant: 4  
User: What is 5 + 3?  
Assistant: 8  
User: What is 7 + 6?  
Assistant:

This encourages the model to continue in the same call-and-response pattern.

4.4 Benefits of In-Context Learning

• Flexibility – You can “train” the model on a new task instantly without modifying its parameters.
• Rapid prototyping – Great for testing new use cases before investing in fine-tuning.
• Control – Helps enforce formatting (e.g., JSON, tables, bullet points).

4.5 Limitations of In-Context Learning

• Context length constraints – Too many examples may exceed the model’s context window.
• Random sampling – Even with examples, the model may ignore instructions if randomness (temperature, top_p) is high.
• Cost & latency – Longer prompts = higher compute and inference cost.
• Inconsistency – The same examples may yield slightly different outputs.

4.6 Advanced Variants of ICL

• Instruction + Demonstration Hybrid: Combine explicit task instructions with few-shot examples for stronger guidance.
• Chain-of-Thought with ICL: Provide examples that include reasoning steps, so the model learns to “think out loud” before answering.
• Style Transfer with ICL: Use few-shot examples to enforce a particular writing style (e.g., Shakespearean, academic, casual).

5. Chain Prompting: Breaking up the Problem

When dealing with complex tasks, asking a large language model (LLM) to solve everything in a single prompt often leads to suboptimal results. The model may lose focus, misinterpret requirements, or generate incomplete answers. Chain prompting is a structured strategy where we break down a large problem into smaller subtasks, design prompts for each subtask, and then link them sequentially, passing outputs from one prompt as inputs to the next. This creates a pipeline of prompts that together achieve the final solution.

This approach mirrors how humans naturally solve complex problems—by breaking them into manageable steps rather than attempting everything at once.

5.1 Key Benefits of Prompt Chaining

1. Better Performance
   • By focusing each prompt on a single subtask, the LLM can generate more accurate and high-quality responses.
   • Reduces cognitive overload for the model.
2. Transparency
   • Each intermediate step in the chain is visible and explainable.
   • Makes it easier for developers and users to trace how the final output was constructed.
3. Controllability and Reliability
   • Developers can adjust or fine-tune only the prompts for the weaker subtasks instead of rewriting the entire large prompt.
   • More control over model behavior.
4. Debugging
   • Since outputs are broken into stages, it’s easier to identify where an error occurs and fix it.
5. Incremental Improvement
   • You can evaluate the performance of each subtask independently and selectively improve weak links in the chain.
6. Conversational Assistants
   • Useful for designing chatbots where conversation naturally involves sequential reasoning (e.g., clarifying intent → retrieving information → generating response).
7. Personalization
   • Chains can be designed to collect user preferences at one step and then apply those preferences consistently across subsequent prompts.

5.2 Common Use Cases

1. Response Validation
   • Prompt 1: Generate an answer.
   • Prompt 2: Ask the model (or another model) to evaluate correctness, consistency, or bias in the answer.
   • Example: LLM generates an explanation of a concept, then another LLM verifies if the explanation is factually correct.
2. Parallel Prompts
   • Sometimes, different subtasks can be run simultaneously.
   • Example: One prompt generates a list of features, another generates customer pain points, and later prompts merge them to design marketing copy.
3. Creative Writing / Storytelling
   • Prompt 1: Generate character descriptions.
   • Prompt 2: Use characters to generate a plot outline.
   • Prompt 3: Expand the outline into a full story.
4. Business Use Case – Marketing Flow
   • Step 1 (Prompt 1): Generate a catchy product name.
   • Step 2 (Prompt 2): Using the product name + product features, generate a short slogan.
   • Step 3 (Prompt 3): Using the product name, features, and slogan, generate a full sales pitch.
   • This modular approach ensures the final pitch is consistent, creative, and logically structured.

5.3 Prompt Chain Example

The following example illustrates the prompt chain that first creates a product name, then uses this name with product features to create a slogan, and finally uses features, product name, and slogan to create the sales pitch.

Figure 7: Example of a prompt chain

Step 1 – Product Naming

Instruction: “Suggest a creative name for a new smartwatch that focuses on health tracking and long battery life.”

Output: “PulseMate”

Step 2 – Slogan Generation

Instruction: “Using the product name ‘PulseMate’ and the features (health tracking, long battery life), create a short catchy slogan.”

Output: “PulseMate – Your Health, Powered All Day.”

Step 3 – Sales Pitch

Instruction: “Using the product name ‘PulseMate,’ its slogan ‘Your Health, Powered All Day,’ and the features (health tracking, long battery life), write a compelling sales pitch for customers.”

Output: “Meet PulseMate, the smartwatch designed to keep up with your lifestyle. Track your health seamlessly while enjoying a battery that lasts for days. PulseMate—Your Health, Powered All Day.”

5.4 Variants of Prompt Chaining

• Sequential Chaining – Output of one prompt feeds directly into the next (step-by-step). The above example in Figure 7 demonstrates sequential chaining.
• Branching Chaining – One output is used to create multiple different paths of prompts.
• Merging Chains – Combine results from different parallel chains into a unified final response.
• Iterative Chaining – Loop a prompt multiple times for refinement (e.g., “revise this until it’s concise and clear”).

6. Reasoning with Generative Models

LLMs don’t “reason” like humans. They excel at pattern completion over very large text corpora. With careful prompting, scaffolding, and verification, we can simulate aspects of reasoning and markedly improve reliability.

6.1 System 1 vs. System 2 (Kahneman) — and LLMs

• System 1 (fast, intuitive): In LLMs this looks like single-shot answers, low token budget, low/no deliberation. Good for well-trodden tasks (grammar fixes, casual Q&A).
• System 2 (slow, deliberate): In LLMs this is multi-step prompting, intermediate reasoning, tool use (calculator/RAG), sampling multiple candidates, and verification. Good for math, logic, policy checks, multi-constraint generation, and anything high-stakes.

In practice: choose System 1 for speed/low risk; escalate to System 2 scaffolds when accuracy, traceability, or multi-constraint synthesis matters.

6.2 Techniques to Induce Deliberation

6.2.1 Chain-of-Thought (CoT): “Think before answering”

Elicit intermediate reasoning steps prior to the final answer.

Zero-shot CoT trigger (minimal):

You are solving a reasoning task.
First, think step-by-step in brief bullet points.
Then, give the final answer on a new line prefixed with "Answer:".
Question: <problem>

Few-shot CoT (when format matters): include 1–3 worked examples showing short, crisp reasoning and a clearly marked Answer line.

Tips

• Keep thoughts succinct to reduce cost and drift.
• For production UIs, you can ask the model to hide the rationale and output only the final answer + a confidence or citation list (see “Reasoning privacy” below).

When to use: arithmetic/logic puzzles, planning, constraint satisfaction, data transformation with edge cases.

The following figure demonstrates standard prompting vs C-o-T Prompting:

Figure 9: Chain-of-thought example; reasoning process is highlighted – source [3]

The following is an example of zero-shot chain-of-thought.

Figure 10: Example of zero-shot chain-of-thought – source[1]

6.2.2 Self-Consistency: sample multiple rationales and vote

Rather than trusting the first reasoning path, sample k solutions and aggregate.

Template

Task: <problem>

Instruction:
Generate a short, step-by-step rationale and final answer.
Vary your approach each time.

[Run this prompt k times with temperature ~0.7–1.0]
Aggregator:
- Extract the final answer from each sample.
- Choose the majority answer (tie-break: pick the one supported by the clearest rationale).
- Return "Final:" <answer> and "Support count:" <m/n>.

Practical defaults

• k = 5–15 (trade accuracy vs. latency/cost)
• temperature: 0.7–1.0
• top_p: 0.9–1.0

When to use: problems with one correct output but many valid reasoning paths (math, logical deduction, label inference).

The following diagram illustrates the concept of self-consistency.

Figure 11: Example of self-consistency in CoT[4]

6.2.3 Tree of Thoughts (ToT): explore and evaluate branches

Generalizes CoT into a search over alternative “thoughts” (states). You expand multiple partial solutions, score them, prune weak ones, and continue until a budget is reached.

Lightweight ToT pseudo-workflow

state0 = problem description
frontier = [state0]

for depth in 1..D:
  candidates = []
  for s in frontier:
    thoughts = LLM("Propose 2-3 next-step thoughts for: " + s)
    for t in thoughts:
      v = LLM("Rate this partial approach 1-5 for promise. Be strict.\nThought: " + t)
      candidates.append((t, v))
  frontier = top_k(candidates, by=v, k=K)

best = argmax(frontier, by=v)
answer = LLM("Given this best chain of thoughts, produce the final answer:\n" + best)

Tuning knobs

• D (max depth), K (beam width), value function (how you score thoughts), and token budget.
• Use “look-ahead” prompts: “Simulate next two steps; if dead-end, backtrack.”

When to use: multi-step planning (itineraries, workflows), puzzle solving, coding strategies, complex document transformations.

The following diagram illustrates the various approaches to problem-solving with LLMs. Each rectangular box represents a thought.

Figure 12: Various approaches to problem-solving with LLMs.

6.2.4 Related, practical reasoning scaffolds

• ReAct (Reason + Act): Interleave “Thought → Action (tool call/RAG) → Observation” until done. Great for tasks that need tools, search, or databases.
• Program-of-Thoughts (PoT): Ask the model to output code (e.g., Python) to compute the answer; execute it; return result. Excellent for math, data wrangling, and reproducibility.
• Debate / Critic-Judge: Have model A propose an answer, model B critique it, and a judge (or the same model) select/merge. Pairs well with self-consistency.
• Plan-then-Execute: Prompt 1 creates a plan/checklist; Prompt 2 executes step by step; Prompt 3 verifies outputs against the plan.
• Retrieval-Augmented Reasoning: Prepend cited context (docs, policies) and require grounded (“quote-and-justify”) answers.

6.3 Putting it together: a robust System-2 pipeline

Use case: Policy compliance check for marketing copy.

1. Extract constraints (CoT):
   “List policy rules relevant to social ads, each with an ID and short paraphrase.”
2. Assess violations (ReAct/PoT):
   For each rule, analyze the ad text; return pass|fail with span references.
3. Self-consistency vote:
   Sample assessments 7× and majority-vote each rule outcome.
4. Summarize & justify:
   Compose a final verdict with a table of rules, decisions, and cited spans.
5. Verifier pass:
   A separate prompt re-checks logical consistency and that every failure has evidence.
6. Guarded output:
   Enforce schema (JSON) and redact PII (Redacting or identifying personally identifiable information).

This gives you accuracy (deliberation), transparency (artifacts per step), and control (schema + verifier).

6.4 Operational Guidance

6.4.1 Prompt templates

CoT (short)

Solve the problem. First give 3-5 brief reasoning bullets. 
Then output the final result as: "Answer: <value>".
Question: <...>

Self-consistency runner (controller code)

answers = []
for i in range(k):
  ans = call_llm(prompt, temperature=0.8, top_p=0.95)
  answers.append(extract_final(ans))
final = majority_vote(answers)

ReAct skeleton

Thought: I need the latest spec section.
Action: search("<query>")
Observation: <top snippet>
Thought: Summarize the relevant passage and apply the rule.
...
Final Answer: <concise verdict + citation>

ToT node expansion

Propose 3 distinct next-step ideas to advance the solution.
For each: give a one-sentence rationale and a 1-5 promise score.
Return JSON: [{"idea":..., "rationale":..., "score":...}]

6.5 Evaluation & QA

Once we design and deploy prompts, evaluation and quality assurance (QA) become critical. Unlike traditional software, where behavior is deterministic, LLM outputs are probabilistic and context-dependent. This means even well-designed prompts may fail in certain conditions. A structured evaluation strategy helps measure reliability, accuracy, and efficiency of your prompt-engineering pipeline.