Hands-on Tutorial on Making an Audio Bot using LLM, and RAG

Updated 26 September 2026: the code runs again. qdrant-client 1.19 removed the add() and query() shortcuts this tutorial was built on, so ingestion now creates the collection explicitly and both scripts go through upsert() and query_points(). LangChain 1.x moved CharacterTextSplitter into its own langchain-text-splitters package. That package is now in the requirements, together with langchain-community, which the downloadable file had been missing. If you ingested with the earlier version, the updated scripts delete that old collection and rebuild it, because the old add() stored vectors in a layout the new calls cannot write to.

1. Learning Outcome

1. Learn about Large Language Models (LLMs), their installation, access through HTTP API, the Ollama framework, etc.

2. Introduction of Retrieval-Augmented Generation (RAG)

3. Learn about the Data Ingestion Pipeline for the Qdrant vector database

4. Learn about the RAG Pipeline

5. Access the prototype using audio-based input/output (audio bot).

6. Audio bot using speech-to-text and text-to-speech

7. Making Qdrant client for making queries

8. Creating context using documents retrieved from the Qdrant database

9. Using Llama 3.2 as a Large Language Model (LLM) using Ollama and Langchain framework

10. Create a prompt template using instruction and context, and make a query using the template and langchain

11. Using Llama to generate the answer to the question using the given context

2. Large Language Model

Large Language Model (LLM) is an artificial intelligence (AI) model with billions of parameters, trained on huge amounts of data to comprehend (human-generated text understanding) and produce language similar to a human's (text generation comparable to that generated by humans). LLMs have learned linguistic structures, relationships, and patterns in human-generated data. It has also gained a huge amount of internal knowledge (in the form of model weights) from the data on which these models have been trained.

Transformer architecture is often the foundation of large language models, allowing them to

1. process sequential data that makes it suitable for tasks like text generation, translation, and question-answering)

2. learn contextual relationships such as word meanings, syntax, and semantics.

3. generate human-like language, i.e., produce coherent, context-specific text that resembles human-generated content

Some key characteristics of large language models include:

1. Trained on vast training data:

2. Scalability: They can handle long sequences and complex tasks.

3. Parallel processing

direction: down

data: "Extensive dataset" {
  shape: cylinder
  style: { fill: "#95A5A6"; stroke: "#6B7B7C"; font-color: "#FFFFFF" }
}

llm: "LLM: billions of parameters" {
  grid-columns: 4
  style: { fill: "#EAF2FB"; stroke: "#2C6FB0"; font-color: "#2C2C2A" }
  u: "Understanding" { style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
  g: "Generation" { style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
  i: "In-context\nlearning" { style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
  k: "Internal\nknowledge" { style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
}

domains: "Fine-tuned for" {
  grid-rows: 3
  style: { fill: "#EAF7F3"; stroke: "#4E9E8B"; font-color: "#2C2C2A" }
  h: "Health" { style: { fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" } }
  e: "Education" { style: { fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" } }
  l: "Law enforcement" { style: { fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" } }
}

families: "Transformer families" {
  grid-columns: 3
  style: { fill: "#F3F0FD"; stroke: "#5A4ECC"; font-color: "#2C2C2A" }
  enc: "Encoder only\nBERT\n\nsentiment\nreview reading\nclassification" {
    style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF" }
  }
  dec: "Decoder only\nGPT\n\ntext generation\ncode generation\ncreative writing" {
    style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF" }
  }
  ed: "Encoder-decoder\nT5\n\ntranslation\nsummarization\nquestion answering" {
    style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF" }
  }
}

data -> llm: "trained on"
llm -> domains: "fine-tune"
llm -> families: "built on"

Transformer language models fall into three families: 1) encoder only, 2) decoder only, and 3) encoder-decoder models. Examples are BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), and T5 (Text-to-Text Transfer Transformer), respectively, and the diagram above shows their typical uses. Today's LLMs (GPT, Llama) are decoder-only. BERT-style encoders are what you reach for to make embeddings, and the bge-small-en model used later in this tutorial is one of them.

Major concerns regarding LLMs are:

1. Data bias: LLMs have the potential to reinforce biases found in the training data.

2. Interpretability: It can be difficult to comprehend the decision-making process of an LLM.

3. Security: Adversarial attacks can target LLMs.

2.1 Llama

Llama is a popular open-weight LLM from Meta, released under the Llama Community License. Because the weights are available, we can fine-tune, distill, and deploy it in our own use cases (provided the above-mentioned concerns are taken care of). Llama is a decoder-only language model, which means that it uses a transformer architecture with only decoder layers to generate text.

When this tutorial was first written (January 2025), the current versions came in three flavors:

1. Llama 3.1: With Multilingual capability and available in three sizes: 8B, 70B and 405B parameters

2. Llama 3.2: Lightweight and Multimodal and available in 1) Lightweight 1B and 3B 2) Multimodal 11B and 90B

3. Llama 3.3: Multilingual with 70B parameters

Meta has since released Llama 4. This prototype still uses Llama 3.2 (the default llama3.2 tag, 3B), because it runs comfortably on a laptop CPU.

2.2 Install, run, and different ways to access Llama

Please refer to my separate post on this topic, Install, run, and access Llama using Ollama.

3. Retrieval-Augmented Generation (RAG)

Large language models are locked in time. It has learned the knowledge that was available till the time when it was being trained and released. These models are trained on Internet-scale open data. So when you ask general questions, it would give very good answers, but it may fail to answer or hallucinate if you go very specific to your personal or enterprise data. The reason is that it usually does not have the right context of your requirements that are very specific to your application.

Retrieval-augmented generation (RAG) combines the strength of generative AI and retrieval techniques. It helps in providing the right context for the LLM along with the question being asked. This way we get the LLM to generate more accurate and relevant content. This is a cost-effective way to improve the output of LLMs without retraining them. The following diagram depicts the RAG architecture:

shape: sequence_diagram

user: "User" { style: { fill: "#95A5A6"; stroke: "#6B7B7C"; font-color: "#FFFFFF" } }
app: "Application" { style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
search: "Search relevant\ninformation" { style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF" } }
sources: "Vector database\n(indexed knowledge sources)" { style: { fill: "#FFD93D"; stroke: "#B8860B"; font-color: "#2C2C2A" } }
llm: "Large language\nmodel" { style: { fill: "#6BCF7F"; stroke: "#3E9A52"; font-color: "#2C2C2A" } }

user -> app: "1. query"
app -> search: "2. query"
search -> search: "2a. embed the query"
search -> sources: "2b. nearest-vector search"
sources -> search: "2c. matching chunk text"
search -> app: "3. relevant information for enhanced context"
app -> llm: "4. prompt + query + enhanced context"
llm -> app: "5. generated text response"
app -> user: "6. answer"

There are two primary components of the RAG:

1. Retrieval: This component is responsible for searching and retrieving relevant information related to the user's query from various knowledge sources such as documents, articles, databases, etc.

2. Generation: This component does an excellent job of crafting coherent and contextually rich responses to user queries.

A question submitted by the user is routed to the retrieval component. Using the embedding model, the retrieval component converts the query text to the embedding vector. After that, it looks through the vector database to locate a small number of vectors that match the query text and satisfy the threshold requirements for the similarity score and distance metric. Each matching vector is stored alongside the original text it was made from, and that stored text becomes the context. This context, along with the prompt and query, is put in the prompt template and sent to the LLM. LLM returns the generated text that is more correct and relevant to the user's query.

4. RAG (Data Ingestion Pipeline)

In order for the retrieval component to have a searchable index of preprocessed data, we must first build a data input pipeline. The following diagram in fig2 depicts the data ingestion pipeline. Knowledge sources can be web pages, text documents, pdf documents, etc. Texts need to be extracted from these sources. I am using PDF documents as the only knowledge source for this prototype.

1. Text Extraction: To extract the text from the PDF, various Python libraries can be used, such as PyPDF2, pdf2txt, PDFMiner, etc. If PDF is scanned PDF, libraries such as unstructured, pdf2image, and pytesseract can be utilized. The quality of the text can be maintained by performing cleanups such as removing extraneous characters, fixing formatting issues, whitespace, special characters, punctuation, spell checking, etc. Language detection may also be required if knowledge sources can have text coming in multiple languages, or a single document may contain multiple languages.

2. Handling Multiple Pages: Maintaining the context across pages is important. It is recommended that the document be segmented into logical units, such as paragraphs or sections, to preserve the context. Extracting metadata such as document titles, authors, page numbers, creation dates, etc., is crucial for improving searchability and answering user queries.

grid-rows: 2
grid-columns: 3
horizontal-gap: 150
vertical-gap: 110

sources: "Knowledge sources\nweb pages, text, PDFs" {
  shape: page
  width: 240; height: 110
  style: { multiple: true; fill: "#FFD93D"; stroke: "#B8860B"; font-color: "#2C2C2A" }
}
chunks: "Chunks of text\n(target 1,000 chars, 30 overlap)" {
  width: 240; height: 110
  style: { multiple: true; fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" }
}
embed: "Embedding model\nFastEmbed, BAAI/bge-small-en" {
  width: 260; height: 110
  style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF" }
}
gap: "" { style: { opacity: 0 } }
vdb: "Vector database\n(Qdrant)" {
  shape: cylinder
  width: 240; height: 130
  style: { fill: "#6BCF7F"; stroke: "#3E9A52"; font-color: "#2C2C2A" }
}
point: "One point per chunk" {
  grid-rows: 3
  vertical-gap: 8
  style: { fill: "#EAF2FB"; stroke: "#2C6FB0"; font-color: "#2C2C2A" }
  id: "id: UUID (file + chunk no.)" { width: 260; style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
  vector: "vector: 384 floats" { width: 260; style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
  payload: "payload: chunk text" { width: 260; style: { fill: "#FFA07A"; stroke: "#D9774F"; font-color: "#2C2C2A" } }
}

sources -> chunks: "extract, split"
chunks -> embed: "each chunk"
embed -> point: "embed"
point -> vdb: "store"

Note: the knowledge source is the book "Democratic Politics" of class IX of the NCERT curriculum. Download its chapter PDFs from the NCERT textbooks page (Class IX, Social Science, Democratic Politics) and put them in a folder named ix-sst-ncert-democratic-politics next to your scripts. Run every command from the folder that contains it.

4.1 Implementation step by step

Each step below shows one piece of the pipeline on its own; to actually run it, use the complete script in 4.2.

Step 1: Install the necessary libraries

1. pip install pdfminer.six

2. pip install langchain-ollama

3. pip install langchain-community langchain-text-splitters "qdrant-client[fastembed]"

Imports:

from langchain_community.document_loaders import PDFMinerLoaderfrom langchain_text_splitters import CharacterTextSplitterfrom qdrant_client import QdrantClient, models

Step 2: Load the pdf file and extract the text from it

path = "ix-sst-ncert-democratic-politics"file_name = "iess401.pdf"  # any one chapter PDF from that folderloader = PDFMinerLoader(path + "/" + file_name)pdf_content = loader.load()

Step 3: Split the text into smaller chunks with overlap

CHUNK_SIZE = 1000 # target size; a paragraph longer than this stays wholeCHUNK_OVERLAP = 30 # a bit of overlap is required for continued contexttext_splitter = CharacterTextSplitter(chunk_size=CHUNK_SIZE, chunk_overlap=CHUNK_OVERLAP)docs = text_splitter.split_documents(pdf_content)# Make a list of split docsdocuments = []for doc in docs:    documents.append(doc.page_content)

Step 4: Embed and store the documents in the vector database

[FastEmbed](https://github.com/qdrant/fastembed) is a lightweight, fast Python library built for embedding generation. Qdrant's Python client calls it to turn text into vectors whenever you pass a models.Document. You pick the model. Here it is BAAI/bge-small-en, and queries have to use the same one or the scores mean nothing. Create the collection before the first insert. Following is the code snippet for inserting in the vector database.

import uuidCOLLECTION = "ix-sst-ncert-democratic-politics"MODEL = "BAAI/bge-small-en"  # use the same embedding model when querying# 3. Create vectordatabase(qdrant) client and the collectionqdrant_client = QdrantClient(url="http://localhost:6333")if qdrant_client.collection_exists(COLLECTION):    vectors = qdrant_client.get_collection(COLLECTION).config.params.vectors    if isinstance(vectors, dict):  # named vectors: left by this post's 2025 version        qdrant_client.delete_collection(COLLECTION)if not qdrant_client.collection_exists(COLLECTION):    qdrant_client.create_collection(        COLLECTION,        vectors_config=models.VectorParams(            size=qdrant_client.get_embedding_size(MODEL),            distance=models.Distance.COSINE,        ),    )# 4. Add document chunks in vectordbqdrant_client.upsert(    COLLECTION,    points=[        models.PointStruct(            # same id on every run, so re-running updates instead of duplicating            id=str(uuid.uuid5(uuid.NAMESPACE_URL, f"{file_name}:{idx}")),            vector=models.Document(text=text, model=MODEL),            payload={"document": text},        )        for idx, text in enumerate(documents)    ],)

Step 5: Making a sample query

# 5. Make a query from the vectordb(qdrant)search_results = qdrant_client.query_points(    COLLECTION,    query=models.Document(text="What is democracy?", model=MODEL),).pointsfor search_result in search_results:    print(search_result.payload["document"], search_result.score)

4.2 Complete Code data_ingestion.py

################################################################ Data ingestion pipeline# 1. Taking the input pdf file# 2. Extracting the content# 3. Divide into chunks# 4. Use embeddings model to convet to the embedding vector# 5. Store the embedding vectors to the qdrant (vector database)################################################################import osimport uuidfrom langchain_community.document_loaders import PDFMinerLoaderfrom langchain_text_splitters import CharacterTextSplitterfrom qdrant_client import QdrantClient, modelspath = "ix-sst-ncert-democratic-politics"filenames = next(os.walk(path))[2]COLLECTION = "ix-sst-ncert-democratic-politics"MODEL = "BAAI/bge-small-en"  # use the same embedding model when querying# 3. Create vectordatabase(qdrant) client and the collection (once)qdrant_client = QdrantClient(url="http://localhost:6333")if qdrant_client.collection_exists(COLLECTION):    vectors = qdrant_client.get_collection(COLLECTION).config.params.vectors    if isinstance(vectors, dict):  # named vectors: left by this post's 2025 version        qdrant_client.delete_collection(COLLECTION)if not qdrant_client.collection_exists(COLLECTION):    qdrant_client.create_collection(        COLLECTION,        vectors_config=models.VectorParams(            size=qdrant_client.get_embedding_size(MODEL),            distance=models.Distance.COSINE,        ),    )for i, file_name in enumerate(filenames):    print(f"Data ingestion for the chapter: {i}")    # 1. Load the pdf document and extract text from it    loader = PDFMinerLoader(path + "/" + file_name)    pdf_content = loader.load()    print(pdf_content)    # 2. Split the text into small chunks    CHUNK_SIZE = 1000 # target size; a paragraph longer than this stays whole    CHUNK_OVERLAP = 30 # a bit of overlap is required for continued context    text_splitter = CharacterTextSplitter(chunk_size=CHUNK_SIZE, chunk_overlap=CHUNK_OVERLAP)    docs = text_splitter.split_documents(pdf_content)    # Make a list of split docs    documents = []    for doc in docs:        documents.append(doc.page_content)    # 4. Add document chunks in vectordb    qdrant_client.upsert(        COLLECTION,        points=[            models.PointStruct(                # same id on every run, so re-running updates instead of duplicating                id=str(uuid.uuid5(uuid.NAMESPACE_URL, f"{file_name}:{idx}")),                vector=models.Document(text=text, model=MODEL),                payload={"document": text},            )            for idx, text in enumerate(documents)        ],    )    # 5. Make a query from the vectordb(qdrant)    search_results = qdrant_client.query_points(        COLLECTION,        query=models.Document(text="What is democracy?", model=MODEL),    ).points    for search_result in search_results:        print(search_result.payload["document"], search_result.score)

Qdrant Dashboard can be accessed at: http://localhost:6333/dashboard

5. RAG (Information Retrieval and Generation) - Audio Bot

I am making an audio bot that will answer questions from the chapters of the book "Democratic Politics" of class IX of the NCERT(India) curriculum. If you want to learn about making an audio bot, you can read my article on the topic "Making a talking bot using Llama3.2:1b running on Raspberry Pi 4 Model-B 4GB".

5.1 Audio Bot Implementation

The following diagram depicts the overall flow of the audio bot and how it interacts with the RAG system. A user interacts with the audio bot using the microphone. The microphone captures the speech audio signal and passes it on to the speech-to-text library (I am using faster_whisper) which in turn converts to a text query that is in turn passed on to the RAG system as a query. When the RAG system comes up with the response text, this text is passed on to the text-to-speech library (I am using pyttsx3) that in turn converts text to audio which is then played by the speaker so the user can listen to the response.

grid-rows: 2
grid-columns: 3
horizontal-gap: 170
vertical-gap: 110

user: "User" { width: 190; height: 80; style: { fill: "#95A5A6"; stroke: "#6B7B7C"; font-color: "#FFFFFF" } }
mic: "Microphone" { width: 190; height: 80; style: { fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" } }
stt: "Speech-to-text\n(faster-whisper)" { width: 190; height: 80; style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
spk: "Speaker" { width: 190; height: 80; style: { fill: "#98D8C8"; stroke: "#4E9E8B"; font-color: "#2C2C2A" } }
tts: "Text-to-speech\n(pyttsx3)" { width: 190; height: 80; style: { fill: "#4A90E2"; stroke: "#2C6FB0"; font-color: "#FFFFFF" } }
rag: "RAG system" { width: 190; height: 80; style: { fill: "#7B68EE"; stroke: "#5A4ECC"; font-color: "#FFFFFF"; bold: true } }

user -> mic: "speech"
mic -> stt: "audio signal"
stt -> rag: "query text"
rag -> tts: "response text"
tts -> spk: "audio signal"
spk -> user: "spoken answer"

5.1.1 Recording Audio from Microphone

I have a detailed blog on this topic. Please refer: how to record audio from a microphone in Python, which covers device selection and the silence checks that catch a dead microphone early.

5.1.2 Speech-to-Text

[faster-whisper](https://github.com/SYSTRAN/faster-whisper) is a reimplementation of OpenAI's Whisper model using [CTranslate2](https://github.com/OpenNMT/CTranslate2/), which is a fast inference engine for Transformer models.

Installation: pip install faster-whisper

Save the following code in Python file say "speech-to-text.py" and run python speech-to-text.py

from faster_whisper import WhisperModelmodel_size = "small.en"model = WhisperModel(model_size, device="cpu",                      compute_type="int8")# Transcribe. vad_filter skips silence, so a silent recording returns no textsegments, _ = model.transcribe(    audio="my_question.wav",    language="en",    vad_filter=True,)# A longer recording comes back as several segments; keep them allseg_text = "".join(segment.text for segment in segments).strip()print(seg_text)

Set audio to any WAV recording of your voice: the recorder from 5.1.1 makes one, and so does 5.1.3, which saves speech.wav.

Output: whatever was said in the recording, as text.

5.1.3 Text-to-Speech

For offline text-to-speech, “pyttsx3” is the simplest option. It wraps the speech engine your operating system already ships (SAPI5 on Windows, NSSpeechSynthesizer on macOS, eSpeak on Linux), which is why it runs happily on a small device like a Raspberry Pi. Want more natural offline voices? Look at Piper.

Installation: pip install pyttsx3

Save the following code in a Python file say "text-to-speech.py" and run python text-to-speech.py

Code Snippet:

import pyttsx3engine = pyttsx3.init()engine.setProperty('volume', 1)engine.setProperty('rate', 130)voices = engine.getProperty('voices')if len(voices) > 1:  # use the second installed voice when there is one    engine.setProperty('voice', voices[1].id)text_to_speak = "I got your question. Please bear " \    "with me while I retrieve the answer."engine.say(text_to_speak)# Folloing is the optional line: If you want # also to save audio fileengine.save_to_file(text_to_speak, 'speech.wav') engine.runAndWait()

Sample input text: “I got your question. Please bear with me while I retrieve the answer.”

5.1.4 Play Audio on the Speaker

I have a detailed blog on this topic. Please refer: playing a WAV file back in Python.

5.2 RAG (Generation) using LLM (Llama3.2)

This snippet queries Qdrant and keeps at most two chunks scoring 0.8 or higher, with Qdrant itself enforcing both limits through limit and score_threshold.

from qdrant_client import QdrantClient, modelsqdrant_client = QdrantClient(url="http://localhost:6333")query = "What is democracy?"search_results = qdrant_client.query_points(    "ix-sst-ncert-democratic-politics",    query=models.Document(text=query, model="BAAI/bge-small-en"),    limit=2,    score_threshold=0.8,).pointscontext = "\n\n".join(result.payload["document"] for result in search_results)

Why 0.8 and not something lower? Scores from bge-small-en bunch up high. In a quick check, a passage about baking bread still scored about 0.7 against "What is democracy?", while the passage that actually answered it scored 0.92. Set it at 0.7 and almost anything gets through. Treat 0.8 as a starting point: run a few questions you know the answers to and look at the scores before settling on a value for your own documents.

The following code snippet creates a template, the template is used to create the prompt, create a reference to the llama model, chain (langchain pipeline for executing the query) is created using prompt and model, and finally chain is invoked to execute the query to get the response formed by LLM using the retrieved context.

from langchain_core.prompts import ChatPromptTemplatefrom langchain_ollama.llms import OllamaLLMtemplate = """Instruction: {instruction}Context: {context}Query: {query}"""prompt = ChatPromptTemplate.from_template(template)model = OllamaLLM(model="llama3.2") # Using llama3.2 as llm modelchain = prompt | modelbot_response = chain.invoke({    "instruction": """Answer the question based on the context below.If you cannot answer the question with the given context,answer with "I don't know."""",    "context": context,    "query": query})

5.3 Complete Code audiobot.py

Following is the code snippet for the audio bot. Save the file as audiobot.py

import pyaudioimport waveimport pyttsx3from qdrant_client import QdrantClient, modelsfrom langchain_ollama.llms import OllamaLLMfrom langchain_core.prompts import ChatPromptTemplatefrom faster_whisper import WhisperModel# Load the Speech to Text Model (faster-whisper: pip install faster-whisper)whishper_model_size = "small.en"whishper_model = WhisperModel(whishper_model_size, device="cpu",                      compute_type="int8")CHUNK = 512 FORMAT = pyaudio.paInt16 #paInt8CHANNELS = 1RATE = 44100 #sample rateRECORD_SECONDS = 7WAVE_OUTPUT_FILENAME = "pyaudio-output.wav"EMBEDDING_MODEL = "BAAI/bge-small-en"  # must match data_ingestion.pydef speak(text_to_speak):    engine = pyttsx3.init()    engine.setProperty('volume', 1)    engine.setProperty('rate', 130)    voices = engine.getProperty('voices')    if len(voices) > 1:  # use the second installed voice when there is one        engine.setProperty('voice', voices[1].id)    engine.say(text_to_speak)    engine.runAndWait()speak("I am an AI bot. I have learned the book \"democratic politics\" of class 9 published by N C E R T. You can ask me questions from this book.")while True:    speak("I am listening now for you question.")    p = pyaudio.PyAudio()    stream = p.open(format=FORMAT,                    channels=CHANNELS,                    rate=RATE,                    input=True,                    frames_per_buffer=CHUNK) #buffer    print("* recording")    frames = []    for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):        data = stream.read(CHUNK)        frames.append(data) # 2 bytes(16 bits) per channel    stream.stop_stream()    stream.close()    p.terminate()    wf = wave.open(WAVE_OUTPUT_FILENAME, 'wb')    wf.setnchannels(CHANNELS)    wf.setsampwidth(p.get_sample_size(FORMAT))    wf.setframerate(RATE)    wf.writeframes(b''.join(frames))    wf.close()    # Transcribe: keep every segment, and let vad_filter drop silence    segments, _ = whishper_model.transcribe(        audio=WAVE_OUTPUT_FILENAME,        language="en",        vad_filter=True,    )    seg_text = "".join(segment.text for segment in segments).strip()    print(f'\nUser: {seg_text}')    if seg_text == '':        speak("Probably you did not say anything.")        continue    else:        text_to_speak = "I got your question. Please bear with me " \            + "while I retrieve about the answer."        speak(text_to_speak)    # 1. Create vector database(qdrant) client     qdrant_client = QdrantClient(url="http://localhost:6333")    # 2. Make a query to the vectordb (qdrant)    #query = "explain democracy in estonia?"    query = seg_text    # At most 2 chunks, each scoring at least 0.8 (see 5.2)    search_results = qdrant_client.query_points(        "ix-sst-ncert-democratic-politics",        query=models.Document(text=query, model=EMBEDDING_MODEL),        limit=2,        score_threshold=0.8,    ).points    for search_result in search_results:        print(f"Retrieved document: {search_result.payload['document']}, "              f"Similarity score: {search_result.score}")    context = "\n\n".join(r.payload["document"] for r in search_results)    #print(f"Retrieved Context: {context}")    if context == "":        print("Context is blank. Could not find any relevant information in the given sources.")        speak("I did not find anything in the book about the question.")        continue    # 4. Using LLM for forming the answer    template = """Instruction: {instruction}    Context: {context}    Query: {query}    """    prompt = ChatPromptTemplate.from_template(template)    model = OllamaLLM(model="llama3.2") # Using llama3.2 as llm model    chain = prompt | model    bot_response = chain.invoke({        "instruction": """Answer the question based on the context below.         If you cannot answer the question with the given context,         answer with "I don't know."        """,                 "context": context,                "query": query        })    print(f'\nBot: {bot_response}')     speak(bot_response)

6. Libraries Used

Following is the list of libraries used in the prototype implementation. These can be installed from the Python pip command.

requirements.txtDownload

1. qdrant-client[fastembed]>=1.19,<2

2. pdfminer.six

3. pyaudio

4. pyttsx3

5. langchain-ollama>=1.0,<2

6. faster-whisper>=1.1,<2

7. langchain-community>=0.4,<1

8. langchain-text-splitters>=1.0,<2

Lower bounds are the versions this code needs; upper bounds stop a future major release from breaking it again. FastEmbed comes in through qdrant-client[fastembed], so it needs no line of its own.

7. Steps

1. Create env python -m venv env1

2. Activate using the activate command in env1\Scripts\activate if on Windows. Activate command is there in the bin directory on the Linux system.

3. python -m pip install -r requirements.txt

4. Start Qdrant (needs Docker): docker run -p 6333:6333 qdrant/qdrant, then check that http://localhost:6333/dashboard opens

5. Install Ollama and pull the model: ollama pull llama3.2

6. Put the chapter PDFs in a folder named ix-sst-ncert-democratic-politics next to the scripts

7. python data_ingestion.py

8. python audiobot.py

8. My Conversation with the Audio Bot

Recorded in January 2025 with the original version of this code. The updated code keeps the same models, prompt and threshold, so the bot behaves the same way.

9. Further Improvement

In the current prototype, the chunk size is of fixed length, CHUNK_SIZE = 1000 and CHUNK_OVERLAP = 30. For further improvement, the document can be split into logical units, such as paragraphs/sections, to maintain a better context.

10. References

1. A Practical Approach to Retrieval Augmented Generation Systems by Mehdi Allahyari and Angelina Yang

2. Install, run, and access Llama using Ollama. - link

3. How to Record, Save, and Play Audio in Python? - link

4. Making a talking bot using Llama3.2:1b running on Raspberry Pi 4 Model-B 4GB - link

5. fastembed library - link

6. Qdrant - link

7. pyttsx3 - link

Related Articles

Genai

• Fact-Checking in LLM Systems: From Hallucinations to Verifiable AI
• Reranking for RAG: Boosting Answer Quality in Retrieval-Augmented Generation

Unstructured Data

• Question Answer Chatbot using RAG, Llama and Qdrant

More Articles

• Why Your Embeddings Are the Wrong Shape for Your Domain

Follow for more technical deep dives on AI/ML systems, production engineering, and building real-world applications: