Building a RAG-Based AI Teaching Assistant

Introduction

Large Language Models are powerful, but they often struggle to answer questions about custom datasets that were never part of their training data. To solve this problem, I built a Retrieval-Augmented Generation (RAG) Teaching Assistant capable of answering questions directly from lecture content.

The goal of this project was to transform raw lecture videos into a searchable knowledge base and generate context-aware answers using local AI models without relying on paid APIs.

Project Objective

The primary objective was to build an intelligent assistant that can:

• Understand lecture content from videos
• Retrieve relevant information based on user questions
• Generate accurate answers using retrieved context
• Operate entirely on local open-source models

This approach mimics how a real teaching assistant first studies course material before answering student queries.

Technology Stack

Programming Language

• Python

AI Models

• Whisper (Speech-to-Text)
• BGE-M3 (Embeddings)
• DeepSeek-R1 1.5B (Answer Generation)

Frameworks and Tools

• Ollama
• Streamlit
• Joblib
• Pandas
• NumPy
• Scikit-Learn

System Architecture

The complete workflow follows the pipeline below:

Lecture Videos
       ↓
Audio Extraction
       ↓
Whisper Transcription
       ↓
Text Chunking
       ↓
BGE-M3 Embeddings
       ↓
Vector Database
       ↓
Semantic Retrieval
       ↓
DeepSeek-R1
       ↓
Generated Answer

Step 1: Converting Lecture Videos into Text

Since language models cannot directly process video files, the first step was extracting audio from lecture recordings.

The videos were converted into MP3 format using FFmpeg.

After audio extraction, Whisper was used to generate transcripts containing:

• Spoken text
• Segment timestamps
• Structured JSON output

These transcripts formed the knowledge source for the assistant.

Step 2: Text Chunking

Raw transcripts are often too large to process efficiently.

To solve this issue, transcripts were divided into smaller context-preserving chunks.

Benefits of chunking:

• Faster retrieval
• Better semantic matching
• Reduced memory usage
• Improved answer quality

Each chunk represents a meaningful portion of lecture content.

Step 3: Creating Embeddings

After chunking, every text segment was converted into vector embeddings using the BGE-M3 embedding model through Ollama.

Embeddings transform textual information into numerical representations that capture semantic meaning.

For example:

Question: "How does gradient descent work?"

The system can retrieve relevant content even if the lecture never uses those exact words.

This semantic understanding is what makes RAG systems powerful.

Step 4: Storing the Knowledge Base

Generating embeddings repeatedly is computationally expensive.

To improve efficiency, all embeddings were stored locally using Joblib serialization.

Benefits:

• Faster startup time
• Reduced computation
• Reusable vector database
• Improved scalability

Step 5: Semantic Search

When a user submits a question:

1. The query is converted into an embedding.
2. Cosine similarity is computed against stored vectors.
3. The most relevant chunks are retrieved.

Instead of searching for keywords, the system searches for meaning.

This allows retrieval of contextually relevant information even when different wording is used.

Step 6: Answer Generation

The retrieved chunks are combined with the user query and supplied to DeepSeek-R1 running locally through Ollama.

The language model then generates a final response using only the retrieved context.

This significantly reduces hallucinations and improves answer accuracy.

User Interface

A Streamlit interface was developed to make the assistant interactive and easy to use.

Features include:

• Simple question-answer workflow
• Real-time retrieval
• Local AI inference
• Clean and lightweight design

The interface allows users to interact with the assistant without needing technical knowledge of the underlying pipeline.

Challenges Faced

Running Whisper Locally

Whisper transcription required substantial computational resources.

To overcome hardware limitations, transcription experiments were performed using cloud-based GPU environments before integrating the outputs into the local workflow.

Embedding Generation Time

Generating embeddings for large transcript collections was time-consuming.

Caching and Joblib serialization were implemented to eliminate unnecessary recomputation.

Retrieval Quality

Finding the right chunk size was critical.

Chunks that were too small lost context, while chunks that were too large reduced retrieval precision.

Several iterations were performed before selecting an optimal strategy.

Key Learnings

This project provided practical exposure to:

• Retrieval-Augmented Generation
• Natural Language Processing
• Semantic Search
• Vector Embeddings
• Local LLM Deployment
• Streamlit Development
• AI System Design

More importantly, it demonstrated how multiple AI components can be combined into a complete end-to-end application.

Future Improvements

Potential enhancements include:

• Multi-document support
• PDF ingestion
• Hybrid search techniques
• Conversation memory
• Advanced vector databases
• Cloud deployment
• Citation-based responses

These improvements would further increase scalability and accuracy.

Conclusion

Building this RAG-Based AI Teaching Assistant provided hands-on experience with modern AI application development. By combining Whisper, BGE-M3, DeepSeek-R1, Ollama, and Streamlit, I created a system capable of understanding lecture content and delivering context-aware responses without relying on commercial APIs.

This project strengthened my understanding of AI pipelines, semantic retrieval, and practical machine learning system design while demonstrating how Retrieval-Augmented Generation can bridge the gap between static datasets and intelligent question answering systems.