🚀 EXAM RAG – AI-Powered Exam Question Search Engine

---

🔥 Overview

Exam RAG is an AI-powered Retrieval-Augmented Generation (RAG) system that automates the process of:

✔ Crawling institutional repositories ✔ Extracting exam questions from PDFs ✔ Structuring them using AI ✔ Enabling semantic search (search by meaning, not keywords)

💡 Built to solve the real problem of finding relevant past exam questions efficiently.

---

✨ Key Features

🌐 Smart Web Crawler

• Automatically discovers and downloads PDF files from repositories (e.g., Amrita DSpace)

📄 PDF Processing Engine

• Extracts text dynamically without storing unnecessary files

🤖 AI Question Extractor

• Uses LangChain + LLMs to convert messy text into structured questions (JSONL format)

🧠 Semantic Search Engine

• Uses vector embeddings to find contextually similar questions

🗂 Modular Architecture

• Separate pipelines for:

  • Crawling
  • Processing
  • Searching

⚡ Fast & Lightweight

• Uses SQLite + NumPy for efficient storage and retrieval

---

🏗️ Tech Stack

🔹 Core

• sqlite3 – Database
• json – Data formatting
• numpy – Vector operations
• os – File handling

🔹 Web & Parsing

• requests – Fetching data
• BeautifulSoup (bs4) – HTML parsing
• PyPDF2 – PDF text extraction
• io – In-memory streams

🔹 AI & Search

• langchain – LLM pipeline
• sentence-transformers – Embeddings
• sklearn – Cosine similarity

🔹 Backend

• flask – Web interface

---

📊 System Architecture

Repository → Crawler → PDF Extractor → AI Processing → SQLite DB → Semantic Search

📂 Project Structure

exam_rag/

│

├── crawler.py              # Crawls repository and collects PDFs

├── processor.py            # Extracts and processes questions

├── database.py             # SQLite operations

├── search.py               # Semantic search engine

├── app.py                  # Flask application

│

├── data/

│   ├── raw/

│   └── processed/

│

├── implementation/

│   ├── cc.png

│   ├── eee.png

│   ├── eng.png

│   └── qm.png

│

├── requirements.txt

└── README.md

---

🖼️ Implementation Preview

---

⚙️ How It Works

1. Crawl Repository

   • Extract all PDF links

2. Process PDFs

   • Convert PDFs → raw text

3. AI Structuring

   • Text → clean questions (JSONL)

4. Embedding Generation

   • Convert questions → vectors

5. Store in Database

   • SQLite stores:

     • Questions
     • Embeddings

6. Semantic Search

   • Input query → find most relevant questions

---

🚀 Getting Started

1️⃣ Clone Repository

git clone https://github.com/backpropbrigade-dev/Exam-rag.git
cd Exam-rag

2️⃣ Install Dependencies

pip install -r requirements.txt

3️⃣ Run Pipeline

python crawler.py
python processor.py
python app.py

---

🔍 Example Use Case

Search: "control systems stability questions" ➡ Returns relevant questions even if exact words don’t match.

---

🧩 Future Improvements

• 🔹 Add subject-wise filtering
• 🔹 Improve UI with React frontend
• 🔹 Support multiple repositories
• 🔹 Add question difficulty classification
• 🔹 Deploy as cloud-based API

---

🤝 Contributing

Contributions are welcome!

1. Fork the repo
2. Create a feature branch
3. Commit changes
4. Open a Pull Request

---

⭐ Support

If you like this project:

• ⭐ Star the repo
• 🍴 Fork it
• 🧠 Share ideas

---