Offline Lecture Transcription, Summarization, and Note-Taking System
SmartNote is a fully offline, end-to-end system that converts lecture or meeting audio into accurate transcripts, semantic summaries, and structured notes. The project emphasizes privacy, reproducibility, and modular design, and was developed as a Master’s-level final project.
- 🎙️ Offline Automatic Speech Recognition (ASR)
- Whisper.cpp (primary inference engine)
- ✍️ Punctuation restoration for cleaner transcripts
- 🧠 Semantic summarization with redundancy reduction
- 📝 Automatic bullet-point notes and PDF export
- 📊 Evaluation using reference-based and reference-free metrics
- ⚙️ One-command CLI pipeline for reproducibility
All processing is performed locally without using cloud APIs.
smartnote/ ├── run.py # One-command pipeline ├── README.md ├── LICENSE ├── requirements.txt │ ├── src/ │ ├── transcribe_whisper.py │ ├── punctuate.py │ ├── summarize_semantic.py │ ├── train_summary_selector.py │ ├── summarize_trained.py │ ├── export_notes.py │ ├── evaluate.py │ └── wer.py │ ├── data/ │ ├── input/ # Input audio files │ ├── output/ # Generated transcripts, summaries, PDFs │ ├── train/ # Labeled data for downstream training │ └── cs50/ # Domain-specific lecture data (sample) │ ├── models/ │ ├── summary_selector.joblib │ ├── summary_vectorizer.joblib │ └── whisper_cs50_finetuned/ # Fine-tuned Whisper weights (Python) │ ├── whisper.cpp/ │ └── models/ │ └── ggml-base.en-cs50.bin # Quantized model for inference │ └── venv/ (optional)