CyberGuard - Intelligent Analysis, Search and Summarization System

CyberGuard is a complete system that allows uploading documents (PDF, texts, articles, emails) and provides advanced information processing capabilities:

• Semantic search based on embedding vectors
• Generation of coherent summaries
• Context-based questions and answers
• Similar content recommendations
• Data visualization and analysis

Architecture

The system is built on a modern architecture:

• Frontend: HTML/JavaScript with Tailwind CSS
• Backend API: FastAPI (Python)
• Vector Database: FAISS for fast search
• Embeddings: SentenceTransformers (all-MiniLM-L6-v2)
• LLM: Integration with OpenAI GPT-3.5 for summarization and Q&A

Features

1. Upload Center: Upload PDF, TXT files
2. Text Extractor: Extract text from documents
3. Chunker: Split text into semantic fragments
4. Embedding Generator: Transform text into vectors
5. Vector Index: Store vectors for fast search
6. Semantic Search: Similarity-based search
7. Summarizer: Generate coherent summaries
8. Q&A Chatbot: Context-based answers to questions
9. Dashboard: View statistics and search history

Installation and Running

Backend

1. Create a Python virtual environment:

python -m venv venv
source venv/bin/activate  # Linux/Mac
venv\Scripts\activate     # Windows

2. Install dependencies:

pip install fastapi uvicorn pydantic faiss-cpu openai sentence-transformers PyPDF2

3. Run the server:

cd astramind
python main.py

The server will run at: http://localhost:8000

Frontend

1. Open the index.html file from the astramind-client directory in a web browser.

Usage

1. Upload documents through the drag-and-drop interface
2. Use the search bar to query the documents
3. Choose between semantic search, summary generation, or questions and answers
4. View usage statistics in the dashboard panel

Technologies Used

• FastAPI: Python framework for fast APIs
• FAISS: Library for efficient vector search
• SentenceTransformers: Models for generating embeddings
• OpenAI API: For generating summaries and answers
• Tailwind CSS: CSS framework for modern design
• Chart.js: Library for data visualization

Future Development

• Add JWT authentication
• Support for more document types (DOCX, HTML, etc.)
• Implementation of local models (Llama2) for independence from external APIs
• Improvement of the interface for mobile devices
• Addition of export and sharing functionalities