InsightStream SaaS Analytics Dashboard

InsightStream is an enterprise-grade, real-time data ingestion and analytics visualization platform. The application leverages a distributed architecture to process large-scale financial CSV data asynchronously in the background, minimizing main-thread execution time and providing users with live dashboard statistical updates via server-driven UI swaps.

---

🚀 Key Features

• Asynchronous Processing Pipeline: High-throughput data parser powered by Celery background workers and Redis event brokers.
• Real-Time UI State Updates: Live data status updates and metric calculations driven by HTMX polling and Out-of-Band (OOB) structural swaps—zero page refreshes required.
• Dynamic Visual Trend Modeling: Interactive time-series line graphing tracking daily aggregated revenue points via Chart.js.
• Defensive Fault-Tolerance Engine: Robust error isolation mechanics that capture, report, and render data serialization exceptions directly on the UI log rows without system interruption.
• Enterprise Data Streaming Exporter: One-click, database-to-client memory-efficient data compiler streaming raw data into formal .csv formats.

---

🛠️ Technology Stack

• Backend Framework: Django (Python 3)
• Task Queue & Broker: Celery + Redis
• Primary Relational Database: PostgreSQL
• Containerization Engine: Docker + Docker Compose
• Frontend UI Layer: HTMX, Bootstrap 5, Chart.js

---

📷 System Interface Showcase

1. Primary Dashboard Analytics Overview

The executive control command center displaying active rolling financial totals, real-time logging, caught processing errors, and responsive trend charts.

2. Django Site Administration Control Panel

The centralized backend data management layer configured to monitor base tables, user authentication levels, and automated periodic scheduling tasks.

---

🏗️ Architectural Flow

 [Client Browser] ──(Uploads File)──> [Django App Engine] ──(Enqueues Task)──> [Redis Broker]
        ▲                                     │                                     │
        │ (HTMX Polling & OOB Swaps)          ▼ (Creates Record)                    ▼
 [Pristine Dashboard View] <────── [PostgreSQL Database] <───(Parses Data)─── [Celery Worker]

---

⚙️ Local Deployment Guide

Follow these steps to spin up the entire distributed infrastructure locally on your machine using Docker Compose.

Prerequisites

Ensure you have the following installed on your system:

• Docker Desktop (Engine v20+)
• Docker Compose V2

Setup Instructions

1. Clone the Repository

git clone [https://github.com/rajneeshpython/insightstream.git](https://github.com/rajneeshpython/insightstream.git)
cd insightstream

2. Spin Up the Docker Infrastructure Run the compose engine in detached mode. This automatically pulls images and initializes PostgreSQL, Redis, the Django Web App, and the Celery Background Worker simultaneously:

docker-compose up --build -d

3. Run Database Schema Migrations Apply database tables to your live PostgreSQL container instance:

docker-compose exec web python manage.py migrate

4. Create an Admin Superuser Account To access the Django Administration portal shown in the showcase, generate an admin credential profile:

docker-compose exec web python manage.py createsuperuser

5. Explore the Applications Once initialization completes, you can access the entry nodes directly:

• Main Analytics Dashboard: 🌍 http://localhost:8080/
• Django Administration Portal: 🔐 http://localhost:8080/admin/

Tearing Down Infrastructure

To gracefully stop and remove all application service containers, run:

docker-compose down

---

📊 Sample Ingestion Frameworks

Successful Template Structure (june_final_pass.csv)

product_name,quantity_sold,unit_price,sale_date
Serverless Database Unit,8,120.00,2026-06-21
CDN Edge Pop License,15,45.00,2026-06-21
IAM Security Audit Tool,3,350.00,2026-06-22

Fault-Inducing Template Structure (june_malformed_fail.csv)

product_name,quantity_sold,unit_price,sale_date
Malicious Exploiter Pod,TEN,150.00,2026-06-23
Corrupted Memory Buffer,4,FREE,2026-06-23