Distributed File System (DFS) 🚀

A high-performance, Master-Worker storage architecture designed for enterprise-grade reliability, data integrity, and real-time fault tolerance. Built with Go for high-concurrency backend logic and React for a live, WebSocket-powered monitoring dashboard.

🏗️ Architecture: "The Brain & The Muscle"

Unlike traditional storage, this system decouples intelligence from physical storage to ensure it is indestructible and infinitely scalable.

• The Master Node (The Brain): The central orchestrator that manages the Metadata Ledger. It handles file chunking, determines replication strategies, and monitors cluster health. It never touches raw data during downloads, preventing network bottlenecks.
• The Data Nodes (The Muscle): Independent, location-agnostic storage servers. They handle high-speed I/O, storing raw chunks and responding to retrieval requests directly from the client.
• The Command Center (Frontend): A real-time dashboard that uses WebSockets to visualize the heartbeat of the entire cluster.

---

✨ Key Features

🧩 Intelligent Chunking & Hashing

Files are transparently sliced into fixed-size 1MB Chunks. Every chunk is stamped with a unique SHA-256 Checksum (digital DNA). This allows the system to detect and fix "silent data corruption" (bit-rot) automatically.

🛡️ N-Way Replication

Every piece of data is automatically cloned across multiple nodes. If one server fails, the system instantly routes requests to the healthy backup, ensuring Zero Downtime.

🩹 Automated Self-Healing

The system is "Alive." A background Healer Service constantly scans for under-replicated chunks. If a node is lost, the Healer automatically restores the required safety factor by re-replicating data to new healthy nodes.

📡 Real-Time WebSocket Monitoring

The dashboard reacts instantly to cluster events. Node failures, storage capacity changes, and upload progress are broadcasted live, providing a "God-eye view" of the distributed network.

---

🛠️ Tech Stack

• Backend: Go (Golang) with Gorilla Mux
• Frontend: React, TypeScript, Vite, TailwindCSS (for aesthetics)
• Real-time: WebSockets (Gorilla)
• Integrity: SHA-256 Cryptographic Hashing
• Containerization: Docker & Docker Compose

---

🚀 Getting Started

1. Prerequisites

• Go (1.21+)
• Node.js (v18+)
• (Optional) Docker Desktop

2. Start the Backend (Manual)

Open separate terminals for the Master and at least two Data Nodes:

# Terminal 1: Start the Master (The Brain)
cd dfs/backend
go run ./master/main.go

# Terminal 2: Start Data Node 1 (The Muscle)
cd dfs/backend
go run ./node/main.go -id n1 -port 9001 -dir ./data/n1

# Terminal 3: Start Data Node 2 (The Muscle)
cd dfs/backend
go run ./node/main.go -id n2 -port 9002 -dir ./data/n2

3. Start the Frontend

cd dfs/frontend
npm install
npm run dev

Navigate to http://localhost:5173 to see your cluster live!

---

🧪 Testing the "Literally Real" Features

1. The Persistence Test: Upload a file, then delete the original from your computer. Click Download in the browser—the file is restored perfectly from the cluster.
2. The Chaos Test: Kill a Node terminal (Ctrl+C) while the dashboard is open. Watch the status turn RED instantly. Download your file again—it still works!
3. The Healer Test: Start a new Node (n3). Within 20 seconds, watch the logs as the Healer automatically moves missing data to the new node to restore safety.

---

🌐 Real-World Use Cases

• Massive Scale: Linking 100 small drives to act as one 10PB hard drive.
• High Availability: Hosting video content that must never go offline (Netflix-style).
• Data Integrity: Storing medical or legal records where even a single bit of corruption is unacceptable.

---

Developed as a high-performance demonstration of Distributed System Design.