README.md

Document Processing System

A comprehensive document processing system with AI-powered analysis, storage, and retrieval capabilities.

---

🚀 Quick Start

1. Clone the repository:

   git clone https://github.com/yourusername/document-processing-system.git
   cd document-processing-system

2. Copy and edit the environment file:

   cp .env.example .env
   # Edit .env with your settings

3. Start all services:

   make dev

---

🖥️ Service URLs

• Node-RED: http://localhost:1880
• OCR API: http://localhost:8081
• MinIO Console: http://localhost:9001
• Kibana: http://localhost:5601
• Elasticsearch: http://localhost:9200
• Tika Server: http://localhost:9998
• (See make help for more)

---

📝 How OCR Automation Works in This System

1. Upload & Trigger

• Upload a PDF via Node-RED HTTP endpoint (/start-ocr) or other input.
• Node-RED flow definition

2. Node-RED Orchestration

• Node-RED receives the file, sends it to the OCR API.
• Node-RED OCR automation flow

3. OCR API Service

• Receives the PDF, extracts text using OCRmyPDF, and returns the result as JSON.
• OCR API implementation
• OCR API Dockerfile

4. Bash Test Script

• Test the OCR API from the command line with a sample PDF.
• Test script

5. Configuration & Compose

• All services are orchestrated via Docker Compose.
• docker-compose.yml
• Environment variables

---

🚦 Step-by-Step Example

1. Start all services:

   make dev

2. Upload a PDF for OCR via Node-RED:
   • POST a PDF to http://localhost:1880/start-ocr
   • Example with curl:

     curl -X POST -F "file=@/path/to/your.pdf" http://localhost:1880/start-ocr

3. Directly test OCR API:
   • Use the test script:

     ./scripts/test_ocr_api.sh /path/to/your.pdf

   • Or use the web form at http://localhost:8081/

---

🔗 Key Files and Links

• Node-RED flows.json
• OCR API Python app.py
• OCR API Dockerfile
• OCR Bash Test Script
• docker-compose.yml
• Makefile
• Environment Variables

---

📁 Directory Structure

.
├── config/                  # Service configurations
│   ├── elasticsearch/       # Elasticsearch config
│   ├── grafana/             # Grafana dashboards and config
│   ├── node-red/            # Node-RED flows and settings
│   └── prometheus/          # Prometheus config
├── data/                    # Persistent data
│   ├── elasticsearch/       # ES data
│   ├── grafana/             # Grafana data
│   ├── minio/               # MinIO data
│   └── redis/               # Redis data
├── docker/ocr-api/          # OCR API implementation
├── scripts/                 # Utility scripts
├── docker-compose.yml       # Main compose file
├── Makefile                 # Project commands
└── .env                     # Environment variables

---

🛠️ Configuration

Create a .env file with the following variables (see .env.example for more):

OCR_HOST=ocr
OCR_PORT=8081
# ... other variables ...

---

🧑‍💻 Contributing & Maintenance

• Use make help to see all available commands.
• To update services:

  git pull origin main
  make dev

• To view logs:

  docker-compose logs -f

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

For support, open an issue in the GitHub repository or contact the maintainers.