Skip to content

vyogotech/erpnext-microservices-lib

Repository files navigation

Frappe Microservice Framework

A Python framework for building secure, isolated Frappe microservices with proper bounded context and multi-tenant support.

Features

  • 🔒 Secure by Default: All endpoints require authentication via Central Site
  • 🔌 Independent Database: Each microservice has its own database (bounded context principle)
  • 👤 User Context Injection: Authenticated user is automatically injected into handlers
  • 🚀 Zero Boilerplate: Create CRUD APIs with a single line of code
  • 🏢 Multi-Tenant Ready: Built-in support for tenant isolation
  • 🛡️ Tenant-Aware Database: Automatic tenant_id filtering prevents cross-tenant data access
  • 🪝 Document Hooks: Frappe-style lifecycle hooks without modifying Frappe core
  • 📦 Frappe-Native: Seamlessly works with Frappe DocTypes and APIs
  • 📚 Auto-Generated Swagger Docs: Interactive API documentation at /apidocs
  • 🔑 OAuth2 + SID Authentication: Support for Bearer tokens and session cookies
  • 🛡️ SQL Injection Prevention: Mandatory parameterized queries
  • ⚙️ Integrated Background Tasks: Built-in RQ support for asynchronous processing

Installation

pip install frappe-microservice

Or install from source:

cd frappe-microservice-lib
pip install -e .

Quick Start

Basic Microservice

from frappe_microservice import create_microservice

# Initialize microservice
app = create_microservice("my-service")

# Create a secure endpoint (authentication required)
@app.secure_route('/hello', methods=['GET'])
def hello(user):
    """User is automatically injected after authentication"""
    return {"message": f"Hello {user}!"}

# Start the service
app.run()

Automatic CRUD with Resource API

from frappe_microservice import create_microservice

app = create_microservice("orders-service")

# Register a DocType for automatic CRUD (zero code!)
app.register_resource("Sales Order")

# This automatically creates:
# GET    /api/resource/sales-order        - List orders
# POST   /api/resource/sales-order        - Create order
# GET    /api/resource/sales-order/{name} - Get order
# PUT    /api/resource/sales-order/{name} - Update order
# DELETE /api/resource/sales-order/{name} - Delete order

app.run()

Custom Business Logic

from frappe_microservice import create_microservice
import frappe

app = create_microservice("signup-service")

@app.secure_route('/signup', methods=['POST'])
def signup_company(user):
    """Create a new company with multi-tenant isolation"""
    from flask import request
    data = request.json

    # Create company with tenant_id for isolation
    company = frappe.get_doc({
        "doctype": "Company",
        "company_name": data['company_name'],
        "tenant_id": generate_tenant_id(),
        "admin_user": user
    })
    company.insert()

    return {
        "success": True,
        "company_id": company.name,
        "tenant_id": company.tenant_id
    }

app.run()

Tenant-Aware Database (Automatic Tenant Isolation)

The TenantAwareDB wrapper automatically adds tenant_id to all queries, preventing accidental cross-tenant data access:

from frappe_microservice import create_microservice

app = create_microservice("my-service")

@app.secure_route('/users', methods=['GET'])
def list_users(user):
    # Get tenant for authenticated user
    tenant_id = get_user_tenant_id(user)
    app.set_tenant_id(tenant_id)

    # ✅ Automatically adds tenant_id filter!
    users = app.tenant_db.get_all(
        'User',
        fields=['name', 'email', 'role']
    )

    return {"data": users}

@app.secure_route('/users/<user_id>', methods=['GET'])
def get_user(user, user_id):
    tenant_id = get_user_tenant_id(user)
    app.set_tenant_id(tenant_id)

    try:
        # ✅ Automatically verifies tenant ownership!
        user_doc = app.tenant_db.get_doc('User', user_id)
        return user_doc.as_dict()
    except frappe.PermissionError:
        return {"error": "Access denied"}, 403

Why TenantAwareDB?

  • Secure by Default: Impossible to forget tenant_id filter
  • Automatic Filtering: All get_all/get_doc calls are tenant-scoped
  • Prevents Leaks: Raises PermissionError if accessing other tenant's data
  • Zero Boilerplate: No manual tenant_id filtering needed

See TENANT_AWARE_DB_EXAMPLE.py for complete examples.

Document Lifecycle Hooks (No Frappe Modifications!)

Register hooks for document lifecycle events without modifying Frappe core. Perfect for microservices!

from frappe_microservice import create_microservice

app = create_microservice("orders-service")

# Global hook - runs for ALL doctypes
@app.tenant_db.before_insert('*')
def ensure_tenant_id(doc):
    """Ensure tenant_id is set on all documents"""
    from flask import g
    if not doc.tenant_id:
        doc.tenant_id = g.tenant_id

# DocType-specific hooks
@app.tenant_db.before_insert('Sales Order')
def set_order_defaults(doc):
    """Set defaults for new orders"""
    if not doc.status:
        doc.status = 'Draft'
    if not doc.order_date:
        doc.order_date = frappe.utils.today()

@app.tenant_db.after_insert('Sales Order')
def send_order_notification(doc):
    """Send notification after order creation"""
    print(f"📧 Order {doc.name} created for {doc.customer}")

@app.tenant_db.before_validate('Sales Order')
def validate_order_amount(doc):
    """Custom validation"""
    if doc.grand_total and doc.grand_total < 0:
        frappe.throw("Order total cannot be negative")

# Use in endpoints - hooks run automatically!
@app.secure_route('/orders', methods=['POST'])
def create_order(user):
    from flask import request
    
    tenant_id = get_user_tenant_id(user)
    app.set_tenant_id(tenant_id)
    
    # All hooks run automatically during insert!
    doc = app.tenant_db.insert_doc('Sales Order', request.json)
    
    return {"success": True, "name": doc.name}

Supported Hook Events:

  • before_validate - Before validation starts
  • validate - During validation
  • before_insert - Before inserting into database
  • after_insert - After inserting into database
  • before_update - Before updating document
  • after_update - After updating document
  • before_delete - Before deleting document
  • after_delete - After deleting document

Why Document Hooks?

  • 🚫 No Frappe Core Changes: Works entirely in microservice layer
  • 🎯 Microservice-Specific: Each service has its own hooks
  • 🔧 Full Control: Easy to test and debug
  • 📝 Clean Code: Separate business logic from endpoints
  • 🔄 Reusable: Write once, applies to all operations

See DOCUMENT_HOOKS_EXAMPLES.py for comprehensive examples.

Traditional DocType Controllers

Use familiar Frappe-style controller classes for your DocTypes! No Frappe core modifications needed.

# controllers/sales_order.py
from frappe_microservice.controller import DocumentController

class SalesOrder(DocumentController):
    def validate(self):
        """Validate order data"""
        if not self.customer:
            self.throw("Customer is required")
        self.calculate_total()
    
    def before_insert(self):
        """Set defaults"""
        if not self.status:
            self.status = 'Draft'
        if not self.order_date:
            self.order_date = frappe.utils.today()
    
    def after_insert(self):
        """Post-creation tasks"""
        self.send_notification()
        self.update_customer_stats()
    
    def calculate_total(self):
        """Reusable method"""
        self.grand_total = sum(item.amount for item in self.items)
    
    def send_notification(self):
        print(f"Order {self.name} created")

# server.py
from frappe_microservice import create_microservice, setup_controllers

app = create_microservice("orders-service")

# Auto-discover and register controllers
setup_controllers(app, controllers_directory="./controllers")

# Controllers run automatically during insert/update!
@app.secure_route('/orders', methods=['POST'])
def create_order(user):
    doc = app.tenant_db.insert_doc('Sales Order', request.json)
    return {"success": True, "name": doc.name}

Features:

  • 🎯 Traditional Pattern: Familiar Frappe controller style
  • 📁 One File Per DocType: Clean code organization
  • 🔄 Auto-Discovery: Automatically loads from directory
  • 🎭 Lifecycle Methods: validate(), before_insert(), after_insert(), etc.
  • 🛠️ Custom Methods: Define reusable business logic
  • 🧪 Easy Testing: Test controllers independently

File Naming Convention:

  • sales_order.pySales Order DocType → SalesOrder class
  • signup_user.pySignup User DocType → SignupUser class

See signup-service/ for a complete example with controllers.

Central Site API Client

The CentralSiteClient provides a standardized way for microservices to communicate back to the Central Site using a Frappe-like API. It is available as app.central in any MicroserviceApp.

from frappe_microservice import create_microservice

app = create_microservice("my-service")

@app.secure_route('/sync', methods=['POST'])
def sync_with_central(user):
    # Fetch a document from the Central Site
    tenant_info = app.central.get_doc("Tenant", "tenant-123")
    
    # Update a record on the Central Site
    app.central.update("Tenant", "tenant-123", {"status": "Active"})
    
    # Call a whitelisted method
    result = app.central.call("some_whitelisted_method", {"param": "value"})
    
    return {"status": "synced"}

Background Task Processing (RQ)

The framework includes integrated support for RQ (Redis Queue) to handle long-running tasks like tenant provisioning or email sending.

Activation

Background task processing is disabled by default. Enable it by setting the following environment variables in your container:

Variable Description
ENABLE_RQ Set to 1 to auto-start the embedded RQ worker thread
REDIS_URL Redis connection URL (default: redis://localhost:6379)

Usage in Code

Any MicroserviceApp instance can enqueue tasks directly:

from frappe_microservice import create_microservice

app = create_microservice("my-service")

@app.secure_route('/task', methods=['POST'])
def trigger_task(user):
    # Enqueue a function to run in the background
    # The framework automatically restores Frappe context in the worker!
    app.enqueue_task(long_running_function, arg1, kwarg1="value")
    return {"status": "enqueued"}

def long_running_function(arg1, kwarg1=None):
    import frappe
    # Full Frappe API available here
    frappe.get_doc({...}).insert()
    frappe.db.commit()

Features:

  • Zero Configuration: No separate worker process needed; it runs as a daemon thread inside your service container.
  • Context Preservation: Automatically restores frappe.local and DB connections in the worker process.
  • Controller Support: Auto-discovery of DocType controllers works seamlessly in background tasks.

Configuration Environment Variables:

Variable Description Default
CENTRAL_SITE_URL URL of the Central Site http://central-site:8000
CENTRAL_SITE_API_KEY API Key for authentication -
CENTRAL_SITE_API_SECRET API Secret for authentication -
CENTRAL_SITE_USER Username (alternative to API Key) -
CENTRAL_SITE_PASSWORD Password (alternative to API Secret) -
CENTRAL_SITE_TIMEOUT Request timeout in seconds 10

The client is lazily initialized, meaning no connection attempts are made until the first time app.central is accessed.

Container Deployment (Library Entrypoint)

The base image runs the library entrypoint so services do not need their own entrypoint.py. The framework discovers your app via environment variables and starts it.

Variable Description Default
SERVICE_PATH Directory containing your service code (e.g. server.py) /app/service
SERVICE_APP Module and app attribute to run (module:attr) server:app

Minimal service Containerfile (e.g. signup-service):

FROM ghcr.io/your-org/frappe-microservice-lib:latest
WORKDIR /app
COPY . /app/service/
RUN touch /app/service/__init__.py
# Expose as Frappe app so the framework can load it (name "my-service" -> my_service)
RUN ln -sf /app/service /app/sites/apps/my_service
ENV SERVICE_PATH=/app/service
ENV SERVICE_APP=server:app
EXPOSE 8000

Your repo only needs a server.py that does app = create_microservice("my-service", ...) and defines routes. The base image ENTRYPOINT is python -m frappe_microservice.entrypoint, which loads the app from SERVICE_PATH / SERVICE_APP and calls run_app(app).

For a different app location or name (e.g. main:app), set SERVICE_APP=main:app.

Configuration

Environment Variables

# Frappe configuration
export FRAPPE_SITE="dev.localhost"
export FRAPPE_SITES_PATH="/home/frappe/frappe-bench/sites"

# Database (for independent microservice DB)
export DB_HOST="signup-db"

# Central Site for authentication
export CENTRAL_SITE_URL="http://central-site:8000"

Programmatic Configuration

app = MicroserviceApp(
    name="my-service",
    central_site_url="http://central-site:8000",
    frappe_site="dev.localhost",
    sites_path="/home/frappe/frappe-bench/sites",
    db_host="my-service-db",  # Independent database
    port=8000
)

Architecture: Bounded Context

Each microservice should have its own database for true isolation:

services:
  # Signup Microservice
  signup-service:
    image: frappe-ms
    environment:
      - DB_HOST=signup-db
    depends_on:
      - signup-db

  # Independent Database
  signup-db:
    image: mariadb:10.6
    environment:
      MYSQL_DATABASE: signup_db
      MYSQL_USER: signup_user
      MYSQL_PASSWORD: signup_pass

## ERPNext Decomposition Strategy

The Frappe Microservice Framework is designed to facilitate the **gradual decomposition of the ERPNext monolith**. Instead of a high-risk full migration, you can incrementally extract specific domains (e.g., Sales, HR, Inventory) into dedicated microservices:

1.  **Define Bounded Contexts**: Create independent databases for specific ERPNext modules.
2.  **Modular Migration**: Use the framework to build services that handle specific DocTypes.
3.  **Unified Authentication**: Keep the Central Site as the single point of entry for user sessions.
4.  **Independent Scaling**: Scale critical services (like Order processing) independently from the rest of the ERP.

This strategy reduces technical debt and enables a more agile, resilient architecture.

## Swagger/OpenAPI Documentation

All microservices built with this framework automatically expose interactive Swagger UI documentation at `/apidocs`.

### Automatic Documentation for Resource API

When you use `register_resource()`, endpoints are automatically documented:

```python
app = create_microservice("my-service")

# This automatically creates fully-documented endpoints
app.register_resource("Sales Order")

# View docs at: http://localhost:8000/apidocs

Manual Documentation for Custom Endpoints

Add YAML frontmatter to docstrings for custom endpoints:

@app.secure_route('/orders/summary', methods=['GET'])
def order_summary(user):
    """
    Get order summary for current tenant
    ---
    tags:
      - Orders
    parameters:
      - name: days
        in: query
        type: integer
        default: 7
        description: Number of days to include
    responses:
      200:
        description: Summary retrieved successfully
    """
    return {"total_orders": 100}

Accessing Documentation

  • Swagger UI: http://localhost:<port>/apidocs/
  • OpenAPI Spec: http://localhost:<port>/apispec_1.json

Authentication

OAuth2 Bearer Tokens

The framework supports OAuth2 Bearer tokens for API clients:

# Obtain token from Central Site OAuth2
curl -X POST http://central-site:8000/oauth/token \
  -d grant_type=client_credentials \
  -d client_id=YOUR_CLIENT_ID \
  -d client_secret=YOUR_SECRET

# Use token in requests
curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  http://localhost:8000/api/resource/sales-order

Session Cookies (SID)

Browser clients can use traditional Frappe session cookies:

// Login via Central Site
fetch('http://central-site:8000/api/method/login', {
  method: 'POST',
  credentials: 'include',
  body: JSON.stringify({ usr: 'user@example.com', pwd: 'password' })
})

// Cookies automatically included in subsequent requests
fetch('http://localhost:8000/api/resource/sales-order', {
  credentials: 'include'
})

SQL Security Best Practices

Use TenantAwareDB Helpers

Prefer TenantAwareDB methods over raw SQL:

# ✅ Good - uses ORM with automatic tenant filtering
orders = app.tenant_db.get_all('Sales Order', filters={'status': 'Draft'})

# ❌ Bad - raw SQL requires manual tenant filtering
orders = frappe.db.sql(
    "SELECT * FROM `tabSales Order` WHERE status='Draft'"
)

Mandatory Parameterized Queries

If raw SQL is unavoidable, always use parameterized queries:

# ✅ Good - parameterized
tenant_id = app.tenant_db.get_tenant_id()
orders = app.tenant_db.sql(
    "SELECT * FROM `tabSales Order` WHERE tenant_id = %s AND status = %s",
    (tenant_id, 'Draft'),
    as_dict=True
)

# ❌ NEVER use string formatting or f-strings
orders = app.tenant_db.sql(
    f"SELECT * FROM `tabSales Order` WHERE tenant_id = '{tenant_id}'"
)

See the Architecture Plan Section 16 for detailed SQL security guidelines.

Multi-Tenant Isolation

All data should include a tenant_id for isolation:

# Create with tenant isolation
company = frappe.get_doc({
    "doctype": "Company",
    "tenant_id": tenant_id,
    "company_name": "Acme Corp"
})

# Query with tenant filter
companies = frappe.get_all(
    "Company",
    filters={"tenant_id": tenant_id}
)

Package Structure

frappe_microservice/
├── app.py          # MicroserviceApp + create_microservice (main entry)
├── hooks.py        # DocumentHooks lifecycle system
├── tenant.py       # TenantAwareDB, get_user_tenant_id
├── isolation.py    # App/module isolation (IsolationMixin)
├── auth.py         # OAuth2 & SID authentication (AuthMixin)
├── resources.py    # Auto-CRUD endpoint generation (ResourceMixin)
├── controller.py   # Traditional DocType controllers
├── core.py         # Re-export shim for backward compatibility
├── entrypoint.py   # create_site_config, run_app, main() (container entrypoint)
└── __init__.py     # Public API surface

All public symbols remain importable from both frappe_microservice and frappe_microservice.core for full backward compatibility. See docs/architecture.md for the dependency diagram and detailed module descriptions.

API Reference

MicroserviceApp

The main class for creating microservices.

Methods

  • secure_route(rule, **options) - Register an authenticated endpoint
  • route(rule, **options) - Register a public endpoint
  • register_resource(doctype, **options) - Auto-create CRUD endpoints
  • run(**kwargs) - Start the microservice

create_microservice(name, **config)

Quick setup function for creating a microservice.

Examples

See the examples/ directory for complete examples:

  • examples/signup-service/ - Multi-tenant user signup
  • examples/orders-service/ - Order management with CRUD
  • examples/notifications-service/ - Event-driven notifications

Development

# Install in development mode
pip install -e ".[dev]"

# Run tests
pytest

# Format code
black frappe_microservice/

# Lint
flake8 frappe_microservice/

License

MIT License

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for details.

About

Foundational library to scale out erpnext/any frappe app into individual services.

Topics

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors