ID PASS DataCollect

A robust offline-first data management system for household and beneficiary data with event sourcing and synchronization capabilities

🚀 Features

• 🔌 Offline-First Architecture - Works seamlessly without internet using IndexedDB
• 🔄 Two-Level Synchronization - Client ↔ Server ↔ External system sync
• 📝 Event Sourcing - Complete audit trail of all data changes
• 🏢 Multi-Tenant Support - Single backend serving multiple applications
• 🔐 JWT Authentication - Secure API access with role-based permissions
• 🎯 TypeScript Throughout - Type-safe development experience
• 📊 Conflict Resolution - Automatic handling of data conflicts during sync
• 🔧 Extensible Architecture - Custom event types and sync adapters

📦 Project Structure

This monorepo contains four main packages:

• packages/datacollect - Core library for offline data management
• packages/backend - Central sync server with PostgreSQL
• packages/admin - Vue.js admin interface for server management
• packages/mobile - Mobile application built with Vue.js and Capacitor

🚀 Quick Start

For setting up on Docker, see the docker/README

Prerequisites

• Node.js 22.x or higher
• PostgreSQL 15+ (for backend)
• pnpm 10.x

Installation

1. Clone the repository:

git clone https://github.com/idpass/idpass-data-collect.git
cd idpass-data-collect

2. Install dependencies:

# Install all workspace dependencies
pnpm install

3. Build the datacollect library (required before using in other packages):

pnpm build:datacollect

4. Set up environment variables:

cp .env.example .env
# Edit .env with your configuration

5. Start the development servers:

# Terminal 1: Backend server
pnpm dev:backend

# Terminal 2: Admin interface
pnpm dev:admin

📖 Documentation

• Getting Started Guide
• Architecture Overview
• API Reference
• Deployment Guide
• Examples
• Glossary

💻 Basic Usage

import { EntityDataManager } from "@idpass/data-collect-core";

// Initialize the data manager
const manager = new EntityDataManager(/* ... */);

// Create a household group
const household = await manager.submitForm({
  type: "create-group",
  data: { name: "Smith Family" },
  // ... other fields
});

// Add members to household
const updated = await manager.submitForm({
  type: "add-member",
  entityGuid: household.guid,
  data: {
    members: [{ name: "John Smith", dateOfBirth: "1980-01-15" }],
  },
  // ... other fields
});

See examples/basic-usage for a complete example.

🏗️ Architecture

graph LR
    A1[App 1 - DataCollect]<--> B[Backend Server]
    A2[App 2 - DataCollect] <--> B
    A3[App 3 - DataCollect] <--> B
    B <--> C[External Systems]
    B <--> D[(PostgreSQL)]

Loading

The system uses event sourcing with CQRS pattern:

• Events represent immutable changes to entities
• Entities represent current state (Groups and Individuals)
• Sync handles bidirectional data synchronization
• Storage Adapters abstract database operations

🧪 Testing

# Run all tests
pnpm test

# Run tests for specific package
pnpm test:datacollect
pnpm test:backend
pnpm test:admin

# Run with coverage (package-specific)
cd packages/datacollect && pnpm test -- --coverage

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on:

• Code of Conduct
• Development setup
• Submitting pull requests
• Coding standards

📋 Policies and Governance

• Privacy Policy
• Governance
• Do No Harm Assessment
• Accessibility
• Security Policy
• Code of Conduct
• Contributing

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🔗 Links

• GitHub Repository
• Issue Tracker
• Website

👥 Authors

Developed and maintained by Association pour la Coopération Numérique

---

For questions or support, please open an issue or contact the maintainers.