DataCROP (Data Collection Routing & Processing) is a configurable framework for real-time data collection, transformation, filtering, and management across IoT and cybersecurity domains. It emphasizes interoperability through a specialized data model for sources, processors, and results, enabling flexible workflow-driven analytics.
- Barley (v1.0): MongoDB, Apache Kafka, RabbitMQ, Kafka Streams, Node.js, React, optional Hyperledger Fabric.
- Farro (v2.0): Builds on Barley; MongoDB, Apache Kafka, RabbitMQ, Node.js, React, and algorithm support (Java, Python, R).
- Maize (v3.0, in progress): MongoDB, Apache Kafka, ELK stack; expanding observability and data services.
Deployable Farro demo: https://github.com/datacrop/farro-demo-deployment-scripts.
- Ensure Ruby and Bundler are installed.
- Install dependencies:
bundle install - Serve the site:
bundle exec jekyll serve - Visit
http://localhost:4000(adjustbaseurlif configured).
_content/index.markdown: Documentation landing page._content/home/: Framework overview and roadmap._content/Setup/: Setup landing page._content/Setup/maize-mvp/: Single-script Maize MVP deployment._content/Setup/manual/: Manual component setup (Keycloak, Airflow, Worker, Model Repository, Workflow Editor)._content/user-guide/: End-user workflows, data models, and overview pages._content/dev-guide/: Developer guidance, including processor integration.