Skip to content

vchaitanyachowdari/Awesome-Workflow-Engines

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

264 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Workflow Engines (Open Source)

image

Full Fledged Product

Screenshot 2025-06-17 at 2 03 29 PM
  1. Open-source no-code automation platform.
  2. Serves as a Zapier alternative with drag-and-drop capabilities.
  3. Enables app-to-app automation (Slack, Gmail, Notion, etc.).
  4. Built for business users with minimal technical background.
  5. Can be self-hosted for security and compliance.
  6. Allows developers to create custom integrations.
  7. Supports triggers, actions, and conditions in workflows.
  8. Lightweight, responsive UI for real-time monitoring.
  9. Strong community support and fast-paced updates.
  10. Suitable for startups and enterprises automating repetitive tasks.
Screenshot 2025-06-17 at 2 04 16 PM
  1. Scientific workflow manager tailored for computational science.
  2. Provides full provenance tracking for all computations.
  3. Optimized for use with high-performance computing (HPC) environments.
  4. Designed to ensure reproducibility of research results.
  5. Plugin architecture for integrating simulation codes.
  6. Uses PostgreSQL and RabbitMQ for backend communication.
  7. CLI and Python API for flexible interaction.
  8. Data lineage and dependency tracking for transparency.
  9. Actively used in materials science and quantum chemistry.
  10. Backed by a strong academic research community.
Screenshot 2025-06-17 at 2 05 43 PM
  1. Python-based platform for authoring, scheduling, and monitoring workflows.
  2. Organizes tasks in directed acyclic graphs (DAGs).
  3. Ideal for ETL, batch processing, and data pipeline orchestration.
  4. Highly extensible with custom operators and plugins.
  5. Rich UI for monitoring task runs and logs.
  6. Integrates well with cloud services (GCP, AWS, Azure).
  7. Built-in support for retries, SLA, and alerting.
  8. Scales horizontally using Celery or Kubernetes executors.
  9. Used in production by Airbnb, Lyft, and many others.
  10. Apache project with wide adoption and community.
Screenshot 2025-06-17 at 2 06 15 PM
  1. Kubernetes-native workflow engine.
  2. Uses YAML for workflow definitions; runs containers as steps.
  3. Ideal for CI/CD, ML workflows, and data processing on Kubernetes.
  4. Supports loops, conditionals, artifacts, and parallel execution.
  5. Can trigger workflows using CRDs and event sources.
  6. Fully cloud-native; scales with Kubernetes.
  7. Offers UI dashboard and CLI for management.
  8. Supports integrations with Argo CD and Argo Events.
  9. Built for DevOps and platform engineering teams.
  10. CNCF project with growing enterprise adoption.
Screenshot 2025-06-17 at 2 07 48 PM
  • Open-source data and workflow management system.
  • Built for large-scale bioinformatics and genomics pipelines.
  • Emphasizes reproducibility, collaboration, and auditability.
  • Handles petabyte-scale data across hybrid cloud/HPC setups.
  • Supports Docker-based workflows.
  • REST API, SDKs, and web UI available.
  • Fine-grained permissions for secure data sharing.
  • Optimized for research and healthcare compliance (e.g., HIPAA).
  • Provides data versioning and automatic logging.
  • Used by research labs and genomics companies.
Screenshot 2025-06-17 at 2 08 17 PM
  • Workflow scheduler for batch job automation.
  • Originally developed at LinkedIn to manage Hadoop jobs.
  • Web-based UI for job management and tracking.
  • Supports job dependencies, retry policies, and failure handling.
  • Uses simple .job files and .properties for configuration.
  • Scalable to thousands of workflows and jobs.
  • Offers security features and LDAP integration.
  • Well-suited for ETL and data warehouse management.
  • Lightweight and easy to deploy.
  • Still used in data-heavy enterprises.
Screenshot 2025-06-17 at 2 08 42 PM
  • Event-driven scripting platform for Kubernetes.
  • Automates CI/CD, testing, and deployments using JavaScript/TypeScript.
  • Lightweight alternative to full-fledged CI tools.
  • Uses "projects" and "events" as core concepts.
  • Tight integration with GitHub, DockerHub, and cloud events.
  • Provides custom pipeline logic using scripting.
  • CLI and dashboard support.
  • Designed for DevOps and cloud-native workflows.
  • Open-source and easily extensible.
  • Backed by the CNCF ecosystem.
  • Open-source low-code automation platform.
  • Designed for API integration and workflow orchestration.
  • Clean UI with drag-and-drop builder.
  • Supports pre-built connectors and custom extensions.
  • Developer-friendly: written in Java/Spring Boot.
  • Runs in Docker and Kubernetes environments.
  • Ideal for automating SaaS and internal tools.
  • Offers form builders, schedulers, and conditional logic.
  • Built for scale and performance.
  • Suitable for both SMBs and enterprise teams.
  • Full-stack Node.js framework integrating frontend and backend workflows.
  • Combines koa (backend), egg (framework), Vue (frontend), and Framework7 (mobile UI).
  • Built-in workflow engine supports approval chains and business logic.
  • Offers modular development with pluggable components.
  • Mobile-first design approach with responsive UI.
  • Suitable for enterprise-grade web apps.
  • Comprehensive developer documentation and examples.
  • Supports multi-tenant architecture and RBAC.
  • Easy to extend and customize.
  • Actively maintained with growing community support.
  • Distributed orchestration engine by Uber.
  • Manages long-running, stateful, asynchronous workflows.
  • Fault-tolerant with strong retry and error-handling support.
  • Guarantees durability and scalability in production.
  • Offers language bindings (Java, Go, etc.).
  • Designed for microservice communication.
  • No single point of failure; supports horizontal scaling.
  • Open-source under MIT license.
  • Backed by production usage at Uber.
  • Foundation for the Temporal framework.
  • Business Process Model and Notation (BPMN) workflow engine.
  • Can be embedded into Java apps or used standalone.
  • Offers graphical modeler and web-based task management UI.
  • Integrates well with Spring Boot and enterprise Java stacks.
  • Provides REST APIs for orchestration.
  • Designed for enterprise use: highly extensible and robust.
  • Features include human task management and decision tables.
  • Actively maintained with commercial support available.
  • Used in telecom, banking, insurance sectors.
  • Good documentation and modeling tools for business users.
  • Open-source continuous delivery system built by OVHcloud.
  • Designed for DevOps workflows and CI/CD pipelines.
  • Supports pipeline as code with YAML configuration.
  • Scales across multiple teams and projects.
  • Includes advanced RBAC, worker management, and secrets handling.
  • Rich dashboard and web UI.
  • Integrates with GitHub, GitLab, Docker, etc.
  • Runs on Docker, Kubernetes, or bare metal.
  • Enterprise-grade reliability with audit and tracking.
  • Enables end-to-end automation of software delivery.
  • C++17-based DAG framework for building graph-based workflows.
  • Lightweight and header-only with no external dependencies.
  • Supports multi-threaded task execution.
  • Modular and customizable graph nodes.
  • Designed for performance-critical applications.
  • Offers visualization tools for debugging.
  • Runs on Windows, Linux, and macOS.
  • Ideal for embedded, gaming, or real-time systems.
  • Simple API for fast prototyping.
  • Well-documented and easy to integrate.
  • Open-source workflow engine for DevOps automation.
  • Based on YAML DSL for defining workflows.
  • Integrates with tools like Jenkins, Docker, and Ansible.
  • Suitable for hybrid cloud and on-prem automation.
  • Strong focus on task orchestration.
  • Web-based IDE for workflow creation.
  • Supports complex error handling and branching logic.
  • Active community and enterprise support options.
  • Lightweight and easy to deploy.
  • Built by HP Software (now Micro Focus).
  • Microservices orchestration platform.
  • Developed originally by Netflix, maintained by Orkes.
  • Designed to build and manage long-running application workflows.
  • Supports JSON/YAML-based workflow definitions.
  • Language-agnostic; integrates with any backend.
  • Offers dynamic task assignment and retry policies.
  • Scales horizontally and supports millions of workflows.
  • Comes with web UI, API server, and rich developer tooling.
  • Used by large-scale cloud-native systems.
  • Backed by an active open-source and commercial ecosystem.
  • Java-based high-performance workflow engine.
  • Built for complex, long-running business processes.
  • Supports BPMN and custom workflows.
  • Compatible with Spring and Java EE ecosystems.
  • Uses persistent state and asynchronous task execution.
  • Fine-grained control over scheduling and execution.
  • Features include correlation handling and transaction support.
  • Pluggable storage backends (DB, JMS, etc.).
  • Lightweight alternative to heavyweight BPM engines.
  • Maintained and battle-tested in real-world applications.
  1. Unified interface to define workflows across multiple engines like Argo, Tekton, and Airflow.
  2. Enables abstraction of workflow logic from execution engines.
  3. Simplifies switching between different workflow orchestrators.
  4. Written in Python and supports Pythonic constructs for defining workflows.
  5. Reduces code duplication when targeting multiple workflow backends.
  6. Highly useful for hybrid-cloud and multi-cloud orchestration.
  7. Offers seamless integration with Kubernetes-native tools.
  8. Lightweight and developer-friendly interface.
  9. Encourages modular workflow design.
  10. Backed by an active open-source community.
  1. Workflow orchestration platform for quantum and high-performance computing (HPC).
  2. Abstracts backend compute environments.
  3. Supports plugins for extending to custom hardware.
  4. Python-native workflow syntax.
  5. Real-time execution tracking and visualization tools.
  6. Built to handle complex DAGs with minimal configuration.
  7. Focus on reproducibility and scientific computing.
  8. Integrates with SLURM and other HPC schedulers.
  9. Supports containerized and virtual environments.
  10. Designed to scale across CPUs, GPUs, and quantum devices.
  1. Developed by the Broad Institute.
  2. Executes workflows defined in WDL (Workflow Description Language) and CWL.
  3. Tailored for genomics and biomedical research workflows.
  4. Provides local and cloud-based execution backends.
  5. High focus on performance and scalability.
  6. Built-in support for Docker containers.
  7. Metadata tracking and logging features.
  8. Offers REST API for external interaction.
  9. Easily integrates into bioinformatics pipelines.
  10. Strong support community and documentation.
  1. Advanced workflow engine for cyclic and acyclic tasks.
  2. Originally designed for weather and climate modeling.
  3. Handles complex dependencies and triggering logic.
  4. Python-based configuration files.
  5. Distributed task orchestration.
  6. Supports HPC and cloud environments.
  7. Includes a powerful GUI dashboard.
  8. Emphasizes repeatability and auditability.
  9. Ideal for scientific research use cases.
  10. Backed by NIWA and open-source contributors.
  1. No-code workflow executor built with simplicity in mind.
  2. Workflows defined using YAML configuration.
  3. Command-line based interface.
  4. Executes workflows locally without external dependencies.
  5. Lightweight and fast.
  6. Ideal for developers preferring configuration over coding.
  7. Supports variable inputs and parallel tasks.
  8. Good for small-scale automations and prototypes.
  9. Integrates easily with other CLI tools.
  10. Maintains high readability and low complexity.
  1. Open-source orchestration platform for ML, analytics, and ETL.
  2. Strong focus on data pipeline observability.
  3. Type-safe and testable pipelines.
  4. Built-in versioning and metadata tracking.
  5. Web UI for pipeline visualization and management.
  6. Native support for software-defined assets (SDAs).
  7. Plugin system for popular data tools.
  8. Emphasizes modularity and reusability.
  9. Cloud-ready with Dagster Cloud offering.
  10. Fast-growing community and enterprise backing.
  1. Durable workflows using Dapr runtime.
  2. Built on top of the Durable Task Framework.
  3. Supports multiple languages: Python, JavaScript, .NET, Java, Go.
  4. Integrates with Dapr pub/sub, state stores, bindings.
  5. Enables event-driven and serverless architectures.
  6. High fault-tolerance and scalability.
  7. Declarative workflow definitions.
  8. Strong consistency guarantees.
  9. Fits well into microservices architecture.
  10. Maintained by the Dapr open-source community.
  1. .NET-native job orchestration framework.
  2. Easy integration into existing .NET applications.
  3. Declarative workflow definitions using C#.
  4. Built-in scheduling and retry logic.
  5. Asynchronous job handling.
  6. Plugin system for extensibility.
  7. Real-time job tracking and status updates.
  8. Lightweight with minimal overhead.
  9. Ideal for enterprise .NET ecosystems.
  10. Open source and community-driven.
  1. Open-source workflow engine for complex pipelines.
  2. Supports multiple backends including Hadoop, Docker, etc.
  3. Simple YAML-based job definitions.
  4. Designed for robustness and repeatability.
  5. Handles retries, dependencies, and scheduling.
  6. CLI-based and embeddable in CI/CD pipelines.
  7. Plugin support for custom tasks.
  8. Good fit for data engineering teams.
  9. Cloud and on-prem deployment ready.
  10. Maintained by Treasure Data.
  1. Enterprise-grade workflow scheduler.
  2. DAG-based UI for job orchestration.
  3. Strong support for big data tasks.
  4. Built-in alerting, retries, and SLA monitoring.
  5. RESTful API support.
  6. Rich set of job types out of the box.
  7. Role-based access control and auditing.
  8. Extensible plugin architecture.
  9. Suited for data integration and batch processing.
  10. Backed by Apache Foundation.
  1. .NET Standard 2.0 workflow library.
  2. Supports long-running and event-driven workflows.
  3. Visual designer for workflow modeling.
  4. API-first architecture.
  5. Easily integrates with ASP.NET Core apps.
  6. Custom activity support.
  7. Built-in persistence and storage providers.
  8. Multi-tenant workflow engine.
  9. Background service hosting.
  10. Ideal for enterprise .NET solutions.
  1. Lightweight Java rules engine.
  2. Based on the concept of rules, facts, and inference.
  3. Embeddable in any Java application.
  4. Enables separation of business rules from application logic.
  5. Easy to define, combine, and prioritize rules.
  6. Supports conditional logic chaining.
  7. Useful in decision engines and fraud detection.
  8. Can be used in workflow conditions.
  9. Simple API and low learning curve.
  10. Actively used in enterprise Java applications.
  • FireWorks Stars - FireWorks stores, executes, and manages calculation workflows.
  • Fission Workflows Stars - A high-perfomant workflow engine for serverless functions on Kubernetes.
  • Flor Stars - A workflow engine written in Ruby.
  • Flyte Stars - A container-native, type-safe workflow and pipelines platform optimized for large scale processing and machine learning written in Golang. Workflows can be written in any language, with out of the box support for Python.
  • ForML Stars - A development framework and MLOps platform for the lifecycle management of data science projects.
  • Galaxy Project Stars - Galaxy is a scientific tool and workflow system that facilitates reproducibility and transparency by providing a graphical user interface (GUI), a multitude of computational dataset tools, a rich API, and a robust scientific community.
  • Goflow Stars - A simple but powerful DAG scheduler and dashboard, written in Go.
  • Huginn Stars - Create agents that monitor and act on your behalf. Your agents are standing by!
  • Imixs-Workflow Stars - A powerful human-centric Workflow Engine based on the BPMN 2.0 standard.
  • Inngest Stars - An event-driven workflow engine that combines event streams, queues, and durable execution into a single reliability layer.
  • iWF Stars - iWF is a WorkflowAsCode API orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes.
  • Kestra Stars - Open source data orchestration and scheduling platform with declarative syntax.
  • Kiba Stars - Data processing & ETL framework for Ruby
  • Kubeflow pipelines Stars - Kubeflow pipelines are reusable end-to-end ML workflows built using the Kubeflow Pipelines SDK.
  • Laravel Workflow Stars - Laravel Workflow is a durable workflow engine that allows users to track job status, orchestrate microservices and write long running persistent distributed workflows in PHP powered by Laravel Queues.
  • Petri Flow Stars - Petri Net workflow engine for Ruby.
  • Martian Stars - An elegant, powerful language and framework for building high-performance computational pipelines.
  • Metaflow Stars - Metaflow is a human-friendly Python/R library that helps scientists and engineers build and manage real-life data science projects.
  • MassTransitStars - .Net Messaging system with Saga Workflow Support
  • Mistral Stars - Workflow service, in OpenStack foundation.
  • N8n-io Stars - Free and open node based Workflow Automation Tool. Easily automate tasks across different services.
  • Nextflow Stars - Develop container-backed, reproducible workflows portable across computational platforms including local, HPC schedulers, AWS Batch, Google Genomics Pipelines, and Kubernetes.
  • Node-RED Stars - Node-RED is a NodeJS based workflow tool featuring a browser based editor for wiring together hardware devices, APIs and online services in new and interesting ways.
  • Oozie Stars - Workflow Scheduler for Hadoop.
  • Pallets Stars - Simple and reliable workflow engine, written in Ruby
  • Parsl Stars - Python framework for workflow orchestration and parallelization based on a dynamic graph of tasks and their data dependencies.
  • Pegasus Stars - Automate, recover, and debug scientific computations.
  • Piper Stars - A distributed Java workflow engine designed to be dead simple.
  • Platformeco - Technology platform, allows Product & Project teams easily build micro services using drag & drop UI and operate it, within out of the box cloud and CI/CD tools with deep tracing & monitoring.
  • Plynx Stars - Interactive platform with drag and drop interface for building and deploying portable and scalable end-to-end data driven workflows.
  • Popper Stars - Lightweight, YAML based container-native workflow engine supporting Docker, Singularity, Vagrant VMs with Docker daemon in VM, and local host.
  • Prefect Stars - Prefect is a new workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine
  • Restate Stars - A low-latency durable execution engine for workflows, event-driven handlers, transactional state machines, and stateful orchestration. Fully self-contained in a single binary. Supports programs in TypeScript, Java/Kotlin, Go, Python, and Rust.
  • River Pro Stars - Postgres-backed workflow engine and web dashboard for the River background queue, written in Go.
  • RunDeck Stars - Job Scheduler and Runbook Automation.
  • Snakemake Stars - Workflow management system to create reproducible and scalable data analyses; python-based inspired by GNU Make.
  • StackStorm Stars - Robust Automation Engine providing Sensors, Triggers, Rules, Workflows, and Actions. StackStorm is how you "glue" your applications together.
  • StepWise Stars - A code-first, event-driven workflow framework for .NET Developers.
  • Temporal Stars - Temporal is a microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or reliability. Temporal is a mature technology, a fork of Uber's Cadence. Temporal is being developed by Temporal Technologies, a startup by the creators of Cadence.
  • Titanoboa Stars - Titanoboa is a platform for creating complex workflows on JVM.
  • Tork Stars - Tork is a very lightweight workflow engine written in Golang.
  • uTask Stars - Automation engine that models and executes business processes declared in yaml.
  • Wexflow Stars .NET Workflow Engine and Automation Platform.
  • Windmill Stars - Turn scripts into workflows and UIs. Open-source alternative to Airplane and Retool.
  • Workflow Engine - A lightweight .NET and Java workflow engine.
  • YAWL Stars - (Yet Another Workflow Language), Java-based, handles complex data transformations, and full integration with organizational resources and external Web Services.
  • Zeebe Stars - A horizontally scalable, cloud-native workflow engine that executes BPMN models and is best operated on Kubernetes; polyglot clients connect via gRPC or available language clients.

BPM Suite

  • Activiti Stars - Activiti is a leading lightweight, java-centric open-source BPMN engine supporting real-world process automation needs.
  • Activiti Cloud - is now the new generation of business automation platform offering a set of cloud native building blocks designed to run on distributed infrastructures.
  • Bonita Stars - BPMN engine that comes with an optional development environment, a designer, an optional user interface and administrative tools.
  • Flowable Stars - The Flowable project provides a core set of open source business process engines that are compact and highly efficient. They provide a workflow and Business Process Management (BPM) platform for developers, system admins and business users.
  • jBPM Stars - The core of jBPM is a light-weight, extensible workflow engine written in pure Java that allows you to execute business processes using the latest BPMN 2.0 specification.

SAAS

  • AWS Step Functions - Clear workflows for modern applications.
  • Azure Logic Apps - Built on a containerized runtime, deploy and run anywhere to increase scale and portability while automating business-critical workflows.
  • Braze - Power customer-centric interactions between consumers and brands in real-time.
  • Camunda Cloud - A workflow service executing BPMN, providing various language clients, based on the open source project Zeebe Stars.
  • Corezoid - Hyperautomation engine.
  • Embed Workflow - Simple native workflow builder for your end-users
  • Orkes Conductor - Orkes provides Netflix Conductor as a cloud services across all the major cloud providers with enterprise features such as security, integrations and Visual workflow code editor.
  • Google Cloud Workflows - Combine Google Cloud services and APIs to build reliable applications, process automation, and data and machine learning pipelines.
  • Inngest - A scalable, event-driven durable execution platform
  • Zenaton - Workflow engine for data processes and background jobs available in PHP, Node.js, Python and Ruby.

Library (embedded usage)

  • Automatiko Stars - a toolkit to build services and functions based on workflows (primarily BPMN2). Introduces and implements concepts: workflow as a service, workflow as a function and workflow as a function flow.
  • C++ Workflow Stars - C++ Parallel Computing and Asynchronous Networking Engine.
  • Camunda Stars - BPMN-based workflow engine that can be embedded as java library (e.g. Spring Boot) or used standalone, including a graphical modeler and operations tooling.
  • Captain Stars - Distributed, light-weight java workflow engine for a microservice architecture.
  • CoreWF Stars - WF runtime ported to work on .NET Core
  • Dagger - Dagger is a distributed, horizontally scalable, durable, and highly available orchestration engine in python based on Faust-Streaming for running millions of long running tasks with direct integration with Kafka
  • Django River Stars - Django workflow library that supports on the fly changes for states, transitions, and authorizations
  • DBOS Transact Stars - DBOS typescript library for writing durable and transactional workflows.
  • Durable Task Framework Stars - Durable Task Framework (DTFx) is a library that allows users to write long running persistent workflows in C#.
  • Kogito Stars - Cloud-native business automation technology. Embeddable to produce JVM or GraalVM artifacts or interacted through Kafka & REST APIs
  • Luigi Stars - Python module that helps you build complex pipelines of batch jobs.
  • nFlow Stars - Embeddable JVM-based workflow engine with high availability, fault tolerance, and support for multiple databases. Additional libraries are provided for visualization and REST API.
  • Oban - Robust job processing in Elixir, backed by modern PostgreSQL or SQLite3
  • SciPipe Stars - A Go library for writing pipelines of Bash commands or Go-code using the dataflow / flow-based programming paradigm.
  • SpiffWorkflow Stars - SpiffWorkflow - a BPMN 2.0 workflow engine implemented in pure Python.
  • Symfony Workflow Stars - Symfony Workflow component - The Workflow component provides tools for managing a workflow or finite state machine in PHP.
  • Unify Flowret Stars - A lightweight Java based workflow / orchestration engine.
  • Viewflow Stars - Reusable workflow library that helps organize people collaboration business logic in django applications.
  • Workflow Core Stars - Lightweight workflow engine for .NET Standard
  • WorkflowEngine.NET Stars - WorkflowEngine.NET - component that adds workflow in your application. It can be fully integrated into your application, or be in the form of a specific service (such as a web service)