Conductor (software)

Last updated
Conductor
Developer(s) Netflix Team (Originally) Orkes & Open Source contributors (Current)
Initial releaseDecember 8, 2016;7 years ago (2016-12-08)
Stable release
3.15.0 / March 20, 2023;9 months ago (2023-03-20)
Repository github.com/conductor-oss/conductor
Written in Java, JavaScript (UI), Python, Go
Platform Linux
License Apache License 2.0
Website www.conductor-oss.org

Conductor is a free and open-source microservice orchestration software platform originally developed by Netflix. [1] [2]

Contents

Conductor was developed by Netflix to solve the problems of orchestrating microservices and business processes at scale in a cloud native environment. [3] It was released under the Apache License 2.0 and has been adopted by companies looking to orchestrate their processes at scale in a cloud native environment. [4] [5]

Conductor belongs to a set of software products that allows developers to build resilient, high-scale, cloud-native stateful applications using stateless primitives.

Architecture

Conductor server is written in Java with APIs exposed over HTTP and gRPC interfaces making it possible to do language agnostic development. [6] A set of client libraries are made available by Netflix and community in Java, Python and Go.

Conductor uses a lightweight JSON based schema with rich programming language constructs such as fork/join, switch case, loops and exception handling to define the flows.

At the heart of Conductor is a queuing system that is used to schedule tasks and manage the process flows.  Conductor leverages a pluggable model allowing different implementations of the queuing system.  Open source version uses Dyno-Queues developed at Netflix for queuing as default implementation [7]

The workflows are defined as the orchestration among the tasks which can be a system level construct such as fork, join, switch, loop, an external HTTP endpoint implementing business logic or a task worker running outside of Conductor servers and listening for work to be scheduled by the server. The workers communicate with the server using pre-defined APIs over HTTP or gRPC. Conductor provides lightweight libraries to manage worker states in Java, Python and Go and additional languages can be used to implement logic using provided APIs.

Conductor uses pluggable architecture model allowing for different databases to store its states. The current version has support for Redis (Stand-alone, Sentinel, Cluster and Dynomite), Postgres, Mysql, Cassandra and uses Elasticsearch as indexing mechanism.

The UI is written in ReactJS and provides ability to search, visualize and manage the workflow states.

Cloud Hosted Services

Orkes - was founded by the creators of Netflix Conductor and provides cloud managed offering of Netflix Conductor in various clouds and in data center. [8]

Related Research Articles

Camunda provides a workflow and decision automation platform offering process orchestration capabilities to organizations of any size. Camunda Platform comes from years of research and development including open source initiatives.

Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating systems and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.

<span class="mw-page-title-main">Aptana</span> Text editor

Aptana, Inc. is a company that makes web application development tools for use with a variety of programming languages. Aptana's main products include Aptana Studio, Aptana Cloud and Aptana Jaxer.

A workflow engine is a software application that manages business processes. It is a key component in workflow technology and typically makes use of a database server.

Nuxeo is a software company making an open source content management system.

Google App Engine is a cloud computing platform as a service for developing and hosting web applications in Google-managed data centers. Applications are sandboxed and run across multiple servers. App Engine offers automatic scaling for web applications—as the number of requests increases for an application, App Engine automatically allocates more resources for the web application to handle the additional demand.

<span class="mw-page-title-main">Apache Qpid</span> Open-source messaging system

Apache Qpid is an open-source messaging system which implements the Advanced Message Queuing Protocol (AMQP). It provides transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support. The Apache Qpid API supports multiple programming languages and comes with both C++ and Java brokers.

<span class="mw-page-title-main">AppScale</span> American cloud infrastructure software company

AppScale is a software company offering cloud infrastructure software and services to enterprises, government agencies, contractors, and third-party service providers. The company commercially supports one software product, AppScale ATS, a managed hybrid cloud infrastructure software platform that emulates the core AWS APIs. In 2019, the company ended commercial support for its open-source serverless computing platform AppScale GTS, however, its source code remains freely available to the open-source community.

<span class="mw-page-title-main">OpenStack</span> Cloud computing software

OpenStack is a free, open standard cloud computing platform. It is mostly deployed as infrastructure-as-a-service (IaaS) in both public and private clouds where virtual servers and other resources are made available to users. The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data center. Users manage it either through a web-based dashboard, through command-line tools, or through RESTful web services.

Apache Attic is a project of Apache Software Foundation to provide processes to make it clear when an Apache project has reached its end-of-life. The Attic project was created in November 2008. Also the retired projects can be retained.

Google APIs are application programming interfaces (APIs) developed by Google which allow communication with Google Services and their integration to other services. Examples of these include Search, Gmail, Translate or Google Maps. Third-party apps can use these APIs to take advantage of or extend the functionality of the existing services.

Kubernetes is an open-source container orchestration system for automating software deployment, scaling, and management. Originally designed by Google, the project is now maintained by the Cloud Native Computing Foundation.

In software engineering, a microservice architecture is a variant of the service-oriented architecture structural style. It is an architectural pattern that arranges an application as a collection of loosely coupled, fine-grained services, communicating through lightweight protocols. One of its goals is that teams can develop and deploy their services independently of others. This is achieved by the reduction of several dependencies in the code base, allowing developers to evolve their services with limited restrictions from users, and for additional complexity to be hidden from users. As a consequence, organizations are able to develop software with fast growth and size, as well as use off-the-shelf services more easily. Communication requirements are reduced. These benefits come at a cost to maintaining the decoupling. Interfaces need to be designed carefully and treated as a public API. One technique that is used is having multiple interfaces on the same service, or multiple versions of the same service, so as to not disrupt existing users of the code.

WAMP is a WebSocket subprotocol registered at IANA, specified to offer routed RPC and PubSub. Its design goal is to provide an open standard for soft, real-time message exchange between application components and ease the creation of loosely coupled architectures based on microservices. Because of this, it is a suitable enterprise service bus (ESB), fit for developing responsive web applications or coordinating multiple connected IoT devices.

Imixs Workflow is an Open-Source-Project, providing technologies for building Business Process Management solutions. The project focus on human based workflows used to execute and control workflows in organisations and enterprises. In difference to task-oriented workflow engines, which focus on automated program flow control (tasks), Imixs Workflow is a representative of an event-based workflow engine. Here, the engine controls the status of a process instance within a defined state-diagram. By entering an event, the state of a process instance can be abandoned or changed. In human-centric workflow engines, events usually occur by an interaction of the actor with the system, for example by approving or rejecting a business transaction. They can also be triggered by scheduled events. An example of this is an escalation of an unfinished task.

gRPC is a cross-platform open source high performance remote procedure call (RPC) framework. gRPC was initially created by Google, which used a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across its data centers from about 2001. In March 2015, Google decided to build the next version of Stubby and make it open source. The result was gRPC, which is now used in many organizations aside from Google to power use cases from microservices to the "last mile" of computing. It uses HTTP/2 for transport, Protocol Buffers as the interface description language, and provides features such as authentication, bidirectional streaming and flow control, blocking or nonblocking bindings, and cancellation and timeouts. It generates cross-platform client and server bindings for many languages. Most common usage scenarios include connecting services in a microservices style architecture, or connecting mobile device clients to backend services.

The Cloud Native Computing Foundation (CNCF) is a Linux Foundation project that was founded in 2015 to help advance container technology and align the tech industry around its evolution.

Quarkus is a Java framework tailored for deployment on Kubernetes. Key technology components surrounding it are OpenJDK HotSpot and GraalVM. The goal of Quarkus is to make Java a leading platform in Kubernetes and serverless environments while offering developers a unified reactive and imperative programming model to optimally address a wider range of distributed application architectures.

References

  1. "Netflix Conductor, an Orchestration Engine for Microservices". InfoQ. Retrieved 2021-09-22.
  2. "Conductor, the newest tool from Netflix for orchestration of…". en.paradigmadigital.com. Retrieved 2021-09-22.
  3. Blog, Netflix Technology (2016-12-12). "Netflix Conductor: A microservices orchestrator".
  4. "Netflix open-sources Conductor, a microservices orchestration engine". JAXenter. 2016-12-13. Retrieved 2021-09-22.
  5. Krill, Paul (16 December 2016). "Netflix open-sources its microservices orchestrator". InfoWorld. Retrieved 23 September 2021.
  6. "Netflix Conductor, an Orchestration Engine for Microservices". InfoQ. Retrieved 23 September 2021.
  7. Blog, Netflix Technology (2017-04-19). "Distributed delay queues based on Dynomite". Medium. Retrieved 2021-09-19.
  8. Lunden, Ingrid (2022-02-28). "Orkes, founded by the creators of Netflix's open source Conductor workflow orchestration tool, comes out of stealth with $9.3M". TechCrunch. Retrieved 2023-03-09.