From Event-Driven Architectures to Reactive Systems
How events have been, and are being, used to build more robust systems
This is the first of three installments of my series on event-driven architectures and reactive systems. Here are part two and part three.
Events have been used to integrate applications and build distributed systems since the late 80s and more notably from the 90s on with the help of Message-oriented Middlewares (MOM). We have observed the rise of different architectural patterns leveraging them: from Event-Driven Architectures (EDA) through Event-Driven SOA (aka SOA 2.0) to Reactive systems more recently.
But what is an event in the first place? According to Wikipedia,
In programming and software design, an event is an action or occurrence recognized by software, often originating asynchronously from the external environment, that may be handled by the software.
So an event has the intent of communicating a significant change in state of some system’s component to others that might be interested in. However, events do not travel, they just occur. Therefore we need a way to broadcast events and the mostly used one is through notification messages.
This explains why EDAs often leverage the messaging integration style which relies on some sort of Message Broker to implement a topology known as Hub-and-Spoke. There are many well-known examples of either on-prem or cloud-based message brokers: RabbitMQ, NATS, Kafka, IBM MQ (former MQSeries), MSMQ, Amazon SQS and SNS, Azure Service Bus, and so on and so forth.
It is worth mentioning we can use Spring Cloud Stream to develop microservices that can be easily ported to use some of the aforementioned message brokers and other streaming data services without any change in the source code.
Spring Cloud Stream is a framework for building message-driven microservice applications which builds upon Spring Boot to create standalone, production-grade Spring applications and uses Spring Integration to provide connectivity to message brokers. It provides opinionated configuration of middleware from several vendors, introducing the concepts of persistent publish-subscribe semantics, consumer groups, and partitions.
Well, all this is not new so the question is why are we witnessing the resurgence of EDAs with microservices?
Reactive Programming
To answer this question, we need to come back to the beginning of the century when the exponential growth of the World Wide Web posed the problem of optimising network sockets to handle a large number of clients at the same time, the famous C10k problem. The term was coined in 1999 by Dan Kegel citing the Simtel FTP host, cdrom.com, serving 10,000 clients at once over 1 Gbps Ethernet in that year. The technique of serving many clients with each server process or thread and use asynchronous I/O to avoid blocking had already been envisaged.
Ryan Dahl released in 2009 Node.js, a JavaScript environment decoupled from the browser. Using Google’s V8 engine and Marc Lehmann’s libev async I/O library, Node.js combined a model of I/O — evented — with a language that was well suited to the style of programming due to the way it had been shaped by browsers. The impressive results in terms of scalability took the industry by storm.
Meanwhile, Reactive programming began to receive special attention. At Microsoft, Erik Meijer created in 2011 a reactive programming framework for .NET called Reactive Extensions, a set of tools allowing imperative programming languages to operate on sequences of data regardless of whether the data is synchronous or asynchronous. In a matter of years, Reactive Extensions (aka Rx) was ported to several languages and platforms, including JavaScript, Python, C++, Swift, and Java.
In the JVM world, Reactive Streams started as an initiative in late 2013 between engineers from Pivotal (now VMware), Netflix and Lightbend (previously Typesafe). In 2015, version 1.0.0 of Reactive Streams was released. Reactive Streams 1.0.1 was released in 2017.
So what is Reactive programming? It is a declarative, event-driven programming paradigm concerned with data streams and the propagation of change. It is the availability of new information that drives the logic forward rather than having control flow driven by a thread-of-execution. It supports decomposing the problem into multiple discrete steps where each can be executed in an asynchronous and non-blocking manner, and then be composed to produce a workflow — possibly unbounded in its inputs or outputs.
The fundamental idea behind reactive programming is that events are data and data are events.
If you want to adopt reactive programming in your projects, I highly recommend Spring Reactor and Spring WebFlux. Spring Reactor is a fourth-generation reactive library that is leading the advance of the Reactive Streams specification.
From Reactive Programming to Reactive Systems
Reactive Programming can be used within components/services and focuses on decomposing the computation into multiple discrete steps to produce a workflow (aka Dataflow programming).
But what about the interaction between components/services? In order to reason about a system at a higher level one has to use another tool[1]:
Reactive Architecture — the process of designing Reactive systems.
Reactive systems leverage the already mentioned messaging integration style and focus on resilience and elasticity through the communication and coordination of distributed systems.
In order to facilitate Responsiveness, two challenges need to be faced: being Responsive under failure, defined as Resilience, and being Responsive under load, defined as Elasticity. The Reactive Manifesto prescribes that in order to achieve this, the system needs to be Message-driven.
The Reactive Manifesto was published in 2014 to describe this new architectural style. I consider it a mandatory reading to anybody working as a software engineer or architect.
To Be Continued
We have covered the origin and evolution of event-driven architectures and how we get where we are today. Hopefully you found it useful. Next time, I am going to share some design patterns that I find extremely useful when developing event-driven systems. Stay tuned.
