It’s a call to arms for event sourcing and a record everything model - you can do it if you have aggressive low latency requirements, so there isn’t a performance reason not to do it. If you have performance problems you’re doing it wrong.

The talk will be presented in general terms; and is applicable to a number of different problems and products, so it will be applicable to Kafka, JMS, Aeron, but when referring to implementation details the product that will be used as an example will be our company’s open-source product, Chronicle Queue.

Logging presents a challenge in your application: you don’t want to slow down the critical path but you want to record what is happening. However, if you log everything you can slow down your application. The key is to be able to log the right amount, so that the application doesn’t get slowed down, but enough to be useful.

So it is necessary to decide up front that you can record enough information to reproduce any problem, and that it’s being recorded by default.

Back-pressure is worth talking about: one of the differentiators of the messaging systems is how they implement back-pressure and this can determine if it is appropriate for your problem or not. In particular as a contrast a very good solution is Reactive Streams, which is what Akka supports. It’s been added to Java 9 as the Flow API, and one of the key elements about this is that the consumer pushes back on the producer in a fairly simply yet elegant way where it says give me another 5, another 2 etc. While it’s receiving data it’s giving the producer permission to give more data while applying back pressure, instead of polling for more information but without the latency downsides of blocking. It works particularly well with GUI clients, because they are running on different machines and with different networks - you don’t have a level of control that allows you to send (or consume) messages at a fixed rate.

With reactive streams you continually say whether how much you can handle, which you can then adjust over time.

In Chronicle Queue we have a different approach, by focussing on upstream, like market data or compliance. Instead of utilising back-pressure, we have an enormous buffer. The assumption is that you can run for a week without it filling, and as a part of your weekly maintenance cycle you can rotate it and delete, compress or archive it..