Introducing Tansu.io -- Rethinking Kafka for Lean Operations

Tansu is an open-source, Apache Kafka®-compatible messaging broker designed to be simpler and more flexible than traditional Kafka clusters. It implements the Kafka protocol so Kafka clients and tools can work with it, but it’s built with stateless brokers and pluggable storage backends rather than relying on Kafka’s built-in replicated storage and consensus protocols.

Key Takeaways:

• Kafka API compatible: Works with standard Kafka clients and tools (topic creation, producers, consumers).
• Stateless brokers: Brokers don’t store persistent data themselves; they rely on external storage systems.
• Pluggable storage engines: Supports storage in PostgreSQL, SQLite (libSQL), S3 (or S3-compatible), or in-memory for different use cases.
• Schema validation: Messages can be validated against Avro, JSON, or Protocol Buffers schemas.
• Open table format support: Can write topics directly into Apache Iceberg or Delta Lake formats for analytics pipelines.
• Single binary & CLI: Distributed as one executable with command-line tools for managing topics and producing/consuming messages.
• Simplicity: Designed to reduce operational complexity (no ZooKeeper/Raft, easy to deploy and scale) while enabling lightweight Kafka-compatible streaming.

In short, Tansu is like a lightweight, Kafka-compatible streaming platform built for simplicity, flexible storage, and modern data pipelines.