Last checked and updated on 06 Jun 2026
What Is New in Apache Kafka 0.11
This release is a major step forward, introducing foundational features that became core to the Kafka ecosystem. It's packed with critical improvements for both developers and operators.
| Category | Key Changes |
|---|---|
| New Features | Exactly-Once Semantics (EOS), Transactional Messaging, New Message Format |
| Improvements | Enhanced AdminClient, Better Quota Management, Log Compaction Optimizations |
| Bug Fixes | Numerous fixes for replication, consumer coordination, and controller stability |
| Deprecations | Deprecation of Scala-based producers and consumers in favor of Java clients |
How does exactly-once semantics change the game?
Exactly-once semantics (EOS) is the headline feature. It fundamentally solves the problem of duplicate or lost messages during producer retries and consumer reprocessing.
Before 0.11, you had at-least-once or at-most-once delivery. EOS gives you atomic writes across partitions, which
is huge for financial transactions or any state-changing operation. You enable it by setting
enable.idempotence=true on your producers and using the new transactional APIs.
What are the new transactional messaging capabilities?
Transactions allow you to atomically publish messages to multiple partitions and topics. This means all messages in a transaction are committed together or none are.
You initiate a transaction with producer.initTransactions(), then bracket your sends with
beginTransaction() and commitTransaction(). On the consumer side, you can use
isolation.level=read_committed to only read committed messages, avoiding those from aborted
transactions.
What improvements were made to the AdminClient?
The AdminClient got a lot more powerful, moving beyond basic topic creation. You can now delete topics directly through the API instead of relying on auto-deletion or ZooKeeper tricks.
This makes operational scripts and tooling much cleaner. You have a proper programmatic interface for cluster management tasks, which is essential for automation.
How is quota enforcement better now?
The quota system was overhauled to be more responsive and fair. It now uses a token-based algorithm to throttle clients that exceed their bandwidth quotas, leading to smoother performance for everyone else on the cluster.
In practice, this means a single misbehaving client is less likely to bring your entire cluster to its knees. The throttling happens more gradually and predictably.
What should I know about the new message format?
The new message format (v2) is the backbone for the transactional features. It includes new record headers for carrying metadata independently of the key/value payload and a new structure to support transactions.
You don't need to manually change your serializers, but the brokers need to be upgraded to understand this format. It's a backward-compatible change, so older clients can still read from topics using the new format.
FAQ
Do I have to use transactions? Will they slow down my producers?
No, transactions are
optional. You only pay the performance overhead if you enable idempotence (enable.idempotence=true)
and use the transactional APIs. For most use cases, the overhead is minimal compared to the reliability gain.
Can I use the new Java client with an older broker?
You can use a 0.11 Java client to talk to
an older broker, but you won't have access to the new features like transactions. The client is
backward-compatible. The reverse (old client, new broker) also works but without new features.
What happens to my existing Scala producers and consumers?
They still work, but they are now
deprecated. The Java clients are the future. You should plan to migrate any new development to the Java clients
for better long-term support and performance.
How do I upgrade my cluster to support the new message format?
You can enable the new
message format on a per-topic basis using the message.format.version configuration. During the
upgrade, you can run both old and new format producers in mixed mode. The brokers will automatically handle
the different formats, but you'll need to plan a maintenance window to switch existing topics to the new
format to fully utilize all the new features.
Do I need to upgrade all brokers at once to use the new features?
No, you can perform a
rolling upgrade. Kafka 0.11 is backward-compatible, so mixed clusters (some brokers on 0.11, others older) can
run temporarily. However, to fully leverage transactional messaging and the new message format, all brokers
should eventually be upgraded to 0.11. This ensures consistent behavior and avoids limitations when using the
new APIs.