Strategies for Aggregation, Data Cardinality, and Batching

Elad Eldor

Chapter Chapter 2 in Kafka Troubleshooting in Production, 2023, pp 17-24 from Springer

Abstract: Abstract This chapter digs into various adjustments you can make to the Kafka producer that can notably increase your Kafka system's speed, response time, and efficiency. The chapter starts by exploring the partitioning strategy, which aims for an equilibrium between distributing messages evenly and clustering related messages together. It will then dive into adjusting parameters like linger.ms and batch.size to improve speed and decrease response time. From there, you learn how the uniqueness and spread of data values, known as data cardinality, impact Kafka's performance. And finally, you explore why, in some cases, duplicating data for different consumers can be a smart move.

Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4842-9490-1_2

Ordering information: This item can be ordered from
http://www.springer.com/9781484294901

DOI: 10.1007/978-1-4842-9490-1_2

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().