This is a real world account from a Druid cluster in production. A story of 48 hours of debugging, learning and understanding Druid better, filing a couple of issues in Druid github and finally a stable production pipeline again thanks to the Druid community. We will discuss the bottlenecks we had in overlord, slot issues for Peons in middle managers, coordinator bottlenecks, how to mitigated task and segment flooding, what configs we changed sprinkled with real world numbers and snapshots from our graphana dashboards. Finally we will list all the latest awesomeness in Druid 25.0 that helped us in this endeavour, how we discovered those midnight and our learnings.
Druid Summit 2024 – Panel: Real-Time Data Experiences
Analytics applications powered by Apache Druid are fun places to be. Take a seat as user experience leaders from Google, the Data Visualisation Society, and Imply bring their minds together, and answer your...
Watch nowDruid Summit 2024 – Panel: Lakehouse Analytics
Apache Iceberg and Delta Lake are coming into the big top, and Apache Druid is ready! Get ready to hear from Apache Druid PMC members and industry experts on just where Apache Druid fits, how it can be used,...
Watch nowDruid Summit 2024 – Panel: Operations and Optimization
Come along and meet people who have implemented and tuned Druid in situations small and large – very very large! Panelists will share some of their key tips and tricks, and be open to your questions, whether...
Watch now