Apache Druid can handle streaming ingestion jobs at any scale. This occasionally poses challenges. Apache Kafka is a popular distributed event streaming platform, so it’s a natural choice for this presentation. As a community, we hope to improve our existing documentation of Kafka errors, so we’re hoping that this presentation will get the ball rolling. We’ll cover basics about setting up a Kafka streaming ingestion job, including a best practice tip or two such as monitoring your deployment with metrics and logs. We’ll also talk about the location of Kafka related logs within your deployment. We’ll conclude with some common Kafka ingestion errors and solutions, specifically lag and parsing.
Druid Summit 2024 – Panel: Real-Time Data Experiences
Analytics applications powered by Apache Druid are fun places to be. Take a seat as user experience leaders from Google, the Data Visualisation Society, and Imply bring their minds together, and answer your...
Watch nowDruid Summit 2024 – Panel: Lakehouse Analytics
Apache Iceberg and Delta Lake are coming into the big top, and Apache Druid is ready! Get ready to hear from Apache Druid PMC members and industry experts on just where Apache Druid fits, how it can be used,...
Watch nowDruid Summit 2024 – Panel: Operations and Optimization
Come along and meet people who have implemented and tuned Druid in situations small and large – very very large! Panelists will share some of their key tips and tricks, and be open to your questions, whether...
Watch now