Apache Druid can handle streaming ingestion jobs at any scale. This occasionally poses challenges. Apache Kafka is a popular distributed event streaming platform, so it’s a natural choice for this presentation. As a community, we hope to improve our existing documentation of Kafka errors, so we’re hoping that this presentation will get the ball rolling. We’ll cover basics about setting up a Kafka streaming ingestion job, including a best practice tip or two such as monitoring your deployment with metrics and logs. We’ll also talk about the location of Kafka related logs within your deployment. We’ll conclude with some common Kafka ingestion errors and solutions, specifically lag and parsing.
Keynote: Powering Event-Driven Data with Apache Druid
The distinction between OLTP and OLAP is becoming less relevant as data architectures shift toward entities and events. In this session, we’ll delve into how Apache Druid’s event-first approach synthesizes...
Watch nowClosing Keynote: Charting the Future of Druid
What lies ahead for Apache Druid? Join us as we explore the evolving landscape of Druid’s query and storage engines, and how they are positioned to address the biggest challenges in event data for the future. Speaker: Gian...
Watch nowSalesforce: Tracing Service Dependencies at Scale with Druid and Flink
At Salesforce, we manage approximately 300 million distributed spans to infer service dependencies. We have successfully utilized a combination of Druid and Flink to handle this scale with high availability....
Watch now