Apache Druid can handle streaming ingestion jobs at any scale. This occasionally poses challenges. Apache Kafka is a popular distributed event streaming platform, so it’s a natural choice for this presentation. As a community, we hope to improve our existing documentation of Kafka errors, so we’re hoping that this presentation will get the ball rolling. We’ll cover basics about setting up a Kafka streaming ingestion job, including a best practice tip or two such as monitoring your deployment with metrics and logs. We’ll also talk about the location of Kafka related logs within your deployment. We’ll conclude with some common Kafka ingestion errors and solutions, specifically lag and parsing.
Physical Hardware, Digital Analytics: IoT Challenges, Best Practices, and Solutions
Electric vehicle maker Rivian and German startup Thing-it were kind enough to talk us through how real-time data and analytics play a key part in the evolving landscape of IoT (Internet of Things). The wealth...
Watch nowAnalyzing streaming data with Apache Druid
Streaming data is not only data in motion—it’s a potential source of valuable insights, ready to be harvested and utilized. The challenge is to analyze streaming data at scale and extract these insights—before...
Watch nowReal-Time Analytics in the Real World
Engineering teams increasingly have to deliver insights in real-time. But as they aim to reduce latency from event-to-insight, they also face the challenge of dealing with larger and more complex data and concurrent...
Watch now