I’ve always thought that an intricate understanding of how something works helps you use it more effectively. In this session, I’ll try to convey what I’ve learned from some of Apache Druid’s creators about the inner workings of streaming ingestion in Apache Druid . We’ll start at the high level with how parallelism throughout the data pipeline drives scalability for real-time analytics. Then we’ll open the hood and see what’s inside each parallel ingestion task. We’ll talk threads, buffers, inner architecture and some of the main configuration parameters that control both its ability to ingest streaming data and process queries on arrival.
Keynote: Powering Event-Driven Data with Apache Druid
The distinction between OLTP and OLAP is becoming less relevant as data architectures shift toward entities and events. In this session, we’ll delve into how Apache Druid’s event-first approach synthesizes...
Watch nowClosing Keynote: Charting the Future of Druid
What lies ahead for Apache Druid? Join us as we explore the evolving landscape of Druid’s query and storage engines, and how they are positioned to address the biggest challenges in event data for the future. Speaker: Gian...
Watch nowSalesforce: Tracing Service Dependencies at Scale with Druid and Flink
At Salesforce, we manage approximately 300 million distributed spans to infer service dependencies. We have successfully utilized a combination of Druid and Flink to handle this scale with high availability....
Watch now