Come along and meet people who have implemented and tuned Druid in situations small and large – very very large! Panelists shared some of their key tips and tricks, covering a range of topics from best practices in alerting and auto-scaling to strategies for handling ingestion and query optimization. This panel provided insights into managing Druid clusters efficiently, from Kubernetes and infrastructure management to data modeling, to ensure stable, high-performance clusters at scale.
Panelists:
– Cinto Sunny, Data Platform Engineer, Apple
– George Wu, Senior Software Engineer, Imply
– Ben Hopp, Lead Solutions Architect, Imply
[Timestamp] Table of Contents:
[00:00] Introductions
[1:00] Best Practices for Monitoring and Alerting
[3:00] Infrastructure as Code and Containerization
[5:00] Key Metrics to Monitor
[7:00] Common Mistakes in Cluster Management
[11:00] Scaling Strategies
[13:00] Metadata Backup and Restore
[15:00] Evaluating New Druid Versions
[17:00] Resource Utilization in Production
[19:00] Rolling Updates vs. Blue-Green Deployments
[21:00] Graceful Node Recycling
[23:00] Filling Feature Gaps in Druid Operators
[24:00] Educating Users on Druid’s Use Cases
[26:00] NVMe vs. Network Block Storage
[27:00] Starting Points for Performance Tuning
[29:00] Disaster Recovery and Failed Ingestion
[30:00] Mitigating Expensive Queries
[33:00] Segment-Related Troubleshooting
[34:00] Query Context for Custom Resource Settings
[36:00] Managing Segment Creation During Ingestion
[38:00] Load Balancing for Ingestion Tasks
[39:00] Closing Remarks
Keynote: Powering Event-Driven Data with Apache Druid
The distinction between OLTP and OLAP is becoming less relevant as data architectures shift toward entities and events. In this session, we’ll delve into how Apache Druid’s event-first approach synthesizes...
Watch nowClosing Keynote: Charting the Future of Druid
What lies ahead for Apache Druid? Join us as we explore the evolving landscape of Druid’s query and storage engines, and how they are positioned to address the biggest challenges in event data for the future. Speaker: Gian...
Watch nowSalesforce: Tracing Service Dependencies at Scale with Druid and Flink
At Salesforce, we manage approximately 300 million distributed spans to infer service dependencies. We have successfully utilized a combination of Druid and Flink to handle this scale with high availability....
Watch now