Salesforce: Tracing Service Dependencies at Scale with Druid and Flink

At Salesforce, we manage approximately 300 million distributed spans to infer service dependencies. We have successfully utilized a combination of Druid and Flink to handle this scale with high availability. Attendees will gain insights into how to leverage Apache Druid for precise service dependency representation and achieve high-performance querying. We also employ Apache Superset to slice and dice the data persisted on the Druid backend according to user requirements. Additionally, we leverage Grafana to represent a time-series view of edge health metrics, such as latencies and errors, which are directly served from the Druid backend. They will also learn how to integrate streaming technologies to manage large-scale data aggregations effectively, optimizing storage and cost efficiency. This session will provide practical knowledge and techniques for leveraging Druid in real-world applications, enabling participants to enhance their data infrastructure and analytics capabilities.

Speakers:
– Sudeep Kumar, Principal Engineer, Salesforce
– Neelesh Korade, Senior Product Manager – A&IE, Salesforce

[Timestamp] Table of Contents:
[0:00] Introduction
[4:40] Distributed Tracing
[9:50] Generating Directed Dependency Graphs Using Flink & Druid
[25:19] Druid for Salesforce Scale