Roblox built a cost-effective analytics solution using Druid to serve millions of experiences on our platform. This system provides free analytics to all creators, handling games ranging from 1 to 100 million monthly active users. Our approach heavily utilizes approximation techniques to manage scale and cost. This talk will cover our year-long journey with this system in production, focusing on key optimizations we’ve implemented.
Speaker:
Willis Kennedy, Senior Software Engineer, Roblox
[Timestamp] Table of Contents:
[0:00] Introduction to Roblox and Creator Analytics
[2:00] Roblox’s Data Infrastructure Overview
[3:00] Creator Analytics Features and Data Served
[4:00] Key Design Goals for Performance and Cost-Efficiency
[5:00] Approximation Techniques with Data Sketches
[7:00] Load Testing and Benchmarking with Druid
[9:00] Challenges with High Cardinality and Query Patterns
[12:00] Average Case Query Optimizations
[15:00] Practical Roll-Up Table Optimizations
[18:00] Cluster Setup and Query Management
[21:00] Data Ingestion and Workflow Management
[22:00] Outcomes and Future Plans
Keynote: Powering Event-Driven Data with Apache Druid
The distinction between OLTP and OLAP is becoming less relevant as data architectures shift toward entities and events. In this session, we’ll delve into how Apache Druid’s event-first approach synthesizes...
Watch nowClosing Keynote: Charting the Future of Druid
What lies ahead for Apache Druid? Join us as we explore the evolving landscape of Druid’s query and storage engines, and how they are positioned to address the biggest challenges in event data for the future. Speaker: Gian...
Watch nowSalesforce: Tracing Service Dependencies at Scale with Druid and Flink
At Salesforce, we manage approximately 300 million distributed spans to infer service dependencies. We have successfully utilized a combination of Druid and Flink to handle this scale with high availability....
Watch now