Building an Open Source Streaming Analytics Stack with Kafka and Druid

AdTech 실시간 광고 분석의 Apache Druid 활용 사례

이 비디오에서는 Apache Druid / Imply를 활용하여 실시간 광고 성과 분석 플랫폼을 구축한 사례, 많은 AdTech 기업들이 Druid / Imply를 선택한 이유, 그리고 실시간 광고...

druid.NEXT

Watch the recording from our druid.NEXT event to see what’s new in Apache Druid 26.0 with technical demos from Apache Druid Committee (PMC) Chair and Imply CTO Gian Merlino along with other Druid committers...

Build Real-Time Analytics for your Kafka Data

Watch this recorded webinar to learn about building real-time analytics for streaming data. Darin Briskman, Imply’s Director of Technology and all around tinkerer, will demonstrate how to use Apache Kafka...

Apache Druid Explained | Core Concepts

This video provides a technical overview of Apache Druid, a popular real-time analytics database used for sub-second queries on streaming and batch data at scale and under load. The intent of this video is...

Apache Druid in 5 minutes

Apache Druid is a real-time analytics database used by 1000s of companies like Netflix, Confluent, Salesforce, and Target. But what's the big deal? Why use Druid instead of a data warehouse - like Snowflake,...

Embedding Polaris Visualizations Demo

Set up an embedded visualization solution in under 5 minutes. This short demo highlights the steps needed to build a highly customized application.

Speeding Up Clickstream Data Analytics at Danggeun Market Using Druid

Danggeun market is an online marketplace in Korea focussed on used goods sales in the neighbourhood. This session will detail out how Daangn uses druid to speed up clickstream data analytics. Realtime data...

Easy onboard with Polaris

Case study to share my experience how easy is to setup a working Analytics DB by using Imply Polaris. Leverage the batch ingestion APIs using python, using Apache airflow scheduler. How we leveraging Theta...

Percentiles at Scale

At Netflix, our engineers often need to see metrics as distributions to get a full picture of how our users are experiencing playback.

Powering Clickstream Analytics at TrueCar with Druid

Come learn about the way that TrueCar uses Druid as the engine to optimize the buying experience through clickstream analytics.

Keynote: Analytics Meets Applications

How do developers at Netflix, Confluent, and Reddit do more with analytics? They joined a community of like-minded people who bring analytics to applications to power a new generation of operational workflows....

Druid SQL for Queries – from an Overview to Tips and Tricks

SQL is a powerful and elegant query language for databases. Druid is increasingly supporting and moving forward with SQL, which is arguably easier to learn and write than the original native (JSON) query language....

How VMware uses Druid for Network & Security Analytics

VMware NSX Intelligence is a distributed analytics platform powered by Druid that can process and visualize network traffic flows, detect anomalous network behaviors, and recommend security policies. We will...

Kafka Ingestion logs

Apache Druid can handle streaming ingestion jobs at any scale. This occasionally poses challenges. Apache Kafka is a popular distributed event streaming platform, so it’s a natural choice for this presentation....

Generating Time-centric Data for Druid

Sometimes we want to understand how Druid will perform using various types of data, but the actual data may not be available. The developer relations team at Imply has created a data generator program that...

Leverage Autonomous Intelligence Driven Decisions Without Using Dashboards

Customer Intelligence facilitates informed decisions to increase business efficiency and success. Out of The Blue™ (OTB) provides SaaS solutions that autonomously and continuously monitor KPIs to create actionable...

Apache Druid 24.0 New Feature! – Nested JSON

Apache Druid 24.0 has some amazing new capabilities. One of the most exciting ones is the ability to load complex nested JSON structures automatically. In this session we review the motivation for this feature;...

User Activity Analytics in Digital Publishing Domain: Success Story by Indian Express

Indian Express is one of the leading media publishing company based in Mumbai is using Imply for user activity analysis. In this talk, we share how Indian Express migrated their use case from Google Analytics...

The present and future of compaction in Druid

Compaction in the Druid database has been evolving into use cases beyond improving query performance and reducing storage. This talk will first go over what is compaction, current use cases for compaction,...

Moving Away from Multi-level Aggregates: Efficiently Scaling Data with Druid

What if I told you moving away from multi-level aggregates could actually speed up your reporting system? In this session, we look at how Druid can be used to retrieve data for reporting quickly and efficiently....

Analyzing Ads Across Medialab’s Global Brands with Druid

In this talk, we will learn more about Medialab's analytics strategy to optimize ad campaigns across several global consumer internet brands using Druid. Juan will highlight the company's journey from Open...

Load, Drop & Kill – The Life of a Segment

In Apache Druid, after ingesting some data and publishing a segment to Deep Storage, it is the Load, Drop rule a Kill process behavior that govern the caching of segments on historicals across multiple tiers....

Why & How We Built An Open-Source Spark Druid Connector – Spark Druid Segment Reader

Spark Druid Segment Reader is a Spark connector that extends Spark DataFrame reader and supports directly reading Druid data from PySpark. By using the DataSource option you can easily define the input path...

Pushing the Limits of Druid Real-time with Data Sharding

Most Druid applications aim for sub-second analytics queries - but this becomes difficult when dealing with large volumes of real-time data.

Real-Time Upserts of Data in Druid

Druid has enabled ThousandEyes to reduce dashboard latency by 10x and provide our customers with a real-time, interactive view of their terabytes of data. In this talk we will learn how the Data Visualization...

Realtime Product Observability with Druid

Statsig enables companies to run experiments so that they can understand the impact of the changes they are making to their product, and do in depth analysis so that they can understand how people use their...

Building Data Pipelines using Druid at Lyft

In this talk, we'll learn more about how Lyft builds data pipelines using Apache Druid, which is useful for several use cases including metrics tracking, model forecasting, and internal tools. We'll also talk...

Principles of Data Modeling in Apache Druid

Apache Druid is often used to power applications that are fun, fluid, and fast. It’s designed for the last mile, sitting behind a UI where customers, suppliers, and employees are waiting for fast responses...

Using Druid at High Throughput for Fast Queries at Low Cost

ironSource exposed a Druid cluster for its external users, called Real Time Pivot, which runs at a scale of 2-3M events/sec, with tens of TBs added per day, while serving parallel queries within 1-2 seconds....

Speeding up ML Model Training Using Druid

Machine learning models require thousands of data points to train models. This session will look at a number of mechanisms druid offers to speed up model training. The tuple sketch can be used to train models...

Keynote: Real time Analytics in Modern SaaS Applications

Real-time analytics has gone from cutting-edge to competitive advantage and is now becoming table stakes. In 8 years and 3 companies of working in the space, Gwen talked to hundreds of companies building real-time...

Production ML Model Quality Monitoring Using Druid

Machine learning and AI have progressed out of the initial technology development phase and into full production. In this new domain, the concept of monitoring has been redefined. Monitoring now fully encompasses...

Apache Druid 24.0 New Feature! – SQL Based Ingestion

Apache Druid 24.0 has some amazing new capabilities. The Multi-stage Query Architecture (MSQA) that is now a part of the database will enable significant new functionality. In this initial release it enables...

Introducing KUDRAS – Kubernetes Druid Autoscaler for Maximum Resource Utilization and Speed

In this session I would like to talk about the huge amount of data we ingested into Druid (raw data is 9 Terra per day) by using EMR, all orchestrated by Airflow. While the data grew we started experiencing...

Atlassian: Delivering insights to Confluence Users at Enterprise Scale

Druid is a powerful tool that has paved the way for a whole new wave of data-focused tech. In the Confluence Analytics team, we ingest millions of events every day and use this tool to power a plethora of features...

New Druid Challenges and Solutions in Pinterest

Pinterest has been using Druid with large ingestion scales and low query latency for a few years. In this talk, we will share recent optimizations we made on querying skewed data, how we deal with large scale...

Low Latency Real Time Ads Pacing Queries

Advertising backend systems need low latency for the efficient delivery and availability of budget data to the ad serving infrastructure. For the reddit team, we helped automate user behavior analysis to serve...

Keynote: Building Next-Gen Observability at Confluent

In this Q&A, Matt Armstrong, Head of Engineering - Observability & Data Platform at Confluent will discuss the use cases and architecture for Confluent's observability platform. Confluent leverages both Druid...

How is Druid so Fast?

A look at how Apache Druid is able to achieve high concurrency subsecond queries at scale.

Apache Druid’s Fit in the Modern Data Stack

Apache Druid is a real-time analytics database for analytics applications. It’s used by 1000s of companies like Netflix, Confluent, Target, and Salesforce. In a crowded database market, it can be hard to...

Handling row level updates in druid using a kafka lookup

This video shows how to configure and populate a kafka lookup and then use it in a query to handle row level updates in druid

The New Database Battle: Apache Druid vs. ClickHouse

They both say they’re fast; they both say they scale. Should you flip a coin or are there real technical differences that matter? Watch this webinar and get the facts that lay out the differences.

How-to Half-hour: Using Multi-value Dimensions in Apache Druid

If you’ve tried, you probably did tons of JOIN operations to build an array. Did you know that with multi-value dimensions you can filter and group complex data quickly and easily?

Keynote – Building Modern Analytics Applications with Apache Druid

Fangjin Yang, Co-Founder and Chief Executive Officer of Imply, presents the opening keynote at the Druid Summit 2021 virtual conference.

Technical deep dive into how Outbrain scales its real-time analytics

Because Outbrain processes billions of impressions and events a day, they risk running into scaling problems.

Imply x Kafka: Capture, interact and scale streaming data

Imply and Kafka is the perfect architecture to capture and surface streaming data through interactive queries and unlimited scale

Confluent Cloud 연동하여 Imply로 실시간 데이터 분석 및 시각화

라이브 데모: 대화형 쿼리 및 제한없는 확장을 통해 스트리밍 데이터를 캡쳐하고 보여주는 아키텍쳐를 구축하는 방법

Making Real-Time Data a Reality for your Business

Technaura takes you through an hour of practical Real-time, data-driven outcomes.

Apache Druid Engine Roadmap

如果你还在每天等待数据刷新，是时候试试Apache Druid 亚秒级数据库了。 该开源数据库在Twitter、Pinterest和Snapchat广泛部署。此次研讨会中， Imply的产品经理Will...

Comparing drill-down workflows between Tableau & Snowflake with Imply

In this video, Jad Naous demonstrates drill-down workflows between Tableau & Snowflake with Imply.

How Spideo turbo-charged data analytics by using Imply

In this video, Spideo, a humanized recommendation provider, shares with us its data analytics journey.

Pivot 2.0 – The Next Gen Visualization Tool

In this webinar, will walk you through the exciting new features that are coming soon to Pivot.

Introducing… Druid’s Components

In this talk, we look at the three families of components that create “Apache Druid.”

积跬步，至千里——详解运用流数据的商业价值

商业用户越来越意识到企业活动产生的数据能够及时帮助他们洞见日常运营对自身现有商业模式的影响以及是否有利于企业的长远决策。

Analyser Des Flux De Donnees Massifs Avec Imply

NOVAGEN présentera un cas avec IMPLY, inspiré de ses projets menés dans le secteur de la grande distribution, qui a su répondre à ces nouveaux enjeux :

Using Imply to prevent fraud

Learn a new methodology for anomaly detection and analysis that can be applied to everything from fraud detection to factory accident prevention.

Inside Apache Druid’s storage and query engine

We’ll cover how Druid stores data, what kinds of compression it uses, how it indexes data, how the storage engine is linked with the query processing engine.

First look: Imply CrossTab (beta)

In this short video, Imply Chief Product Officer Vadim Ogievetsky demonstrates a cool new beta feature, Imply CrossTab.

Imply CrossTab: PivotTables to infinity and beyond

In this webinar we will introduce Imply CrossTab, a new visualization that makes the pivot table user experience feel instantaneous.

Elasticsearch: pros and cons for real-time analytics

In this video, solution architect Itai Yaffe shares what Elasticsearch provides for analytic use cases compared to Imply.

Where do Imply and Elasticsearch fit in my big data stack?

Itai Yaffe shares how Imply and Elasticsearch fit in a modern architecture for big data.

How Imply fulfills the requirements of real-time analytics

Data wizard Itai Yaffe explains how Imply's platform fills the requirements of general-purpose operational analytics.

Near term Apache Druid roadmap for 2021

Gian Merlino, Apache Druid PMC Chair, lays out the near team Druid roadmap.

What’s new in Apache Druid?

Looking for a recap of the key improvements to Apache Druid in its most most recent releases? Here’s the video for you.

5 minute explanation of Imply’s advantages for self service analytics

We describe the Imply full-stack, multi-cloud data platform, and Itai breaks down the unique combination of features Imply brings to self-service analytics.

The Superstars of Apache Druid meetup

The Superstars of Apache Druid meetup was recorded live on 1-21-2021. Talks from Dan Prince from Target, Gian Merlino, and Vadim Ogievetsky

Comparing Elasticsearch and Imply for operational analytics

If you’re considering using it for interactive analytics, be advised that it’s quite common to see companies struggling with Elastic-based analytic solutions.

3 different approaches to multi-tenant applications

The three main challenges with multi-tenant applications are cost, performance, and data management.

How Apache Druid ensures quality of service

Gian Merlino covers Druid’s use of tiers, lanes and priorities to address QoS and deliver consistent performance as you scale demand.

Building interactive data applications for event streams w/Confluent

Data apps let business users explore and investigate all of a company's event data and come to insights that impact day-to-day and long-range decision making.

Outbrain’s real-time analytics architecture

Outbrain’s real-time analytics architecture consists of modern big data technologies like Kafka, Spark, Druid and Imply Pivot for query and visualization.

Lyft’s modern data architecture feat. Apache Druid, Kafka and Flink

Take a look at Lyft’s modern data architecture. They have an app implemented in Flink that reads real-time event data from Kinesis and transforms the data.

Technical reasons why Lyft chose Apache Druid for real time analytics

In this video, Sharanya Santhanam from Lyft explains the key technical reasons why they chose Druid as the engine for their real-time data pipeline.

Lyft’s Apache Druid uses cases

Lyft uses Apache Druid for three core use cases: geo-spatial data lookups for rideshare customer service, analytics on AWS cloud infrastructure spend.

Achieve the event-driven Nirvana with Apache Druid

We will explain how Apache Druid enables self-service BI on event data and allows business users to ask their own questions leading to real-time insights.

Apache Druid and database evolution

A look at how Druid builds on the long history of database engines, and what that means for technologists and decision-makers.

Apache Druid data optimisations

A telling of the tale of how Druid can optimize data as it is ingested, turning it into a format perfect for interactive, real-time analytics.

Virtual Druid Summit 2020 highlights

Catch all of the amazing highlights from our free Virtual Druid Summit. We had four amazing events that took place between April 10 to November 20, 2020.

A demo of how Imply powers real-time data applications

Learn more about how Imply powers real-time data applications. In this demonstration video, Sales Engineer Vasilis Vagias walks you through the product.

Scaling Apache Druid as a multi-tenant platform

In this talk I will present: Our data model in Druid, based on UNION of several different types of events.

Outbrain: Moving real-time analytics to the next level

In this talk I will present: Our data model in Druid, based on UNION of several different types of events.

Enabling real-time querying using Druid & Flink @ Lyft

In this talk, we’ll learn more about how Apache Druid powers alerting against real-time data at Lyft.

Innowatts: Analyzing electric meters using Druid

Learn how they use Druid to power visualizations that users can interactively explore, shortening the time it takes to diagnose data, and off-loading Spark.

Building a real-time eventing engine at FullContact

In this talk, Jeremy Plichta will outline FullContact’s journey with Apache Kafka and Druid

Imply Cloud on AWS demonstration

Want to see how to set up, use and manage Imply Cloud for real-time self-service analytics at scale?

Apache Druid 101

This talk provides an introduction to Apache Druid including: Druid's core architecture and its advantages.

What is a data application?

Data applications are visually-oriented and designed for non-analysts to investigate and explore large, fast-moving data sets.

The 5 things data applications need

What do data applications need? Imply CTO Gian Merlino boils it down to 5 key things.

How Apache Druid addresses the 5 requirements of data applications

Data applications are visually-oriented and designed for non-analysts to investigate and explore large, fast-moving data sets.

SuperAwesome Returns for a SuperAwesome Apache Druid Meetup

In this talk, we walk through how we run Druid on spot instances.

Three reasons why Target built a custom enterprise analytics platform

Target explains why they built a custom analytics platform, focusing on self-service development, discovery /collaboration of data, and to ensure scalability.

Target explains unions in Apache Druid

In this short video, Jeremy Woelfel of Target explains how unions allow them to blend their data.

Key Apache Druid features Target utilizes

Jeremy Woelfel of Target explains how they ingest their data, the impact of cardinality, metrics collection, Thetasketch approximations, and more!

ApacheCon @Home Thursday Keynote: Gian Merlino, Imply

Gian Merlino discuss why you can never find that single perfect system and how to think about & evaluate data on a temperature-based spectrum.

A explainer about Druid rollup, cardinality and segments from Netflix

In this short video, Ben Sykes of Netflix explains Druid roll-up, the impact of high cardinality, and segment sizing.

What’s inside Netflix’s metrics pipeline & Apache Druid cluster?

Ben Sykes discusses how Netflix created its metrics pipeline to ensure a high-quality streaming experience & how they structure their Apache Druid cluster.

Building Data Applications with Apache Druid

One of the most popular use cases for Apache Druid is building data applications.

GameAnalytics: Building a Real-Time Gaming Analytics Service w/ Druid

In this talk, you will learn how we managed to migrate our legacy backend system from using an in-house built streaming analytics service to Apache Druid.

Zeotap: Data Modeling in Druid for Non temporal and Nested Data

Druid has been the production workhorse for the past 2+ years at Zeotap powering the core Audience planning across our Connect and Targeting products.