Dec 06, 2023

Enhancing Druid’s Analytics with Apache Arrow and Flight SQL

Currently Druid has two result formats: a JSON format and a protobuf-based format using Apache Calcite Avatica. Both of these formats are row-oriented while all of Druid’s internal data representations are column-oriented, as expected for high-performance analytics. This means that performance is being left on the table due to these transpositions.

This talk will describe a host of possible benefits for Apache Druid that would come by supporting Apache Arrow as an output format. Apache Arrow is an in-memory, columnar data format. It is used as the backing memory representation for utilities like pandas and polars, and allows for zero-copy data communication across many libraries. This session will cover various ways that we could improve the performance, interoperability and flexibility of Apache Druid, along with making it easier to integrate Druid with new data sources and analytics pipelines by leveraging Arrow, FlightSQL and ADBC (Arrow Database Connectivity).

Given the competitive landscape of data computation engines right now (Snowflake, BigQuery, Druid, Dremio, DuckDB), embracing support for Arrow will help Druid “keep up with the competition” and open doors for enhanced connectivity and performance!

See similar videos

No records found...
Oct 03, 2024

Demo Video: Imply Polaris Overview

Imply Polaris is a cloud-native database-as-a-service that simplifies the developer experience by providing all the advantages of Apache Druid, plus auto-scaling for seamless data ingestion, enhanced security,...

Watch now
Aug 16, 2024

Imply Polaris + Natural Intelligence: Powering Real-time Analytics & Observability

Imply Polaris provides the fastest path to real-time analytics by offering seamless data ingestion, auto-scaling capabilities, built-in visualization, and a secure, fully managed infrastructure. Watch this...

Watch now
Jan 29, 2024

Physical Hardware, Digital Analytics: IoT Challenges, Best Practices, and Solutions

From supporting ad-hoc queries across massive time series datasets to ensuring data freshness and scalability, our panelists will share insights into selecting the right database solutions that can deliver...

Watch now

Let us help with your analytics apps

Request a Demo