📚 Download the Whitepaper: A Data Team's Guide to Real-time Analytics for Apache Kafka® ⇒
Integrations

Import database objects easily

Druid easily ingests data from object stores or common data formats including JSON, CSV, TSV, Parquet, ORC, Avro, and Protobuf. Simply execute the INSERT INTO statement for SQL import and in-database transformations.

See documentation

Batch ingestion with Druid

Druid supports blazing batch ingestion via SQL. Multiple options exist for most databases, including writing data to a file for batch ingestion, using ELT tools such as Apache Nifi, Fivetran, or Ascend.io., Fivetran, or Ascend.io, or exposing data changes as a stream.

integrations-database-diagram
  • icon-Simple-Fast-SQL-ingestion

    Simple fast SQL ingestion

    Use common SQL statements for auto-tuned easier and faster data ingestion

  • icon-Flexible-Data-Schema

    Flexible data schema

    Choose to either flatten data or ingest nested data directly from JSON files.

  • icon-In-Database-Transformation

    Flexible data transformation

    Transform data during ingestion or load raw data to Druid and transform in-database

  • Integrations for Batch Ingestion
logo-Oracle

Oracle

The initial copy of the data from Oracle can be completed using Oracle SQL developer which provides a wizard for exporting data and metadata from the database.  That data is then ingesting into Druid via SQL.  Subsequent changes to the Oracle analytical data can then be replicated in Druid using LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration, using various ELT tools or the Debezium Oracle connector to automate the CDC (Change Data Capture process).

Snowflake

The initial copy of data from Snowflake to Apache Druid can be completed using the Snowflake bulk unloader and ingesting that data into Druid via SQL. After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration or using various ELT tools.

logo-PostgreSQL

PostgreSQL

Data in PostgreSQL can be exported using the pg_dumpall command line utility.  The .sql export can then be imported into Druid using SQL ingestion. After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration, using various ELT tools or the Debezium Postgres connector which automates the CDC (Change Data Capture) process.

logo-MySQL

MySQL

Data in MySQL can be exported using mysqldump utility. The .sql export can then be imported into Druid using SQL ingestion. After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration, using various ELT tools or the Debezium MySQL connector which automates the CDC (Change Data Capture) process. 

logo-MariaDB

MariaDB

Data in MariaDB can be exported using the mariadb-dump/mysqldump utility.  The .sql export can then be imported into Druid using SQL ingestion. After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration or using various ELT tools.

logo-Microsoft-SQL

SQL Server

Data in SQL Server can be exported using the SQL Server import and export wizard utility.  The data can then be imported into Druid.  For example, a .csv export can be ingested into Druid via the load data option in the Druid UI.  After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration, using various ELT tools or the Debezium SQL Server connector which automates the CDC (Change Data Capture) process.

logo-InfluxDB

InfluxDB

Data stored in InfluxDB buckets can be ingested into Druid using Druid batch ingestion. Since the indexing is parallel, extremely large files will ingest faster if they are split into multiple smaller files. For incremental ingesting a data pipeline can be established using ELT tools such as Apache Nifi, Fivetran, or Ascend.io.

logo-DataStax Cassandra

DataStax Cassandra

Data in Apache Cassandra can be exported using the COPY command. The data can then be imported into Druid.  For example, a .csv export can be ingested into Druid via the load data option in the Druid UI.  After the initial ingest data updates can be handled by a simple change data capture process using the LAST_UPDATED date time field to determine the updated or new records along with code to facilitate the migration, using various ELT tools or the Debezium Cassandra connector which automates the CDC (Change Data Capture) process.   

logo-Elasticsearch

Elasticsearch

Objects stored in Elasticsearch can be exported in JSON format using the export API.  The data can then be imported into Druid using Druid batch ingestion.  After the initial ingest data updates can be handled using various ELT tools, such as Apache Nifi, Fivetran, or Ascend.io.

logo-AWS-S3

AWS S3

AWS S3 buckets can be ingested into Druid using Druid batch ingestion. Since the indexing is parallel, extremely large

files will ingest faster if they are split into multiple smaller files. For incremental ingesting a data pipeline can be established using ELT code such as Apache Nifi, Fivetran, or Ascend.io.

logo-Azure

Azure Blob & Azure Data Lake Store

Objects stored in Azure Blob and ADLS can be ingested into Druid using Druid batch ingestion. Since the indexing is parallel, extremely large files will ingest faster if they are split into multiple smaller files. For incremental ingesting a data pipeline can be established using ELT code such as Apache Nifi, Fivetran, or Ascend.io.

Let us help with your analytics apps

Request a Demo