Ingestion from Confluent Cloud and Kafka in Polaris
Apr 20, 2023
Timmy Freese
One of the foundational pillars of real-time data analytics is the ability to move data from one place to another quickly; Apache Kafka® is a widely adopted solution for this functionality. Confluent Cloud provides a fully-managed solution for Kafka. Whether you use Confluent Cloud, self-hosted Kafka, or alternative hosting platforms such as AWS MSK or Aiven MAK, Imply Polaris offers a solution for you to ingest data.
For Confluent Cloud users, Polaris offers a native, pull-based ingestion capability. Users only need to specify a few fields to establish a connection to their Confluent Cloud account and can ingest data in no time—every event enters the database and is queryable with sub-second latency. Additionally, this method offers exactly-once semantics and infers the schema from available data using Druid’s sampler.
For self-hosted and other Kafka solutions, Polaris offers an HTTP-based Kafka Connector which uses our Push API endpoint. This connector will automatically batch and compress your data and handles auth token renewal for you. Additionally, this solution can be used in tandem with Polaris’s PrivateLink offering to provide enhanced security.
Regardless of which solution you choose, you can expect high throughput and low latency results. We provide OLAP functionality with OLTP performance.
The Most-Read Imply Blogs of 2025 (and what they signal for 2026)
Before we take on 2026, let’s rewind. 2025 was the year observability teams stopped asking, “How do we reduce data?” and started asking the real question: “How do we build an architecture that can keep...
Observability is at a crossroads For years, observability has promised to give teams the visibility they need to keep digital services resilient. But as data volumes explode, many leaders are realizing the...
The Observability Warehouse that helps you keep more data, move faster, and spend less without changing how you work Observability Is Hitting Its Limits Splunk has long been the system of record for observability...