Make Imply Polaris the New Home for your Rockset Data

Co-author: Charles Smith

By now, you’ve heard the news: OpenAI has purchased Rockset, and has chosen to sunset Rockset services by September 2024 for existing customers. This tight timeline means that Rockset users are under immense pressure to migrate to another database, learn the new technology, and go live—within weeks. 

Any replacement for Rockset must:

• Support real-time analytics,a key operational advantage for many organizations, with an architecture optimized for fast, concurrent queries.

• Provide compatibility with SQL, a popular language that remains the preferred way for most developers to query and work with data.

• Accommodate a wide variety of data formats, from JSON to CSV to Parquet, and their associated indexes and schema.

• Support nested structures, a way to model complex relationships between data (and typically associated with JSON).

• Effortlessly autoscale to keep pace with demand.

• Offer continuous uptime, as disruptions, slowdowns, or outages can lead to customer churn or financial penalties for organizations.

• Include a managed service that can automatically configure, run, and monitor database clusters—so that developers don’t have to. 

What is Polaris—and why is it an ideal choice for Rockset users?

Built on Apache Druid, one of the world’s fastest analytical databases, Imply Polaris is a fully managed database-as-a-service—and an ideal alternative for Rockset users. 

From the very beginning, Polaris was designed for high concurrency (think 1000+qps), low latency queries (think milliseconds),at the petabyte scale. Polaris also provides native Kafka and Kinesis integration, enabling a true, real-time query capability. This means that Imply is ideal for real-time analytics applications and systems, as response times will remain consistent even in challenging conditions.

Like Rockset, Druid uses SQL for querying and managing data, easing the learning curve. In fact, Polaris supports a wide range of ANSI SQL expressions. 

In addition, Imply can automatically discover  schema changes during ingestion. With the inferred schema, Druid will also automatically index the data, alleviating the need to manually manage indices. This makes loading new data into the platform straightforward.

Lastly, Polaris abstracts away most of the operational work associated with provisioning, deploying, and troubleshooting clusters. Polaris can autoscale ingestion based on demand, rightsizing resources so that users will use only what they need, and thus pay only what they use. As a plus, migrating from Rockset to a managed service will require lower effort than an open source database.

How to migrate from Rockset to Polaris

With minimal effort, you can transition your data to Imply Polaris. If you have any questions, our dedicated professional team is on hand to assist you at each step of the way.

Sign up for Polaris

To migrate your data onto Polaris, you need a Polaris organization. 

Sign up for a Polaris trial.

Start reading data from your stream

If you utilize a streaming service like Apache Kafka or Amazon Kinesis, you can start reading streaming events directly into Polaris; because it is natively compatible with both technologies, Polaris will not require additional plugins or workarounds to do so.

If you push data directly into Rockset, you should integrate with the Polaris API, which can directly push data from an application source into Polaris over HTTP(S). The process is fairly straightforward, and consists of creating a connection through the UI or API, starting a job to ingest via the connection, and finally, sending event data to Polaris.

Backfill data

The next step is to backfill, or to move, data from Rockset into Polaris. In essence, the backfill process occurs by exporting data to S3 and then loading it into Polaris—for the full process, visit our dedicated documentation page.

As a note, you can simultaneously backfill data and migrate queries. You can even go further and serve production traffic and backfill at the same time.

Connect your client

To run queries: https://docs.imply.io/polaris/query 

To connect your apps, use the JDBC API or HTTP API.

Migrate your queries

Use the Query API to rewrite your queries in Druid SQL.

Imply will help you migrate your queries for free! If you have any questions at all, please don’t hesitate to contact our professional services team—they’re here to make migrations as easy as possible.

Conclusion

OpenAI’s purchase of Rockset, and the subsequent deprecation of its service, doesn’t necessarily have to be a setback for your organization. In fact, Imply Polaris, built on the tested Apache Druid architecture, is a fantastic alternative to existing Rockset users. With Polaris, you can support massive growth in user numbers, query traffic, and data volumes; maintain subsecond response times for aggregations and other operations; and enjoy improved performance at a lower cost.

Best of all, Polaris features both a lower learning curve (thanks to its extensive SQL support) and a lighter developer workload (due to its heavy automation). Any Rockset migrations are fast and straightforward, so get started today—and contact us if you have any questions at all.