Overview
Atlassian is a well-known software company based in Australia that provides collaboration, development, and issue tracking software for teams. One of their most popular products is Confluence, an industry-leading team workspace cloud application used for collaboration, knowledge sharing, and more.
Atlassian’s Confluence Analytics Experience team is responsible for building and maintaining the user-facing analytics features within Confluence. These surface user behavior insights to the end-user, resulting in increased adoption and higher engagement on the Confluence product.
Challenge
The development team needed a high-powered database that could meet the needs of the analytics features embedded within Confluence. The database had to deliver an interactive experience at high concurrency to support the large number of users of their cloud application, and serve as an underlying database that was flexible and extensible for new features and data and user growth without compromising performance. The team started with Postgres, but soon ran into challenges.
“We started with PostgreSQL, but its scale limitation became really painful to deal with. Very happy we switched to Apache Druid with Imply.” – Puneet Arora, Engineering Manager, Atlassian
Postgres has many use cases as an OLTP database, but it’s not an analytical processing database. It required the team to write a lot of custom code for handling large aggregations because it was unable to scale beyond two years of data retention at varying granularities. This meant queries that were error-prone and maintenance-heavy.
In addition, pre-aggregations of data in PostgreSQL provided aged data to users resulting in sub-optimal user experience. The application’s response time was several seconds, whereas the application was required to return results within milliseconds.
Solution
To meet their critical requirements, the Confluence Analytics Experience Team chose to deploy Imply Enterprise Hybrid, a complete, real-time database built from Apache Druid® that runs in Atlassian’s VPC with Imply’s management control plane. This replaced usage of the Postgres database for event-based data, which was plagued with limitations. The new solution provided the Confluence Analytics Experience Team with the subsequent query response time and high concurrency they needed to provide their customers with the best analytics experience possible.
With Imply, Atlassian’s Confluence Analytics Experience Team found a partner who could assist them in their rapid and continuous growth. Imply delivered the enterprise distribution of Apache Druid, which was not only fast and reliable, but was also effortless to manage.
Results
With Imply, the Atlassian Confluence Analytics Experience Team has achieved more than they anticipated. They have seen massive performance improvements of up to 5X previous results with low latency of 100 milliseconds or less. No more “spinning wheel of death” for their users. They’re now able to increase data retention up to five years or more. They also have the ability to do funnel analysis on up to two years of data with no issues on real-time data ingestion and aggregation. Better still, they can now easily manage cluster configurations using Imply Manager.