Community Spotlight: Mindhouse Achieves Monitoring Nirvana with Apache Druid
Sep 10, 2020
Matt Sarrel
Focus on the power of real-time user behavior and application performance monitoring
Mindhouse is a meditation and yoga app available for Android on Google play and for iPhone on the App Store. The app encourages mental wellness through a variety of guided meditation sessions and techniques, and yoga classes. Content is developed by a team of instructors and provided in an interactive format along with pre-recorded audio and video content. The app recommends content based on the user’s previous experience and selected goals such as better sleep, increased patience, sharper focus, relaxed mind, and relaxed body.
We recently discussed Mindhouse’s use of Apache Druid for clickstream data analysis and user behavior funnel analysis with Ankur Gupta, the company’s engineering technical lead. Ankur’s team relies on Druid to “segment users and understand how they are using our app”, and finds that “it’s especially helpful when we launch a new feature because we can understand the acceptance of the feature based on current user activity”.
Mindhouse streams data from Apache Kafka into Druid and runs dashboards as well as ad-hoc SQL queries to gain insights from user behavior and pinpoint potential issues in their meditation app. It takes about 30 to 40 seconds between the user activity taking place and when it appears in the dashboards. Mindhouse’s small team, consisting of one data engineer and one data analyst, is highly collaborative and productive, rapidly rolling out new dashboards and connecting new data sources. The data engineer is focused on the pipeline and data ingestion (ingestion specs, new data sources), and is responsible for the upkeep and maintenance of the overall system. The data analyst writes SQL queries and builds dashboards.
“I’ve used Druid for almost five years now. From an on-premises point of view, I found it easy to host and manage compared to other analytics tools. It really
doesn’t burn a hole in your pocket in terms of infrastructure cost.
Druid provides some of the key features of data warehouses and key features
of search tools. That hybrid nature is helpful enough to make Druid
Lumi Loglake lets Splunk teams query logs directly in object storage — AWS S3, Delta Lake, Apache Iceberg — using standard SPL, with results returned as native Splunk events that work with existing dashboards,...
Supercharging Schema-On-Read: Logs in Object Storage Don’t Need a Data Catalog
Machine data architectures are rapidly changing. As telemetry volumes continue to grow and as costs rise, organizations are increasingly moving logs and other machine data into object stores such as AWS S3....
Imply Lumi Loglake vs Splunk Federated Search for S3
Teams are increasingly moving log data into AWS S3 to reduce costs and extend retention. Both Lumi Loglake and Splunk Federated Search to S3 help you query data in AWS S3 to lower costs, however the two technologies...