Global Asset Manager

Challenge

The company is a global asset manager providing investment, advisory, and risk management services to institutional and individual clients. Its technology infrastructure supports a large and growing population of internal users, including investment, sales, and trading teams.

The team was preparing for what it described internally as a “data dust storm.” As data volumes grew, Splunk remained central to the company’s security environment, but the cost of storing and searching that data was becoming harder to justify.

Several terabytes of data in Splunk were rarely queried but still expensive to retain, while other potentially valuable logs never made it into Splunk in the first place. The team wanted to move infrequently accessed data to cheaper storage while keeping it immediately searchable for investigations.

Its users were also committed to SPL. They didn’t want to learn a new query language or bounce between tools to correlate data. Replacing Splunk outright would have introduced the risk of a migration lasting more than 12 months while disrupting established queries, dashboards, and operational processes built up over years.

Compounding all of this, the company operates in a tightly controlled security environment. Internal services can’t be exposed to the public internet, and its networking, certificate-validation, and vulnerability-management policies ruled out a standard cloud deployment.

In short, the team needed something that would lower the cost of retention while preserving SPL and existing dashboards, and that could hold up at scale. Just as important, it had to keep every record at full fidelity and run entirely inside the company’s own AWS environment with no public exposure.

Solution

The company deployed Imply Lumi in its AWS environment as a more efficient data layer for security and observability workloads.

Lumi separates storage and compute, letting the company retain growing data volumes efficiently while scaling query resources independently. Existing users continue working in SPL against familiar dashboards, with no new interface or query language to learn.

The combination of faster SPL performance, a significantly smaller storage footprint, and no workflow changes initially sounded too good to be true. Rather than asking the customer to take those claims on faith, the teams tested Lumi head-to-head against the company’s Splunk Cloud environment using an identical workload.

Under those conditions, Lumi delivered subsecond responses for interactive dashboard queries, a materially smaller footprint at the same retention.  Existing SPL queries and dashboards ran unmodified, with full-fidelity data,  no filtering or sampling, and sustained ingestion with no drop in performance.

Results varied by query type. Count and aggregation queries saw the largest gains, with analytical searches running up to 50× faster than Splunk. Across the full query set, queries ran roughly 9× faster.

Storage told a similar story. Using the same source data and retention window, Lumi’s footprint came in over 80% smaller than Splunk’s, with an 18:1 compression ratio compared with Splunk’s 2:1.

Impact

With Lumi now in production, the asset manager has improved performance and storage efficiency without changing how its teams work. SPL queries and dashboards run against Lumi unmodified, so users keep the language and workflows they already know while getting meaningfully faster answers on analytical searches. Dashboard queries return in under a second, and nothing has to be filtered or sampled to get there.

Lumi also matches Splunk’s ingestion performance while using a fraction of the footprint Splunk needed for the same data. That gives the asset manager room to keep more data for longer without scaling its Splunk footprint at the same rate.

For the asset manager, Lumi is not about shrinking its investment in Splunk. In fact, the company expects its use of Splunk to grow rather than shift those workloads to competing tools. Lumi provides a more efficient data engine underneath Splunk, allowing the team to expand its use without forcing every byte of data into the same architecture.

Splunk remains part of the experience its teams already know, while Lumi provides the engine underneath the security and observability workloads where it makes the most sense. The production deployment is designed to expand to multiple terabytes per day as the company brings additional teams and data sources onto Lumi.

This gives the asset manager a more flexible foundation for modernizing its data infrastructure and supporting future analytics and AI workloads.