BTG Pactual + Imply Lumi: More Data. Longer Retention. Lower Cost. Without Replacing Splunk.

BTG Pactual, a global financial institution, ran into a familiar problem.

As their Splunk environment scaled, so did the pressure:

• More data to ingest
• Longer retention requirements
• More complex investigations

But the underlying system hadn’t changed.

Detection and investigation were still running on the same infrastructure.  Steady workloads and burst workloads were treated the same way.

That worked—until it didn’t.

Common Splunk tradeoffs: cost, retention, and performance

As data volumes grew, BTG was forced into a set of tradeoffs most security teams recognize:

• Index everything in Splunk and overspend.
• Reduce retention and accept investigative risk.
• Push data to cheaper storage and accept slower access.

None of these options solved the real problem.

Why Splunk struggles with investigation workloads

Investigations don’t behave like detection.  They arrive in bursts. They require deep historical access. They drive spikes in compute demand.

Treating them the same forces teams to size infrastructure for their worst day, every day.

That’s where the model starts to break.

How to Scale Splunk without increasing infrastructure costs

Instead of replacing Splunk, BTG changed the architecture underneath it.

Here’s what happened:

+10 TB/day Data Ingest

The team was able to bring significantly more telemetry into their Splunk environment, including data that was previously too expensive or impractical to index. This expanded visibility without requiring additional always-on Splunk infrastructure.

4× Data Retention

Retention in Splunk extended from 90 days to one full year, giving analysts access to a much deeper historical record during investigations. This eliminated the need to choose between cost and visibility.

60% Lower Cost per GB

By shifting how data supporting Splunk is stored and queried, the cost to manage each unit of data dropped significantly. This allowed the Splunk environment to scale without cost growing at the same rate as data volume.

Zero Workflow Changes Required

Existing Splunk dashboards, alerts, and SPL workflows remained unchanged. Analysts continue working in Splunk exactly as before, with no retraining or disruption.

This wasn’t the result of tuning or cost-cutting.
It was the result of aligning infrastructure to how workloads actually behave.

“With Imply Lumi, we can ingest more data, retain it longer, pull in telemetry from platforms beyond Splunk, and still understand what our costs will look like as we scale,” said Rafael Hass, Security Information Manager at BTG Pactual. “Lumi lets us expand what we can do with our observability data and has fundamentally changed how our team operates.”

Splunk architecture best practices: separating detection and investigation

What changed was not the tool.  It was the data layer.

Detection and investigation workflows remained in Splunk, using the same dashboards, alerts, and SPL.  What changed was how the data supporting those workflows is stored and accessed.

Detection and investigation no longer had to share the same infrastructure.

Detection runs on static, always-on compute optimized for continuous monitoring and alerting. Investigations run on elastic compute that scales on demand for large, bursty queries.

Each workload now runs in an environment designed for how it actually behaves.

Teams could access more data.  And they no longer had to trade off cost, performance, and retention.

“We adopted Imply Lumi because we needed to ingest more data, retain it longer, run faster queries, and improve cost efficiency — all while keeping our existing dashboards and SPL workflows intact,” Hass said.

How security teams need to rethink their environments

BTG’s experience reflects a broader change in how security teams are thinking about their environments.

The problem isn’t Splunk.  It’s forcing fundamentally different workloads to share the same system.

Detection is steady and predictable.  Investigation is bursty and exploratory.

They shouldn’t be treated the same.

What is an observability data layer for Splunk?

This is the model behind Imply Lumi.

Imply Lumi is a high-performance data layer that works with Splunk, fundamentally changing where data lives and how it is queried.

How Imply Lumi extends Splunk for scale and retention

Instead of forcing all telemetry into a single always-on indexed model, Imply Lumi lets teams align storage and compute to the needs of each workload. Detection workloads can run on static compute for continuous monitoring, while investigation workloads can run on elastic compute that scales on demand. Splunk remains the interface for detection, dashboards, and investigation workflows, with queries executed through federated search using SPL.

This allows teams to ingest more data, retain it longer, and run large-scale investigations without being constrained by the cost and limits of traditional architectures.

Splunk remains where teams work. Lumi becomes where the data lives.

“Imply Lumi has expanded what we can do with Splunk,” he added. “We’re bringing in data that wasn’t there before, scaling investigations more efficiently, and it gives us flexibility in how we evolve our SIEM strategy over time.”

Learn more about scaling Splunk for security investigations

We put together a detailed guide on how this architecture works in practice.

Read the full ebook: Scaling Security Investigations for Modern Splunk Environments

If you want to see how to scale Splunk for higher data volumes, longer retention, and faster investigations, request a demo of Imply Lumi.