Part 2: Operational Analytics in Practice

by Fangjin Yang · August 1, 2018

In the previous blog post, we discussed the need for operational analytics. Operational analytics is used to explain why any particular trend, anomaly, or pattern occurs in data. Operational analytics isn’t about asking one question of data, it is about asking many, often iterative questions, and using the answer of one question to inform the next question, until an actionable insight is reached.

Vis image

Operational analytics covers many different forms of analysis, including:

  • Root cause analysis: In root cause analysis, we are often faced with a problem that needs fast resolution, but there is often no clear indication of what is causing the problem. To determine the root cause of the problem, we usually start with a broad view of data, and gradually introduce more variables to narrow down our view of data until we can pinpoint a set of attributes that is explaining the data pattern.
  • Sensitivity analysis: In sensitivity analysis, we change attributes in data to measure how results vary based on those changes. A common use case is in A/B testing, where we may want to compare how changing product attributes impact user engagement. Understanding the impact of product changes allows us to optimize our product to drive further engagement.
  • Top-K/heavy hitter analysis: In topK/topN analysis, we want to rank the top members of a set based on some metric, and see how the metric changes when filtered by different attributes in the data. For example, we may want to find the top devices in an enterprise network based on average bytes per second. Finding top/bottom outliers is important if we want to diagnose or troubleshoot an issue in the network, or if we want to optimize network throughput by providing more resources to the top devices.
  • Behavioral analysis: Behavioral analysis focuses on understanding the behaviours of users. For example, if we were developing a mobile app, we’d want to track unique users on the app. We’ll likely want to know how usage changes from week to week, and how many users used feature X but didn’t use feature Y. Measuring distinct counts, measuring retention rates, and building funnels are all a part of behavioral analysis. By better understanding users, we can build better products.

Use Cases for Operational Analytics

Operational analytics comes up in many use cases across many different verticals. The core idea in operational analytics is always about understanding why patterns exist.

Some examples of operational analytics include:

Diagnosing problems

  • Troubleshoot networks. Detect spikes in network flows, and root cause the spikes.
  • Diagnose software crashes. Track software usage and root cause the crashes.
  • Analyze security data. Collect communication data between different services and detect anomalies. Pinpoint data properties to explain the anomalies.

Analyzing performance

  • Monitor performance of digital products. Analyze performance metrics such CPU load on servers, numbers of cache requests/hits/misses, and network throughput.
  • Track mobile product use. Create ad-hoc views on how users are engaging with a mobile app, including what fractions of users are running on different mobile devices, operating systems, and versions of an app.
  • Monitor website reliability. Analyze performance filtered by any combination of attributes including specific endpoints, datacenters, racks, or servers.

Finding commonalities

  • Find commonalities in failures and what is in common among events. For example, find the common data attribute among a set of failing devices in a manufacturing process.
  • Identify commonalities in sales data. Find what group of users are most engaged with a product. Identify what top selling products have in common.

Increasing efficiency

  • Optimize ad spend. Measure ad views, clicks, and conversions. Optimize ad campaign performance based on user location, ad type, or user demographic.
  • Increase product engagement. Measure how ad-hoc groups of users are engaging with online products. Improve products based on learnings.

For more information, you can read a full case study for operational analytics for network flows.

In next blog post, we will cover the technology behind operational analytics.

Back to blog

How can we help?