Off The Charts: Ideas in Analytical Design

Is this normal? Comparing observations with expectations

At TellApart, the first question we hope to answer with our dashboard is “are we performing as expected?” For instance, consider the chart below:

Is This Normal?

The middle of the week seems healthy in the left chart, but unhealthy in the second where a 10 week average ‘lookback’ is added in blue. Healthy and unhealthy are defined in reference to expectations. Explicit expectations make anomalies obvious.

The above data is representative of a week in retail – consumers shop and browse more often during the day and on certain days of the week. Retailers also have specific times for new product launches or promotions. To consistently find gaps like the one above, an Account Manager would need to recall retailer-specific patterns for several clients across several metrics.

We found that time of day and day of week reliably predicted a majority of normal behavior. The underlying data is a cumulative ten-minute sample (each data point represents the total number of events in the past ten minutes). The expectation in our case is built from the prior ten weeks – each data point is the average of the corresponding time of day and day of week weighted by recency.

When you present the expected data in a loud color underneath current data, anomalies become apparent at a glance. Likewise, when the expectation matches the observation, it fades into the background. Formalizing an expectation makes it simple for anyone to find anomalies. If the provided expectation is accurate in predicting normal behavior, issues hiding in plain sight are revealed.

In the next post in this series, we’ll consider the application of this technique to sparse data.

Join the team turning terabytes of information into revenue. Check out our careers site.

Pratik Prasad is a Software Engineer at TellApart.