Web Tortoise

2012-Sep-20

Hey Hey, Check the Overlay!

Response:

Hello!  This #WebTortoise post was written 2012-SEP-20 at 03:43 PM ET (about #WebTortoise).

Main Points

#- Use programs like Excel or Tableau to get additional formatting value from your third-party-provided chart data.

#- Continually measure website Performance to identify Pattern Changes, be them by minute of hour, by hour of day, by day of week or by some other dimension.  Point-in-time measurements have value, where continually measuring will tell more of the story.

#- Use overlay charts (a.k.a. “Compare to Previous” charts) to complement time-based line charts.   Use overlay charts to also identify especially subtle Pattern Changes.

Story

Time-based line charts are probably the most popular chart type (at least in Web Tortoise world).  In them, some period of time will be along the X axis where we read it from the left to the right, matching to a corresponding Y axis value as we go.  The chart could be showing anything (e.g. from website response time by the hour to number of website hits by the day) but they are all read the same way.

When reading a time-based line chart, will generally look at the value for a given period and compare to the previous value.  Was it the same as the previous value?  Was it more or less than the previous value?  If the value was different, by how much?  Is there a blip, or is there a sustained Pattern Change?

The identification of sustained Pattern Change is the focus of this Web Tortoise post.  The other day, was fortunate to discover a sustained Pattern Change in one of my customer’s key metrics.  The problem is the Pattern Change was so subtle, it almost went unnoticed!  In this Performance Index chart (screenshot taken from Google Analytics), able to identify which day the Pattern Change occurred?  How about which hour?  No?  Then “Check the Overlay!”

Note this chart is showing a Performance Index and the values of the Y axis are materially irrelevant.  What’s to be focused on is the general shape(s) of the daily patterns.

First, download the chart data and load into Excel; then apply custom formatting.  Might say the below chart is prettier with additional formatting, but still can’t quite identify the day or hour of the Pattern Change.

Second, take the above time-based line chart and turn it into an Overlay chart.  Since this chart is for 2-weeks, by the hour, there are 336 data points (24 hours in a day * 14 days = 336).  To create the overlay, take the latter 168 data points and plot along the Primary (bottom) left-to-right X axis; take the earlier 168 data points and plot along the Secondary (top) left-to-right X axis.

Note in the below Overlay chart, using both Primary and Secondary axis titles takes up a lot of real estate.  May remove the Secondary axis title if comfortable doing so.  Note I also chose to override Excel’s auto-sizing of the chart to ensure its length and width were using the Golden Ratio.  Here again, this takes up a lot of real estate, but adjust accordingly.

Because we used some semi-advanced Excel formatting features, because we continually measured Performance and because we Checked the Overlay!, we are able to report the sustained Pattern Change started on Wednesday, August 15th in the 08:00 AM PT hour.  At this point, consider engaging Release, Change or some other Management Function to figure out why.

Document Complete / OnLoad:

_The following is optional reading material._

Download the Excel file(s) here:  https://docs.google.com/open?id=0B9n5Sarv4oonYW8zQ0NjS2lqM2c

LinkedIn:  http://www.linkedin.com/in/leovasiliou

Twitter:  @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #ExcelOverlayChart #Overlay #CompareToPrevious #PatternChange

1 Comment »

  1. […] Check the overlay. Comparing the latter set of Response Times to the earlier set of Response Times. Was there a […]

    Pingback by WebTortoise Year in Review 2012 « Web Tortoise — 2012-Dec-20 @ 03:37 PM EDT


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: