Web Tortoise


RUM Charts Side-by-Side With Synthetic Charts


Hello! This #WebTortoise post was written 2013-FEB-28 at 06:15 PM ET (about #WebTortoise).

Main Points

#- Consider the instrumentation of different Performance measurement tools before looking at their respective measurement data.

#- Measure web assets (e.g. websites, pages and/or apps) as an output of many different inputs (In Webtortoise World, we are talking about Real User Measurements (“RUM”) and Synthetic Measurements). Use these external, outside-in measurements to complement what is done internally.

#- The Response Times of the different Performance measurements are relative to a number of factors (e.g. distance, geography, browser cache, versions, infrastructure, application, ISP, CDN). These factors may also be different for each web asset.

#- See ‘Arithmetic Mean’ versus ‘Geometric Mean’ versus ‘Median’


In this Webtortoise post, will be looked at the various Response Times of the Ask.com homepage (Thank you, Ask.com). Have chosen this page because:

01. The URL http://www.ask.com/ was easy enough to measure Synthetically and RUMally (is that a word?) ;

02. It has a good mix of both first-party and third-party asset/object calls ; and

03. It has a good mix of both cacheable and non-cacheable asset/object calls.

Screenshot of the Ask.com homepage (2012-DEC-05):

Ask Home Page

In this post, the RUM data comes from Google Analytics and the Synthetic data comes from Catchpoint (thank you Google and Catchpoint). The RUM settings have been filtered to Geography=United States and Browser=Internet Explorer. Have also taken the metric ‘names’ directly from each provider, so folks may reference respective definitions themselves.

This first chart is showing [RUM: ‘Page Load Time’] metric and [Synthetic: full ‘Webpage Response’] metric:


Should not be surprised to see the RUM Response Times are higher than Synthetic Response Times. Was curious, though, why the RUM times on occasion dipped below the Synthetic times. After looking around, found GeoDB to be the culprit.

This second chart is showing [RUM: ‘Server Response’] metric and [Synthetic: ‘Server Response’] metric:


Was a bit surprised the RUM times here were lower than the Synthetic times. After looking around, discovered the RUM ‘Server Response Time’ did not include redirect or connect times, where the Synthetic ‘Server Response’ did.

When looking at these charts, one could almost remove the Y axis values and look at the lines by themselves. Did the next value in the series increase, decrease or remain the same versus the previous value? If there was a change, was it sustained or was it transient?

Here’s where is considered the instrumentation of your Performance measurements, to figure what may cause the hills and valleys. Remember, “If you do not measure Performance, then Performance will not be measured”. May or may not always be able to tell why the Response Times change, but that’s part of the fun!

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance #SiteSpeed

#RealUserMeasurements #RUM #SyntheticTests


WebTortoise Year in Review 2012

Filed under: Availability, Performance, Review — Tags: , , , , , , , — leovasiliou @ 03:37 PM EDT


Hello! This #WebTortoise post was written 2012-DEC-20 at 12:53 PM ET (about #WebTortoise).

Main Points

#- Because saying it once sometimes just isn’t enough! Here’s the WebTortoise 2012 Year in Review.


Once in a while, will have to retrain or refresh on a particular subject matter. This may be the result of an organizational change, may be the result of using something only occasionally or may be the result of any number of factors. In that vein, here are some select WebTortoise 2012 posts:

#- How do I calculate the geometric mean in Excel?

#- Excel: Use color to add value to your Performance charts.


#- Arithmetic Mean Average versus Geometric Mean Average: Knowing when to choose which calculation.


#- Excel Frequency Distribution: How many Response Times were between 0-1,000ms? How many Response Times were between 1,001-2,000ms? And so on?

Frequency Distribution

#- Excel Heat Map: Making it easier to find patterns in website Response Time. Applying Excel conditional formatting (red/yellow/green) to detect website’s “hot” times.


#- Always consider the different between Performance versus Availability when choosing your measurement instrument(s).

#- Check the overlay. Comparing the latter set of Response Times to the earlier set of Response Times. Was there a Pattern Change?


#- The Excel Hockey Stick Chart: Looking at Response Times across the entire % percentage range.

Excel Hockey Stick Chart RE Web Performance

#- Studying Prior Rates of Change to configure “Site is Slow” Performance alerts.

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #FrequencyDistribution

Blog at WordPress.com.