Web Tortoise


Don’t Forget About Availability


Hello! This #WebTortoise post was written 2013-MAY-16 at 11:34 AM ET (about #WebTortoise).

Main Points

#- Analyze Availability by various Dimensions, e.g. Hour of Day or Minute of Hour, to look for patterns.

#- Performance infers Availability. Performance may be measured if and only if Availability = 1 (your choices are either 1 or 0; either something’s available or something’s not available).

#- We monitor Availability; we measure Performance.

#- Don’t go it alone. When working to uncover patterns in your Availability and Performance data, will need the help of others in the Organization.


I thought I’d break away from the normal second-and-third-person writing style of Webtortoise to write this more intimate, first-person post. Lately, I’ve been feeling bad for my buddy, Availability (In this Webtortoise Story, my buddy’s name is, “Availability”). You see, Availability’s cousin, Performance, has been getting all of the limelight. I mean, don’t get me wrong, Performance IS sleek and sexy while Availability IS binary and boring, but the only reason we’re able to talk about all these advancements in Performance is because of their JOINT efforts!

So much attention has been given to Performance lately that I am seeing more and more folks forget, or casually glaze over, Availability! The problem here is: Performance _infers_ Availability. That is, if it’s not available, then you cannot measure it [for Performance].

So please, help me spread the word and remind folks to never forget about their ol’ buddy and friend, “Availability”.

And now, your obligatory Webtortoise chart:

In this chart, we counted the number of Availability strikes (a.k.a. errors) for several days. Then we plotted the COUNT by Minute of Hour.

In this first chart, there is no special formatting. But can still see some high errors counts.

Blog Post Availability by Minute of Hour - 1

In this second chart, have highlighted and called out the discovered pattern! At first, the guessed pattern was incorrect because was trying to find a *single*. However, after pulling in some more people resources, was able to figure out there were *multiples*.

Blog Post Availability by Minute of Hour - 2

In this specific case, these patterns were caused by *two separate* log shippings, across two different subsystems, affecting page load (i.e. the page was not available)! And had it not been for a Performance Management Program, may never have discovered these Patterns!

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

Download Excel sheet here:  https://docs.google.com/file/d/0B9n5Sarv4oonTjR0Zk9oYzI1bGc/edit?usp=sharing

#CatchpointUser #KeynoteUser #Webtortoise #Performance #WebPerformance #SiteSpeed #ChartsAndDimensions #Availability


Why Did The Statistician Cross the Road


Hello! This #WebTortoise post was written 2013-APR-30 at 09:35 AM ET (about #WebTortoise).

Main Points

#- Here’s to the statisticians of the world!


– Why did the statistician cross the road?
— He wasn’t sure.

– A statistician can have his head in an oven and his feet in ice, and he will say that on the average he feels fine (http://math.bnu.edu.cn/~chj/Statjokes.htm).

– A new government 10 year survey costing $3,000,000,000 revealed 3/4 of the people in America make up 75% of the population (http://www.ahajokes.com/m027.html).

– According to recent surveys, 51% of the people are in the majority (http://www.ahajokes.com/m027.html).

– Statistics play an important role in genetics. For instance, statistics prove that numbers of offspring is an inherited trait. If your parents didn’t have any kids, odds are you won’t either (One passed by Gary Ramseyer, taken from http://stats.stackexchange.com/questions/1337/statistics-jokes).

– Final Exam: A statistics major was completely hung over the day of his final exam. It was a true/false test, so he decided to flip a coin for the answers. The statistics professor watched the student the entire two hours as he was flipping the coin… writing the answer… flipping the coin… writing the answer. At the end of the two hours, everyone else had left the final except for the one student. The professor walks up to his desk and interrupts the student, saying, “Listen, I have seen that you did not study for this statistics test, you didn’t even open the exam. If you are just flipping a coin for your answer, what is taking you so long?”
The student replies bitterly (as he is still flipping the coin), “Shhh! I am checking my answers!” (http://math.bnu.edu.cn/~chj/Statjokes.htm)

– Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital (Aaron Levenstein, taken from http://www.workjoke.com/statisticians-jokes.html).

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #Webtortoise #Performance #WebPerformance #SiteSpeed #ChartsAndDimensions



Traffic Load vs Response Times – Hour of Day Dimension


Hello! This #WebTortoise post was written 2013-MAR-31 at 09:35 PM ET (about #WebTortoise).

Main Points

Question: How do I tell if the Response Time of my website is affected by traffic load (e.g. peak versus non-peak)?

Answer: Use an Hour of Day chart to correlate whether or not web traffic load affects Response Times. These non-time-based dimension charts allow you to aggregate data over more than one day if, for example, you wanted to look at several days/weeks/etc., but without having to plot several data in a time series.

A traditional time-based line chart may very well answer the asked question. However, at times, may be easy or necessary to look at long periods of time by Hour of Day, especially if there are subtleties to discover. In these examples, are being looked at three months data.


Consider the following two statements, which convey the same idea of change each in a different way.

ABSOLUTE:  Our sales went from $1 last year to $2 this year!
RELATIVE:  Our sales increased 100% year-over-year!

In this Webtortoise post, will look at Response Times, as they vary through the day, in both Absolute (chart 2) and in Relative (chart 3) terms.  The effect of saying the same thing in a different way may be more profound, but must “remember to remember” the context of the overall picture.

Hour of Day charts, similar to Day of Week, Minute of Hour or other non-time-based charts are powerful ways to analyze the Performance and Availability data of your website. Was asked this question and, in researching, discovered a particular page performing worse than intended, especially compared to another like page on The Company’s site.

This first chart shows the average number of hits (for a 3-month period).

Chart Dimension Hour of Day -1 of 4

This second chart shows the Response Times for two pages on The Company’s site (for the same 3-month period).

Chart Dimension Hour of Day -2 of 4

This third chart shows the Response Times for the same two pages as in Chart 2. In this chart, however, the Response Times have been converted to percentages to make them relative on the same scale.

Chart Dimension Hour of Day -3 of 4

This fourth chart shows all three chart series in one location, with the # Visits on the Primary Axis and the Response Times on the Secondary Axis. Fair warning, this chart is misrepresenting because [intentionally] was removed the Primary and Secondary Axis labeling to avoid confusion.

Chart Dimension Hour of Day -4 of 4

Now, are talking about the second and third charts for a moment. Because Page 1 and Page 2 (on the second chart) are on the same Y axis, was not so easy to see Page 1 performing substantially worse during peak traffic. However, when changed to a relative % in the third chart, was more easily able to see the Performance delta.

Document Complete / OnLoad:

_The following is optional reading material._

Download Excel Sheet Here.

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ChartDimensions #HourOfDay #MinuteOfHour #DayOfWeek #Percentile #Histogram


RUM Charts Side-by-Side With Synthetic Charts


Hello! This #WebTortoise post was written 2013-FEB-28 at 06:15 PM ET (about #WebTortoise).

Main Points

#- Consider the instrumentation of different Performance measurement tools before looking at their respective measurement data.

#- Measure web assets (e.g. websites, pages and/or apps) as an output of many different inputs (In Webtortoise World, we are talking about Real User Measurements (“RUM”) and Synthetic Measurements). Use these external, outside-in measurements to complement what is done internally.

#- The Response Times of the different Performance measurements are relative to a number of factors (e.g. distance, geography, browser cache, versions, infrastructure, application, ISP, CDN). These factors may also be different for each web asset.

#- See ‘Arithmetic Mean’ versus ‘Geometric Mean’ versus ‘Median’


In this Webtortoise post, will be looked at the various Response Times of the Ask.com homepage (Thank you, Ask.com). Have chosen this page because:

01. The URL http://www.ask.com/ was easy enough to measure Synthetically and RUMally (is that a word?) ;

02. It has a good mix of both first-party and third-party asset/object calls ; and

03. It has a good mix of both cacheable and non-cacheable asset/object calls.

Screenshot of the Ask.com homepage (2012-DEC-05):

Ask Home Page

In this post, the RUM data comes from Google Analytics and the Synthetic data comes from Catchpoint (thank you Google and Catchpoint). The RUM settings have been filtered to Geography=United States and Browser=Internet Explorer. Have also taken the metric ‘names’ directly from each provider, so folks may reference respective definitions themselves.

This first chart is showing [RUM: ‘Page Load Time’] metric and [Synthetic: full ‘Webpage Response’] metric:


Should not be surprised to see the RUM Response Times are higher than Synthetic Response Times. Was curious, though, why the RUM times on occasion dipped below the Synthetic times. After looking around, found GeoDB to be the culprit.

This second chart is showing [RUM: ‘Server Response’] metric and [Synthetic: ‘Server Response’] metric:


Was a bit surprised the RUM times here were lower than the Synthetic times. After looking around, discovered the RUM ‘Server Response Time’ did not include redirect or connect times, where the Synthetic ‘Server Response’ did.

When looking at these charts, one could almost remove the Y axis values and look at the lines by themselves. Did the next value in the series increase, decrease or remain the same versus the previous value? If there was a change, was it sustained or was it transient?

Here’s where is considered the instrumentation of your Performance measurements, to figure what may cause the hills and valleys. Remember, “If you do not measure Performance, then Performance will not be measured”. May or may not always be able to tell why the Response Times change, but that’s part of the fun!

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance #SiteSpeed

#RealUserMeasurements #RUM #SyntheticTests


Synthetic Test Runs – What Time Is It


Hello! This #WebTortoise post was written 2013-FEB-14 at 02:30 PM ET (about #WebTortoise).

Main Points

#- Various monitors and measurements can help assure Quality; Use them in creative ways.

#- The question, “What Time Is It?” is relative. So have a little fun with it.


When discussing Synthetic Test Runs or Real User Measurements, are often referring to either monitoring Availability or to measuring Performance (see, “Availability versus Performance“). These attributes are very powerful, valuable data on their own, but they may also feed into [things like] quality.

In this Web Tortoise Story, asked the question, “What Time Is It” of a handful of large websites. The catch: the question was asked from Catchpoint’s US-based Synthetic Node network and are able to see geography-based web services are not perfect!

Note in each of these below examples, a different website was used.

Asked from a Synthetic Node in Atlanta, GA.

What Time Is It - Atlanta

Asked from a Synthetic Node in New York City.

What Time Is It - NYC

Asked from a Synthetic Node in Washington, DC.

What Time Is It - DC

Asked from a Synthetic Node in Los Angeles, CA.

What Time Is It - Los Angeles


Additionally, some Synthetic Nodes were redirected to other countries! And still other Synthetic Nodes didn’t get any time at all (instead, they were given links to other sites giving the time)!

Now, the example of “What Time Is It” may not be the best practical example, but the underlying principle is paramount. That is, when used in creative ways, your various monitors and measurements may give you more than just Availability or Performance data.

Next up in the #CreativeUses series: Image Search and DNS Takeovers.

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#CreativeUses #WhatTimeIsIt #AintNobodyGotTimeForThat


Arithmetic Mean Versus Geometric Mean Versus Median


Hello! This #WebTortoise post was written 2013-JAN-31 at 09:06 PM ET (about #WebTortoise).

Main Points

#- An Arithmetic Mean will, for all intent and purpose in WebTortoise World, result in a higher value than its Geometric Mean counterpart. Relative to “faster is better” in web performance, might say an Arithmetic Mean is a pessimistic calculation.

#- A Geometric Mean will, for all intent and purpose in WebTortoise World, result in a lower value than its Arithmetic Mean counterpart. Relative to “faster is better” in web performance, might say a Geometric Mean is an optimistic calculation.

#- Define: What is a Percentile?

#- See, “How do I calculate the Geometric Mean in Excel”?


Had an opportunity to discuss which statistical calculation should be used when looking at Performance charts. The discussion summary goes something like this.

First, assume consideration for a central-tendency calculation. Then:

If, in fact, looking for spurious outliers, consider plotting the Arithmetic Mean average.

Otherwise, consider plotting either the Geometric Mean or the Median, as they are very good central-tendency calculations.

To start, see this XY scatter plot taken from a day’s worth of synthetic test runs. In this Story, are using data from Catchpoint’s US node network (Thank you, Catchpoint), measuring @ 3,500 times a day (about 170 per hour). Intentionally chose this webpage as it contained a third-party ad network having particular host issues (the waterfall data was invaluable for troubleshooting, but that’s a Story for another day).

Eyeballing the chart, notice the thick band of majority data is less than 5,000 ms (right around 1,500 – 3,000 ms) with thinner pockets and bands throughout. Also notice around between 10:00 AM – 02:00 PM, there were no measurements higher than around 14,000 ms.

XY Scatter Plot

Second, will take the above XY scatter plot and draw a bar graph representing the middle 25th-75th percentile range (See, “What is a Percentile”). The idea here is to show a middle range (which might better represent overall Performance) versus just a single line (which can sometimes ‘lie’ or misrepresent).

Middle Range

Third, using the same data from the XY scatter plot, overlay line charts showing respective Arithmetic Mean, Geometric Mean and Median calculations.

Arithmetic Mean VS Geometric Mean VS Median

Critical thing to notice is the height of the Arithmetic Mean (Y axis) versus either the Geometric Mean or the Median. Notice how the Arithmetic Mean is, at times, either very near the upper limit of the middle range or, in some cases, even above the upper limit of the middle range! Now notice the Geometric Mean and Median are always comfortably between the middle range.


Notice the 12:00 AM and 07:00 AM hour’s Arithmetic Mean is above the Middle Range. Now, quickly glance back at the XY scatter plot to see the measurement data.

Notice the middle range for the 02:00 PM and 03:00 PM hours are smaller than other hours. Glancing back at the XY scatter plot, can see the thick band of measurement data is more tightly packed.

Last, want to give a fair warning when looking at these types of charts: The amount of the data will generally affect the height and patterns of the lines and bars. Do not be caught off guard if, for example, the Arithmetic Mean average is always above your middle range. This is a function of the amount of data.

Document Complete / OnLoad:

_The following is optional reading material._

Download Excel document: https://docs.google.com/file/d/0B9n5Sarv4oonaDZSZXNURzZrd00/edit?usp=sharing

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #ExcelXYScatter #ArithmeticMean #GeometricMean #Median


WebTortoise Year in Review 2012

Filed under: Availability, Performance, Review — Tags: , , , , , , , — leovasiliou @ 03:37 PM EST


Hello! This #WebTortoise post was written 2012-DEC-20 at 12:53 PM ET (about #WebTortoise).

Main Points

#- Because saying it once sometimes just isn’t enough! Here’s the WebTortoise 2012 Year in Review.


Once in a while, will have to retrain or refresh on a particular subject matter. This may be the result of an organizational change, may be the result of using something only occasionally or may be the result of any number of factors. In that vein, here are some select WebTortoise 2012 posts:

#- How do I calculate the geometric mean in Excel?

#- Excel: Use color to add value to your Performance charts.


#- Arithmetic Mean Average versus Geometric Mean Average: Knowing when to choose which calculation.


#- Excel Frequency Distribution: How many Response Times were between 0-1,000ms? How many Response Times were between 1,001-2,000ms? And so on?

Frequency Distribution

#- Excel Heat Map: Making it easier to find patterns in website Response Time. Applying Excel conditional formatting (red/yellow/green) to detect website’s “hot” times.


#- Always consider the different between Performance versus Availability when choosing your measurement instrument(s).

#- Check the overlay. Comparing the latter set of Response Times to the earlier set of Response Times. Was there a Pattern Change?


#- The Excel Hockey Stick Chart: Looking at Response Times across the entire % percentage range.

Excel Hockey Stick Chart RE Web Performance

#- Studying Prior Rates of Change to configure “Site is Slow” Performance alerts.

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #FrequencyDistribution


Configuring “Site is Slow” Performance Alerts


Hello!  This #WebTortoise post was written 2012-NOV-28 at 10:55 AM ET (about #WebTortoise).

Question & Answer

Question: I have various measurements continually recording the Response Time of my website. Now, though, I’d like to configure some Performance alerts to know if there is a Performance degradation, but I don’t know the exact settings to choose. So, how should I configure them?

Answer: First, notice this question is about Performance versus Availability. This is an important distinction because the alert settings would be configured differently for one versus the other.

Second, this question is looking for a “good enough” place to start. For example, if there is already a Response Time threshold set by Management, then the below Webtortoise Story may or may not be considered.

Now, regarding the question, the suggested answer is, “Consider using a Bayesian approach and study prior Rates of Change (explained in the below Story)”. Then consider how sensitive to configure the settings.

Fair warning, each measurement vendor would implement their alert modules in different ways and this below Story is only one specific example. The principle answer still applies, though:

Study prior Rates of Change.


In Webtortoise World, is continually discussed how to measure website Performance and how to alert if it degrades. Have these conversations a lot, particularly with various Operation and Production folks who’d be receiving the alert emails (even in the middle of the night!).

Have all types of Availability alerts in place, but what if the site just slows (while still being technically available)? Maybe just need to tighten the settings a bit as the holiday season approaches? Maybe just getting a bit too many alert emails and people are starting to ignore them? Maybe just this? Maybe just that? Well, without further ado…

Step 01. Have a test measurement in place and let it run for a few days or a few weeks (the larger the sample size the better). The idea here is we’ll be looking “back” at the data to help determine the “forward” setting of the Performance alert.

Step 02. Decide the alert attributes. In this Story, we’ll be alerting on the Full Webpage Response Time metric, comparing the delta between the latter hour and the former hour. If the Rate of Change from one hour to the next is above a certain threshold, then send an alert email.

As mentioned, each measurement vendor would implement their alert modules in different ways. Please remember the attributes in this Story are only one specific example.

Step 03. Calculate the Rates of Change from one hour to the next. For example, if Response Time for the Midnight hour is 1,517 ms and if Response Time for the 01:00 AM hour is 1,503 ms, then the Rate of Change is 0.92% (1,517 minus 0.92% of 1,517 equals 1,503)(this Excel sheet contains the formulas for calculating this Rate of Change). If Response Time for the 01:00 AM hour is 1,503 ms and if Response Time for the 02:00 AM hour is 1,532 ms, then the Rate of Change is 1.93% (1,503 plus 1.93% of 1,503 equals 1,532).

May have noticed is being discarded whether the Rate of Change is positive or negative. For the purpose of this Story, that is okay.


Note to all Performance measurement providers: Most have capabilities to alert on only Response Time INCREASES. Consider adding capability to alert also on Response Time DECREASES as they can be just as indicative of a problem.

Finish calculating the Rates of Change (In this Excel sheet, is calculated the Rates of Change for six weeks of test measurement data, by the hour (total of 1,008 hours). The formula in column D will always give a positive number (except when the Rate of Change is zero) and column D has been formatted to display a Percentage %).

Step 04. Now use a Frequency Distribution on the Rates of Change (for a refresher on Frequency Distributions, consider reading Webtortoise: What the Frequency?) to answer the question(s), “How many Rates of Change were less than 1%? How many Rates of Change were between 1-2%? How many Rates of Change were between 2-3%?” And so on.

The Frequency Distribution will answer these questions and, in the same Excel sheet, can see most Rates of Changes are between zero thru twenty’ish percent %. Now, given most Rates of Change, from one hour to the next, are less than 20%, should the alert threshold be set to less than 20%? …

Probably not.  Unless many alert emails are desired.

If the threshold setting is meant to alert in the most egregious of Performance degradations, then maybe set the alert threshold to 50% or greater. Looking again at the Frequency Distribution, can see a Rate of Change greater than 50% occurred eight times in the last six weeks. If the threshold setting is meant to alert in some other condition, then can look at the Frequency Distribution to get an idea of how sensitive the setting should be. At this point, consider other relative items to determine how sensitive the threshold setting should be. Otherwise, the threshold setting will come down to making a choice and iterating.

Document Complete / OnLoad:

_The following is optional reading material._

Here’s the traditional, time-based line chart for the test measurement used in this post.  It is for a 6-week period, by the hour, totaling 1,008 data.

Download the excel sheet here:  https://docs.google.com/open?id=0B9n5Sarv4oonZWJVMU9QTTlzSGM

Webtortoise Author on LinkedIn:  http://www.linkedin.com/in/leovasiliou

Webtortoise Author on Twitter:  https://twitter.com/Lvasiliou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #FrequencyDistribution

« Newer Posts

Blog at WordPress.com.