Web Tortoise

2013-Mar-31

Traffic Load vs Response Times – Hour of Day Dimension

Response:

Hello! This #WebTortoise post was written 2013-MAR-31 at 09:35 PM ET (about #WebTortoise).

Main Points

Question: How do I tell if the Response Time of my website is affected by traffic load (e.g. peak versus non-peak)?

Answer: Use an Hour of Day chart to correlate whether or not web traffic load affects Response Times. These non-time-based dimension charts allow you to aggregate data over more than one day if, for example, you wanted to look at several days/weeks/etc., but without having to plot several data in a time series.

A traditional time-based line chart may very well answer the asked question. However, at times, may be easy or necessary to look at long periods of time by Hour of Day, especially if there are subtleties to discover. In these examples, are being looked at three months data.

Story

Consider the following two statements, which convey the same idea of change each in a different way.

ABSOLUTE:  Our sales went from $1 last year to $2 this year!
RELATIVE:  Our sales increased 100% year-over-year!

In this Webtortoise post, will look at Response Times, as they vary through the day, in both Absolute (chart 2) and in Relative (chart 3) terms.  The effect of saying the same thing in a different way may be more profound, but must “remember to remember” the context of the overall picture.

Hour of Day charts, similar to Day of Week, Minute of Hour or other non-time-based charts are powerful ways to analyze the Performance and Availability data of your website. Was asked this question and, in researching, discovered a particular page performing worse than intended, especially compared to another like page on The Company’s site.

This first chart shows the average number of hits (for a 3-month period).

Chart Dimension Hour of Day -1 of 4

This second chart shows the Response Times for two pages on The Company’s site (for the same 3-month period).

Chart Dimension Hour of Day -2 of 4

This third chart shows the Response Times for the same two pages as in Chart 2. In this chart, however, the Response Times have been converted to percentages to make them relative on the same scale.

Chart Dimension Hour of Day -3 of 4

This fourth chart shows all three chart series in one location, with the # Visits on the Primary Axis and the Response Times on the Secondary Axis. Fair warning, this chart is misrepresenting because [intentionally] was removed the Primary and Secondary Axis labeling to avoid confusion.

Chart Dimension Hour of Day -4 of 4

Now, are talking about the second and third charts for a moment. Because Page 1 and Page 2 (on the second chart) are on the same Y axis, was not so easy to see Page 1 performing substantially worse during peak traffic. However, when changed to a relative % in the third chart, was more easily able to see the Performance delta.

Document Complete / OnLoad:

_The following is optional reading material._

Download Excel Sheet Here.

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ChartDimensions #HourOfDay #MinuteOfHour #DayOfWeek #Percentile #Histogram

2013-Feb-28

RUM Charts Side-by-Side With Synthetic Charts

Response:

Hello! This #WebTortoise post was written 2013-FEB-28 at 06:15 PM ET (about #WebTortoise).

Main Points

#- Consider the instrumentation of different Performance measurement tools before looking at their respective measurement data.

#- Measure web assets (e.g. websites, pages and/or apps) as an output of many different inputs (In Webtortoise World, we are talking about Real User Measurements (“RUM”) and Synthetic Measurements). Use these external, outside-in measurements to complement what is done internally.

#- The Response Times of the different Performance measurements are relative to a number of factors (e.g. distance, geography, browser cache, versions, infrastructure, application, ISP, CDN). These factors may also be different for each web asset.

#- See ‘Arithmetic Mean’ versus ‘Geometric Mean’ versus ‘Median’

Story

In this Webtortoise post, will be looked at the various Response Times of the Ask.com homepage (Thank you, Ask.com). Have chosen this page because:

01. The URL http://www.ask.com/ was easy enough to measure Synthetically and RUMally (is that a word?) ;

02. It has a good mix of both first-party and third-party asset/object calls ; and

03. It has a good mix of both cacheable and non-cacheable asset/object calls.

Screenshot of the Ask.com homepage (2012-DEC-05):

Ask Home Page

In this post, the RUM data comes from Google Analytics and the Synthetic data comes from Catchpoint (thank you Google and Catchpoint). The RUM settings have been filtered to Geography=United States and Browser=Internet Explorer. Have also taken the metric ‘names’ directly from each provider, so folks may reference respective definitions themselves.

This first chart is showing [RUM: ‘Page Load Time’] metric and [Synthetic: full ‘Webpage Response’] metric:

RUM.and.Synthetic.1

Should not be surprised to see the RUM Response Times are higher than Synthetic Response Times. Was curious, though, why the RUM times on occasion dipped below the Synthetic times. After looking around, found GeoDB to be the culprit.

This second chart is showing [RUM: ‘Server Response’] metric and [Synthetic: ‘Server Response’] metric:

RUM.and.Synthetic.2

Was a bit surprised the RUM times here were lower than the Synthetic times. After looking around, discovered the RUM ‘Server Response Time’ did not include redirect or connect times, where the Synthetic ‘Server Response’ did.

When looking at these charts, one could almost remove the Y axis values and look at the lines by themselves. Did the next value in the series increase, decrease or remain the same versus the previous value? If there was a change, was it sustained or was it transient?

Here’s where is considered the instrumentation of your Performance measurements, to figure what may cause the hills and valleys. Remember, “If you do not measure Performance, then Performance will not be measured”. May or may not always be able to tell why the Response Times change, but that’s part of the fun!

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance #SiteSpeed

#RealUserMeasurements #RUM #SyntheticTests

2013-Feb-14

Synthetic Test Runs – What Time Is It

Response:

Hello! This #WebTortoise post was written 2013-FEB-14 at 02:30 PM ET (about #WebTortoise).

Main Points

#- Various monitors and measurements can help assure Quality; Use them in creative ways.

#- The question, “What Time Is It?” is relative. So have a little fun with it.

Story

When discussing Synthetic Test Runs or Real User Measurements, are often referring to either monitoring Availability or to measuring Performance (see, “Availability versus Performance“). These attributes are very powerful, valuable data on their own, but they may also feed into [things like] quality.

In this Web Tortoise Story, asked the question, “What Time Is It” of a handful of large websites. The catch: the question was asked from Catchpoint’s US-based Synthetic Node network and are able to see geography-based web services are not perfect!

Note in each of these below examples, a different website was used.

—-
Asked from a Synthetic Node in Atlanta, GA.

What Time Is It - Atlanta

—-
—-
Asked from a Synthetic Node in New York City.

What Time Is It - NYC

—-
—-
Asked from a Synthetic Node in Washington, DC.

What Time Is It - DC

—-
—-
Asked from a Synthetic Node in Los Angeles, CA.

What Time Is It - Los Angeles

—-

Additionally, some Synthetic Nodes were redirected to other countries! And still other Synthetic Nodes didn’t get any time at all (instead, they were given links to other sites giving the time)!

Now, the example of “What Time Is It” may not be the best practical example, but the underlying principle is paramount. That is, when used in creative ways, your various monitors and measurements may give you more than just Availability or Performance data.

Next up in the #CreativeUses series: Image Search and DNS Takeovers.

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#CreativeUses #WhatTimeIsIt #AintNobodyGotTimeForThat

2013-Jan-31

Arithmetic Mean Versus Geometric Mean Versus Median

Response:

Hello! This #WebTortoise post was written 2013-JAN-31 at 09:06 PM ET (about #WebTortoise).

Main Points

#- An Arithmetic Mean will, for all intent and purpose in WebTortoise World, result in a higher value than its Geometric Mean counterpart. Relative to “faster is better” in web performance, might say an Arithmetic Mean is a pessimistic calculation.

#- A Geometric Mean will, for all intent and purpose in WebTortoise World, result in a lower value than its Arithmetic Mean counterpart. Relative to “faster is better” in web performance, might say a Geometric Mean is an optimistic calculation.

#- Define: What is a Percentile?

#- See, “How do I calculate the Geometric Mean in Excel”?

Story

Had an opportunity to discuss which statistical calculation should be used when looking at Performance charts. The discussion summary goes something like this.

First, assume consideration for a central-tendency calculation. Then:

If, in fact, looking for spurious outliers, consider plotting the Arithmetic Mean average.

Otherwise, consider plotting either the Geometric Mean or the Median, as they are very good central-tendency calculations.

To start, see this XY scatter plot taken from a day’s worth of synthetic test runs. In this Story, are using data from Catchpoint’s US node network (Thank you, Catchpoint), measuring @ 3,500 times a day (about 170 per hour). Intentionally chose this webpage as it contained a third-party ad network having particular host issues (the waterfall data was invaluable for troubleshooting, but that’s a Story for another day).

Eyeballing the chart, notice the thick band of majority data is less than 5,000 ms (right around 1,500 – 3,000 ms) with thinner pockets and bands throughout. Also notice around between 10:00 AM – 02:00 PM, there were no measurements higher than around 14,000 ms.

XY Scatter Plot

Second, will take the above XY scatter plot and draw a bar graph representing the middle 25th-75th percentile range (See, “What is a Percentile”). The idea here is to show a middle range (which might better represent overall Performance) versus just a single line (which can sometimes ‘lie’ or misrepresent).

Middle Range

Third, using the same data from the XY scatter plot, overlay line charts showing respective Arithmetic Mean, Geometric Mean and Median calculations.

Arithmetic Mean VS Geometric Mean VS Median

Critical thing to notice is the height of the Arithmetic Mean (Y axis) versus either the Geometric Mean or the Median. Notice how the Arithmetic Mean is, at times, either very near the upper limit of the middle range or, in some cases, even above the upper limit of the middle range! Now notice the Geometric Mean and Median are always comfortably between the middle range.

Other:

Notice the 12:00 AM and 07:00 AM hour’s Arithmetic Mean is above the Middle Range. Now, quickly glance back at the XY scatter plot to see the measurement data.

Notice the middle range for the 02:00 PM and 03:00 PM hours are smaller than other hours. Glancing back at the XY scatter plot, can see the thick band of measurement data is more tightly packed.

Last, want to give a fair warning when looking at these types of charts: The amount of the data will generally affect the height and patterns of the lines and bars. Do not be caught off guard if, for example, the Arithmetic Mean average is always above your middle range. This is a function of the amount of data.

Document Complete / OnLoad:

_The following is optional reading material._

Download Excel document: https://docs.google.com/file/d/0B9n5Sarv4oonaDZSZXNURzZrd00/edit?usp=sharing

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #ExcelXYScatter #ArithmeticMean #GeometricMean #Median

2012-Dec-20

WebTortoise Year in Review 2012

Filed under: Availability, Performance, Review — Tags: , , , , , , , — leovasiliou @ 03:37 PM EST

Response:

Hello! This #WebTortoise post was written 2012-DEC-20 at 12:53 PM ET (about #WebTortoise).

Main Points

#- Because saying it once sometimes just isn’t enough! Here’s the WebTortoise 2012 Year in Review.

Story

Once in a while, will have to retrain or refresh on a particular subject matter. This may be the result of an organizational change, may be the result of using something only occasionally or may be the result of any number of factors. In that vein, here are some select WebTortoise 2012 posts:

#- How do I calculate the geometric mean in Excel?

#- Excel: Use color to add value to your Performance charts.

Excel.Waterfall.Snip.2012-MAR-06-2114ET

#- Arithmetic Mean Average versus Geometric Mean Average: Knowing when to choose which calculation.

Comparing.Mean.Calculations

#- Excel Frequency Distribution: How many Response Times were between 0-1,000ms? How many Response Times were between 1,001-2,000ms? And so on?

Frequency Distribution

#- Excel Heat Map: Making it easier to find patterns in website Response Time. Applying Excel conditional formatting (red/yellow/green) to detect website’s “hot” times.

Heat.Map.Side.by.Side

#- Always consider the different between Performance versus Availability when choosing your measurement instrument(s).

#- Check the overlay. Comparing the latter set of Response Times to the earlier set of Response Times. Was there a Pattern Change?

Blog.Post.Check.the.Overlay-3

#- The Excel Hockey Stick Chart: Looking at Response Times across the entire % percentage range.

Excel Hockey Stick Chart RE Web Performance

#- Studying Prior Rates of Change to configure “Site is Slow” Performance alerts.

Document Complete / OnLoad:

_The following is optional reading material._

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #FrequencyDistribution

2012-Nov-28

Configuring “Site is Slow” Performance Alerts

Response:

Hello!  This #WebTortoise post was written 2012-NOV-28 at 10:55 AM ET (about #WebTortoise).

Question & Answer

Question: I have various measurements continually recording the Response Time of my website. Now, though, I’d like to configure some Performance alerts to know if there is a Performance degradation, but I don’t know the exact settings to choose. So, how should I configure them?

Answer: First, notice this question is about Performance versus Availability. This is an important distinction because the alert settings would be configured differently for one versus the other.

Second, this question is looking for a “good enough” place to start. For example, if there is already a Response Time threshold set by Management, then the below Webtortoise Story may or may not be considered.

Now, regarding the question, the suggested answer is, “Consider using a Bayesian approach and study prior Rates of Change (explained in the below Story)”. Then consider how sensitive to configure the settings.

Fair warning, each measurement vendor would implement their alert modules in different ways and this below Story is only one specific example. The principle answer still applies, though:

Study prior Rates of Change.

Story

In Webtortoise World, is continually discussed how to measure website Performance and how to alert if it degrades. Have these conversations a lot, particularly with various Operation and Production folks who’d be receiving the alert emails (even in the middle of the night!).

Have all types of Availability alerts in place, but what if the site just slows (while still being technically available)? Maybe just need to tighten the settings a bit as the holiday season approaches? Maybe just getting a bit too many alert emails and people are starting to ignore them? Maybe just this? Maybe just that? Well, without further ado…

Step 01. Have a test measurement in place and let it run for a few days or a few weeks (the larger the sample size the better). The idea here is we’ll be looking “back” at the data to help determine the “forward” setting of the Performance alert.

Step 02. Decide the alert attributes. In this Story, we’ll be alerting on the Full Webpage Response Time metric, comparing the delta between the latter hour and the former hour. If the Rate of Change from one hour to the next is above a certain threshold, then send an alert email.

As mentioned, each measurement vendor would implement their alert modules in different ways. Please remember the attributes in this Story are only one specific example.

Step 03. Calculate the Rates of Change from one hour to the next. For example, if Response Time for the Midnight hour is 1,517 ms and if Response Time for the 01:00 AM hour is 1,503 ms, then the Rate of Change is 0.92% (1,517 minus 0.92% of 1,517 equals 1,503)(this Excel sheet contains the formulas for calculating this Rate of Change). If Response Time for the 01:00 AM hour is 1,503 ms and if Response Time for the 02:00 AM hour is 1,532 ms, then the Rate of Change is 1.93% (1,503 plus 1.93% of 1,503 equals 1,532).

May have noticed is being discarded whether the Rate of Change is positive or negative. For the purpose of this Story, that is okay.

However:

Note to all Performance measurement providers: Most have capabilities to alert on only Response Time INCREASES. Consider adding capability to alert also on Response Time DECREASES as they can be just as indicative of a problem.

Finish calculating the Rates of Change (In this Excel sheet, is calculated the Rates of Change for six weeks of test measurement data, by the hour (total of 1,008 hours). The formula in column D will always give a positive number (except when the Rate of Change is zero) and column D has been formatted to display a Percentage %).

Step 04. Now use a Frequency Distribution on the Rates of Change (for a refresher on Frequency Distributions, consider reading Webtortoise: What the Frequency?) to answer the question(s), “How many Rates of Change were less than 1%? How many Rates of Change were between 1-2%? How many Rates of Change were between 2-3%?” And so on.

The Frequency Distribution will answer these questions and, in the same Excel sheet, can see most Rates of Changes are between zero thru twenty’ish percent %. Now, given most Rates of Change, from one hour to the next, are less than 20%, should the alert threshold be set to less than 20%? …

Probably not.  Unless many alert emails are desired.

If the threshold setting is meant to alert in the most egregious of Performance degradations, then maybe set the alert threshold to 50% or greater. Looking again at the Frequency Distribution, can see a Rate of Change greater than 50% occurred eight times in the last six weeks. If the threshold setting is meant to alert in some other condition, then can look at the Frequency Distribution to get an idea of how sensitive the setting should be. At this point, consider other relative items to determine how sensitive the threshold setting should be. Otherwise, the threshold setting will come down to making a choice and iterating.

Document Complete / OnLoad:

_The following is optional reading material._

Here’s the traditional, time-based line chart for the test measurement used in this post.  It is for a 6-week period, by the hour, totaling 1,008 data.

Download the excel sheet here:  https://docs.google.com/open?id=0B9n5Sarv4oonZWJVMU9QTTlzSGM

Webtortoise Author on LinkedIn:  http://www.linkedin.com/in/leovasiliou

Webtortoise Author on Twitter:  https://twitter.com/Lvasiliou

#CatchpointUser #KeynoteUser #GomezUser #Webtortoise #Performance #WebPerformance

#ExcelStatistics #FrequencyDistribution

2012-Oct-29

The Web Performance Hockey Stick Chart!

Response:

Hello! This #WebTortoise post was written 2012-OCT-29 at 03:30 PM ET (about #WebTortoise).

Main Points

#- Percentile-based Histograms complement time-based line charts, and should be part of your Web Performance data analysis arsenal.

#- Percentile-based Histograms look at overall Performance, independent of time.

#- Percentile-based Histograms are a good way to look at A/B test results (In Web Tortoise World, we’re looking at a change affecting various Response Times. Did the change improve or worsen Response Times?).

#- Excel Chart: Excel Hockey Stick Chart.

#- Excel VBA: Add drop lines to chart in this story: http://t.co/zSKTAy4Q

#- Define: What is a Percentile?

Story

In this story, we’re looking at whether a change in Apache improved or worsened our overall Webpage Response Times. We started by looking at a traditional time-based line chart, but there were periods during the day where the Performance lines oscillated around each other. Because of this, we couldn’t really get an idea of the overall Performance change. So, we then decided to look at a percentile-based Histogram (because we wanted an idea about the overall change (if in fact there was one)).

After making the change in Apache, measurements ran for a week. In the below chart, we’re looking at the Earlier week VS the Latter week, where the X axis dimension is based on Percentages and the Y axis is the actual Percentile values.

WebTortoise Excel Hockey Stick Chart

Reading the Chart

Here’s how you read the above chart:  Example, forty percent % of the Webpage Response Times, for both the Before and After week, were just under 1,000 ms (find 40% on the X axis, and follow it up to the corresponding Y axis value).  You can start to see the delta in the Before and After weeks get larger and larger as you move further right along the X axis.

NOTICE how the Median values are nearly the same.  If you were looking at only a single calculation versus this above full distribution, you may have said there was no Performance change at all!

Note download the excel sheet here to see the actual Percentile values.

If we’re eyeballing the above chart, might say right around 50-55’ish percent of the Response Times were about the same (or very close), before and after our change was made.  However, as we move further right along the Percentage X axis, we notice that for about 40-45’ish percent of our users, they are getting a slower Response Time!

Download the excel sheet to see the actual values for all the Percentiles.  But here are the 99% for each “Before” and “After” week:
In the “Before” week, 99% of our Response Times were below 2,299 ms.  In the “After” week, 99% of our page loads were below 4,022 ms.  We can confidently say the change we made for our A/B Performance testing *worsened* our overall Performance.

Document Complete / OnLoad:

_The following is optional reading material._

Download the excel sheet here:  https://docs.google.com/file/d/0B9n5Sarv4oonYUJFaTVRMlZpZDQ/edit?usp=sharing

LinkedIn: http://www.linkedin.com/in/leovasiliou

Twitter: @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #ExcelOverlayChart #Overlay #CompareToPrevious #PatternChange #WebPerformance #ExcelHockeyStickChart #PercentileBasedHistogram

2012-Sep-20

Hey Hey, Check the Overlay!

Response:

Hello!  This #WebTortoise post was written 2012-SEP-20 at 03:43 PM ET (about #WebTortoise).

Main Points

#- Use programs like Excel or Tableau to get additional formatting value from your third-party-provided chart data.

#- Continually measure website Performance to identify Pattern Changes, be them by minute of hour, by hour of day, by day of week or by some other dimension.  Point-in-time measurements have value, where continually measuring will tell more of the story.

#- Use overlay charts (a.k.a. “Compare to Previous” charts) to complement time-based line charts.   Use overlay charts to also identify especially subtle Pattern Changes.

Story

Time-based line charts are probably the most popular chart type (at least in Web Tortoise world).  In them, some period of time will be along the X axis where we read it from the left to the right, matching to a corresponding Y axis value as we go.  The chart could be showing anything (e.g. from website response time by the hour to number of website hits by the day) but they are all read the same way.

When reading a time-based line chart, will generally look at the value for a given period and compare to the previous value.  Was it the same as the previous value?  Was it more or less than the previous value?  If the value was different, by how much?  Is there a blip, or is there a sustained Pattern Change?

The identification of sustained Pattern Change is the focus of this Web Tortoise post.  The other day, was fortunate to discover a sustained Pattern Change in one of my customer’s key metrics.  The problem is the Pattern Change was so subtle, it almost went unnoticed!  In this Performance Index chart (screenshot taken from Google Analytics), able to identify which day the Pattern Change occurred?  How about which hour?  No?  Then “Check the Overlay!”

Note this chart is showing a Performance Index and the values of the Y axis are materially irrelevant.  What’s to be focused on is the general shape(s) of the daily patterns.

First, download the chart data and load into Excel; then apply custom formatting.  Might say the below chart is prettier with additional formatting, but still can’t quite identify the day or hour of the Pattern Change.

Second, take the above time-based line chart and turn it into an Overlay chart.  Since this chart is for 2-weeks, by the hour, there are 336 data points (24 hours in a day * 14 days = 336).  To create the overlay, take the latter 168 data points and plot along the Primary (bottom) left-to-right X axis; take the earlier 168 data points and plot along the Secondary (top) left-to-right X axis.

Note in the below Overlay chart, using both Primary and Secondary axis titles takes up a lot of real estate.  May remove the Secondary axis title if comfortable doing so.  Note I also chose to override Excel’s auto-sizing of the chart to ensure its length and width were using the Golden Ratio.  Here again, this takes up a lot of real estate, but adjust accordingly.

Because we used some semi-advanced Excel formatting features, because we continually measured Performance and because we Checked the Overlay!, we are able to report the sustained Pattern Change started on Wednesday, August 15th in the 08:00 AM PT hour.  At this point, consider engaging Release, Change or some other Management Function to figure out why.

Document Complete / OnLoad:

_The following is optional reading material._

Download the Excel file(s) here:  https://docs.google.com/open?id=0B9n5Sarv4oonYW8zQ0NjS2lqM2c

LinkedIn:  http://www.linkedin.com/in/leovasiliou

Twitter:  @LvasiLiou

#CatchpointUser #KeynoteUser #GomezUser #ExcelOverlayChart #Overlay #CompareToPrevious #PatternChange

2012-Aug-22

Are You Available?

Filed under: WebTortoiseAnchor001 — Tags: , , , — leovasiliou @ 01:59 PM EST

Response:

Hello!  This #WebTortoise post was written 2012-AUG-22 at 01:39 PM ET (about #WebTortoise).

<==>
Consider the difference between Availability versus Performance when choosing which assets to monitor and/or measure.
<==>

Hello, Everyone.  This is a Web Tortoise anchor post.  Its academic material will be referenced in future posts.

In order for us to measure a website’s Performance, the website must first be Available.  Sometimes Availability and Performance are mistakenly discussed as a single item, but they are clear and distinct from each other:  We _monitor_ Availability, whereas we _measure_ Performance.  In Web Tortoise World, the distinction goes something like this:

Availability:  Was the website available?

Performance:  If the website was available, how long did it take to download?

After is distinguished between Availability versus Performance, decide which assets to monitor and/or measure.

Document Complete / OnLoad:

_The following is optional reading material._

Here are the ITIL Version 3 definitions of Availability and Performance.

Availability:

Ability of a Configuration Item or IT Service to perform its agreed function when required

Performance:

A measure of what is achieved or delivered by a system, person, team, process, or IT Service

As previously distinguished, we _monitor_ Availability, whereas we _measure_ Performance.  Here are some non-IT, anecdotal examples:

Availability:  Was the coffee shop open?

Performance:  If the coffee shop was open, how long was the wait?

Availability:  Did you make it to work this morning?

Performance:  If you made it to work this morning, how long was the drive?

Availability:  Did you read this post?

Performance:  If you read this post, how long did it take?

As the answers to the Availability questions are either only “yes” or “no”, say Availability is a binary attribute.  That is, it’s either “available” or “not available”.  Performance, however, is a Measurement.  That is [ example ], “It took me 20 minutes to get a cup of coffee this morning” or, “Traffic was terrible, it took me 90 minutes to drive 22 miles!”  Alternatively, [ example ], “The coffee shop was closed, so there was no wait” or, “There was a major accident and the roadway was not available; I did not make the drive to work.”  In those last two examples, we are unable to measure Performance because the coffe shop and roadway were not available.  In other words, the ability to measure Performance depends on the asset being Available.

#CatchpointUser #KeynoteUser #GomezUser #WebTortoiseAnchor001

2012-Jul-12

What Are Your Website’s Hot Times?

Filed under: Performance — Tags: , , , , — leovasiliou @ 12:19 PM EST

Response:

Hello! This #WebTortoise post was written 2012-JUL-12 at 11:56 AM ET.

==> Use color to add value to your Performance charts.

In this Heat Map example, web site’s Performance data is overlay with color, where red means “slowest” and green means “fastest” (with various color grades in between). I added this Heat Map example because the chart data was difficult to “crunch”, to analyze for patterns; adding color made this easier. Download the Excel sheet here. Otherwise, see the below “Before” and “After”.

Before: Raw Chart Data:

After: Heat Map:

Side by Side:

The chart data is the arithmetic mean average of six weeks’ worth of web site response time measurements (in milliseconds), broken down by Day of Week and by Hour of Day. I then used Excel’s built-in /Conditional Formatting/Color Scales/Red – Yellow – Green Color Scale/.

Document Complete / OnLoad:

_The following is optional reading material._

Here are some potential fact statements we get from looking at this Heat Map:

01. The weekends are the “fastest”.

02. The weekdays are the “slowest”.

03. Response Time slows starting at around 07:00 AM PT and Response Time speeds starting at around 07:00 PM PT.

04. The “slowest” (i.e. the most red) “Day of Week”/”Hour of Day” combination is Wednesday at 05:00 PM PT.

05. The “fastest” (i.e. the most green) “Day of Week”/”Hour of Day” combination is Wednesday at 02:00 AM PT.

06. Interestingly enough, the “fastest” and “slowest” “Day of Week”/”Hour of Day” combination was on a Wednesday.

07. Friday afternoons were “faster” when compared to other weekday afternoons (insert comment here RE: folks getting ready for the weekend!).

Download the Excel file here: https://docs.google.com/open?id=0B9n5Sarv4oonMEdtOTlvOHNib0E

#CatchpointUser #KeynoteUser #GomezUser #HeatMap #ExcelConditionalFormatting

Older Posts »

Blog at WordPress.com.