Broadband Speed Tester Benchmarks November 23, 2006
Posted by paragonhost in Internet, VOIP.add a comment
Broadband Speed Tester Benchmarks
A comparison of Internet connection speed testers
by Jerry Jongerius - March 4, 2005 - v16
| All Bandwidth Testers are not Created Equal |
 Fig.1 - myspeed.visualware.com |
Is your Internet speed tester accurate?
Writing an Internet bandwidth speed tester on the surface appears to be very easy. Just download a test data file, time it, and calculate the resulting bandwidth, right? No, even major companies get it wrong, as we will see below. The result is many speed testers that report an incorrect bandwidth for your connection.
The need for speed test accuracy
Are you getting the bandwidth that you paid for? If you paid for a 8 Mbps service, but are only testing at 6 Mbps, is the problem with your broadband connection, or with your speed tester?
The need for broadband bandwidth consistency - Why QOS Matters
Your broadband connection is only as reliable as its weakest link. So if you paid for a 3 Mbps connection, but the performance of the connection jumps around from 5 Mbps to 1 Mbps (over time averaging 3 Mbps), you will effectively be limited to 1 Mbps for planning purposes. Because if you deploy an application that needs more than 1 Mbps, that application will fail, because at times it really will not have more than 1 Mbps to use. In other words, it is far easier to deploy a ‘race car’ (application) on a ‘track’ (broadband) that is ‘flat’ (consistent) rather than ‘rolling hills’ (excellent, then poor, then excellent).
Measuring broadband consistency/QOS
In order for your speed tester to provide an accurate quality of service (QOS) measurement for your broadband connection, it first needs to be able to very accurately measure your bandwidth. The concept of QOS is illustrated below.
Fig.2 - Provider A, 100% QOS
Fig.3 - Provider B, 20% QOS
Both provider A (Fig.2) and provider B (Fig.3) deliver an average of 3 Mbps performance. Provider A provides an excellent consistent service, obtaining a QOS of 100% (3/3). However, provider B does not provide consistent results, resulting in a QOS of 20% (1/5) (details). Most high-quality broadband connections today easily deliver a QOS over 90%. Test your broadband QOS now using MySpeed.
The bottom line
In order to correctly evaluate your Internet broadband connection bandwidth and consistency, you need a speed tester that is very accurate and precise.
| Benchmarking Various Speed Testers |
In Fig.4 you can see the results of the author’s evaluation of various widely-used Internet broadband speed testers:
| Speed Tester | Run #1 | Run #2 | Run #3 | Run #4 | Run #5 |
Fig.4 - ‘Mbps / Kbps’ for various Internet speed testers
All tests were run from the author’s PC7 via a cable modem broadband connection8 with a known download speed around 4.23 Mbps and an upload speed around 370 kbps.
As discussed, a credible speed tester must provide both consistent and accurate results. From the table above you can see that only MySpeed provided consistent and accurate test results.
Why are some speed testers not as accurate and consistent as compared to other testers? Unfortunately, many companies write their own bandwidth speed test without understanding how easy it is to create a speed tester which contains hidden incorrect assumptions, resulting in a speed test which produces inaccurate test results. Writing an accurate speed tester at a minimum requires:
- a detailed working knowledge of certain aspects of TCP/IP
- a knowledge of timer units vs accuracy in the programming language used
- an understanding of basic mathematical concepts, like precision
- a server with an appropriate connectivity for multiple simultaneous client speed tests
- an ability to validate one’s own speed test results
Let’s examine some of the speed testers and analyze the test results.
| Evaluating Speed Tester Two |
 Fig.5 - Speed Tester Two results |
After reviewing the HTML source code for this speed tester, it appears to be timing (cable modem connection type) how long it takes for a 518,095 byte JPEG image to download within the context of a web page, and calculating the bandwidth like:
TIME = (time in seconds for image to download)
KBYTES = 500;
kbps = Math.round(KBYTES/TIME*8*10*1.02) / 10;
There are a couple of issues with this code. The first is an error in the KBYTES (in 1000 byte units) of the JPEG image. It is 518.095, not the 500 seen in the code. The second is the 1.02 multiplier, which could be an attempt by the test author to account for TCP/IP header overhead, but as that multiplier is not accurate the true intent of this multiplier is not known.
Timing how long it takes to download an image is a classic, well known and published speed test method. However, this technique is flawed in that it incorrectly assumes that:
- downloading a moderately sized image utilizes 100% of the download bandwidth immediately
- JavaScript language is a good choice for timing events
 Fig.6 - Speed Tester Two bandwidth utilization |
In reality, downloading a moderately sized image only utilizes 100% of the download bandwidth after an initial startup time.
The reason why is left as an exercise for the reader (requires a moderate understanding of how TCP/IP works).
This initial under-utilization can clearly be seen in the bandwidth utilization over time graph in Fig.6, of the actual image download during this speed test. This initial short under-utilization is one of the reasons that this speed test is not accurate. This speed test assumes 100% bandwidth utilization at all times, when in reality that is not the case. The result (as can be seen in the table above) is a large error (because of the relatively small image file downloaded) by this speed tester in underestimating the true bandwidth speed (a speed of 3.2 Mbps reported for a 4.2 Mbps link).
Interestingly, when the author used a download performance verification program to download exactly the same JPEG image as is used by this speed tester, the calculated download speed was 4,234,000 bps! So, if you avoid the pitfalls and use an accurate speed test program, accurate results can be achieved. Everything (the client computer, the server computer, the network) was the same except the program used to calculate the bandwidth speed. Clearly, the bandwidth calculation in this speed tester needs to be changed.
Finally, how accurate is JavaScript at timing events? The JavaScript Date() object provides millisecond units, but not necessarily millisecond accuracy. Under Windows, the accuracy is only to (at best) 0.01 seconds. This means that depending upon the final download time, there can only exist two or to three digits of precision. So when the test displays a download speed like “3306.3 Kbps” (which implies a precision to 5 digits), given that they only have two (or three) digits of precision in their time measurements, it is clear that this speed test author did not apply the concept of mathematical precision (the end result cannot be more precise than the minimum precision of the intermediate values).
Understanding Precision: Your boss gives you a digital thermometer which displays the temperature accurate to the degree. He asks you to calculate average temperature outside your office at high noon in May. Every day in May you take a measurement. At the end of the month you calculate the average and present your results to your boss. Do you report the average with any decimal digits, like 85.47°, or do you just report the temperature rounded to the degree, like 85°.
The answer is 85° because you can never have more digits of precision in a calculated answer than the minimum digits of precision in any value used to calculate the answer. So if all temperatures have two digits of precision, you can only present an answer with two digits of precision.
Another way of understanding the answer is to look at 85.47°, which is pretty much saying that the average could not have been 85.45° or 85.49°. But given the error possible in each daily temperature measurement, you really don’t know that. So reporting 85.47° as the average temperature is implying a precision that does not exist.
| Evaluating Speed Tester Three/Four |
 Fig.7 - Speed Tester Three/Four results |
After reviewing the HTML source to this speed tester, it appears to be using a very similar technique to Speed Tester Two, in that it embeds a lot of text within a web page, then times how long it takes for the web page to download, as in:
start = new Date().getTime();
(614405 bytes in the HTML)
end = new Date().getTime();
elapsed = (end-start)/1000;
KBps = Math.round(614405*100/(elapsed*1024)) / 100;
This speed test is making very similar incorrect assumptions that timing a moderately sized web page will provide you with a measure of your bandwidth. For all of the same reasons that Speed Tester Two is not accurate, neither is this speed test.
Moreover, this speed test reports a Mbps value that has been calculated incorrectly. The text on this speed tester’s web page appears very clear on the issue of bits vs bytes:
- Kbps - kilobits per second, or thousands of bits per second
- KBps - kilobytes per second
- Mbps - megabits per second, or millions of bits per second
- MBps - megabytes per second
As you can see in Fig.7 (ignoring mathematical precision issues for now), downloading 614405 bytes in 1.609 seconds is 372.91 kilobytes per second, but that is not 2.913 megabits per second (it is that many megabytes per second). It is instead 3.055 megabits per second (614405*8/1.609/1000000).
The bottom line is that Speed Tester Three/Four is not accurate, and the Mbps value reported to the end user is calculated incorrectly.
| Evaluating Speed Tester Five |
 Fig.8 - Speed Tester Five results |
A Java applet is used, which comes the closest to being the correct choice for a speed test, however the results are still far from accurate. A likely cause is that their servers (or network) are over-utilized, making it very difficult to provide an accurate speed test.
When the author ran some tests with this speed tester a couple of days later, the results seemed much better, but they were still were not accurate. After reviewing the HTML source code, the test download files were found.
Once again, when using an authenticated download speed program the download results were 4.23 Mbps. This confirms that this speed tester also incorrectly assumes 100% download utilization during the entire download.
| Evaluating Speed Tester Six |
 Fig.9 - Speed Tester six results |
This speed tester appears to have been written using Macromedia Flash. Given how quickly the test completed (under one second), this tester appears to be using a very small download file size. Also, given the wildly varied results, the web server or network to the web server appears to be highly utilized.
When the author tested the download speed from this tester’s server using a speed test verification program, results were mostly in the upper 3 Mbps range, sometimes above 4 Mbps, and once around 1 Mbps, confirming that this speed test is assuming 100% bandwidth utilization during the entire download, resulting in a bandwidth measurment that is a lot more incorrect than it should be. Also, since 4.23 Mbps results were not obtained every time (except once), this confirms that either the web server or network of Speed Tester Six is over utilized.
In Fig.9, this speed tester places “Cable” below that of T1, which is around 1.5 Mbps. Given that almost all cable companies provide cable modem download speeds above 3.0 Mbps, it appears that this speed tester was written some time ago and not updated to deal with today’s true broadband connections.
| How Accurate is Your Speed Tester? |
The key benchmark for any bandwidth speed tester is: Can it deliver consistent and accurate results for a highly accurate broadband connection of known speed? If not, the speed tester cannot be trusted.
The results of this benchmark analysis determined that only the MySpeed speed tester provided accurate and consistent Internet bandwidth speed test measurements.
| References |
- MySpeed: 4,233,464/370,888, 4,233,336/370,912, 4,232,408/370,648, 4,232,984/370,600, and 4,232,008/370,440. These results are fully auditable and can be viewed in the online enhanced reporting database. See tests #190078, #190080, #190081, #190083, and #190084. [Feb 28, 2005]
- CNET bandwidth meter: 3434.3, 3225.3, 3148.1, 3182.5, and 3225.3 kbps. [Feb 28, 2005]
- McAfee Speedometer (restart via web browser restart): 2.858, 2.885, 2.856, 3.001, and 2.804 Mbps. [Feb 28, 2005]
- McAfee Speedometer (restart via the ‘retest’ button): 2.856, 3.296, 4.223, 4.167, and 4.481 Mbps. [Feb 28, 2005]
- speedtest.dslreports.com 3,359,457/368,459 3,093,440/370,851 3,329,373/369,480 bps from server 1 and 679,936/363,967 620,765/363,967 bps from server 2. [Feb 28, 2005]
- Intel Broadband Speed Test: 732.2 Kbps, 767.9 Kbps, 1.3 Mbps, 339.4 Kbps, 700.6 Kbps. [March 3, 2005]
- Client computer was a Windows XP Professional, 3.20 Ghz Pentium 4, 1.0 GB RAM system. The ethernet card (Realtek RTL8169/8110 Gigabit) was connected at 100 Mbps to a LinkSys 10/100 Workgroup Switch, connected to a LinkSys WRT54G router, connected to a LinkSys BEFCMU V3 cable modem, connected to Comcast; see 8 below.
- Tests were performed on a broadband cable modem with a known download speed right around 4.23 Mbps and a known upload speed right around 370 kbps. In the test environment, the cable network amazingly provides virtually 100% consistent bandwidth all the time, which is very unusual for a broadband cable system, but which is great for running benchmarks. This speed was validated by downloading and uploading large files. 51,726,024 bytes were downloaded in 97,786 milliseconds (4,231,800 bps) and 8,626,342 bytes were uploaded in 186.36 seconds (370,310 bps).
Source: http://www.myspeed.com/support/benchmarks.html
World Class Internet Services: http://www.paragonhost.com
Business Class VOIP Services: http://www.telxvoip.com
What is Quaility of Service (QOS) Why do we need it? November 23, 2006
Posted by paragonhost in VOIP.add a comment
Why is Quality of Service (QOS) a crucially important metric for business applications?
| A tale of two airlines |
Consider for a moment two competing cargo airlines (A and B) operating out of Washington Dulles airport with service to London. Both airlines have the same aircraft, both airlines charge the same rate per package shipped and both airlines offer seven flights a week. On the surface it appears that there is very little to differentiate the service offered by these two companies. It would also be fair to say that, on paper at least, these competing airlines offer the same throughput capacity in terms of packages shipped per week.
Now consider that airline A offers one flight per day for each day of the week whereas the airline B offers all of its seven flights on a single day of the week. So while both airlines provide the same throughput capacity over the weekly period, they differ greatly in the actual service provided.
Depending on your delivery needs these different airline service models will succeed or fail in quite dramatic fashion. If your business is an online real-time mail order business, choosing airline B would mean that your customers would be unlikely to get next day delivery and at worst would have to wait an entire week for their order to be shipped. In contrast, by choosing airline A with its daily service to London you would obviously be able to deliver a much more consistent next day delivery service to your customers, which is often critical for a mail order business to remain competitive.
A different delivery requirement would not necessarily provide the same benefit however. If, for example, your delivery needs were simply to supply stock to your retail outlet in London on a monthly basis you could argue that airline B would offer the better overall service depending on your capacity needs. If the quantity of packages to ship each week exceeded the capacity of a single aircraft, then airline A would take two or more days to ship your goods to London whereas airline B would more than likely be able to accommodate the entire shipment in a single day.
Obviously, the business requirements for delivery define which service model works best. Real-time (daily) demand needs a very regular and consistent service whereas irregular (weekly) batch demand does not.
| QOS and ISPs |
Now consider two competing Internet Service Providers (ISPs A and B) offering, what on the surface seems to be, the identical broadband services of 3 Mbit per second. However, ISP A delivers a consistent throughput of 3 Mbps for each and every second, whereas ISP B offers an inconsistent throughput of 5 Mbps in the first second, 1 Mbps in the second, 3 Mbps in the third, 4 Mbps in the fourth and 2 Mbps in the fifth, as depicted in the graphs below.
Fig. 1. While both ISPs average 3 Mbps, ISP A provides a consistent
throughput of 3 Mbps, while ISP B provides fluctuating throughput.
Over a full five second period both ISPs have delivered 15 Mbps of data (average of 3 Mbps), but which ISP is offering the better Internet service overall? The answer to that question is identical in concept to the airline answer: It depends entirely on the business application.
If your Internet usage is simply browsing websites and downloading files then you are unlikely to notice much difference in the services offered by these two competing ISPs. It could be argued that ISP B would provide a better browsing service as more data can transfer in the first second making web page retrieval a little quicker. However if, as many businesses do, you run applications that are geared to a real-time environment such as multimedia applications, video, online conferencing and VoIP telephony, then the consistent service offered by ISP A will deliver a dramatically better quality of service than the fluctuating service offered by ISP B.
So while these two ISPs may provide similar throughput, they have entirely different Quality of Service (QOS) models, with QOS being defined as:
minimum speed
————————————— = Quality of Service
maximum speed
In Fig. 1 above, the QOS for provider A is 100% (5/5), whereas the QOS for provider B is 20% (1/5).
The end result is that you are more likely to get ‘jerky’ video and sound breaks using ISP B than with ISP A even though both ISPs, on paper, offer 3 Mbps service.
| Q. Should you be concerned about the QOS of your networks? |
A. Absolutely
Obviously there are many different types of applications that run over the Internet, especially when it comes to business B2B type applications. In addition to the multimedia type applications, there are many other very QOS-sensitive applications that are commonly used across the Internet. These applications will deliver a very poor customer experience when operated on networks that have a poor QOS rating. As an example, Citrix (delivers the Windows desktop environment), a business application framework widely-used across the Internet and corporate extranets, is sensitive to changing latencies and as a result can easily disconnect when run on networks with a poor QOS rating.
Finally, understanding the QOS needs of an application and the QOS provided by the ISP is a crucial step in delivering a good overall customer experience. Understanding the QOS requires accurate measurement and reporting of throughput over time, along with a periodic review of the application requirements vs. the actual service provided
Source: http://www.myspeed.com/support/qos_matters.html
Content Powered By: ParagonHost, LLC http://www.paragonhost.com
Voip Services Powered By: TelxVoip http://www.telxvoip.com
MySpeed PC - Internet Speed Test Program November 23, 2006
Posted by paragonhost in VOIP.add a comment
ParagonHost: http://www.paragonhost.com/intro.html
“World Class Internet Services”
TelxVoip: http://www.telxvoip.com
“Total VOIP Solutions for your Business”
MySpeed PC is a downloadable speed test program that accurately measures your Internet connection bandwidth, and provides an easy to understand report of your download and upload speeds and connection quality.
FREE for personal use!
For additional speed test features see MySpeed PC Advanced. For information on hosting MySpeed on your website see the MySpeed Server and MySpeed Server Professional options.
KEY FEATURES:
|
|
MySpeed PC Advanced
Download a FREE trial of MySpeed PC Advanced Edition! |
