Hi,

Came across this discussion from a link to a later article...so sorry for the latness of reply, but felt I had to correct the misconception that a Baseline and Benchmark are one an the same, because they're not.

http://en.wikipedia.org/wiki/Benchmark_(computing)

http://en.wikipedia.org/wiki/Baseline_(configuration_management)

A benchmark is used to assess the performance of a system using a set of tests that are standardised, so as to give comparable results. Often different system architectures are tested, such as Oracle, SQL Server, DB2, etc., in the Database TPC tests (www.tpc.org). Often here, you'll find that those who participate are looking to achieve the highest performance and set the benchmark. When I benchmark a set of systems presented by different vendors as the best system/setup I often use a tool something like SQLIO, which I will run in a pre-defined and standard way. I then compare the results to see which gave the best performance characteristics for the type of workload I'm looking for the system to do.

With baselines, you are looking to set a starting point from which to compare future performance of a particular system, for example. This can be seen in the DMVStats tool for performance monitoring of SQL Server 2005. In this tool you can create baselines for different time intervals on the same system and then compare the results. Often in baselining you create a 'before change' baseline, carry out some change to the system, and then create an 'after change' baseline. From these you can compare and see what effect, if any, the change you made had.

So, IMHO, baselines and benchmarks are not the same thing.

HTH