To Sun bloggers, about benchmarks
By walterbays on Dec 14, 2005
Speaking as a SPEC representative, we love to have people using our benchmarks, but everyone has to follow the rules - bloggers too. Standard benchmarks have reporting and fair use rules that require for instance that performance claims be backed up by fully rule compliant test runs, and in some cases by prior independent results review; and that comparisons among products must include the appropriate information to offer a fair comparison.
Speaking as a Sun engineer, we have some great products and I don't want you to stop talking about them! Just please take care to follow the rules. If you're not sure what they are, ask. Inside Sun go to SAE on SunWeb for information on the benchmarks. Email them. Or email me.
See BMSeer's postings for examples of footnotes saying what he's comparing, the basis for comparison, when he looked at the competitive numbers, and substantiation of the numbers - like this entry about SPECweb2005. You may find the required disclosure inelegant, but bits are free. Put them in. It might just be exactly the information some reader really needs to know.