Wednesday May 27, 2009

Paula Smith is SPECtacular

Another SPECtacular award from the SPEC annual meeting: Paula Smith (VMware) was honored for her tireless, competent and patient work managing the SPEC office and the people there. Paula consistently exhibits what make SPEC an unique place. The attention and enthusiasm she brings to her volunteer work make her a pleasure to interact with. She goes above and beyond in everything she does, and is often able to turn emergencies into opportunities. Most impressive is how she maintains this over time and in every interaction, despite many competing pressures for her attention. Beyond this management work, she also manages to handle the organizational and technical work of chairing the Virtualization committee, and of course her day job at VMware.

Tuesday May 26, 2009

Cloyce Spradling is SPECtacular

Another SPECtacular award from the SPEC annual meeting: Cloyce Spradling (Sun Microsystems) was honored for continued timely support of SPEC CPU, HPG and editorial tools. The key factor is his timeliness, in that he responds to unplanned, asynchronous requests, if not with a solution then at least with a map to help people find their way out of the woods. And that's on top of his day job at Sun.

Friday May 22, 2009

SPEC awards, power performance

More 2009 SPECtacular awards. The SPECpower committee has been busy. They released version 1.10 of the SPECpower_ssj2008 benchmark as a no-cost upgrade to existing licensees. It adds support for measurement of multi-node (blade) servers, improves usability, and adds a graphical display of power data during benchmark execution. Review and publication of benchmark results continues apace, with a spirited competition for first place, and with ever more power analyzers accepted for testing, and more test labs qualified for independent publication. They have also been assisting several other benchmark committees inside SPEC, and other industry standard benchmark organizations, to implement energy measurement for their benchmarks. SPECpower is more than just a benchmark; it is a methodology, and the methodology is modified and expanded as necessary over time to accommodate energy measurements for all the different workloads which are relevant to the real world in those market segments. In alphabetical order SPEC recognizes:

  • Chris Boire (Sun Microsystems) – As release manager he coordinated and integrated development activities to keep the deliverables on schedule.

  • David Schmidt (HP) – He created stand-alone and network integrated tools for automated results checking to help insure that results submissions are correct and complete.

  • Greg Darnell (Dell) – Author of the PTDaemon, he helped many other groups get started measuring power for their benchmarks. He helps out with whatever needs to be done, technical or organizational.

  • Hansfried Block (Fujitsu Technology Solutions) - He automated the process of determining power analyzer precision, handled the acceptance of several new power analyzers, and was instrumental in getting multi-channel analyzers accepted.

  • Harry Li (Intel) – He was primary developer of the Visual Activity Monitor, giving an unique view of the system's activity.

  • Jeremy Arnold (IBM) – If I tried to recount all the accomplishments Jeremy was cited for I'd probably run into some internal blog size limit. Suffice it to say he is a primary developer on many parts of the code, who never turns down a plea for help, and who is never satisfied until the entire benchmark package is right.

  • Karl Huppler (IBM) – As primary author/editor of the Power and Performance Methodology, he organized the document to capture deep technical consensus in the committee, and made it readable and understandable for people new to the field.

  • Matthew Galloway (HP) – He designed the control software to drive multiple JVMs, enabling multi node (blade) testing.

  • An engineer (AMD) – Who created and maintained much of the web content explaining the benchmark and methodology to the public.

Thursday May 14, 2009

SPEC awards, virtualization

More 2009 SPECtacular awards. SPEC's forthcoming virtualization benchmark will provide meaningful metrics of hardware and software performance in data center consolidation. As complex as this benchmark is, running several different benchmarks together in virtual machines on a host system under test, the code is only half the story. As with all benchmarks the workload is vital, to represent realistic usage scenario(s) so that performance improvements made on the benchmark will also benefit real world users. And the run rules are vital, needing to accommodate technology improvements over the lifetime of the benchmark, while precluding unrepresentative optimizations exploiting rule loopholes. (Or what the layman might call “cheating”) There is spirited debate from companies representing rather diverse user communities, all with an interest in seeing that their customers' needs are addressed by the benchmark. In the end when this group of top engineers reaches a consensus you know they've come up with a benchmark that is as rock solid as is possible to make. From among this great team of partners and competitors, three were singled out for SPECtacular awards:

Andrew Bond of HP always steps forward when a person is needed to test new code, features, parameter tuning. He performed many experiments whose results showed the committee the sensitivity of the benchmark to various parameters, sizes, and configuration options, so that the right choices could be made for fair benchmark comparisons. He also created scripts to set up and configure new guest VMs for each workload.

Chris Floyd of IBM improved and tailored the mail server and application server workloads for the new benchmark. He's revamped these workloads several times to improve the I/O profiles and add burstiness to the application server transaction injection. He helps the other developers at regular on-line coding sessions, explaining new features, and resolving problems. He even helps out when on vacation.

Greg Kopczynski of VMware developed a (necessarily) complex and feature extensive harness for the benchmark. He responds to countless pleas for help, assistance, debugging, etc., in true SPEC fashion without asking whether the help is for a partner or a competitor. He added burstiness to the web server workload. And he integrates new code and changes from all the developers for each development kit revision.

Thanks for your great efforts!

Wednesday May 13, 2009

SPEC awards, graphics

More 2009 SPECtacular awards. Sometimes even success doesn't succeed, at first. SPEC developed a workstation energy consumption benchmark, and a lot of people worked extra hard to deliver it in time for EPA to consider using it in the Energy Star program which is being extended beyond PC's to also include workstations, servers, thin clients, and storage. Although EPA decided not to use our test for the workstation program at this time, the work is still important and I am confident it will be used in some way. A graphics processor can easily use more energy than a CPU, especially a high performance accelerated 3D processor. For their exceptional work in producing this benchmark I thank David Reiner of AMD, Joerg Grosshennig of Fujitsu Technology Solutions, Paul Besl of Intel, and an engineer from NVIDIA.

Tuesday May 12, 2009

SPEC awards, HPC

More 2009 SPECtacular awards. SPEC released an update to our MPI2007 benchmark of Message Passing Interface performance. It allows evaluation of MPI-parallel, floating point, compute intensive performance across a wide range of cluster and SMP hardware. MPI2007 continues the SPEC tradition of giving HPC users the most objective and representative benchmark suite for measuring the performance of SMP (shared memory multi-processor) systems. The update, provided at no cost to existing MPI2007 licensees, improved compatibility, stability, documentation and ease of use. SPEC gave awards to:

  • Brian Whitney of Sun Microsystems for meticulous care as release manager in scheduling, and putting it all together,

  • Carl Ponder of IBM for the development and management of documentation, especially with respect to the run rules, FAQ, and the configuration file.

  • Håkon Bugge of Platform Computing for outstanding testing skills during the benchmark development.

Monday May 11, 2009

SPEC award, mail server benchmark

At SPEC's 2009 annual meeting, awards were given for SPECtacular contributions. When your competitors and partners alike join to honor one of your own it indicates that person has truly excelled. There are 77 member organizations in SPEC including hardware vendors, software vendors, universities, government agencies, and more. We are joined by a common belief that the industry as a whole is well served by a common base of reliable and representative measures of computer system performance and energy. Thereby our companies benefit from more effective test results at lower cost. And for academia, the SPEC benchmarks provide a common reference point from which to begin performance and energy related studies.

It takes a lot of hard work to produce these benefits. Each year the individuals who are recognized by their peers as having done the most to advance SPEC's mission are singled out for awards. And now I have the pleasure of thanking these exceptional people publicly. I won't list everyone since some people don't want their names posted; but you know who you are.

I'll start today by thanking Michael Abbott of Apple who carried the brunt of the new profile and code changes to the SPECmail2009 benchmark, and contributed invaluable insight and analysis on message characteristics to improve the representativeness of the benchmark. SPECmail2009 simulates corporate mail server workloads ranging from 250 to 10,000 or more users, using industry standard SMTP, IMAP4, SSL v3.0, and TLS 1.0 protocols. Folder and message MIME structures accomodate traditional office documents and a variety of rich media content.

The SPECtacular award winners like Michael are making a positive difference in the industry, and so I say thank you! (More award winners coming...)

Sunday Apr 05, 2009

personal blog on Wordpress

I signed up for a blog at WordPress, though there's nothing there as yet. From time to time I've thought of getting a personal blog. There are few restrictions on what I can write here, beyond the basic guidance to use good judgment. But I believe that since it's hosted on sun.com it reflects somewhat on the company, and also that readers here are more likely to be interested in information somehow related to computers. So some topics seem out of place here to me, though different bloggers make different judgments on what they will write about.

It was hard to choose between Wordpress and Blogspot and both looked like they would do the job. More serious blog sites and hosted software would be overkill for my casual and intermittent writing. The main drawback of Wordpress.com I read about was that it's difficult to customize the blog beyond the basic templates. No problem for me since I'm not overly concerned with the look and feel, other than an extreme dislike of uber-chic sites which use dark grey 7 point font on mottled black backgrounds.


Friday Apr 03, 2009

R&D Spending - IEEE report

In my second blog posting I wrote about IEEE Spectrum's survey of 2004 R&D spending. Today I looked at their latest report, for 2007, to see what happened to those top companies in the intervening years. Ford slipped from #1 to #8 despite holding R&D at 4.3% of sales, meanwhile Toyota climbed from #3 to #1 even while their spending dropped from 4.1% to 3.6% of sales. Size matters. GM clearly takes R&D seriously, climbing from #5 to #2 by increasing R&D spending from 3.4% to 4.5% of sales - an increase of $1.6 billion. Maybe if they survive, they really will bring the Volt to market and recapture lost market share in the new energy economy.

In the computer world, Microsoft is on top at #9 overall, with R&D spending increasing a bit more slowly than sales, but spending an impressive $93k per employee. Sun slipped slightly in rank and in spending as percent of sales, on a modest increase in R&D dollars. IBM increased R&D both in absolute dollars and in percent of sales, but slipped in rank. With their size comes a big R&D budget. AMD is a real standout, doubling R&D spending, moving up from #89 to #65, and increasing spending as percent of sales from 18.7% to 30.0%. AMD's R&D per employee at $112k exceeds even that of the pharamaceutical companies which traditionally lead in this measure.

In the table below I colored the 2007 "Rank" column red if the company declined both in rank and in R&D spending as percent of sales. I colored it green if the company increased both in rank and in R&D spending as percent of sales. In the 2007 "$k/ employee" column I colored it red if the spending was below the median of IEEE's top 100 companies, and green if it was above the 80% percentile.



2004

2007

Company Rank R&D ($B) % of Sales Rank R&D ($B) % of Sales $k/ employee
Ford 1 7.4 4.3% 8 7.5 4.3% 30
Daimler (Chrysler) 2 7.2 4.0% 22 4.7 3.2% 17
Toyota 3 7.1 4.1% 1 8.8 3.6% 28
Pfizer 4 6.6 12.6% 3 8.1 16.8% 93
GM 5 6.5 3.4% 2 8.1 4.5% 30
Siemens 6 6.4 6.7% 19 5.1 4.7% 13
Microsoft 7 6.2 15.5% 9 7.1 13.9% 90
Matsushita 8 5.7 7.1% 18 5.1 6.1% 17
Glaxo Smith Kline 9 5.3 13.9% 12 6.3 13.9% 61
Johnson & Johnson 10 5.2 11.0% 6 7.7 12.6% 64
IBM 11 5.2 5.4% 15 5.8 5.9% 15
Intel 13 4.8 14.0% 14 5.8 15.1% 67
HP 25 3.5 4.4% 35 3.6 3.5% 21
Fujitsu 46 2.2 5.0% 54 2.4 5.0% 14
TI 51 2 15.7% 58 2.2 15.9% 71
Sun 56 1.8 16.1% 62 2 14.4% 59
AMD 89 0.9 18.7% 65 1.8 30.0% 112

Monday May 05, 2008

ROI for Sun

Not NASDAQ - solar power. The ASES annual solar energy conference is in San Diego this week. The top question I get about my solar panels is how long is my return on investment? I did calculate it before we installed them. At current electricity prices and time value of money they will just break even over their useful life. (And we live near the ocean where morning fog obscures the panels on many summer mornings.) Still, if we had another price shock equivalent to the 1973 oil embargo they would pay back about twice the initial investment, in current dollars. And if we had another price shock equivalent to Kenny-Boy Lay's market manipulation, they would pay back about five times the initial investment. Economically, call the panels zero cost insurance.

Now what's the ROI on an SUV? Our solar panels cost about a quarter to a half the price of a big SUV. Will that Escalade have a productive life of 20 years? And over that time how many dollars will it return to your pocket? Or will it perhaps take more money out of your pocket? For the price of the SUV you could instead buy solar panels, zero out your electric bill, buy a Chevy Malibu (which sits in clogged traffic equally well as the SUV), and have enough money left over to pay for over 200,000 miles worth of gas for it, at $4/gallon.

So why aren't there more solar panels in sunny Southern California? Why is Germany, in the cloudy wintery north, so far ahead of the U.S.? Two reasons: (1) money, and (2) money.

(1) Lots of people don't have the luxury of deciding whether to spend discretionary money on a new SUV or on solar panels; they're deciding whether to pay the mortgage, pay the electric bill, or fill up the gas tank. Ditto businesses hard pressed to show a profitable bottom line. Increasingly solar energy entrepreneurs are in effect buying energy "drilling rights" on rooftops. LA's electric utility Edison is building the equivalent of a new generating plant by putting panels on the roofs of commercial and industrial buildings. The building owners pay nothing, and get a good long term locked in electricity rate. Here in San Diego, Hewlett-Packard is converting its campus to solar power. HP stockholders will pay nothing for it, and HP will get substantial energy cost savings in the future. While they're at it, HP is matching the rebate to their employees who want to put solar panels on their homes.

(2) The recent earth shaking discovery that people are more willing to give goods and services in exchange for money, than to give them with nothing in return. (See capitalism.) The biggest barrier to local development of solar energy in San Diego has been a convoluted rate structure that in many cases actually made businesses that installed solar generators pay more money to use less electricity, than before they installed them. Small wonder that northern California is far ahead of sunnier southern California in solar power installations. Now that crazy rate structure is changing, which could bring a boom in locally generated solar power.

For homeowners in San Diego no change is forthcoming. Germany has all those solar installations because of a rate structure that pays for solar electricity at much higher than market rates. In San Diego you see many solar panel installations like ours covering a small portion of the roof. The rate structure here is fair up to the point that you replace your total annual electricity usage with solar power. Produce more than you use, however, and all the excess is just "donated" to the utility without compensation. So you're okay if your solar installation is a bit smaller than you need, but it's economic madness to make it any larger than you need. If not for this rate structure, our solar panel installation could have produced enough electricity for one or two of our neighbors in addition to our own needs.

 

Tuesday Apr 15, 2008

Will it run on multi-chip CMT?

Cooltst v3.0 is out, updated to assess workload suitability for single- and multi-chip CMT. When the UltraSPARC T1 was released, Cool Threads Selection Tool (cooltst) was developed to help gauge how well given workloads might run on the new chip which traded speed for throughput, allowing cooler, lower power, lower cost computing for many applications. But which applications? A single threaded application would tap just a tiny fraction of the 8 cores and 32 hardware threads of the UltraSPARC T1 processor.

Iguazu FallsMuch has changed since then. There is much empirical data showing various applications running well on CMT. The UltraSPARC T2 processor was released, increasing CMT power to 64 hardware threads. This processor also added dedicated floating point units per core so that, far from being relegated to a niche web server market, it claimed (and still holds) a high performance computing record.

Now UltraSPARC T2 Plus systems have been released, further extending CMT power to 2 chips, 8 cores per chip, 8 hardware threads per core - 128 virtual CPU's in a 1RU box. Cooltst helps you assess how well your workload may tap that throughput potential. You can read about it and download it starting at sunsource.net.

There's nothing magical about cooltst's heuristics. You can make much the same assessment yourself using ordinary tools like ps (to look at the software threads) and cpustat (to look at instruction characteristics). All the source code is included so you can see what it's doing. On Linux systems a loadable kernel module is included to measure instruction characteristics in place of Solaris' built-in cpustat command. The output of cooltst is tabular data and a narrative description and  of your workload characteristics, and a bottom line recommendation.


Disclosure Statement:

SPEC and SPEComp are registered trademarks of Standard Performance Evaluation Corporation. Results are current as of 11/11/2007. Complete results may be found at the url referenced above or at http://www.spec.org/omp/results/ompm2001.html


My photo:

Iguazu Falls, on the border of Argentina and Brazil. It's over twice as big as Niagara Falls in terms of water flow, because it covers such a wide area.

Friday Apr 11, 2008

autonomous robots come to San Diego

The International Autonomous Robotics Competition is coming to San Diego in June, at the Del Mar Fairgrounds! Thanks to the San Diego Java Users Group and Wintriss Technical Schools, kids can compete in building and programming robots. The kids will use Sun SPOT which are - of course - the open source robot tool kits. They'll program the robots in Java. Eric Arseneau writes all about the robot competition. I think my boy is a bit young to be writing software, but he's taken me by surprise many times before. Contest or not, he wants me to get him a robot kit, which he thinks I must be able to pick up in the office any day.


 

Wednesday Apr 09, 2008

SPEC does not certify results

Nothing is more fun than arguing with BM Seer. He usually helps me more than anyone in keeping everyone at Sun in compliance with SPEC's fair use rules. But in a recent posting on SPECweb2005 for Sun SPARC Enterprise T5220 he refers to SPEC published results as "Certified." Actually as the official SPEC disclaimer spells out, "the contents of any SPEC reporting page are the submitter's responsibility. SPEC makes no warranties about the accuracy or veracity of this data."

Most SPEC benchmark results can be used without SPEC review. They must comply with all the run and reporting rules, including the requirement for a full disclosure report. And their rules compliance can be challenged on the basis of the details in that report. There is a real value to readers, and hence to vendors, of publishing a result at spec.org. Such results are peer reviewed by other SPEC committee members including competitors, prior to publication. If a result is found to be not in compliance with the run rules it is not published, and the result cannot be used elsewhere either. However, passing this review is not a guarantee or certification that the result is accurate.

Instead of a paid independent audit process, SPEC relies on full disclosure and peer review to increase confidence in the reliability of results. From the details in the full disclosure report anyone should be able to reproduce the performance experiment and obtain substantially the same results. From time to time competitors will conduct such replication experiments on each others' systems, and if they cannot get the same number they bring it to SPEC to either get some details of the test configuration that were erroneously left out of the full disclosure report, or to have the published result marked non-compliant. By this method SPEC dramatically lowers the cost of benchmarking, making it possible to have the thousands of results posted on spec.org, while keeping vendors honest by the fear of exposure and humiliation.


Sunday Apr 06, 2008

♪ Nuevo Flamenco and Reggaeton ♪

Jeff Tamarkin: "Simply labeling Barcelona's Ojos de Brujo a nuevo flamenco group is a little like calling Disney World an amusement park: it's way too inadequate a description." OdB is available at emusic.com. This music recommendation is specifically targeted to Alan Adamson, who enjoyed Bach por Flamenco. Coincidentally, Alan lives in Canada, and my favorite classical guitarist is Daniel Cox from Edmonton. Unfortunately as far as I can tell, Cox isn't recording now. He produced several albums during the early days of mp3.com when its business model was to give all the music free, and hope a few people would, like me, also buy some CD's of the same music in the hope that some of the money would make its way back to the artist and encourage them to keep playing.

Pick two is the Reggaeton Kings, also on emusic.com. Fortunate name for the band. If you've heard some Reggaeton and would like some more, where do you find it? In the old days you'd ask the knowledgeable clerk at the local record store. (Are you old enough to remember what a record store was?) The emusic categories are far too crude to find it. You could search for Don Omar, and along with some music find postings by purists sniffing that Don Omar isn't real Reggaeton. Or you can just search for the word, and the Kings come up. I'm surely missing lots of Reggaeton from bands with less fortunate names, but that's okay because the Kings are as the album cover says, Lo Mejor.

 

Saturday Apr 05, 2008

Forward to the 32-bit past

After wrestling with incompatibilities of 64-bit Linux for a while, I finally downgraded my home PC to 32-bit Ubuntu 7.10 (Gutsy). I found some nice and less nice workarounds, like running the Windows version of Firefox under Wine in order to get Flash to work. I hadn't found a workaround for the Java browser plugin, or for Skype, and was considering a 32-bit chroot environment.

Finally one of those automatic updates decided it for me. You know the ones, the  messages offering later versions of software, critical security updates and recommended updates. Being able to just click OK to automatically be upgraded to the latest software is part of what makes Ubuntu so friendly. But this time it wasn't so friendly. Something left my PC unable to boot to multi-user, unable to start networking, and unable to start graphics. I don't know what because I didn't keep the disk image around for a post-mortem. It was much much faster simply to blow away my root partition with a complete new OS installation. So while I was at it, I dropped down to 32-bit.

Lots of things started working, but some got worse. I had always had a problem on Gutsy that after suspend/resume the Ethernet driver would get a reversed MAC address, complain that it was invalid, and switch to a new eth instance with a random MAC. Of course this played havoc with my router trying to keep track of where my PC was in order to provide DNS. This problem occurs in the forcedeth driver, reverse engineered for the nForce chipset. Some people worked around the problem with limited success by adding commands in the suspend/resume scripts to stop and restart networking.

But now on 32-bit Gutsy it got worse. Upon resuming the screen stayed black, and since the network seemed to be down I couldn't remotely login to find out what was wrong. I found lots of reports on the web about suspend/resume problems with the same error message in my .xsession-errors

Gtk-WARNING \*\*: This process is currently running setuid or setgid.

 

This seems related to my NVIDIA GeForce 6150 LE graphics. Like many others who posted their experiences, the problem occurred for me both with the generic open source driver and with the Nvidia proprietary accelerated driver. One person mentioned a workaround by logging out, logging in to a failsafe X-terminal, and suspending manually from there.

Irony: The main reason I'm running Ubuntu instead of Solaris is that Solaris doesn't yet have power management, and for a home PC, suspend and resume are essential. I've been eagerly watching the power mangement project Tesla at opensolaris.org, wondering why it's taking so long. I guess like most things it's easier to do, than it is to do right. By comparison, my PC's when running Windows 98SE often fail to wake up at all, and those running Windows XP tend to wake up by themselves, unbidden. The only systems where suspend/resume always worked were Linspire and, of course, MacOS.

Neither workaround by itself would work for me, but putting them both together I end up with a clumsy workaround that lets me suspend/resume, and may possibly point the way towards a less cumbersome workaround.

  1. Disable networking via gnome panel
  2. Logout
  3. Select failsafe X-terminal session
  4. Login
  5. sudo /etc/acpi/sleep.sh
  6. (system sleeps)
  7. (normal wakeup by pressing ENTER)
  8. Logout
  9. Select normal gnome session
  10. Login
  11. Enable networking via gnome panel

 



About

I am a software engineer in San Diego, president of the Standard Performance Evaluation Corporation (spec.org), formerly a mathematician and a violist.

Search

Categories
  • General
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today