A few poorly designed websites will cost you serious money
By was on Feb 25, 2005
So over the past few weeks we launched a new site at Sun. During the evaulation process, engineering suggested they trim the video on the homepage (1MB video, single file). The marketing folks didn't like our suggestion so they left it that way and we launched.
Shame on me for not testing the site a bit more - it took us a while to figure out what was going on, but it turns out that handy little 1MB file doesn't get cached. It's not cacheable by browsers. That's right, if you stay on the home page of this site, you get that file downloaded every 15 seconds or so. Good times.
We spiked our traffic 10 fold for this one site - from 30mb/s to over 130mb/s. The good news is that I get to charge back the group that owns the site. The bad news is it put some stress on my infrastructure that I didn't need. It's fun to watch the traffic graphs for our cage now - it's the first time we've spiked over 300mb/s. Of course, like I said, that costs real money. Serious money too (although if you buy bandwidth in bulk you get a MASSIVE discount :)
Lessons Learned: test pages for caching before release; don't allow video on the home page of a popular site; and don't ever allow creative freedom to marketing - now they want video everywhere on the site...
We were going to wait to upgrade the net until the big move. Now I've got to upgrade several parts of the network right now instead of the more leisurely pace I prefer (ok, leisurely is a misnomer, controlled, tested, etc. is a better way to phrase it :)