Wednesday Dec 14, 2005

Why Does the Sun Fire T1000 use 180 Watts?

Why Does the Sun Fire T1000 use 180 Watts?
If you looked at computers in datacenters 5, 10, 15 or 20 years ago the thing most cabinets had in common was one 30 Amp 220V power plug, from this we could see that each cabinet could draw 6600 Watts. So for years we had a simple relationship between the floor area and the power/cooling requirements of the datacenter. We could just roll cabinets in and out without having to worry about power or cooling. Around 5 years ago small 2 processor servers started entering the datacenter we could get 12 to 16 of these in a rack which was great but often power and cooling requirement per rack increased up to 20000 Watts, this is just too much for the typical datacenter so we end up putting servers in small piles giving us the lilliput server farm. So once power and cooling are factored in the TOC per server is high.

Designing a 1U server for server farms if we are constrained by the server room door 42RU (Rack Units) and one power socket 6600 Watts, add in power sequencers, smaller racks etc we may have only have 32RUs free giving a power range of 140 Watts to 200 Watts per RU. The T1000 comes in at 180 Watts putting it in the sweet spot of the power vs space trade off.

[ Technorati: ]

Tuesday Dec 06, 2005

Oracle Embraces CoolThreads

Oracle Embraces CoolThreads

I just saw this on yahoo

“customers using Oracle products with CPU-based licenses on Sun Fire T1000 and T2000 systems will be able to count cores as .25 percent of a processor”

So your new Sun Fire T2000 with 8 cores, 4 threads per core giving 32 simultaneous processing threads only requires 2 Oracle CPU licences!

Is A Sun Fire T2000 SMP ?

Is A Sun Fire T2000 SMP (Symetric MultiProcessor) ?

I've seen this question asked a few times. This question is need to be broken down into 2 parts.

How many sockets does the T2000 have? - There is 1 socket for a UltraSPARC T1 processor .

The T1 processors contains 8 cores running 4 threads per core. Giving us 32 simultaneous processing threads on 1 die.

Symetric MultiProcessing describes the operating system feature of scheduling any task to any processing thread (Previous we would used CPU but the term CPU has become confused) it requires some underling hardware features cache coherency as an example to function efficently.

So the T2000 is a SMP computer but all of the SMP functions happen on 1 piece of silicon. This is a wow moment it! When you think of a T2000 as computer having 8 cores and the cache coherency and other SMP processes only takes a few cycles you see why it can outperform a multi-socket Xeon box .

First login to a Sun Fire T2000

So what do you see when you first login to a Sun Fire T2000.

lets do a simple test

which sh
file /usr/bin/sh
/usr/bin/sh: ELF 32-bit MSB executable
SPARC Version 1, dynamically linked, stripped

Do the same thing on any SPARC solaris machine from Sun or others you will get the same result this means that an executable complied today or ten years ago will run! So even though the Sun T2000 is the biggest thing to hit the computer industry in ten years it has over 10 years worth of software out of the box. The next question is how will it scale. When you look here and here you can see that the T2000 scales very well.

So lets have a quick look around.

System Configuration: Sun Microsystems sun4v Sun Fire T200
System clock frequency: 200 MHz
Memory size: 32760 Megabytes

========================= CPUs ===============================================

Location CPU Freq Implementation Mask
------------ ----- -------- ------------------- -----
MB/CMP0/P0 0 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P1 1 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P2 2 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P3 3 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P4 4 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P5 5 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P6 6 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P7 7 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P8 8 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P9 9 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P10 10 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P11 11 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P12 12 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P13 13 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P14 14 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P15 15 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P16 16 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P17 17 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P18 18 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P19 19 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P20 20 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P21 21 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P22 22 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P23 23 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P24 24 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P25 25 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P26 26 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P27 27 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P28 28 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P29 29 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P30 30 1200 MHz SUNW,UltraSPARC-T1
MB/CMP0/P31 31 1200 MHz SUNW,UltraSPARC-T1

========================= IO Configuration =========================

Location Type Slot Path Name Model
----------- ----- ---- --------------------------------------------- ------------------------- ---------
IOBD/NET0 PCIE IOBD /pci@780/pci@0/pci@1/network@0 network-pciex8086,105e
IOBD/NET1 PCIE IOBD /pci@780/pci@0/pci@1/network@0,1 network-pciex8086,105e
IOBD/PCIE0 PCIE 0 /pci@780/pci@0/pci@8/fibre-channel@0fibre-channel-pciex1077,2432
IOBD/PCIE0 PCIE 0 /pci@780/pci@0/pci@8/fibre-channel@0,1fibre-channel-pciex1077,2432
IOBD/PCIE-1 PCIE IOBD /pci@7c0/pci@0/pci@1/pci@0/isa isa
IOBD/PCIE-1 PCIE IOBD /pci@7c0/pci@0/pci@1/pci@0/usb@5 usb-pciclass,0c0310
IOBD/PCIE-1 PCIE IOBD /pci@7c0/pci@0/pci@1/pci@0/usb@6 usb-pciclass,0c0310
IOBD/PCIE-1 PCIE IOBD /pci@7c0/pci@0/pci@1/pci@0/ide ide-pci10b9,5229
IOBD/PCIE-1 PCIE IOBD /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2 LSILogic,sas-pci1000,50 LSI,1064
IOBD/PCIX1 PCIX 1 /pci@7c0/pci@0/pci@2/network@0 network-pciex8086,105e
IOBD/PCIX1 PCIX 1 /pci@7c0/pci@0/pci@2/network@0,1 network-pciex8086,105e

The first thing to note is this Sun Fire T2000 has 32G of memory. The next thing is wow 32 processors and four gigabyte ethernet ports.

This reminds Me of the first time I logged into an Enterprise 10000 (E10K) in 1997, from memory it was half loaded with 32 UltraSPARC II and 32G of memory. The E10K was a bit bigger than a large refrigerator. Now we have a similar system but only requiring 2U of rack space. Back then the first question was how are we going to use all that processing power.

We had a few large applications that needed that much processing power and back then we needed to do some databases, web servers tuning to get them to scale with 32 CPU, Today after 8 years all these applications have defaults that make them run well on Sun Fire E25K where we can have over 140 cores. So to most solaris applications a 32 “CPU” machine is mid sized – no tuning necessary.

Another great thing about the E10K was you could partition them on the fly the unit of granularity was 4 CPUs so you could have up to 16 OS running in one E10K chassis, people loved this feature and that drove Sun to develop a rich resource management framework. In Solaris 10 you can now have psrset, containers and zones and other resource management tools so if you currently have a whole rack of lightly loaded systems they can keep their IP addresses and resource QOS and move onto one T2000.

So as the E10K was years ahead of the competition in the late 90s the T2000 is years ahead of the competition Today.

Thursday Nov 17, 2005

ZFS and Application Managament

When you ask people to rank the difficulty/risk of tasks required to manage applications the list typically goes like this.

Apply a patch to our code – Easy – We do it all the time, the developers make the fix, the QA folks test it, We know how to back it out if things go wrong, We have this process documented, We all understand it.

Apply a patch to the application software - Medium - We get several patches a year for software X we test in in the development environment then QA then production. We can test backing it out if we like on our development or QA environments. We shut down the application back it up apply the patch. We know how to back it out if things go wrong, We have this process documented.

Apply a patch to the OS - Medium – We apply OS upgrades a few time a year We test in in the development environment then QA then production. We can test backing it out if We like on our development or QA environments. We shut down the application back it up apply the patches. We know how to back it out if things go wrong, We have this process documented. Its a little more complicated the application and OS teams have to coordinate.

Change the storage configuration – Hard – We don't like to do this its risky. We have faster storage in our production environment, the software We use to manage the storage array's LUNs is different than the development or QA environments. The volume manager has to be used in a different way on production as well. Oh the backup software is different and We need to coordinate with the SAN team. We have the process documented but it is different from the development and QA process so We can't test it. Shutdown the application, back it up. Get the volume manager team/OS Team/SAN Team/Storage team to work together doing a time critical complicated interdependent task that they can't test and only do infrequently.

So what does ZFS do for you? It takes you untestable time critical complicated interdependent task and makes it testable while reducing the degree of interdependency dramaticly. All the features of ZFS are always available regardless of platform Sparc/x64 or the underlying storage.

Add/remove storage/filesystems/bandwidth on the fly – Yes

Take backups/clones/snapshots on any platform regardless of storage – Yes

Have exactly the same procedures in all environments - YES

Technorati Tag: Solaris OpenSolaris ZFS

Wednesday Sep 14, 2005

Cores, Dies and Parallelism

My definition of a good magazine is one where I read half or more of the articles. Being in the IT industry I receive a dozen or so IT magazines a month. From time to time they all contain something worth reading but mostly they are just fluffy marketing. Only a few are consistently good, Queue is one of the good ones. I just finished reading the September issue. They have a very good collection of articles around multiprocessors, including one from our own Richard. If you subscribe to queue this month's edition is excellent - read it now! If you don't it looks like the website lags by one issue so the September 2005 issue should be online in about 3 weeks - read it then.

Thursday Sep 08, 2005


Technorati Profile

Tuesday Sep 06, 2005

GPL in context

A topic that comes up quite often on news sites is the upcoming GPL v3 license, the GPL v2 has been around for 10+ years. While this should be discussed I haven't seen an article that puts it into context. When we look at the opensource software projects that are having a big impact many of them don't use the GPL - firefox, apache, php for example or dual license like mysql. Why did these projects create there own license in the first place? Most of these projects have updated their licenses in past few years why? Is there a trend? Is there really only five or six different positions on licensing or do we really need the forty or so OSI licenses? Are for profit organizations heading in one direction and not for profit organizations in another? Why do some dual license? How does all of this fit into the IT landscape of developers, ISVs, resellers, consultants and users?

I've read many articles but haven't had any of these questioned answered.

Technorati Tag: GPL

Friday Sep 02, 2005

How long does it take to install and compile OpenSolaris?

I've been planning to install OpenSolaris for a few months and now I've finally done it. I've got a reasonably new laptop for this install, 512M Memory Intel(r) Pentium(r) 4 CPU 2.80GHz with a 30G hard disk and DSL. The most time consuming part is downloading the DVD image of Nevada. This took me about 5 hours. The next step was to do an interactive install, answering the questions correctly & knowing what you want as far as disk partitioning could be made easier. Downloading the compiler & apply patches took about 30 minutes, finally downloading the tools and source then compiling the source, 15 minutes for the downloading and then about 15 minutes reading and following the directions in the release notes then 2 hours to compile. So the answer to the original question is about 8 hours of total time with about 1 hour of interactive time.

Technorati Tag: OpenSolaris

Monday Jul 25, 2005

Filesystem Activity

DTrace and OpenSolaris have made it so easy to find out what the kernel is up. I wanted to know how much activity the filesystem was doing with afew different workloads. I spotted Joh's script and poked around opensolaris source browser to find the filesystem flush counters to understand what the counters mean and put together this script. The whole problem solved in 30 minutes :)

#!/usr/sbin/dtrace -s

#pragma D option quiet



printf("%10s %10s %10s %10s %10s %10s %10s\\n", "SCANNED", "EXAMINED",


lexam = `fsf_total.fsf_examined;

lscan = `fsf_total.fsf_scan;

llock = `fsf_total.fsf_locked;

lmod = `fsf_total.fsf_modified;

lcoal = `fsf_total.fsf_coalesce;

ltime = `fsf_total.fsf_time;

lrel = `fsf_total.fsf_releases;




printf("%10d %10d %10d %10d %10d %10d %10d\\n", `fsf_total.fsf_scan - lscan,

`fsf_total.fsf_examined - lexam, `fsf_total.fsf_locked - llock,

`fsf_total.fsf_modified - lmod, `fsf_total.fsf_coalesce - lcoal,

`fsf_total.fsf_releases - lrel, `fsf_total.fsf_time - ltime);

lexam = `fsf_total.fsf_examined;

lscan = `fsf_total.fsf_scan;

llock = `fsf_total.fsf_locked;

lmod = `fsf_total.fsf_modified;

lcoal = `fsf_total.fsf_coalesce;

lrel = `fsf_total.fsf_releases;

ltime = `fsf_total.fsf_time;


Technorati Tag: OpenSolaris Solaris DTrace

Friday Jul 22, 2005

Open Source

NetHack was my introduction to open source. I came across this afew days ago, like many "movements" if you have to ask you don't understand.

Tuesday Jun 21, 2005

Linux Experts

The passion and robustness of the Linux SIG meeting was the highest I saw at the OAUG. Many experts debated what worked & what didn't a lot of questions revolved around interoperability/storage/costs/stability. With a little surprise it came out the linux was far from unbreakable if you didn't get your server and storage hardware from a vendor that supplies tested firmware/drivers , much like all other Operating Systems! Like Sun, Sadly I didn't see a linux expert running linux on their laptop and only afew of these linux experts had sites that would render correctly on my linux/solaris laptop :(

Tuesday Jun 14, 2005


I'm at the Oracle Applications Users Group Conference. It good to see what other folks are doing with big databases. I've spent most of my time in the Database SIG and at other sessions. The most impressive stuff I've see so far is during a tuning session the guy from oracle logged into the Oracle production ERP system and poked around on the cluster of Solaris E25Ks – wow. Jserv was the next issue that people seem to be having trouble with these days. Understanding the problems you can have in a multi tier environment is hard. Oracle trace files make my life hard the Hotsos profiling method was a simple way out of the mess without looking at the tracefile yourself.

Monday May 16, 2005

First Post

With a bit of luck this blog will become something that others find useful. My main interests are ERP systems, I've spent almost all of my career working with Oracle Applications mostly on Sun hardware.  I would say I've seen more of this "world" that most people, having worked as a Developer, Programmer, System Administrator, done Implementations and Upgrades, worked on benchmarks with Customers and Industry Benchmarks, worked in a Sun/Oracle lab, and with a joint support organization and now in a Solaris Performance and Availability group



« July 2016