Wednesday Jan 20, 2010

Sun rules in PeopleSoft NA Payroll Benchmarks

Sun previously had published record numbers with PeopleSoft NA Payroll  240K benchmark using Sun F5100 Flash Storage Array .  However Giri just announced a follow-on benchmark which beats even that number using twice the number of streams using the Sun Enterprise M4000 server and Sun F5100 Flash Storage Array.

Read more on Giri Mandalika's blog.



Monday Oct 12, 2009

Accelerate your Payroll Performance with F20 PCIe Card

I guess you already heard about Sun Storage F5100 Flash Array and its world record benchmarks.  

But it's not F5100 that I am going to talk about but its smaller sibling called Sun Flash Accelerator F20 PCIe Card.  The name is a mouthful like all Sun product names so I will just call it "The Accelerator Card" in the remainder of this blog entry.  Of course the idea is not to start with the answer and find a problem with it. But I am going to narrate  is how we saw a problem and then thought of using this answer to solve the problem.

Recently our group ISV-E was doing our standard thing of  making applications run best on Sun. In this particular project with PeopleSoft Enterprise 9.0 on M5000 system using Sun Storage 6540, we encountered a problem that certain batch jobs where taking a long time to execute. Peoplesoft Enterprise 9.0 actually have ways to breakup jobs and run them in parallel so as to use the multi-core of the multi-processor system. But yet we could not really leverage the system enough to be satisfactory.  In this project they were using Oracle Database 11g. I got to give it to Oracle, they do have good tools. We used Oracle Enterprise Manager and saw for the troubled batch process, it was showing lot of blue color in its output.

Also looking at the top Objects, the tool reported which tables and index were  troublesome which was causing that amount of blue appear in the chart. This "Blue" problem is what led us to an idea to test out the Accelerator Card in the system and see if can help out here. What we did was created a few tablespaces and spread them out on the four Flash Modules on the Accelerator Card and moved the highly active (or "hot" ) tables and indices to the newly created tablespace. What we saw was simply huge reduction in the blue area and more green. That lead to the slogan in our team

"Go Green with the Accelerator Card !"

The Accelerator card not only reduced the time on this process but many other batch processes which had high IO components.  Here is a relative comparison of how it helped (with additional slight boost from upgrading SPARC64 VII from 2.4Ghz to 2.53Ghz CPUs).

Of course the next question is what if you take the same thing to its bigger sibling, Sun Storage F5100 Flash Array, well that's exactly what we did and as they say the rest is history.(Hint: Read the world records link and search for PeopleSoft)  For more information check out Vince's blog entry on  PeopleSoft Enterprise Payroll 9.0 NA and also  Why Sun Storage F5100 is good for PeopleSoft 9.0 NA Payroll application.

Truly if you use Oracle and use Oracle Enterprise Manager to monitor your application performance and are turning blue by seeing lot of Blue area in the chart then just remember

"Go Green with the Accelerator Card !"

Tuesday Jul 21, 2009

Olio on 6-core Opterons (Istanbul) based Sun Systems

Sun is launching systems with multisocket  6-core Opterons (Istanbul) today. Last week I got access to  Sun Fire X4140 with 2 x 6-core Opterons with 36GB RAM. It is always great to see such a 1RU system packaged with so many x64 cores.

# psrinfo -vp
The physical processor has 6 virtual processors (0-5)
  x86 (chipid 0x0 AuthenticAMD family 16 model 8 step 0 clock 2600 MHz)
    Six-Core AMD Opteron(tm) Processor 8435
The physical processor has 6 virtual processors (6-11)
  x86 (chipid 0x1 AuthenticAMD family 16 model 8 step 0 clock 2600 MHz)
    Six-Core AMD Opteron(tm) Processor 8435

I decided to take the system for a test drive with Olio. Olio is a Web 2.0 toolkit consisting on a web 2.0 event calendar application  which can help stress a system. Depending on your favorite scripting language you can use either PHP, Ruby on Rails, Java as the language used to create the application. (I took the easy way out and selected Olio PHP's prebundled binary kit)

Please don't let the small 2MB kit size fool you thinking it will be a easy workload to test it out. While setting it up I figured that to generate the data population for say 5000 users you will need space with atleast 500GB disk space for the content that it generates for it. Yes I quickly had to figure out how to get a storage array for Olio with about 800GB LUN.

Olio requires a webserver, PHP (of course) and  a database for its metadata store (it has scripts for MySQL already in the kit). The system came preconfigured with Solaris 10 5/09. I downloaded MySQL 5.4.1 beta  and also the Sun WebStack kit which has Apache Httpd 2.2, PHP 5.2 (and also MySQL 5.1 which had not used since I had already downloaded MySQL 5.4 Beta). Memcached 1.2.5 is part of the WebStack download and Olio is configured to use it also by default (but can be disabled too).

Eventually everything was installed and configured in the same X4140 and using the Faban Harness on another system started executing some runs with file store and the meta store preconfigured to handle all the way up to 5000 concurrent users. The results are as follows:


Here are my observation/interpretations:

  • Eventually beyond 10 cores run I find that the system memory (36GB) is not enough to sustain more concurrent users to fully utilize the remaining cores. I would probably need RAM  in the range of 48GB or more to handle more users. (PHP is not completely thread-safe and hence the web server used here spawns processes)
  • This 1RU system can handle more than 3200 users  (with everything on the same system) with CPU cycles to spare is pretty impressive. It means you still have enough CPU to log into the system without seeing degraded performance.
  • Actually you can see here that SMP (or should be called  SMC - Scalable Multi Cores) type system helps when the initial cores are added  instead of using multiple single core systems (ala in Cloud).

 In an upcoming blog entries I will talk more about the individual components used.

Monday Sep 26, 2005

Solaris 10 x64 FCS with Sun Fire V40z and 4 3510s (2+2)

Recently I spent some time configuring a Sun Fire V40z (with single core Opterons) and four Sun StorEdge 3510s. The configuration was "custom made" since I had specifically ordered two dual RAID controller 3510s and two JBOD 3510s. I had also ordered two PCI-X Dual 2Gbit FC Network Adapter (X6768A).
The actual configuration is as follows: The PCI-X Dual 2Gbit FC Network Adapter HBA cards went into slot 6 & 7 of V40z. The 3510s configuration was bit tricky in my case. I wanted two set of 1 Dual RAID Controller with 1 expansion JBOD 3510.Hence I attached two (diagonal) ports from the JBOD into the ports (2,3 - similarly diagonal) of the dual controller 3510. And then ports (0) which essentially is my primary controller port connected to one port of the HBA and the port (5) to the other HBA (I purposely selected both HBA). The other set was similarly configured and attached to both the HBAs simultaneously. (My logic for connecting ports to both the HBAs: Even if the HBA dies I still have access to the both the sets (albeit different controller).
I had already installed Solaris 10 03/05 (FCS) on the V40z. After creating 1 huge RAID-5 LUNs assigned to each controller, I finally rebooted Solaris 10 x64 with the "touch /reconfigure" option. This is where I landed into various bugs in Solaris 10 FCS. Now I have done the hard work of identifying bugs and seeking out corresponding patches so for the sake of simplicity for someone who has similar configuration just apply the following patches. I will try to mention the symptom that it solves also for you. (If you don't have this patches already installed then working with Sun StorEdge 3510FC will be a pain on Solaris x64)
  • 119255-06 SunOS 5.10_x86: Install and Patch Utilities Patch
  • 119375-04 SunOS 5.10_x86: sd patch
  • 118997-02 SunOS 5.10_x86: patch usr/sbin/format
  • 119131-09 SunOS 5.10_x86: Sun Fibre Channel Device Drivers

You can download these patches from SunSolve. (HINT: These patches are all included in Solaris 10 x86 recommended and security updates patch cluster, so it makes sense applying the whole cluster which will apply all patches in the right order)

Just to be more complete I will give you symptoms on the lack of these patches. If you don't install 119255 then other patch installs may fail. So that should be the first patch to install. The next patch 119375 fixes the geometry view of the RAID LUNS. (If you ever saw RAID LUNS reported much smaller than what they really are... then this patch solves that problem). 118997 fixes format specially if you see it dumping "core" when >1TB LUNS are created on 3510s. And finally my favorite 119131-09 apart from all other bugs mentioned in the bugreport it solves bug 6261601 high xcall rates from fcp driver kill v40z performance. The symptom of this bug is IO performance difference of your application when it runs on any cpu other than cpu-p0 where mpstat shows high interrupts in the system.
I also added the following parameters to the /etc/system file.

set maxphys=1048576
set md:md_maxphys=1048576
\* set ufs:ufs_WRITES=0
\* set ufs:freebehind=0
\* set segmapsize=10737418240
set msgsys:msginfo_msgmni = 3584
set semsys:seminfo_semmni = 4096
set shmsys:shminfo_shmmax = 15392386252
set shmsys:shminfo_shmmni = 4096

Now I am ready for real Database load on this Sun System with huge amount of Storage. (The commented parameters are useful when I use PostgreSQL on this system and these parameters optimizes File System Buffer Cache for PostgreSQL)
More on this Sun System setup later on another blog topic. Stay tuned..


Jignesh Shah is Principal Software Engineer in Application Integration Engineering, Oracle Corporation. AIE enables integration of ISV products including Oracle with Unified Storage Systems. You can also follow me on my blog


« July 2016