Wednesday Jan 14, 2009

Perforce runs best on ZFS

Perforce is a commercial source code management software available on many platforms including Solaris Sparc and Solaris x86. As one of our ISV partner experienced last year, the storage subsystem is likely the performance bottleneck of a Perforce Server installation. With entry-level systems featuring a minimum of 4 CPU cores, 4GB memory and dual GigE network nowadays --I am looking e.g. at the base configuration of the Sun Fire X2250 server--, CPU, memory and network are no longer bottlenecks. Unless you run Windows and are subject to the 2GB limit of addressable memory in Windows --Solaris will, by the way, happily let 32-bit applications allocate most of the theoretical 4GB of addressable memory, and you can go beyond that with the 64-bit Perforce Solaris binaries.

So our partner was experiencing poor performance and blocking situations under high load (280 users, 16M files, 15 server instances) with Perforce Server 2006.2 when running off a Sparc V440 server, Solaris 10 and UFS filesystem --with logging enabled. Benchmark results at the 2003 Perforce User Conference had already pointed out the low performance of UFS with Perforce, where synchronous directory updates are a key factor of performance. Linux performs better because it executes directory updates asynchronously --at the risk of data loss, of course. Our partner rather wanted to run off a more reliable Solaris Sparc server --the source code repository and management system is the number one mission-critical application in a software house--, tuned the kernel parameter segmap_percent to 80 but that yields very limited gains in Solaris 10. At that point, we offered to test ZFS, the novel filesystem introduced in Solaris 10.

We benchmarked Perforce Server (P4D) 2007.2 on a Sun Fire X4200 server and …

[Read More]

Friday Oct 31, 2008

Zones + ZFS = Zen

The US Center for Disease Control (CDC) BioSense data warehouse for national bio-surveillance undergoes nightly updates from an SQL server to a (data mart) SAS® SPD Server which is made available to the researchers community each morning. During that time, the SPD Server goes offline. As data volume grows, nightly updates may eventually spill into the next business day, so when in 2007 CDC upgraded their server to a Sun Fire 2900 system, they knew that a simple processor upgrade would not be enough in the long run. The solution of replicating SPD Server domains was however complex (software redesign) and/or costly (full server/storage duplication).

Enter Zones and ZFS. Zones, a.k.a. Solaris Containers, are an operating system abstraction for partitioning Solaris systems --they are kinda lightweight logical instances-- allowing multiple applications to run in isolation from each other on the same physical hardware. ZFS is a new kind of filesystem that provides simple administration, transactional semantics and immense scalability. A ZFS snapshot is a consistent point-in-time image of a filesystem. A clone is a writable copy of a snapshot. Solaris creates ZFS clones quickly using no additional disk space to start with. Both Zones and ZFS were technological innovations originally introduced with Solaris 10.

Leveraging Zones and ZFS, the following solution was designed for CDC. ZFS clones of the Biosense 1TB datasets can be created within minutes in a separate Solaris Container, where one can run a second read-only instance of the SPD Server with no modifications of config files or metadata, since Zones appear as brand new Solaris instances to applications. Zones & ZFS combined are solving CDC's problem very elegantly (built-in Solaris features, 4-hour proof-of-concept!), effectively (maintenance window is confined) and at no incremental cost (no extra server/storage, no SPD Server metadata redesign). In addition, the SAS admins are able to define access rights such that the few SAS programmers can update the live SPD Server in the global zone while the general scientific community accesses the cloned data.

Maureen Chew from Sun's ISV Engineering presented this work at the SAS Global Forum 2007 in Orlando, FL. Check out her paper and presentation for more details.

Friday Oct 24, 2008

Fast, reliable, cheap: pick 2? ZFS brings all!

Today's post is the first instance of an on-going series of technology adoption proofpoints. They are short real-life success stories about our application partners leveraging Sun technologies and innovation to create value for the end-customer. They will be, for the most part, the result of our developer support work at ISV Engineering. Unless there had been some public coverage of that work, proofpoints will be anonymized; only the industry and application segment will be identified --so you can still relate them to your own market and customers. So there we go...

After a server crash at a major European bank, two companies got a phone call; Sun because the logo is on the box, and the ISV whose application --a SWIFTnet messaging platform-- was running on it. The call came very quickly because the crash had led to corrupted data. Our joint investigation showed that one VxFS extend --the bank was using the Veritas File System 4.1-- contained old/bad financial data mixed with correct data. The problem was believed to happen after a new VxFS extend was allocated but before its metadata was written. By default, VxFS does not zero newly-allocated extends. For business environments where data integrity is critical, Veritas recommends the mount -o blkclear option that guarantees that uninitialized data does not appear in a file. That solution induced however an 15% performance loss with the application, unacceptable for the customer, and increased disk footprint of the C-ISAM (embedded database) data files.

The Solaris 10 operating system and ZFS filesystem was tested as an alternative solution and used as is with no additional flag. Due to its guaranteed data integrity, application data was found 100% correct after a stringent series of (20) crash tests. The application performance increased by 10% with ZFS compared to VxFS. In addition, the application's cold recovery process, which is started after a power failure, was taking only 15 seconds on ZFS when all C-ISAM database files have been checked. Finally, it was demonstrated that the ZFS built-in snapshot mechanism could be used to implement a Point-in-Time solution eliminating the need of expensive storage with snapshot capabilities.

As a result of this joint engineering work, the ISV listed ZFS as the recommended filesystem to all its customers. Based on this recommendation, customers have migrated to Solaris 10, from earlier releases of Solaris on Sparc, or are planning to do it. Based on ZFS, which is an integral part of the (free) Solaris and OpenSolaris distributions, the ISV solution is more competitive, guaranteeing data integrity --equals higher availability-- and reducing cost of ownership --no VxFS license, no expensive storage-- while delivering full performance.

About

How open innovation and technology adoption translates to business value, with stories from our developer support work at Oracle's ISV Engineering.

Subscribe

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
5
6
8
9
10
11
12
13
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Feeds