Friday Sep 12, 2008

Updating your OpenSolaris to the latest build

One of the nicest features of OpenSolaris is the new package management feature.  Using the pkg command you can quickly update your system to the latest bits available in the repository.  It turns out, however, that with OpenSolaris 2008.05 there is a workaround that you must use in order for this to work properly.  It caught me by surprise recently (not reading those forums thoroughly enough).

Like the rest of the world, I downloaded the OpenSolaris 2008.05 ISO image to my MacBook Pro and installed it into (Sun's free and open source hypervisor) VirtualBox 2.0.  The 2008.05 edition is based upon build 86. To get the complete update to the latest build 97, I simply:

time pfexec pkg image-update -v

About 35 minutes later the system has been updated, a ZFS snapshot of my original system has been made and the grub menu automatically updated to add a new boot image.  All I need to do now is reboot.  This is where the pain started.  After the initial Solaris banner, the system simply reset itself repeatedly.

Luckily, thanks to the snapshot, I can still choose the original boot environment from the GRUB menu.

Thanks to the great community of OS Ambassadors within Sun, I had my solution within hours as posted at this forum.

  • beadm list
  • pfexec beadm mount <my boot env> /mnt
  • pfexec /mnt/boot/solaris/bin/update_grub -R /mnt
 
  

Final step was getting my favorite Gnome theme to help my Solaris box look more like a Mac and place the close widget in the upper left corner where God and Steve Jobs intended it to be.

Finally, if you are a Linux user and unfamiliar with the "pfexec" command, see Glenn Brunette's blog about the benefits of pfexec vs. sudo.


Thursday Aug 14, 2008

Updated: Solaris 10 and OpenSolaris enhanced for Intel XEON

I added some additional YouTube video links to my blog on enhancements for Intel in Solaris 10 and OpenSolaris. 

Monday Aug 11, 2008

Updated: Solaris 10 and OpenSolaris enhanced for Intel XEON

Many of you have heard that Solaris 10 and open-source OpenSolaris runs on both SPARC and X86/X64 architectures.  You probably even know Solaris is available on both AMD and Intel processors in Sun servers as well as non-Sun platforms. In fact, Dell, IBM and Fujitsu/Siemens are Solaris OEMs on their platforms. You may even know that Solaris has set a number of world record benchmarks for scalability and performance on the Intel processor.  But do you really know how we did it? 

Sun and Intel work together on a number of areas in the Solaris OS and development tools including:

  • I/O optimizations
  • Scalability and performance
  • Power Management
  • Compiler optimizations
  • Virtualization enhancements
  • Fault Management

There are a number of resources available where you can learn why Solaris is a great choice on Intel XEON processors.

These are just a few of the projects that make Solaris run better than any other OS on Intel Xeon based processors.  Many more have been completed or are planned in the future including enhancement specifically for the Intel Nehalem microarchitecture

Download Solaris 10 or OpenSolaris today and try it out on your favorite Intel based PC, Server or Virtual Machine.




Wednesday Jul 02, 2008

Using your Mac as a Sun Ray server

Like most System Engineers at Sun, I'm often called upon to demonstrate Sun's technology especially Solaris 10 and Sun Ray thin clients.  In the past, demonstrating Sun Rays meant bringing a customer into our Sun office OR setting up a network server and device at the customer's location. 

To make this much easier, I decided to follow the example of others and turn my Sun issued MacBook Pro into a Sun Ray server.  As a result of this configuration, I can set two devices on my customer's desk with only one ethernet cord and no power cords (have to keep those batteries charged) to display the power of the Sun Ray thin client.  I also have a configuration (thanks to Matt) the provides a multi-level Solaris environment via Solaris 10 Trusted Extensions along with the ability to display an MS Windows desktop using Win2003 running in a separate virtual machine on the same Mac.  Very Cool!

To do this I needed:

Here's how I did it:

  • Install Solaris 10 using VMware Fusion and these settings.
    • 1024 MB of RAM
    • Bridged networking
  • Install the Solaris 10 Entire Distribution
  • Configur the Solaris IP address as 192.168.1.3
  • Download the Sun Ray Server Software (it's free)
  • unpack the downloaded tar image, this creates a directory srss_4.0
  • install the apache tomcat server.  In my case:
    • su
    • cd /opt
    • tar xvf /Documents/srss_4.0/Supplemental/Apache_Tomcat/apache-tomcat-5.5.20.tar
    • mv apache-tomcat-5.5.20 apache-tomcat
  • install Sun Ray Server Software

    • cd ~jlaurent/Dcouments/srss_4.0
    • ./utinstall  (installs the Sun Ray server tools in /opt/SUNWut)
  • patchadd 127554-02
  • reboot
  • PATH=$PATH:/opt/SUNWut/sbin
  • Use utadm to add the 192.168.1.0 subnet as a shared Sun Ray network.  Make sure to choose the option to offer IP addresses.
 # utadm -A 192.168.1.0
### Configuring /etc/nsswitch.conf
### Configuring Service information for Sun Ray
### Disabling Routing
  Selected values for subnetwork "192.168.1.0"
    net mask:           255.255.255.0
    no IP addresses offered
    auth server list:   192.168.1.3
    firmware server:    192.168.1.3
  Accept as is? ([Y]/N): n
  new netmask: [255.255.255.0]
  Do you want to offer IP addresses for this subnet? (Y/[N]): y
  new first Sun Ray address: [192.168.1.245]
  number of Sun Ray addresses to allocate: [10]
  auth server list:     192.168.1.3
To read auth server list from file, enter file name:
Auth server IP address (enter <CR> to end list):
If no server in the auth server list responds,
should an auth server be located by broadcasting on the network? ([Y]/N):
  new firmware server: [192.168.1.3]
  new router: [192.168.1.1]
  Selected values for subnetwork "192.168.1.0"
    net mask:           255.255.255.0
    first unit address: 192.168.1.245
    last unit address:  192.168.1.254
    auth server list:   192.168.1.3
    firmware server:    192.168.1.3
    router:             192.168.1.1
  Accept as is? ([Y]/N): y
### Configuring firmware version for Sun Ray
### Successfully enabled tftp for firmware downloads
        All the units served by "sunray" on the 192.168.1.0
        network interface, running firmware other than version
        "4.0_127553-02_2008.03.06.15.04" will be upgraded at their next power-on.

### Configuring Sun Ray Logging Functions
### Turning on Sun Ray LAN connection

NOTE: utrestart must be run before LAN connections will be allowed

DHCP is not currently running, should I start it? ([Y]/N): y
  • utrestart -c
  • utconfig

Configuration of Sun Ray Core Services Software

This script automates the configuration of the Sun Ray Core Services
software and related software products.  Before proceeding, you should
have read the Sun Ray Core Services 4.0 Installation Guide and filled
out the Configuration Worksheet.  This script will prompt you for the
values you filled out on the Worksheet.  For your convenience, default
values (where applicable) are shown in brackets.

Continue ([y]/n)? y
Enter Sun Ray admin password:
Re-enter Sun Ray admin password:

Configure Sun Ray Web Administration? ([y]/n)?
Enter Apache Tomcat installation directory [/opt/apache-tomcat]:
Enter HTTP port number [1660]:
Enable secure connections? ([y]/n)?
Enter HTTPS port number [1661]:
Enter Tomcat process username [utwww]:
Enable remote server administration? (y/[n])?

Configure Sun Ray Kiosk Mode? (y/[n])? y

Enter user prefix [utku]:

Enter group [utkiosk]:

Enter userID range start [150000]:

Enter number of users [25]:
Configure this server for a failover group? (y/[n])?
About to configure the following software products:

Sun Ray Data Store 3.0
    Hostname: sunray
    Sun Ray root entry: o=utdata
    Sun Ray root name: utdata
    Sun Ray utdata admin password: (not shown)
    SRDS 'rootdn': cn=admin,o=utdata

Sun Ray Web Administration hosted at Apache Tomcat/5.5.20
    Apache Tomcat installation directory: /opt/apache-tomcat
    HTTP port number: 1660
    HTTPS port number: 1661
    Tomcat process username: utwww
    Remote server administration: Disabled

Sun Ray Core Services 4.0
    Failover group: no
    Sun Ray Kiosk Mode: yes

Sun Ray Kiosk Mode 4.0
  User name prefix:   utku
  Base user ID:       150000
  Number of accounts: 25
  Kiosk group name:   utkiosk
  Kiosk group ID:     auto

Continue ([y]/n)? y
Updating Sun Ray Data Store schema ...
Updating Sun Ray Data Store ACL's ...
Creating Sun Ray Data Store ...
Restarting Sun Ray Data Store ...
Starting Sun Ray Data Store daemon .
Wed Jul  2 11:02 : utdsd starting

Loading Sun Ray Data Store ...
Executing '/usr/bin/ldapadd -p 7012 -D cn=admin,o=utdata' ...
adding new entry o=utdata
adding new entry o=v1,o=utdata
adding new entry utname=sunray,o=v1,o=utdata
adding new entry utname=desktops,utname=sunray,o=v1,o=utdata
adding new entry utname=users,utname=sunray,o=v1,o=utdata
adding new entry utname=logicalTokens,utname=sunray,o=v1,o=utdata
adding new entry utname=rawTokens,utname=sunray,o=v1,o=utdata
adding new entry utname=multihead,utname=sunray,o=v1,o=utdata
adding new entry utname=container,utname=sunray,o=v1,o=utdata
adding new entry utname=properties,utname=sunray,o=v1,o=utdata
adding new entry cn=utadmin,utname=sunray,o=v1,o=utdata
adding new entry utname=smartCards,utname=sunray,o=v1,o=utdata
adding new entry utordername=probeorder,utname=smartCards,utname=sunray,o=v1,o=utdata
adding new entry utname=policy,utname=sunray,o=v1,o=utdata
adding new entry utname=resDefs,utname=sunray,o=v1,o=utdata
adding new entry utname=prefs,utname=sunray,o=v1,o=utdata
adding new entry utPrefType=resolution,utname=prefs,utname=sunray,o=v1,o=utdata
adding new entry utPrefClass=advisory,utPrefType=resolution,utname=prefs,utname=sunray,o=v1,o=utdata

Added 18 new LDAP entries.

Creating Sun Ray Core Services Configuration ...
Adding user account for 'utwww' (ut admin web server user) ...done
Sun Ray Web Administration enabled to start at system boot.
Starting Sun Ray Web Administration...
See /var/opt/SUNWut/log/utwebadmin.log for server logging information.

Unique "/etc/opt/SUNWut/gmSignature" has been generated.

Restarting Sun Ray Data Store ...
Stopping Sun Ray Data Store daemon
Sun Ray Data Store daemon stopped
Starting Sun Ray Data Store daemon .
Wed Jul  2 11:02 : utdsd starting
Adding user admin ...
User(s) added successfully!

Creating new Sun Ray Kiosk Mode configuration ...

Validating new user ids.
Validating new user accounts.
Creating kiosk group utkiosk
Configuring new kiosk user accounts:
.........................
25 users configured

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
The current policy has been modified.  You must restart the
authentication manager to activate the changes.
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
Configuration of Sun Ray Core Services has completed.  Please check
the log file, /var/adm/log/utconfig.2008_07_02_11:01:42.log, for errors.

In MacOS

  • Apple Menu > System Preferences > Network
  • Location > Edit Locations
  • Click the '+' Sign to create a new location and name it.
  • Click on Ethernet
  • Configure Manually
  • IP address 192.168.1.1
  • Netmask 255.255.255.0
  • Click Apply
  • Turn your Airport Wireless connection OFF. (This appears to interfere with the networking path to Solaris)

Connect the Sun Ray device directly to the Mac with a single ethernet cord.  No hub required.

If you have done this correctly, when you power on the Sun Ray device it will get an IP address from Solaris and display a login screen.

Access the Sun Ray web based management tool by pointing your browser to http://localhost:1660


Thursday May 15, 2008

Importing Solaris VMDK image into Virtual Box

Virtual Box 1.6 has been released and is no longer in beta for MacOS X.  One of the advertised features is the ability to import VMDK image files from VMware into Virtual Box.  Being the eternal optimist, I decided to try it.  How long could it take?  A few minutes maybe?  I have quite a few different VMs in Fusion and did this with Solaris 10 08/07.

Virtual Box is:

  • a type 2 hypervisor
  • Free
  • Open Source
  • supported on a variety of host OSes (Windows, Linux, Macintosh and OpenSolaris)
  • capable of running a variety of guest OSes
  • now owned and being developed by Sun Microsystems as part of the open source xVM family of virtualization products

The first part was easy. Extract the VMDK file and import it into Virtual Box

  • Right click on your chosen VM.  Choose "Show Package Contents"
  • Find a file with a .vmdk suffix.  Click once to select
  • Command-D (duplicate it) Wait a few minutes while Mac OS copies the multi-GB file
  • Drag the copied file to another location
  • Start Virtual Box
  • File > Virtual Disk Manager
  • Click Add.  Locate and select the copied .vmdk file. Click OK.
  • Create a New VM as usual using the added vmdk file
  • Boot the VM

That should have been it, right?  Unfortunately, after seeing the grub screen and attempting to boot Solaris, I entered an infinite loop of rebooting OS.  Obviously, it's mostly working but something is still wrong.  Luckily, inside of Sun, we archive our mail aliases and Rudolf Kutina had already posted a solution to the problem.

The rebooting sequence resulted from the fact that VMware Fusion emulates SCSI disks (c0t0d0s0) while VBox emulates IDE disks (c0d0s0).  Because of this, the Solaris device trees and vfstab mount entries are not correct.  Rudolf's solution is not for the weak of heart but DOES work.  After all, it's all just a virtual machine and if I screw it up, I just make another copy.  What have you got to lose?

  1. Boot into Solaris Safeboot mode. You can get access at the Grub menu, usually is the 2nd or 3rd option.
  2. Mount the found Solaris partition on /a , Safeboot will usually find the slice on the disk with Solaris and ask if you want it to mount on /a. Select Yes.
  3. Move /a/dev, /a/devices, and /a/etc/path_to_inst to another name (I just append .orig) and then create new directories, (mkdir) /a/dev and /a/devices, and touch file /a/etc/path_to_inst.
  4. Run "devfsadm -r /a" to rebuild the device tree
  5. set TERM so we can use 'vi', TERM=vt100; export TERM
  6. Now we need to fix boot disk patch changes Edit /a/boot/solaris/bootenv.rc and fix the line with "setprop bootpath '/pci@0,0....' to match the path you'll find mounted for /a (i.e. run a 'df -k' command, and you should see /a mounted from /dev/dsk/c1d0s0 or something, then run 'ls -l /dev/dsk/c1d0s0' or whatever your device listed was, and you should see the actual link point to ../../devices/pci@0,0/...ide..)
  7. Fix also disk naming in /a/etc/vfstab to match IDE "c0d0sx" scheme. Change each instance of c1t0d0s0 to c0d0s0 etc.
  8. Recreate archive "bootadm update-archive -v -R /a" to rebuild the boot-archive on /a
  9. Force to reconfigure on next boot with 'touch /a/reconfigure'
  10. Delete /etc/dhcp.e1000g0 /etc/hostname.e1000g0 create /etc/dhcp.pcn0.
  11. Run "cd /; sync; sync; sync; umount /a"
  12. reboot with 'init 6'

Enjoy your new Virtual Box machine.

Instructions are also available for importing a Windows XP .vmdk file to Virtual Box.

Wednesday May 14, 2008

FAQ: Using ZFS for Swap

You may have seen my earlier blog entry on myths and facts about swap space in which I mentioned that ZFS file systems cannot be used for swap files.

# cd /zpool1
# mkfile 10g swapfile
# swap -a /zpool1/swapfile
"/zpool1/swapfile" may contain holes - can't swap on it.

You can, however, use zvols to add swap space onto a ZFS pool:

#
# Add swap partition in the /export/home zfs partition
#
echo "adding zfs swap"
if [ ! -L /dev/zvol/dsk/export/swap ]
then
       echo "creating swap area"
       zfs create -V 1gb export/swap
fi
echo "/dev/zvol/dsk/export/swap -  -  swap  -  no   -" >> /etc/vfstab
/usr/sbin/swap -a /dev/zvol/dsk/export/swap
 

 Thanks to Jim Litchfield for pulling this info from the documentation for zpool

 


Wednesday Apr 23, 2008

What's new in Solaris 10 5/08 video

Solaris 10 5/08 is now available on the Sun Download center.  It's free for commercial use and based on an open source development project. Watch this video by Larry Wake of Solaris Marketing team to learn what's new.

 


Tuesday Apr 01, 2008

Solaris 10 receives DoD IPv6 certification

Solaris 10 has become the first Unix or Linux Operating System to receive IPv6 Certification from the DoD Joint Interoperability and Test Command (JITC).  JITC is the DoD organization responsible for validating products for use in the US DoD.  This most recent certifcation of Solaris for IPv6 standards extends our earlier IPv6 logo certification performed at the University of New Hampshire Interoperabity Lab.

Solaris is the ONLY product currently listed in the "Advanced Server" Category.  Testing was completed on SPARC as well as x86/x64 platforms.

Why should you care?

Sun's continuing commitment to standards in support of the Federal Government means that our customers will be able to move quickly into their transition to the next generation of the internet.

If you'd like to try out Solaris 10 or our next generation of Solaris, known as Solaris Express, they are both available via free downloads and include free right-to-use license.  If you are not sure of the difference between the various Solaris editions, please see my earlier blog entry.


Wednesday Mar 26, 2008

FAQ: Difference between OpenSolaris, Solaris Express etc, etc.

 

I'm often asked the relationship between the various Solaris named products that Sun provides.  Here is my view on them:

OpenSolaris is a SOURCE code project at opensolaris.org from which a number of actual products may be derived including:

  • Portions of Solaris 10
  • Solaris Express and SX Dev. Edition
  • xVM Server
  • Project Indiana

Solaris Nevada is the portion of Open Solaris community code that includes only the kernel (OS and Networking consolidations). Running uname on this build indicates SunOS 5.11.

Solaris Express Community Edition is Sun's binary release for OpenSolaris developers (code named "Nevada"). It is built from the latest OpenSolaris source and additional technology that has not been published in the OpenSolaris source base. This release is unsupported. Developers can build the OpenSolaris source by using this release as the base system. It is updated every other Friday.

Solaris Express Developers Edition, includes Solaris Express Community Edition along with the development tools (Netbeans, Studio etc) in a single installation to simplify life for developers. The Developer Edition is released every three to four months and replaces the Solaris Express monthly release.

Project Indiana is currently in preview edition two.  The OpenSolaris Developer Preview is the first milestone of Project Indiana. It is a single CD combined live/install image: a core operating system, kernel, system libraries, a desktop environment and a package management system. It is not a final release and is intended for developers to try, test, and provide feedback.

Solaris 10
is our enterprise ready, supported version of Solaris.  It is updated less frequently and provdes a stable platform for deployment of long term applications.
 

They are ALL free to download use in a production environment.  If you need support for Solaris 10 you can choose from a variety of Solaris 10 subscriptions on Sun or non-Sun hardware (Sparc, Intel or AMD based).

Monday Mar 10, 2008

Updated: Type Enforcement security project joins the OpenSolaris security community

Update:  Our own architect of Solaris 10 Trusted Extensions corrected me on my statements about MLS capability and Type Enforcement.  I've corrected my table.  Glenn writes in a comment:

It isn't accurate to state that Type Enforcement enables multilevel security. Although you could define relationships between various types that have similar semantics to Bell & Lepadula rules, this is not practical in general. Types, unlike sensitivity labels, don't have implicit hierarchical relationships. Instead the flexibility of the relationships between types is seen as an advantage over the more rigid MLS rules.

One reason this is confusing is that FLASK in SELinux supports both Types and MLS labels, whereas the Solaris implementation of FLASK will just focus on Types since MLS labels are already associated with zones.

 -----

Great News! 

One of the benefits of open sourcing Solaris is the ability to take advantage when "Innovation Happens Elsewhere" (to quote Sun co-founder Bill Joy).  One of the innovative projects that originated elsewhere is an implementation of Type Enforcement (aka "Flask") for OpenSolaris.  Type Enforcement is a form of Mandatory Access Control that has already appeared in the Security Enhanced Linux project first developed at NSA.  SELinux has worked its way from a science project into major Linux distributions today.

What does this mean for Open Solaris?

  • First, it means that we have active development and external contributions to the OpenSolaris community.
  • Secondly, it means that (when completed), customers and governments who prefer the Type Enforcement to Sun's own Solaris 10 Trusted Extensions model, will have that choice without having to give up the other advanced features of Solaris.

Who is doing this work?

When can I get it?

The project has only recently been created at in the OpenSolaris security community.  The source code has yet to be written and posted.   Nothing has been integrated in to the next version (Nevada) of the Solaris kernel yet and there are no plans yet for it to be in Solaris 10.  As the project progresses it may be fully integrated into the Nevada kernel and eventually find its way into a commercial release of Solaris.  Join the community to keep up to date on the latest information.

How will Type Enforcement complement the current Solaris security model?

Read Glenn Faden's most recent blog entry.

Why should I care?

If you have been looking at using SELinux in your project, you should join the community and contribute your comments, feedback, testing and even code to the project creating a better Solaris.


Thursday Mar 06, 2008

FAQ: What Solaris/Unix conferences should I attend?

 

 I received this question from a customer today:

We are looking into Solaris or Unix conferences that are held yearly (such as the MS tech net conference) that you feel are worthwhile for learning or finding out cutting edge news. 

Are there any you would recommend or is there a schedule with prices you could direct me to?

I have a number of suggestions but I'm certain that others out there have their own favorites.  Here's my list. 

Have your own favorite conference?  Jump into the comments section and let us all know about it.

Why should you care? 

To quote Sy Sims, "An educated consumer is our best customer."

Go out and get educate.  Some of the brightest and most enthusiastic Sun Engineers speak and attend many of these conferences.


Saturday Mar 01, 2008

DTrace Toolkit makes DTrace easier to use

As an OS Ambassador at Sun, I have spoken hundreds of times around the country about the Dynamic Tracing facility built in (no extra charge) to Solaris 10 since 2005 and part of the Open Solaris community.  I've described it as a "CAT Scan" into the system when we previously only used X-Ray.  I've said that this allows us to be good doctors (healing the sick) rather than coroners (diagnosing the dead).

Many customers, however, are put off by the programming language or 400 page manual that describes DTrace, however and therefore never really get started.  They don't always realize that we have enhanced PostgreSQL, Ruby, Java, PHP and other higher level languages to make good use of DTrace.  They haven't felt the power of being able to root cause any problem in their system.

While DTrace will never be an "Easy" or "Go Fast" button for your system, there are a number of tools that make it more palatable to the casual user.

Dtrace Toolkit

This collection of pre-written scripts provide some easy tools for collecting the type of data that  system administrators are starving for.

DExplorer

DExplorer automatically runs a collection of DTrace scripts to examine many areas of the system, and places the output in a meaningful directory structure that is tar'd and gzip'd.

Chime Visualization tool

Chime is a graphical tool for visualizing DTrace aggregations. It provides an alternative to similar CLI-based tools (such as intrstat) that is more visually appealing and potentially more useful. In particular, its ability to display data over time adds a missing dimension to system observability. Among its recent new features is the ability to display moving averages.

DTrace NetBeans GUI Plugin

Graphical User Interface (GUI) for running DTrace scripts that can be installed into the Sun Studio 12 IDE, NetBeans IDE 5.5, NetBeans IDE 5.5.1, and NetBeans IDE 6.0.

DTrace BidAdmin community

Includes a collection of tips, tricks, documentation and discussions on DTrace

Why should you care?

Want to be a hero?  Use DTrace to determine why your system isn't working properly.  Save you boss money.  Get more transaction through your systems.  We've done this at a number of customers on live, production systems and you can to.  Download the free DTrace Toolkit today and get started.

PS.  For those who think that System Tap in the Linux community is "just like DTrace," see Adam's rebuttal.

Wednesday Feb 27, 2008

Updated: Playing with ZFS, USB memory disks and VMware Fusion

 

Update 2/28: Made some minor corrections.  Provided an English and high quality version of the German video.  Added a ZFS GUI screenshot and instructions.  Added a link to Constantin's ZFS and Virtual Box blog entry.


This week I am at "Immersion Week" in suburban Chicago.  Immersion Week is an annual training event for Sun Technical staff in the field sales and professional services organizations.  Included in our "goodie bags" was a USB hub and three USB memory sticks along with the suggestion that we use them to demonstrate the open source ZFS file system included with Solaris 10.

Being a Solaris (and Mac) propeller head and fueled by a few Coronas, I found it hard to refuse this challenge. For an advanced version of this, check out this YouTube video (high quality MP4 version) from my colleagues across the pond.  Here are the steps that I followed.

System under test:  MacBook Pro running MacOS 10.5.2, VMware Fusion 1.1.1 and Solaris 10 08/07.

 1. Enable USB device access per the VMware Fusion instructions: <script type="text/javascript" language="JavaScript1.2">WebWorks_WriteArrow(WebWorksRootPath, "wwdd1825234", true);</script>

2
Choose Virtual Machine > Settings or click the Settings button in the toolbar to open the virtual machine Settings sheet.
3
Select + and Add USB controller.
<script type="text/javascript" language="JavaScript1.2">WebWorks_WriteDIVOpen("wwdd1825234", true);</script>
5
Click Apply.

2. Boot the Solaris VM. Login. Open a Solaris terminal window.  Assume root privileges.  Disable the Volume Management service volfs.  This prevents Solaris from automounting the removable disks. This stays in effect across reboots until you "enable" it.

    svcadm disable volfs 

3. Insert the USB hub with 3 sticks into the Mac's USB port

4. Fusion menus: Virtual Machine > USB > Connect ....  for each of the 3 USB devices.  This "grabs" them away from MacOS into Solaris control.

5. Find out the device names for the three USB disks:

# rmformat
Looking for devices...
     1. Logical Node: /dev/rdsk/c0t0d0p0
        Physical Node: /pci@0,0/pci-ide@7,1/ide@1/sd@0,0
        Connected Device: NECVMWar VMware IDE CDR10 1.00
        Device Type: DVD Reader/Writer
     2. Logical Node: /dev/rdsk/c2t0d0p0
        Physical Node: /pci@0,0/pci15ad,790@11/pci15ad,770@2/storage@1/disk@0,0
        Connected Device: CBM      Flash Disk       5.00
        Device Type: Removable
     3. Logical Node: /dev/rdsk/c3t0d0p0
        Physical Node: /pci@0,0/pci15ad,790@11/pci15ad,770@2/storage@2/disk@0,0
        Connected Device: USB      Flash Disk       1100
        Device Type: Removable
     4. Logical Node: /dev/rdsk/c4t0d0p0
        Physical Node: /pci@0,0/pci15ad,790@11/pci15ad,770@2/storage@3/disk@0,0
        Connected Device: CBM      Flash Disk       5.00
        Device Type: Removable

6.  Create a zpool using RAID Z on the three devices.

# zpool create usbdisk raidz c2t0d0p0 c3t0d0p0 c4t0d0p0
invalid vdev specification
use '-f' to override the following errors:
raidz contains devices of different sizes

Wasn't that nice of ZFS to warn us!
# zpool create -f usbdisk raidz c2t0d0p0 c3t0d0p0 c4t0d0p0
# # zpool status

  pool: usbdisk
 state: ONLINE
 scrub: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        usbdisk       ONLINE       0     0     0
          raidz1      ONLINE       0     0     0
            c3t0d0p0  ONLINE       0     0     0
            c2t0d0p0  ONLINE       0     0     0
            c4t0d0p0  ONLINE       0     0     0

errors: No known data errors

# zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
usbdisk                 360M     91K    360M     0%  ONLINE     -


7.  Now lets have some fun......

8. Create a 5 MB file

cd /usbdisk
mkfile 5m test
# ls -l
total 10245
-rw------T   1 root     root     5242880 Feb 27 23:43 test
# du -ak
5122    ./test
5124    .

Notice how du and ls agree on sizes.

9. Enable compresssion

zfs set compression=on usbdisk
# pwd
/usbdisk
# mkfile 5m testcompression
# ls -l
total 10246
-rw------T   1 root     root     5242880 Feb 27 23:43 test
-rw------T   1 root     root     5242880 Feb 27 23:48 testcompression
# du -ak
5122    ./test
0       ./testcompression
5124    .

 Notice that ls shows a 5 MB file but du -ak shows a zero size file because zero filled files compress so well.

10.  Now remove one of the USB memory sticks from the hub and attempt to create file.

# mkfile 5m test2
# zpool status

  pool: usbdisk
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        usbdisk       ONLINE       0     0     0
          raidz1      ONLINE       0     0     0
            c2t0d0p0  ONLINE       0     0     0
            c3t0d0p0  ONLINE       0   156     0
            c4t0d0p0  ONLINE       0     0     0

errors: No known data errors

zpool status reports that although a device is missing, data is intact.

Re-insert the removed memory stick and...

# zpool scrub usbdisk
# zpool status

  pool: usbdisk
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: resilver completed with 0 errors on Thu Feb 28 00:37:03 2008
config:

        NAME          STATE     READ WRITE CKSUM
        usbdisk       ONLINE       0     0     0
          raidz1      ONLINE       0     0     0
            c2t0d0p0  ONLINE       0     0     0
            c3t0d0p0  ONLINE       0   254     0
            c4t0d0p0  ONLINE       0     0     0

errors: No known data errors
# zpool clear usbdisk
# zpool status

  pool: usbdisk
 state: ONLINE
 scrub: resilver completed with 0 errors on Thu Feb 28 00:37:03 2008
config:

        NAME          STATE     READ WRITE CKSUM
        usbdisk       ONLINE       0     0     0
          raidz1      ONLINE       0     0     0
            c2t0d0p0  ONLINE       0     0     0
            c3t0d0p0  ONLINE       0     0     0
            c4t0d0p0  ONLINE       0     0     0

errors: No known data errors

zpool scrub examines all data in the specified pools to verify that it checksums correctly. For  replicated  (mirror  or raidz)  devices,  ZFS  automatically  repairs any damage discovered during the scrub.

11.  Now for some real fun with export and import.

# cd /
# zpool export usbdisk
# zpool list

Note that the pool usbdisk is no longer listed.  Remove all three memory sticks.  Mix them up.  Re-insert them.

# zpool import
  pool: usbdisk
    id: 13155150575270542445
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        usbdisk       ONLINE
          raidz1      ONLINE
            c2t0d0p0  ONLINE
            c4t0d0p0  ONLINE
            c3t0d0p0  ONLINE
# zpool import usbdisk
# zpool status
 
  pool: usbdisk
 state: ONLINE
 scrub: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        usbdisk       ONLINE       0     0     0
          raidz1      ONLINE       0     0     0
            c2t0d0p0  ONLINE       0     0     0
            c4t0d0p0  ONLINE       0     0     0
            c3t0d0p0  ONLINE       0     0     0

errors: No known data errors

Notice how politely, ZFS tells you the name of the pool (even if you forgot it) and asks you to import it by name.  It doesn't matter that the actual "disks" have changed location.

12.  Transfer the disks to another systems (in this case a MacOS system). First note the files that exist and then export the file system. 

 On the Solaris system....

# ls -l
total 20473
-rw------T   1 root     root     5242880 Feb 28 00:32 test
-rw------T   1 root     root     5242880 Feb 28 00:49 testcompression
# du -a
10236   ./test
1       ./testcompression
20477   .
# cd /
# zpool export usbdisk

Shutdown the virtual machine and exit VMware to avoid confusion. Remove the USB hub from the Mac.

Now on Mac OS X 10.5 Re-insert the USB hub. MacOS X Finder produces an error: "Disk inserted was not readable by this computer."

Click "Ignore." Open the MacOS X terminal applications.

$ sudo -s
Password:
bash-3.2# zpool import
  pool: usbdisk
    id: 13155150575270542445
 state: ONLINE
status: The pool is formatted using an older on-disk version.
action: The pool can be imported using its name or numeric identifier, though
    some features will not be available without an explicit 'zpool upgrade'.
config:

    usbdisk     ONLINE
      raidz1    ONLINE
        disk4   ONLINE
        disk3   ONLINE
        disk5   ONLINE
bash-3.2# zpool import usbdisk
bash-3.2# cd /Volumes/usbdisk
bash-3.2# ls
test        testcompression
bash-3.2# du -a
10236    ./test
1    ./testcompression
10241    .

# zfs get all usbdisk
NAME     PROPERTY       VALUE                  SOURCE
usbdisk  type           filesystem             -
usbdisk  creation       Thu Feb 28  0:32 2008  -
usbdisk  used           5.14M                  -
usbdisk  available      200M                   -
usbdisk  referenced     5.03M                  -
usbdisk  compressratio  1.00x                  -
usbdisk  mounted        yes                    -
usbdisk  quota          none                   default
usbdisk  reservation    none                   default
usbdisk  recordsize     128K                   default
usbdisk  mountpoint     /Volumes/usbdisk       default
usbdisk  sharenfs       off                    default
usbdisk  checksum       on                     default
usbdisk  compression    on                     local
usbdisk  atime          on                     default
usbdisk  devices        on                     default
usbdisk  exec           on                     default
usbdisk  setuid         on                     default
usbdisk  readonly       off                    default
usbdisk  zoned          off                    default
usbdisk  snapdir        hidden                 default
usbdisk  aclmode        groupmask              default
usbdisk  aclinherit     secure                 default
usbdisk  canmount       on                     default
usbdisk  shareiscsi     off                    default
usbdisk  xattr          on                     default
usbdisk  copies         1                      default


Like magic, the USB-based ZFS array is now accessible (read-only) to MacOS X 10.5.  A future update is expected to support R/W access. The compression property is still turned on as it was in Solaris.

PS.  I tried mounting the devices in Solaris using Virtual Box by Innotek (recently acquired by Sun).  This software for MacOS X is currently in Beta test.  I received some rather nasty messages about: Failing to create proxy device for USB device.  Virtual Box also runs on Linux, Windows and OpenSolaris hosts.

 See here what Constantin has done with Virtual Box on Open Solaris with ZFS.

Using the ZFS GUI.

I used the command line but ZFS also has a fully capable browser interface.  To use it the webconsole service must be enabled:

 

# svcadm enable webconsole

Point your browser to:  https://localhost:6789.  Login with the root username and password.

ZFS BUI Screenshot








Tuesday Feb 05, 2008

Solaris resurgent in European financial community

For those who think Solaris is dead and "Linux" will take over the world, a recent survey by Forrester Research (NOT paid for by Sun) points out that Solaris is one of the top three "strategic" OS platforms. This shows the value of communities and openness in the software space.  More about this at Jonathan Schwartz blog.

Some interesting quotes include:

Solaris is back on the winner's podium. Sun Solaris has regained its "historical significance" in European financial services.

Linux has lost traction.

Pure J2EE is still strategically very important.
 

Want to get Solaris for free?  Download Solaris 10 today or participate in the OpenSolaris community. 

Sun invites you to read the independent Forrester report titled "European Financial Services Architecture Shows Clear Strategic Direction"(January 2008) in which Forrester reports Solaris as one of the top 3 most strategically positioned operating systems in European Financial Services Firms.

Wednesday Jan 23, 2008

Caustic comments about my blog

The internet is a wonderful egalitarian place where everyone can have their say.  Who am I to complain?  I get to put my information up here on blogs.sun.com and actually asked for corrections and comments regarding my comparison chart between Solaris 10 and RHEL 5.  Naturally, I got some comments and corrections.  Information week picked it up on Jan 2nd (must have been a slow holiday in the old newsroom.)  Today, while googling a totally unrelated topic (I wasn't googling myself, I promise) I ran across an entry titled: So Mr. Laurent, Solaris is all that \*and\* a bag of chips?

Written by "Spencer Shimko, Real Genius" who describes himself as "the source of this dribble." Spencer is currently working with technologies related to security and SELinux for Tresys Technology, LLC. We always like to thank and credit those who comment an help improve our information.

While I fully admitted my Solaris bias in creating the chart, I did try to be as complete and factual as I could.  I hardly think that I fit his description of Sun Guys who are (expletives deleted.)

I do have to take issue with some of his counterpoints however:

Platform support.  Mr. Shimko seems to be implying that I'm playing fast and loose with HW and SW support numbers.  I try to deal in facts and tried only to quote numbers that I could verify. Both Sun and RHAT have issues here because ISVs are so darn "Independent!"  They don't always tell vendors when they port a product to a platform and the information that they provide us changes rapidly and is not always accurate. I had to work with numbers at Sun's and RHATs ISV pages because for me to make up any other number for would truly be lying.  As far as his reference to 3000 RHEL applications, my comparison is only with RHEL 5.  Because they don't guarantee binary compatibility and vendors don't always support the latest OS version, I refuse to extrapolate all available RHEL apps to be available for RHEL 5.

Life cycle support.  While we might argue about what "support" and updates" consist of, I can provide a number of examples of our actual timelines for the last 4 EOL versions. It's true that our Solaris lifecycle page quotes 10 years, but as you can see, support lifespans range from 10-12 years.  This varies based upon customer "acceptance" of OS versions.  Solaris 8 was heavily adopted and Solaris 10 even more so and may end up with a lifespan longer than 12 years.  You can see from this that Sun has a long history of extended life cycle support for our OS.

 OS Version
First shipped
End of phase one support
End of phase two support
Solaris 8
Feb 2000
 March 2009
March 2012
Solaris 7
November 1998
August 2005
August 2008
Solaris 2.6
July 1997
July 2003
July 2006
Solaris 2.5.1
May 1996
Sept 2002
Sept. 2005


Commercial license costs.  Apparently there was no argument here.  Solaris just costs less than RHEL 5 and is available free for download and production use to all of our customers.

Subscription costs.  I was NOT attempting to compare the cheapest Solaris subscription to the cheapest RHEL subscription but the most comparable subscription level.  Solaris is cheaper at the enterprise level.

Unique OS Advanced technologies.  Mr. Shimko would like to remove certain items from the Solaris list such as binary compatibility guarantee, massive scalability, memory placement optimizations etc.  I could find no references to proof of these items in  RHEL 5.  Solaris, however, is proven in all these areas.  Even Linus Torvads admits that he would like to have ZFS in the Linux codebase.

Virtualization.  He calls Solaris zones "stupid, pointless," but I can assure you that a wide variety of enterprise customers including the US DoD find containers useful, easy and cost effective in their data center environments for consolidation of applications.

Application containment.  He predicts the death of Solaris Trusted Extensions and again disrespects containers.  Solaris TX, however, provide capabilities that SElinux cannot, that is a true multi-level Gnome (or CDE) desktop environment that can be displayed on an ulta-thin client.  This technology is currently going through a Protection Level 5 (highest) accreditation at a government customer.  I'll add here that because Solaris is developed using an open source process, the ability to add Type Enforcement is certainly there.  A little bird tells me that there may already be an effort underway to do just that.

Meanwhile, look forward to an updated version of the chart coming to a blog near you.  This time, we will be adding Windows 2003 server to the list since it is one of the OS platforms that Sun can sell and support now.

Thanks for listening and keep those cards and letters coming.


 


Monday Jan 07, 2008

Great Solaris Security recommendations by Glenn Brunette

Glenn Brunette just published an excellent blog listing his 5 favorite Solaris security features.  Among the valuable quotes are:

  • Solaris has had its auditing facility in place since Solaris 2.3, but I can't even begin to count how often I talk with people who do not know that it exists.  (I frequently get this question)
  • Zones are IMHO one of the most significant security features in the Solaris 10 OS. Kernel and most user-land forms of root kits are essentially rendered non-effective when running your applications in a sparse-root non-global zone. (I even recommend to customer when only running one application on a box to run it in a local zone for enhanced security.)
  • For those wanting something a little more advanced, you can use RBAC to implement a two-person (or four-eyes) access control scenario.  (An excellent recommendation for security conscious DoD customers

He also points you to a number of learning resources on Solaris:

Why should you care?

You chose Solaris because of its stellar reputation for security.  Don't be "living in the 90s."  Take the time to learn the new features of Solaris 10 so that you can build and maintain a more robust and secure infrastructure for your organization.

If security is your main area of interest, join the OpenSolaris security community and participate.  Don't forget to get your free download of Solaris 10 or OpenSolaris for Sparc or X64 platforms.

Thursday Jan 03, 2008

Thanks for the quotes at Information Week

I always get a little concerned when I walk into the office and my boss tells me, "Congratulations on being quoted in Information Week."  Although I admit my mind is still fuzzy from a week and a half away from work, I am positive that I never sat for an interview with an InfoWeek reporter.  Nonetheless, there is the article in black and white electrons under the title: Sun Shines In Solaris 10, Linux Comparison.  I guess I can't complain about the title can I?

This serves as a good reminder to us all what risks and potential problems can result from blog entries that are poorly written, researched or misrepresented.  Thankfully, although I admit that my original entry and the chart are not perfect, I haven't yet been accused of outright lies or propaganda.

As a Sun stockholder, I can't complain when the company gets more good publicity and attention driven to our products and services.

Read my original blog entry and see the Solaris vs. RHEL 5 comparison document.  Feel free to comment.

Bill Vass (SunFederal President and COO) also makes reference to this in his blog entry.

Thanks to Information Week for picking this up.

Wednesday Dec 12, 2007

Comparing Solaris 10 with Red Hat EL 5

As an employee of Sun Microsystems Federal, my big boss is Bill Vass.  Bill recently posted a blog entry which references a comparison chart between Solaris 10 and Red Hat EL 5.  As the primary author of the comparison chart I felt that I should come out from behind the veil of my COO.  Admittedly, the list is composed from the point of view of a long time (12 years) Sun employee and Solaris ambassador.  Although I tried to be as complete as possible in collecting the relevant RHEL 5 information, there may be items that I missed.

Feel free to let me know where I made mistakes and provide your input and comments so that the list can continue to be as complete as possible.  It's somewhat like using the "open source" methodology to put many eyes on the code to ensure correctness.  Go ahead!  I can take it!

The general point of the chart should lead you to the conclusion that I've stated before, namely:

  • Solaris costs less than Red Hat
  • Solaris does more than Red Hat
  • Solaris runs on more SPARC and X86/X64 platforms than Red Hat
  • Solaris is developed as an open source project 

Download Solaris today or check out the OpenSolaris source code.  While you're at it, you might want to join the xVM community for open virtualization server and management development.

Why should you care?

There are a wide variety of products on which you can base you computing infrastructure.  Having the most complete and correct information can help you to make decisions based upon facts rather than religious factors. 

Monday Nov 19, 2007

FAQ: Solaris Kernel tunables

On the internal mail aliases within Sun, I see these questions asked frequently about Solaris kernel tunables.

Where can I find out about kernel tunables?

In the documentation, naturally.  Make yourself familiar with the Solaris Tunable Parameters Reference Manual for Solaris 10. 

What should I do with my /etc/system file when I upgrade from Solaris 8 or 9  to 10?

Solaris kernel tunables change in their usage and default values from one revision to the next or even one update to the next.  To help you keep up with these changes we include a change history appendix for the manual. For example, in Solaris 10 we completely removed some parameters (more on that later) and added new ones.  In fact, if you ask most of our kernel engineers about /etc/system, they'll describe it as a "bug that needs to be removed."  Their goal is to make the OS kernel completely adaptive and (where it can't determine the best value automatically) tunable online without requiring the reboot that /etc/system needs to take effect.  Just as a modern car doesn't need manual choke (remember that?), manual spark advance or carb tuning, we would like Solaris to adjust dynamically to changes in memory size, CPU configuration and I/O load.

We suggest that you review all of the tunable parameters in the file to see if they still apply or need to be adjusted.  In many cases, Solaris 10 will perform for you perfectly well if you remove the prior parameters, get a new performance baseline and then (if necessary) make your modifications.

What happened to the shared memory and semaphore settings required by Oracle?

You'll be happy to learn that these are NO LONGER adjusted in /etc/system. In Solaris 10 release, all System V IPC facilities are either automatically configured or can be controlled by resource controls.  Resource controls allow IPC settings to be made on a per-zone, per-project or per-process basis on the local system or in a name service environment. Many applications that previously required system tuning to function might now run without tuning because of increased defaults and the automatic allocation of resources.  This change has several specific benefits including:

  • Reboot is no longer required to change them increasing availability
  • The larger defaults may mean that no operator intervention is required simplifying systems management.
  • They can be tuned differently for different Oracle instances or Containers within a single system increasing flexibility
  • Allows centralized control via a naming service such as LDAP

 How do I know what to change and when?

The actual process of performance management and tuning the OS is beyond the scope of this article.  Keep in mind, however, that a "well behaved" system should show between 20-25% "system time" in vmstat.  Tuning the kernel can reduce this "system time" overhead.  However, even if you are really good at it and get a 10% improvement, that only takes your system down to 18-23%. Your time is probably better spent looking at your application or disk layout.

Jim Mauro and Rich McDougall have written excellent books about Solaris Internals which address some of the actions you can take in excruciating detail.  See their Solaris Internals wiki for more FAQ and to purchase the books.

Why should you care?

The enhancements made to system tunables are designed to make your life easier when using Solaris, reduce your downtime and simply system management and performance analysis.  By removing as many of the "wacky knobs" as possible, we reduce the potential for errors and downtime.

Dell joins other Sun OEMs Fujitsu, Intel and IBM

Dell has joined Fujitsu, IBM and Intel in becoming yet another provider of Solaris on non-Sun hardware.  You have a wide variety of sources to purchase your Solaris based systems today.  Read the press release or listen to the audiocast.

 Would you like to test it out first?  Download your free copy of Solaris 10 or OpenSolaris today.  It is supported on a wide variety of Sparc or X64 based hardware.


 

Monday Nov 12, 2007

Solaris FAQ: Myths and facts about Solaris swap space

As one of the 60 or so OS Ambassadors in Sun world wide, I frequently see the question asked about how to configure Solaris swap.  Apparently, there is quite a bit of mystery about swap space even though it is clearly documented in the Solaris administrator collection.  I decided to publish a collection of my favorite myths and facts about Solaris swap space.  Note that certain applications (such as Oracle) that use "Intimate Shared Memory" will require more swap than most applications. Please refer to the application docs for swap size recommendations.

Myth:  Always set Solaris swap to 2 x RAM size

This myth is clearly a case of users who have been around since the SunOS 4.x days.  Virtual memory today consists of the sum total of physical RAM and swap space on disk.  Solaris DOES NOT require any swap space to be configured at all.  If you choose this option, once RAM is full, you will not be able to start new processes.  There are recommendations for swap space size in the Solaris documentation but the rule of thumb in general is that swap should be configured about 30% of physical RAM. 

Myth: Solaris swap requires raw partitions to be available

Swap can easily be added using standard UFS files in addition to raw disk slice, online without a reboot.  The added swap space takes effect immediately.  The instructions are documented but because I'm a nice guy (and it is so easy) I'll put an example here.

  • mkfile 500m /swapfile
  • swap -a /swapfile
  • Make this added swap area persistent across reboots by adding a new entry in /etc/vfstab

There now, that didn't hurt a bit did it?  The file can be any size you choose and any location in a UFS file system.  You can add as many swap files as you like. ZFS is not currently supported for swap files.  You can use the vmstat or swap commands to show the changes.  Swap space is used in a round robin rotation.

Myth: Swap partitions are also dump partitions

It was back in the Solaris 8 timeframe (late 1999) that the dumpadm command was added to Solaris. To quote the S8 documentation (because I'm lazy): The new dumpadm command, which allows system administrators to configure crash dumps of the operating system. The dumpadm configuration parameters include the dump content, dump device, and the directory in which crash dump files are saved.  See the Solaris 10 dumpadm documentation for more information.

Myth:  You can't control swap space for Solaris 10 containers

 With the latest update of Solaris 10 08/07, we added new resource controls for swap space and containers.  These provide significantly better control and help eliminate denial of service attacks caused by memory leaks and "malloc bombs."

  • zone.max-locked-memory
  • zone.max-msg-ids
  • zone.max-sem-ids
  • zone.max-shm-ids
  • zone.max-shm-memory
  • project.max-locked-memory - Replaces project.max-device-locked-memory
  • zone.max-swap - Provides swap capping for zones through the capped-memory resource

Fact: Swap and tmpfs are the same

This is true.  This design has a number of benefits but we also offer a number of options for controlling tmpfs usage.  I'll refer you directly to the documentation again. 

Fact:  Using swap is bad for performance

Think of swap space as an overflow area for RAM.  It's OK if non-active processes are using swap space, however, if actively used processes are constantly having their pages moved back and forth from RAM to disk based swap areas, performance will suffer.  You can monitor this using the vmstat FREE column.  In Solaris 7 and earlier this number wass relatively meaningless.  Since Solaris 8, however, the FREE column provides an accurate indicator of your free memory.  If the number is too low, page scanning begins (as indicated by the 'sr' column in vmstat).  Any non-zero number in the 'sr' column for an extended period of time is an indicator that it's time to buy more RAM. 

Jim Maura and Rich McDougall have written excellent books about Solaris Internals which described memory utilization in excruciating details.  See their Solaris Internals wiki for more FAQ and to purchase the books. 

 Why should you care?

Solaris continues to be updated and improved with every update based upon feedback from our customers.  If you are not staying up with the latest technology, you're still "living in the 90s" and not getting the most from your compute resource.  We work hard to provide you the facilities in Solaris to increase your availability and utilization of you compute farms.

Tuesday Nov 06, 2007

Testing MacOS X read only ZFS capability

When I first heard Jonathon Schwartz announce that MacOS 10.5 (aka Leopard) would include ZFS, I was psyched!  As a Microsoft free user of Macs and Unix since the late 1980s, I was looking forward to seeing Sun's open source file system in MacOS and was convinced that its snapshot capability would be the basis of Time Machine, Apple's new backup facility.  Imagine my disappointment when news trickled out that the first release of Leopard would only included a basic, read-only implementation of ZFS.  What good is a read only file system?

Leopard shipped two weeks ago and ZFS is almost impossible to find by anyone but developers and OS nuts like me.  It's completely invisible to the typical Mac user.  Then I heard a different piece of news.  Apple shipped 2 MILLION copies of Leopard in the first weekend!  Once ZFS becomes a more prominent part of MacOS, they will be able to touch many more people than Sun ever could in our enterprise ready Solaris OS.  I feel confident that Apple will continue to innovate on top of ZFS.  And in typical Apple style, the end user (like my 82 year old mother who loves her Mac and has no idea that she's running Unix) may never know what ZFS is, but they will appreciate the benefits that they get.   The same will no doubt be true  in their  implementation of Sun's Dtrace technology.

With that in mind, I set about to find a way to prove to myself that ZFS is in there and compatible with ZFS in Solaris 10.  Here's what I did using my MacBook Pro, VMware Fusion 1.1RC1 beta and Solaris 10 08/07.

  • Halt Solaris and shut down the VM
  • VM > Settings > + > Add USB controller
  • Boot Solaris
  • Plug in the USB memory stick. (the VM must have focus)
    • This was actually the most time consuming part of the whole exercise.  It did not mount reliably)
  • If you're lucky, mount shows: /rmdisk/noname on /vol/dev/dsk/c2t0d0/noname:c
  • umount /rmdisk/noname 
  • zpool create usbpool /vol/dsk/noname
  • zpool list
    NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
    usbpool                 120M     88K    120M     0%  ONLINE     -

  • zfs list
    NAME         USED  AVAIL  REFER  MOUNTPOINT
    usbpool       85K  87.9M  24.5K  /usbpool

  • zpool export usbpool
  • Suspend the VM and quit Fusion to avoid confusion
  • Re-insert the USB stick.
  • Finder complains that the disk is not readable.  Click Ignore
  • Open a terminal on the Mac.
  • sudo bash
  • zpool import
      pool: usbpool
        id: 13927799406997242219
     state: ONLINE
    status: The pool is formatted using an older on-disk version.
    action: The pool can be imported using its name or numeric identifier, though
        some features will not be available without an explicit 'zpool upgrade'.
    config:

        usbpool     ONLINE
          disk2     ONLINE
  • zpool import usbpool 
  • Mount shows:
    • usbpool on /Volumes/usbpool (zfs, local, read-only)
  • I was then able to view and copy files from the newly mounted pool
  • Woooo Hoooo! 

Why should you care?

ZFS is a truly easy to use, open source, endian independent, scalable, reliable file system.  This is the first example of it being ported to a commercial, consumer oriented product.

Things to like about ZFS:

Learn more at the ZFS learning center.

Saturday Nov 03, 2007

Using ZFS to expand my virtual Solaris disk in VMware Fusion

Here you will find my chronicles of several hours of failed attempts to add disk space to a Solaris VM disk image.  It turns out that some "newthink" was required.  If you want the correct solution, just skip to the end.

I'm running my Solaris images under VMware Fusion on a MacBook Pro.  The question has come up on how to expand the virtual disk size. 

  • Download the VMware Virtual Disk manager for MacOS X. This is a GUI to command line tools provided with Fusion.  If you really like command lines, you can find it at: /Library/Application\\ Support/VMware\\ Fusion/vmware-vdiskmanager. Figure it out yourself.  I know you're man enough!
  • Duplicate your virtual machine.  Only work on the copy! Select it in the Finder and choose Edit > Duplicate. (Apple-D). The VM must NOT be running or even in use and suspended when you make the copy. Fusion complains about this.
  • Start Fusion
  • File Open... your new VM Copy
  • Fusion notices that the name has changed and asks you if you have copied it. 
  • Suspend the VM
  • You must discard any snapshots before expanding this disk. Virtual Machine > Discard Snapshot.
  • Start the Vdiskmanager GUI
  • Click Expand and locate the vmdk file in your VM.  Select your desired size.
  • Click Go (the GUI echoes the command line it uses at the bottom of the windows for cheaters)
  • The GUI does NOT show the progress of this activity.
  • The Results Tab will open when complete with the status.

Now the real fun begins.  Format, however, shows my disk at its original 10 GB size rather than the new 18 GB size.  This is where fdisk comes into play.

fdisk /dev/rdsk/c8t0d0p0 shows that my disk has one partition that is 56% of the entire disk.  This proves that the operation worked. Now we will attempt to delete the partition and recreate it with a larger size while the OS is running (holding breath). Unfortunately, this attempt failed, if you don't care about learning from my failures, skip to the next section.

  • fdisk /dev/rdsk/c8t0d0p0
  • Select 3 to delete the partition, select partition 1 and confirm
  • Select 1 to create a partition. Specify 100% of the Disk.
  • Select 5 to exit and pray!
  • Run Format and crash (Oh crap!  Glad it was only a copy!)
  • System reboots and Grub has no menu. All attempts to boot the kernel fail.Oops. try again.

Ok, so Solaris doesn't like me removing and recreating it's fdisk partition while it's running.  How about creating a separate partition and mounting it?  Throw away this VM and make another copy of the original.  Repeat the steps to enlarge the disk, then... This attempt also failed, if you don't care about learning from my failures, skip to the next section.

  • reboot is required for fdisk to recognize the new larger size
  • fdisk /dev/rdsk/c8t0d0p0
  • 1 to create new partition, enter size, do NOT make active

Now I'm stuck again.  I can't find a way to get format to recognize the disk in order to build slices.  newfs refuses to write a new file system with no partition table.

In SunSolve, I found this bug 6307998 which has been closed with these comments.

I have verified that fundamentally Solaris has a limitation in that 
it does not allow more than one physical Solaris partition on the same disk.

This lack of functionality goes beyond the installer, it's something lacking in
Solaris in general. Having 2 Solaris partitions on the same disk is not
supported in Solaris because the disk driver assumes there's only one
Solaris partition per disk. For example, if we reference /dev/dsk/c0d0s0, how do
we determine which Solaris partition we're intending to access on c0d0.

 ZFS to the rescue

Who needs that nasty old format and mkfs stuff when you have ZFS! 

  • reboot is required for fdisk to recognize the new larger size
  • fdisk /dev/rdsk/c8t0d0p0
  • 1 to create new partition, enter size, do NOT make active
  • zpool create mypool /dev/dsk/c8t0d0p1
  • zfs create mypool/jim

I've successfully increased by virtual storage!

 Alternative method:  Add a second disk to the image

In order to add second hard disk with Fusion.

  • solaris must be halted.
  • VM must be shut down.
  • Click the + sign, add disk and enter a size.
  • devfsadm  (almost typed reboot -- -r but that would be "old think" so that format sees the new device.)

format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63>
          /pci@0,0/pci1000,30@10/sd@0,0
       1. c1t1d0 <DEFAULT cyl 2557 alt 2 hd 128 sec 32>
          /pci@0,0/pci1000,30@10/sd@1,0

# zpool create mypool /dev/dsk/c1t1d0
# zfs create mypool/jim

# zpool status
  pool: mypool
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        mypool      ONLINE       0     0     0
          c1t1d0    ONLINE       0     0     0

# zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
mypool                 4.97G    116K   4.97G     0%  ONLINE     -
 

Why should you care

I found myself guilty here if something that my customers also do frequently.  That is, deal with Solaris 10 as if it were Solaris 2.2.  The new capabilities of the open sourced ZFS are not only easier to use, they support a wider variety of options for the user.


 

Thursday Oct 18, 2007

Thanks IBM for the good words on Solaris!

Thanks IBM for becoming a good OEM for Solaris and providing a great endorsement of its benefits to the enterprise.  To summarize, their list of competitive advantages:

  • Great product
  • Great price
  • Open
I couldn't have said it better.

Wednesday Oct 10, 2007

BUSTED! 11 Myths about Solaris on X86/X64 platforms

Most Sun employees and Solaris fans know that Solaris has run on X86 platforms since 1994  However, in my visits to customer sites as an OS ambassadors I hear these questions frequently.  Today, I'd like to dispel some of the most common myths about Solaris.

Myth: Sun is not serious about the X86 market.

At this time, Sun is the 3rd largest server vendor in the world and #5 in the x86 server market. We have a variety of hardware platforms using the Intel and AMD chips from under $1000 to complete blade server systems.  We have two OEMs for Solaris signed up (IBM and Intel) with more expected to come in the near future. Intel recommends Solaris as the enterprise OS for their Xeon processor family. We have also agreed to become an OEM for MS Windows server software. We can sell, train, support and take your trade-ins on our complete line of SPARC and X86 systems.  We are QUITE serious.

Myth: Solaris on SPARC and X86 platforms are different OSes.

There is only one Solaris source code base.  You can see and contribute to it at the OpenSolaris web site. 95% of the code is common.  Examples of code that is NOT common includes chip specific features such as memory management, cache, hardware features, boot proms and virtualization technologies.  Features such as Solaris containers, SMF, ZFS, Trusted Extensions, resource management and more work the same on Sparc, X86 or virtualized platforms such as VMware.

Myth: Sun's support organizations are different for SPARC and X86 platforms

The same engineering and customer support team is used to design, develop, test and support Solaris.  You can call 800-USA-4-SUN and get support for Solaris whether it is on Sun systems or any of the over 900 systems on our hardware compatibility list.  Many of our engineers actually do their development work on PC hardware. 

Myth: Solaris for X86 platforms is not on the same schedule as SPARC platforms

Solaris updates and patches are released at the same time for each platform.  The only exception to this is when a patch ONLY applies to a specific platform, such as an Intel memory management fix that does not apply to AMD or SPARC chips.

Myth: You can only get Solaris from Sun

IBM recently announced that they will be selling Solaris for their blade and rack mounted servers.  We anticipate more companies to announce OEM agreements in the future.  Intel has also announced that Solaris is its preferred OS for enterprise deployment on Xeon platforms.

Myth: Solaris only runs on a few X86 platforms.

The hardware compatibility list has hundreds of platforms from Sun, HP, Dell, IBM and others.  It includes the latest Quad-core Intel and AMD chips, blade servers and more. Of our 10 million Solaris registrations, 63% of them were on non-Sun platforms.

Myth: Solaris doesn't work on VMware

Solaris 10 is a supported platform in the VMware support matrix. VMware is also listed at the Sun hardware compatibility list. Pre-built Solaris vmware images at the Sun Download Center

Myth:  It's too hard to move code from Solaris 8 to 10 or from SPARC to X86 platforms.

Solaris is guaranteed to be binary compatible moving forward from Solaris 2.5.1 on each platform.  This means that a binary running on an Ultra 2/2.5.1 can be transferred to Solaris 10 on Sun's latest T2000 and is GUARANTEED to run.  In addition, Solaris is source code compatibile between the two instruction sets.  If you need assistance on the best compiler practices for building 64-bit applications or using the proper performance options see the huge collection of white papers at our Solaris developer's portal.

Myth:  Only Solaris Nevada, OpenSolaris or Solaris Express run on X86 hardware.

Our production ready distribution of Solaris (known as Solaris 10) was first released in March 2005 with complete support for SPARC and X64/X86 platforms just as it has since 1994.  Available for Solaris 10 is enterprise level support and a long life cycle.  Our development for the next version of Solaris (known as Nevada) is currently ongoing as an open source project at www.opensolaris.org.  Periodically we produce binary versions known as Solaris Express community edition or developer edition for users to try out new features. Download Solaris 10 now for free for both SPARC and X86 platforms.

Myth: Solaris is hard to install

If you've heard this before, please check out our latest Solaris Express Developer's Edition.  It has an updated installer, improved wireless networking support and simpified networking setup. Our Flash archive, Live upgrade and jumpstart technologies simplify data center practices for patching and upgrades.

Myth:  You have to dedicate an entire PC to test out Solaris

Solaris works quite well in multi-boot or virtualized environments.  The vast majority of Sun system engineers run Solaris either on a Windows or Linux-based PC or in a virtual machine such as VMware Fusion or Parallels on MacOS X. Solaris include the GRUB boot loader to allow you to choose between multiple OS images to boot. We make virtulization easy with pre-built Solaris vmware images at the Sun Download Center

Bonus Myth (thanks to Bob for Suggesting)

Myth: There are no open source or ISV applications for Solaris on X86 platforms.

You can find a prepackaged and easy to install repository of the most common open source applications from blastwave.org.  Solaris on X86 has more ISV packages than Red Hat and other competitors.  Many freeware packages like PostGreSQL, gcc, gmake, perl, apache, webmin and more are built into Solaris or included on the companion CD.

Monday Sep 17, 2007

Why use Solaris? It's for the Application support!

Once again Solaris 10 (on either Sparc or X64 platforms) features a larger collection of available third party ISV applications than the competition.  The numbers speak for themselves.  Want freeware?  Go to blastwave.org or sunfreeware.com

 Chart of ISV Applications

Why should you care?

 When choosing an enterprise operating system platform, you need to be sure that you have the widest choice of workloads available to you.  Add that to the fact the Solaris 10 is developed as an open source project, runs on commodity Sparc and X64 hardware and is free to download and the choice is simple.

 

Tuesday Aug 28, 2007

Linux Shop Embraces Solaris and Chip multi-threading on T2000

See this excellent article at ServerWatch that describes how web startup Real Time Matrix found that Solaris 10 met their needs better than a Linux based OS.

ServerWatch quoted the CEO: "We need a stable, robust infrastructure to process millions of items a day, match against millions of preferences and run 24/7," said Jeff Whitehead, CEO of The Real Time Matrix Corp of Oakland, Calif. "For high-speed, high-performance, 100 percent raw computing, we are finding it is cheaper and better on Sun and Solaris." 

"For us, Solaris was a no-brainer," Whitehead said. "We immediately went from a couple of hundred to 10,000 matches per second and up to 32 concurrent processes."

He couldn't argue with the results obtained on the T2000 running Solaris. It replaced six x86 boxes and is cheaper to run.

"Our administrative costs went down with the one big machine as we can segment it," Whitehead said. "But with our business model requiring high-speed, high-performance, 100 percent uptime and maximum raw computing power, we are finding it is cheaper and better to be on Sun and Solaris. I'd estimate that we spend 50 percent less than if we had we gone with our original power, hardware and leasing arrangement."

Why you should care

If you want an open source OS that runs on Intel, AMD and Sparc commodity processors, Solaris 10 is free to download and put into production.  Check out our Startup Essentials program to help you get started.


 

 

Thursday Aug 16, 2007

IBM Chooses Solaris!

IBM has agreed to be an OEM for Solaris subscriptions on it's X-series and Blade Center servers.  Read the complete press release and audiocast

Excerpts:

Bill Zeitler, senior vice president & group executive, IBM Systems & Technology Group says, "IBM is the first major x86 vendor to have such an agreement with Sun; and the first big vendor apart from Sun to offer Solaris on blade servers. 

"We're thrilled to be working with IBM to bring the Solaris OS to the broadest market possible - they are a natural partner for Sun," said Jonathan Schwartz, president and CEO of Sun Microsystems. "Solaris adoption continues to accelerate, among both the open source and commercial communities -- driven by bundled virtualization for servers and storage, support for thousands of ISVs, including nearly the entirety of IBM's software portfolio, and outstanding operational economics. Solaris is clearly a choice customers are demanding." 

 Why should I care?

 I frequently get told that a customer's reason for choosing Red Hat over Solaris is that RHEL can be purchased from multiple sources.  Customers now have the choice to purchase Solaris subscription support from Sun or from IBM (can HP and Dell be far behind?)


 

Wednesday Aug 01, 2007

Update: Anti virus software for Solaris? What are they thinking?

You may have read my earlier blog discussing the requirement for Anti-virus scanning software on Solaris in the US DoD.   The Field Security Office of DISA maintains Security Checklists to help their system admins secure a system before attaching to a DoD network. 

The good news is that this requirement was lowered from a Category I finding (highest) to a Category II finding (out of a possible IV).  Although Sun would prefer an even lower rating, we are happy to see this change.

The bad news is that the checklist requires, "An approved DOD virus scan program" with no definition of which products are approved.  Many of the most popular products do not yet support Solaris on X64 platforms.  I've updated my earlier blog with products that I've been able to identify so far.  Some of these products support Solaris 10 on both the Sparc and X86 platforms.

At this point I'm trying to find out what the "approved DoD products" are from DISA FSO. 

Tuesday Jul 31, 2007

How did Solaris 10 and Dtrace help Twitter improve performance?

For a great presentation on how Solaris 10 and DTrace helped Twitter improve their performance by 15% see Adam Levanthal's  most recent blog entry along with his presentation provided at OSCON.

 In this case, the application was written in Ruby and the DTrace provider in Ruby really helped.

Why should you care?

 If you are building a fast growing company (as Twitter is) it's inevitable that you will run into performance issues.  Solaris 10 and Dtrace provide the kind of visibility into your application's performance that can help you grow smoothly.
 

About

Jim Laurent is an Oracle Sales consultant based in Reston, Virginia. He supports US DoD customers as part of the North American Public Sector hardware organization. With over 17 years experience at Sun and Oracle, he specializes in Solaris and server technologies. Prior to Oracle, Jim worked 11 years for Gould Computer Systems (later known as Encore).

Search

Categories
Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today