Oracle Open World - Hands-on Lab : Configuring ASM and ACFS on Solaris - Part 1

A quick introduction

I have been invited by Dominic Kay, Product Manager for Solaris Storage sub-systems, for an hands-on lab at OOW. For those of you who will assist at this session, next Monday, at 11:00am, in Marriott Marquis - Salon 5/6, here are the gory details to get you through this lab. For the others that won't have the opportunity to be there, we hope it will be usefull for you to set it up on your own environment. 

The reasoning behind this Lab

I already posted on this blog many times, about ZFS, and all its benefits, including the deployment of Oracle Database. And Dominic found very valuable to develop the knowledge of ASM (and ACFS) deployment on Solaris, as you can look at ASM in a way, as the "ZFS" from a DBA perspective, with another interesting benefit : the ability to deploy an Oracle Database in a shared multi-nodes environment with Oracle RAC, which is what's is running on Solaris on Exadata and on this week's new Engineered System announced, SPARC Supercluster .

[You can look at this lab, as a follow up of the one done by Alain Chereau last year, which was covering ZFS.]

The content and goal

"Oracle Automatic Storage Management gives database administrators a storage management interface that is consistent across all server and storage platforms and is purpose-built for Oracle Database.

Oracle Automatic Storage Management Cluster File System is a general-purpose file system for single-node and cluster configurations. It supports advanced data services such as tagging and encryption.

This hands-on lab shows how to configure Oracle Automatic Storage Management and Oracle Automatic Storage Management Cluster File System for installation of an Oracle Database instance on Oracle Solaris 10 8/11.

You'll learn how to install the software, build Oracle Automatic Storage Management volumes, and configure and mount Oracle Automatic Storage Management Cluster File System file systems."

Preparing for the Lab : downloads !

1 : Download your favorite Virtualbox and install it

[WARNING: you will need to enable Intel VT or AMD-V for your VM to be able to run in 64bits, which is required for this lab, as we need to run Solaris x86 in 64bits]

2 : Get Solaris 10 8/11

3 : Get your latest Oracle 11gR2 release for Solaris x86 64bits (at the current writing of this post this is in the patchet P10098816 that you will get from your Oracle support).

Setup your Solaris 10 8/11 Virtualbox VM

1 :  Create your VM

- name - asmdemo

- type = solaris 10 10/09 and later 64 bit

- memory size = 4096 mb

- create disk type vdi, fixed size of 16 GB (boot/root)

2 : Add additionnal disks to your VM

[Those disks will be used for your Oracle binaries, your ASM and ACFS storage.]

Under "settings", select "storage", then "add hard disk"

- create 1 fixed size disk, type vdi, size 10 GB called asmdemo1 (this will be /u01)

- create 6 fixed size VDI disks of 3 GB (asmdemo2-asmdemo7)

- add the downloaded Solaris .iso in the storage settings as vitual CD/DVD

3 : Setup the network of your VM as NAT under "settings" > "network"

 Tips :

On a Solaris laptop you need to run Virtualbox as root to enable all the networks plumbing, if you forget this stage, just export your VM and Import it back in the Virtualbox of root user.
You also need to disable "nwam" as you don't want him to get in the way. disable nwam :

root@solaris-laptop:~# svcadm disable nwam 

plumb your interface manually :

root@solaris-laptop:~# ifconfig e1000g0 dhcp start

4 : Start the VM & Install Solaris

- use option 1 (solaris interactive default) to create a UFS root or choose option 3 to create a ZFS root

- keyboard is US-English [or French (AZERTY) if required : WARNING this will become the default for your Vbox VM setup and may impact if you plan to export your VM to someone using a different keyboard setup]

- language is English

- choose DHCP networking

- say no to ipv6

- say no to kerberos and name services

- use a system derived NFSv4 domain

- choose the appropriate timezone (example : US Pacific timezone)

- setup the root password

- enable remote services

- choose not to register

- no proxies

- accept the license terms :-)

- choose a Custom (not Default) install

- no software localizations

- no web start ready products

- Solaris Software Group = “Entire Group”

- choose just the first disk (c0t0d0)

- one partition for whole disk (accept defaults)

- In the “Lay Out File Systems” configure at least 8 GB of swap, taking space from /export/home [This is often a requirement from Oracle Database installer to have at least 2x swap space for the memory configured. This can be useful when using Dynamic Intimate Shared Memory, but otherwise not require. In this lab we will run the installation so it pass even if we don't set up 2x swap space, and we will also pay attention during the installation on the SGA and PGA size]

- after install you will have to stop the system rebooting into install again and remove the DVD from the attached devices

- boot into the system

Setup of your Solaris to be able to configure ASM, ACFS and install an Oracle DB on it

1: Configure your disks

Check for your additional disks, partition and label each one . [Note : if you don't see your additional disks, run # devfsadm; disks;]. Run format and fdisk and label for disks 1-7 to put a TOC on them, accepting the defaults. Put all the space except the first cylinder in s0 on each disk, to prevent Oracle ASM from overwriting the partition table.

# format
        > disks
	> (select a disk)
	> fdisk => yes
	> partition (leave first cylinder)
	> label 

If you chose UFS boot setup

- create /u01 as a UFS filesystem using all of c0t2d0 (10 Gb) :  newfs c0t2d0s0, and mount it. Then add an entry to /etc/vfstab for it.

If you chose ZFS boot setup

# zpool create u01_pool /dev/dsk/c0t2d0s0
# zfs create u01_pool/u01
# zfs set mountpoint=/u01 u01_pool/u01

2: Create groups for Oracle account

# groupadd oinstall                                          	 
# groupadd dba                                               	 

3: Create the oracle account, preparing directories and mount points for ACFS

This requires setting /etc/default/login variables as follows before changing (this is for Lab purpose only, to allow easy passwd setting):



# useradd -g oinstall -G dba -d /export/home/oracle -s /usr/bin/ksh oracle
# passwd oracle  (oracle) 
# mkdir -p /export/home/oracle
# chown oracle:dba /export/home/oracle
# chown -R oracle:dba /u01
# mkdir -p /u01/app/oraInventory ; chown oracle:oinstall /u01/app/oraInventory
# mkdir -p /u02/acfsmounts ; chown oracle:dba /u02/acfsmounts

4: Establish necessary kernel parameters

# projadd -U oracle
# projmod -s -K "project.max-sem-ids=(priv,100,deny)"
# projmod -s -K "project.max-sem-nsems=(priv,256,deny)"
# projmod -s -K "project.max-shm-memory=(priv,429967296,deny)"
# projmod -s -K "project.max-shm-ids=(priv,100,deny)"
# projmod -s -K "project.max-file-descriptor=(priv,65536,deny)"

5: Get the Oracle distribution in your VM

You need to copy in your VM (in /u01/staging) the dowloaded Oracle Patchset.

We will do that through ftp. First enable ftp on your laptop, and permit external access (for Lab purpose only) :

root@solaris-laptop:~# svcadm enable ftp
root@solaris-laptop:~# cd /etc/ftpd
root@solaris-laptop:/etc/ftpd# cat > ftpusers 

From within your VM :

# su - oracle
$ cd /u01/staging
$ sftp @IP-your-laptop
> prompt
> bi
> mget *.zip
> bye

You will have under /u01/staging the following zip files :

1 and 2 are database

3 is grid

4 is client

5 is examples

6 is deinstall

- unzip them to create subdirectories client, database, deinstall, examples, grid. That leaves us with 5.6 Gb to play with.

Currently in format output disk 0 is the boot disk, disk 1 is now /u01 leaving disks 2-7 for ASM. This gives us c0t3d0s0 c0t4d0s0 c0t5d0s0 c0t6d0s0 c0t7d0s0 c0t8d0s0 for the ASM exercise

Set the screensaver for oracle user to off.

!! At this stage we have all we need in the way of packages, accessible disks and oracle software so we shutdown the VM and do an export before going further. This will be our starting point for the lab exercice (as we only have one hour) !!

This end the Part 1 of the Lab... and this post. Stay tuned for Part 2 to play with...(coming soon)

Sincerely yours,

Eric Bezille & Dominic Kay


The system requirements are pretty high...

I'm okay on the RAM, but most laptops don't have 8 GB.

I am having trouble coming up with 44 GB disk space, so will try with dynamic disks. I hope this is okay!

Posted by Steve Scharf on octobre 01, 2011 at 11:44 PM CEST #

Hi, some minor notes about this first part.

1. Well, I am sure you know that already, but if you provide the -m flag to useradd, it will create and change the ownership of the home directory for you (instead of doing it yourself).

2. If the operating system hosting VirtualBox is a UNIX system (and it seems to be a Solaris system), why still using plain old FTP? More, the listing says sftp instead of ftp, so...

3. Last, remember that new defaults were sets for lots of kernel tunables with Solaris 10, almost seven years by now. In particular, here are some new defaults for proposed rctl:
- project.max-sem-ids : 128 (against 10 previously), 100 required by Oracle.
- process.max-sem-nsems : 512 (against 25 previously), 256 required by Oracle.
- project.max-shm-ids : 128 (against 100 previously), 100 required by Oracle.
So, when setting these as required by Oracle, you are lowering them instead of growing them which was the expected result in the first place.

Nonetheless, thank you Eric and Dominic for this first entry... I am waiting for the second part!

Posted by guest on octobre 02, 2011 at 12:03 PM CEST #


Thanks for your comments.

Point 1 : you are right, we just wanted to make it clear about the ownership and let it in plane text

Point 2 : yes we are using sftp and not ftp as you saw it (will update the blog accordingly)

Point 3 : we wanted to comply with Oracle required setup and did as such.

Posted by Eric Bezille on octobre 02, 2011 at 12:23 PM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed

Eric Bezille


« avril 2014