Oracle Open World - Hands-on Lab : Configuring ASM and ACFS on Solaris - Part 1
By Eric Bezille-Oracle on sept. 30, 2011
A quick introduction
I have been invited by Dominic Kay, Product Manager for Solaris Storage sub-systems, for an hands-on lab at OOW. For those of you who will assist at this session, next Monday, at 11:00am, in Marriott Marquis - Salon 5/6, here are the gory details to get you through this lab. For the others that won't have the opportunity to be there, we hope it will be usefull for you to set it up on your own environment.
The reasoning behind this Lab
I already posted on this blog many times, about ZFS, and all its benefits, including the deployment of Oracle Database. And Dominic found very valuable to develop the knowledge of ASM (and ACFS) deployment on Solaris, as you can look at ASM in a way, as the "ZFS" from a DBA perspective, with another interesting benefit : the ability to deploy an Oracle Database in a shared multi-nodes environment with Oracle RAC, which is what's is running on Solaris on Exadata and on this week's new Engineered System announced, SPARC Supercluster .
[You can look at this lab, as a follow up of the one done by Alain Chereau last year, which was covering ZFS.]
The content and goal
"Oracle Automatic Storage Management gives database administrators a storage management interface that is consistent across all server and storage platforms and is purpose-built for Oracle Database.
Oracle Automatic Storage Management Cluster File System is a general-purpose file system for single-node and cluster configurations. It supports advanced data services such as tagging and encryption.
This hands-on lab shows how to configure Oracle Automatic Storage Management and Oracle Automatic Storage Management Cluster File System for installation of an Oracle Database instance on Oracle Solaris 10 8/11.
You'll learn how to install the software, build Oracle Automatic Storage Management volumes, and configure and mount Oracle Automatic Storage Management Cluster File System file systems."
Preparing for the Lab : downloads !
1 : Download your favorite Virtualbox and install it
[WARNING: you will need to enable Intel VT or AMD-V for your VM to be able to run in 64bits, which is required for this lab, as we need to run Solaris x86 in 64bits]
2 : Get Solaris 10 8/11
3 : Get your latest Oracle 11gR2 release for Solaris x86 64bits (at the current writing of this post this is in the patchet P10098816 that you will get from your Oracle support).
Setup your Solaris 10 8/11 Virtualbox VM
1 : Create your VM
- name - asmdemo
- type = solaris 10 10/09 and later 64 bit
- memory size = 4096 mb
- create disk type vdi, fixed size of 16 GB (boot/root)
2 : Add additionnal disks to your VM
[Those disks will be used for your Oracle binaries, your ASM and ACFS storage.]
Under "settings", select "storage", then "add hard disk"
- create 1 fixed size disk, type vdi, size 10 GB called asmdemo1 (this will be /u01)
- create 6 fixed size VDI disks of 3 GB (asmdemo2-asmdemo7)
- add the downloaded Solaris .iso in the storage settings as vitual CD/DVD
3 : Setup the network of your VM as NAT under "settings" > "network"
| Tips :
On a Solaris laptop you need to run Virtualbox as root to enable all the networks plumbing, if you forget this stage, just export your VM and Import it back in the Virtualbox of root user.
root@solaris-laptop:~# svcadm disable nwam
plumb your interface manually :
root@solaris-laptop:~# ifconfig e1000g0 dhcp start
4 : Start the VM & Install Solaris
- use option 1 (solaris interactive default) to create a UFS
choose option 3 to create a ZFS root
- keyboard is US-English [or French (AZERTY) if required : WARNING this will become the default for your Vbox VM setup and may impact if you plan to export your VM to someone using a different keyboard setup]
- language is English
- choose DHCP networking
- say no to ipv6
- say no to kerberos and name services
- use a system derived NFSv4 domain
- choose the appropriate timezone (example : US Pacific timezone)
- setup the root password
- enable remote services
- choose not to register
- no proxies
- accept the license terms :-)
- choose a Custom (not Default) install
- no software localizations
- no web start ready products
- Solaris Software Group = “Entire Group”
- choose just the first disk (c0t0d0)
- one partition for whole disk (accept defaults)
- In the “Lay Out File Systems” configure at least 8 GB of swap,
taking space from /export/home [This is often a requirement from Oracle Database installer to have at least 2x swap space for the memory configured. This can be useful when using Dynamic Intimate Shared Memory, but otherwise not require. In this lab we will run the installation so it pass even if we don't set up 2x swap space, and we will also pay attention during the installation on the SGA and PGA size]
- after install you will have to stop the system rebooting into install again and remove the DVD from the attached devices
- boot into the system
Setup of your Solaris to be able to configure ASM, ACFS and install an Oracle DB on it
1: Configure your disks
Check for your additional disks, partition and label each one . [Note : if you don't see your additional disks, run # devfsadm; disks;]. Run format and fdisk and label for disks 1-7 to put a TOC on them, accepting the defaults. Put all the space except the first cylinder in s0 on each disk, to prevent Oracle ASM from overwriting the partition table.
> disks > (select a disk) > fdisk => yes > partition (leave first cylinder) > label
If you chose UFS boot setup
- create /u01 as a UFS filesystem using all of c0t2d0 (10 Gb) : newfs c0t2d0s0, and mount it. Then add an entry to /etc/vfstab for it.
If you chose ZFS boot setup
# zpool create u01_pool /dev/dsk/c0t2d0s0 # zfs create u01_pool/u01 # zfs set mountpoint=/u01 u01_pool/u01
2: Create groups for Oracle account
# groupadd oinstall # groupadd dba
3: Create the oracle account, preparing directories and mount points for ACFS
This requires setting /etc/default/login variables as follows before changing (this is for Lab purpose only, to allow easy passwd setting):
# useradd -g oinstall -G dba -d /export/home/oracle -s /usr/bin/ksh oracle # passwd oracle (oracle) # mkdir -p /export/home/oracle # chown oracle:dba /export/home/oracle # chown -R oracle:dba /u01 # mkdir -p /u01/app/oraInventory ; chown oracle:oinstall /u01/app/oraInventory # mkdir -p /u02/acfsmounts ; chown oracle:dba /u02/acfsmounts
4: Establish necessary kernel parameters
# projadd -U oracle user.oracle # projmod -s -K "project.max-sem-ids=(priv,100,deny)" user.oracle # projmod -s -K "project.max-sem-nsems=(priv,256,deny)" user.oracle # projmod -s -K "project.max-shm-memory=(priv,429967296,deny)" user.oracle # projmod -s -K "project.max-shm-ids=(priv,100,deny)" user.oracle # projmod -s -K "project.max-file-descriptor=(priv,65536,deny)" user.oracle
5: Get the Oracle distribution in your VM
You need to copy in your VM (in /u01/staging) the dowloaded Oracle Patchset.
We will do that through ftp. First enable ftp on your laptop, and permit external access (for Lab purpose only) :
root@solaris-laptop:~# svcadm enable ftp root@solaris-laptop:~# cd /etc/ftpd root@solaris-laptop:/etc/ftpd# cat > ftpusers
From within your VM :
# su - oracle $ cd /u01/staging $ sftp @IP-your-laptop > prompt > bi > mget *.zip > bye
You will have under /u01/staging the following zip files :
1 and 2 are database
3 is grid
4 is client
5 is examples
6 is deinstall
- unzip them to create subdirectories client, database, deinstall, examples, grid. That leaves us with 5.6 Gb to play with.
Currently in format output disk 0 is the boot disk, disk 1 is now /u01 leaving disks 2-7 for ASM. This gives us c0t3d0s0 c0t4d0s0 c0t5d0s0 c0t6d0s0 c0t7d0s0 c0t8d0s0 for the ASM exercise
Set the screensaver for oracle user to off.
!! At this stage we have all we need in the way of packages, accessible disks and oracle software so we shutdown the VM and do an export before going further. This will be our starting point for the lab exercice (as we only have one hour) !!
This end the Part 1 of the Lab... and this post. Stay tuned for Part 2 to play with...(coming soon)
Eric Bezille & Dominic Kay