Oracle Open World - Hands-on Lab : Configuring ASM and ACFS on Solaris - Part 2

Oracle Open World - Hands-on Lab - Participant Guide

Content and Goal

"Oracle Automatic Storage Management gives database administrators a storage management interface that is consistent across all server and storage platforms and is purpose-built for Oracle Database.

Oracle Automatic Storage Management Cluster File System is a general-purpose file system for single-node and cluster configurations. It supports advanced data services such as tagging and encryption.

This hands-on lab shows how to configure Oracle Automatic Storage Management and Oracle Automatic Storage Management Cluster File System for installation of an Oracle Database instance on Oracle Solaris 10 8/11.

You'll learn how to install the software, build Oracle Automatic Storage Management volumes, and configure and mount Oracle Automatic Storage Management Cluster File System file systems."

Overview

This tutorial covers the installation of Oracle Grid Infrastructure for a standalone server. In the Oracle 11g Release 2, the Grid Infrastructure contains, amongst other software:

  • Automatic Storage Managment (ASM)

  • ASM Dynamic Volume Manager (ADVM)

  • ASM Cluster File System (ACFS)

This lab is divided into 4 exercises.

Exercise 1: We install the ASM binaries and grid infrastructure. As part of the install we create a diskgroup of three disks called DATA. DATA will later be used to store the database data files.

Exercise 2: We use ASM Configuration Assistant (ASMCA) to create a second diskgroup called MYDG. From MYDG we create a ADVM volume called MYVOL and from that we create a ACFS file system called u02.

Exercise 3: We use the installer to install the Oracle database binaries into our new ACFS filesystem (u02).

Exercise 4: We then use the database configuration assistant to create a database with the tablespaces populating the DATA ASM diskgroup.

In our setup we use "External" redundancy for our disks. This implies that the RAID or mirroring is performed in an external RAID array which of course for our laptops is not true. However for practical reasons, we demonstrate using this choice.

Note: In the following text, text displayed in the terminal window is in courier font. Text you are expected to input at the terminal window is in courier bold.

The password for the oracle account is oracle. The password for the root account is provided during the lab

Exercise 1 - Prepare for ASM & Grid installation

-If not logged in, log in as oracle.

Some of the operations we need to perform require administrator access (root).

- Start a terminal for root. Using the existing Terminal window menu "File->Terminal->Default. In the new window type su – root and the root password. You should now see a root prompt " # ".

A point that is key to installing ASM is that the ASM software recognises disk devices as being available to it through the fact that they are owned by oracle (or other ASM install user account). Do not expect to start the GUI and find useable disk devices if the permissions on their device files have not been altered as a pre-requisite step. This tutorial will change the disk device ownership appropriately, below.

All of the devices in an Oracle ASM disk group should be the same size and have the same performance characteristics.

Do not specify multiple partitions on a single physical disk as a disk group device. Oracle ASM expects each disk group device to be on a separate physical disk.

- In the root terminal browse the available disks using format(1)

root@asmdemo:~# format

For this exercise there are 8 disks.

Disk 0 (c0t0d0) is the boot disk.

Disks 1 contains a filesystem mounted under /u01.

The 6 disks c0t3d0...c0t7d0 are for use by ASM/ACFS.

To prevent Oracle ASM from overwriting the partition table, we cannot use slices that start at cylinder 0 (for example, slice 2) so we are using slice 0 which has been created for the exercise to begin at cylinder 1. Examine the disk table as follows:

 (still under format) 
Specify disk (enter its number): 7
selecting c0t7d0
[disk formatted]
FORMAT MENU:
    	disk   	- select a disk
        type   	- select (define)a disk type
        partition  - select(define) a partition table
    	current	- describe the current disk
   	format 	- format and analyze the disk
    	fdisk  	- run the fdisk program
   	repair 	- repair a defective sector
   	label  	- write label to the disk
	analyze	- surface analysis
  	defect 	- defect list management
	backup 	- search for backup labels
  	verify 	- read and display labels
  	save   	- save new disk/partition definitions
  	inquiry	- show vendor, product and revision
   	volname	- set 8-character volume name
   	!<cmd> 	- execute <cmd>, then return
   	quit
format> p
PARTITION MENU:
   	0  	- change `0' partition
   	1  	- change `1' partition
   	2  	- change `2' partition
   	3  	- change `3' partition
   	4  	- change `4' partition
   	5  	- change `5' partition
   	6  	- change `6' partition
   	7  	- change `7' partition
   	select - select a predefined table
   	modify - modify a predefined partition table
    	name   - name the current table
    	print  - display the current table
   	label  - write partition map and label to the disk
   	!<cmd> - execute <cmd>, then return
    	quit
partition> p
Current partition table
(original):
Total disk cylinders available:
1021 + 2 (reserved cylinders)
Part  	Tag	Flag 	Cylinders   	Size        	Blocks
  0    	usr	wm   	1 - 1020	1020.00MB	(1020/0/0) 2088960
  1 unassigned	wu   	0         	0     	(0/0/0)      	0
  2 	backup	wu   	0 - 1020	1021.00MB	(1021/0/0) 2091008
  3 unassigned	wu   	0         	0     	(0/0/0)      	0
  4 unassigned	wu   	0          	0     	(0/0/0)      	0
  5 unassigned	wu   	0          	0     	(0/0/0)      	0
  6 unassigned	wu   	0         	0     	(0/0/0)      	0
  7 unassigned	wu   	0          	0     	(0/0/0)      	0
  8   	boot	wu   	0 -	0   	1.00MB	(1/0/0)   	2048
  9 unassigned	wu   	0          	0     	(0/0/0)      	0
partition>
[type “q” twice or control-d to exit the format utility]

Take the disk devices into oracle ownership (owner : oracle, group: dba) as follows:

# cd /export/home/oracle/scripts
# cat ./change-perms.sh
    #!/bin/ksh
    for i in  c0t3d0s0 c0t4d0s0 c0t5d0s0 c0t6d0s0 c0t7d0s0 c0t8d0s0
    do
    chown oracle:dba /dev/rdsk/${i}
    done

# chmod +x ./change-perms.sh
# ./change-perms.sh

- Confirm that the device owners have changed:

# find /dev -user oracle -print
# find /devices -user oracle -print

Exercice 2 - Install ASM (and Grid)

Switch from the root window to the oracle user terminal window.

1 : Examine and set the environment variables for grid install (note the dot in the final command below):

$ cat scripts/vars-for-grid.sh

# For Grid
ORACLE_OWNER=oracle
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/11.2.0/grid
PATH=/usr/bin:/u01/app/oracle/product/11.2.0/grid/bin
ORACLE_SID=+ASM

export ORACLE_OWNER ORACLE_BASE ORACLE_HOME PATH ORACLE_SID
$ chmod +x scripts/*
$ . scripts/vars-for-grid.sh

2 : Running the installer.

All of the Oracle binary distribution (11.2.0.2) is in /u01. There is more than we need there so will remove some elements of it to create needed space.

For these exercises we only require the database and grid install binaries.

When we run the installer we do so with the -ignorePrereq option because we have not configured as much swap as the installer would like to see for this exercise.

$ cd /u01/staging
$ rm -rf deinstall client examples
$ cd grid
$ ./runInstaller -ignorePrereq

Screen 1 (Download Software Updates). Click on button “Skip software updates”. Click on Next

""

Screen 2 (Select Install Option). Click on button “Configure Oracle Grid Infrastructure for a Standalone server”. Click on Next.

""

Screen 3 (Select Product Languages). Accept default of “English”. Click on Next.

Screen 4 (Create ASM Disk Group). Click on the “All disks” button to show which disks are available. Check the “External Redundancy” option. Leave the Disk Group Name field as DATA. Choose the first 3 of the disks the software has identified (c0t3-c0t5) by clicking on their check boxes. Leave c0t6, c0t7 and c0t8 unchecked. Click on Next

""

Screen 5 (Specify ASM Password). Check the “Use same passwords for these accounts” button and enter the same password as given for the oracle user account. Click on Next and dismiss the warning notice by clicking "Yes".

Screen 6 (Privileged Operating System Groups) Accept the defaults Click Next. Click on Yes to dismiss the warnings dialog box.

"<span id=

Screen 7 (Specify Installation Location). Accept the Oracle Base of /u01/app/oracle and the Software Location of /u01/app/oracle/11.2.0/product/grid. Click Next.

""

Screen 7a (Create Inventory) Accept the default of /u01/app/oraInventory. Click Next.

""

Screen 8 Summary. Click on Install - and read the box below entitled "While you are waiting".

""


While You Are Waiting

There are times during this lab when you just have to wait for things to install.
During these times you are encouraged to move to the root Terminal window
and try some commands to explore the Solaris environment.
Read the manual pages for commands to find out more about them - for command foo

# man foo

Here is a selection to get you going:

Where am I: # uname -a

Devices/RAM: # prtconf | more

Devices: # prtdiag -v | more

Memory usage: # echo "::memstat" | mdb -k

mdb is the kernel debugger,
Network Interfaces # ifconfig -a

Networks (many options) # netstat
Disks # echo | format

Disk tables (quickly) # prtvtoc /dev/rdsk/c8d0s2

Filesystems # df -h

The following are ZFS equivalents, if ZFS is configured on your system.

# zfs list

# zpool status -v

# zpool list

# zpool iostat 5


Screen 8a Execute Configuration Scripts.

""

When this appears, Execute the following scripts in the root terminal window:


# /u01/app/oraInventory/orainstRoot.sh
# /u01/app/oracle/product/11.2.0/grid/root.sh (accept defaults)


and then click OK on Screen 8a to dismiss it.

Screen 9 (Finish). Click Close to dismiss the GUI.

The ASM instance (by default on a single node having the SID ASM+) is running as can be seen from the ps(1) output:

$ ps -fuoracle

UID PID PPID C STIME TTY TIME CMD

oracle 2197 2133 0 18:57:42 pts/1 0:00 ksh -o vi

oracle 4321 1 0 11:42:43 ? 0:01 /u01/app/oracle/112/bin/ocssd.bin

oracle 3883 1 0 11:39:18 ? 0:11 /u01/app/oracle/112/bin/oraagent.bin

oracle 4497 2447 0 12:06:14 pts/2 0:00 -ksh

oracle 4673 4497 0 12:30:59 pts/2 0:00 ps -fuoracle

oracle 3894 1 0 11:39:18 ? 0:01 /u01/app/oracle/112/bin/evmd.bin

oracle 3724 1 0 11:39:05 ? 0:10 /u01/app/oracle/112/bin/ohasd.bin reboot

oracle 4308 1 0 11:42:43 ? 0:03 /u01/app/oracle/112/bin/cssdagent

oracle 4186 1 0 11:42:25 ? 0:01 /u01/app/oracle/112/bin/tnslsnr LISTENER -inherit

oracle 3984 3894 0 11:39:22 ? 0:00 /u01/app/oracle/112/bin/evmlogger.bin -o /u01/app/oracle/112/evm/log/evmlogger.

oracle 4332 1 0 11:42:44 ? 0:01 /u01/app/oracle/112/bin/diskmon.bin -d -f

oracle 4310 1 0 11:42:43 ? 0:03 /u01/app/oracle/112/bin/orarootagent.bin

oracle 4410 1 0 11:43:29 ? 0:00 asm_diag_+ASM

oracle 4408 1 0 11:43:29 ? 0:00 asm_gen0_+ASM

oracle 4402 1 0 11:43:29 ? 0:01 asm_psp0_+ASM

oracle 4414 1 0 11:43:30 ? 0:00 asm_mman_+ASM

oracle 4412 1 0 11:43:29 ? 0:01 asm_dia0_+ASM

oracle 4416 1 0 11:43:30 ? 0:00 asm_dbw0_+ASM

oracle 4418 1 0 11:43:30 ? 0:00 asm_lgwr_+ASM

oracle 4420 1 0 11:43:30 ? 0:00 asm_ckpt_+ASM

oracle 4422 1 0 11:43:30 ? 0:00 asm_smon_+ASM

oracle 4424 1 0 11:43:30 ? 0:01 asm_rbal_+ASM

oracle 4426 1 0 11:43:30 ? 0:01 asm_gmon_+ASM

oracle 4404 1 1 11:43:29 ? 0:06 asm_vktm_+ASM

oracle 4428 1 0 11:43:31 ? 0:01 asm_mmon_+ASM

oracle 4430 1 0 11:43:31 ? 0:01 asm_mmnl_+ASM

oracle 4400 1 0 11:43:29 ? 0:01 asm_pmon_+ASM

oracle 4440 1 0 11:43:38 ? 0:00 oracle+ASM (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

You can use ipcs(1) to view she shared memory and semaphore structures for the ASM SGA:

$ ipcs -s

IPC status from <running system> as of Mon Sep 5 17:45:06 PDT 2011

T ID KEY MODE OWNER GROUP

Semaphores:

s 7 0x496bed6c --ra-ra---- oracle oinstall

$ ipcs -m

IPC status from <running system> as of Mon Sep 5 17:45:10 PDT 2011

T ID KEY MODE OWNER GROUP

Shared Memory:

m 29 0xfa55c7d8 --rw-rw---- oracle oinstall

There does not need to be an oracle database instance for an ASM instance.

Notice two files in the /u01/app/oracle/product/11.2.0/grid/dbs directory provided for ASM:

  • ab_+ASM.dat This is generated when the ASM instance starts and used by the RDBMS instance to determine environment information.

  • hc_+ASM.dat Enterprise Manager uses this file for health-check monitoring.

We connect as follows:

$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.2.0 Production on Wed Aug 31 12:34:46 2011

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Automatic Storage Management option

Try running some obvious queries:

SHOW SGA

SELECT INSTANCE_NAME FROM V$INSTANCE;

SELECT NAME, VALUE FROM V$DIAG_INFO;

SELECT NAME, PATH, GROUP_NUMBER FROM V$ASM_DISK ;

SELECT PATH, HEADER_STATUS, MODE_STATUS FROM V$ASM_DISK ;

QUIT

Exercice 3 - Using ASMCA to make an ASM Diskgroup, ADVM Volume and ACFS Filesystem

Run the ASM Configuration Assistant:

$ asmca

Screen 1: It has three tabs - Disk groups, Volumes and ASM cluster filesystems. In the Disk Groups Tab you already have a disk group called DATA which you created when you installed the grid software. Click the Create button to bring up a disk group creation dialog.

""

Screen 2: Diskgroup Creation Dialog to create the Disk Group we will use for our ACFS filesystem

""

Complete the fields of the form as follows:

Disk Group Name: MYDG
Redundancy: External

Check disks c0t6-c0t8
Click OK and a dialog box should (eventually) indicate succesful completion.

Screen 3: Select the Volumes tab. Click the Create button to bring up a Create Volume dialog box.

""

Fill in the fields as follows:

Volume name: MYVOL

DiskGroup name: Use pulldown to select MYDG
Size: Enter 8.3 and leave G Bytes selected.
Click OK and a dialog box should (eventually) indicate succesful completion.

Screen 4: Select the ASM Cluster Filesystems tab. Click on Create to bring up a Create ASM Cluster Filesystem dialog:

""

Volume: MYVOL (this will already be selected)
Check the Database Home Filesystem button if not already checked.
Database home mount Point: /u02/acfsmounts
Click on the Show Command button and a dialog will appear to tell you that the following commands should be run as privileged user :

/sbin/mount -F acfs /dev/asm/myvol-333 /u02/acfsmounts (number may vary)
chown oracle:oinstall /u02/acfsmounts
chmod 775 /u02/acfsmounts
Do not run these commands.

Dismiss the dialog

Click on the OK button to create the filesystem.

Screen 5: A dialog will appear telling you to run the following script as root:

/u01/app/oracle/cfgtoollogs/asmca/scripts/acfs_script.sh

Run the script in the root Terminal and dismiss the dialog box.

 # /u01/app/oracle/cfgtoollogs/asmca/scripts/acfs_script.sh

""

Click on Exit to leave ASMCA and reply Yes.

Examine the script you ran: /u01/app/oracle/cfgtoollogs/asmca/scripts/acfs_script.sh

It contains the same commands as the Show Commands dialog (from root terminal):


#!/bin/sh
/sbin/mount -F acfs /dev/asm/myvol-168 /u02/acfsmounts
if [ $? = "0" ]; then 
chown oracle:oinstall /u02/acfsmounts
chmod 775 /u02/acfsmounts
exit 0
fi

Run the following comands:

$ df -h 
$ df -n

and observe the output.

So to recap we now have

- an ASM disk group called DATA with 17 Gb of space: this will hold our database

- A volume called MYVOL composed of the diskgroup MYDG

- a file system at /u02/acfsmounts composed of MYVOL: this will hold our oracle binaries

Exercice 4 - Creating The Database

In the oracle user terminal window.

1 : Set the parameters for the Database that you will install

$ cd

$ cat scripts/vars-for-database.sh

# For DB
ORACLE_OWNER=oracle 
ORACLE_HOME=/u02/acfsmounts/product/11.2.0/dbhome_1
ORACLE_SID=orcl
PATH=/usr/bin:$ORACLE_HOME/bin
export ORACLE_OWNER ORACLE_HOME ORACLE_SID PATH
$ . scripts/vars-for-database.sh

2 : Install Oracle Database distribution

$ cd /u01/staging/database
$ ./runInstaller -ignorePrereq

Screen 1 (Configure Security Updates) Uncheck the “I wish to receive...” box and click Next. Click "Yes" on the warning dialog that appears.

Screen 2 (Download Software Updates) Check the “Skip Software Updates” button and click Next.

""

Screen 3 (Select Installation Option) Check the “Install Database Software Only” button and Click Next. [DO NOT CHECK Create and configure a Database, as depending on your VM setup you may run out of memory in the installation process]

""

Screen 4 (Grid Installation Options) Check the Single Instance Database Installation Click Next

""

Screen 5 (Select Product Languages) Accept the default of English and click Next.

Screen 6 (Select Database Edition) Check the Enterprise Edition and click Next

""

Screen 7 (Specify Install Location) Accept Oracle base of /u01/app/oracle. Accept Software Location of /u02/acfsmounts/product/11.2.0/dbhome_1

""

Screen 8 - (Privileged Operating Systems Group) Accept default button and click Next

""

Screen 9 (Summary) - Click on Install

""

Screen 10: Installion in progress - this will take few minutes

""

Screen 11: Execute Configuration Scripts:

""

Run the script as root :

 # /u02/acfsmounts/product/11.2.0/dbhome_1/root.sh 
(hit enter to accept the default path)

and then click Close in the Finish screen to exit the installer.

3: Install Oracle Database

Run dbca as oracle user :

 $ dbca

Screen 1 : (Welcome) – Click Next

Screen 2 : (Operations) - Chose "Create a Database"

""

Screen 3 : (Database Templates) - Chose "General Purpose or Transaction Processing" [For the Lab, we pick up this one, as it is essentially an RMAN restore, and it is the fastest to fit our purpose. In real life, you may chose "Custom Database", and be ready for a 45 minutes to 1 hour installation process]

""

Screen 4 : (Specify Database Identification) Global Database name = orcl, Oracle Service Identifier (SID) = orcl. Click Next

""

Screen 5 : (Management Options) - As this is not the purpose of this Lab and we want to fit in 1 hour, uncheck Configure Enterprise Manager. Click Next.

""

Screen 6 : (Database Credentials) – Check "Use Same Administrative Passwords for all accounts". Enter the oracle password as the passwords for SYS and SYSTEM. Click Next. Click "Yes" to dimiss the warning dialog.

Screen 7 : (Database File Locations) - Select Storage Type: Automatic Storage Management (ASM) and Use Oracle-Managed Files, Browse for a Database Area of DATA. Click Next

""

Screen 8 (Choose the recovery option) - As this is not the purpose of this Lab, uncheck all [For a real production Database it is highly recommended to specify both. The Fast Recovery Area, would have required another ASM Disk Group to be provided to store the files to be used in case of "Fast  Recovery". The archive log are *mandatory* to recover the Database on a system crash.]

""

Screen 9 (Sample Schema) - uncheck

""

Screen 10 (Specify Configuration Options) Check the “Custom" box under the Memory tab , specify "Automatic Shared Memory Management", SGA size = 350 MB, PGA size = 250 MB.

""

Screen 11 (Storage) - Have a look at the Storage Datafiles layout. This should differ from what you are used to in a regular filesystem. This simplify the Oracle installation process thanks to the tight integration with ASM. Click Next.

""

Screen 12 (Select the Database Creation Option) - Check all and click finish

""

Screen 12a (Installation in Progress) - This will take several minutes

""

Screen 12b (Database Creation Complete) - We are done (nearly :), with a running Database on ASM. You can click on Exit.

""

As oracle user, connect to your newly created instance

$ sqlplus / AS SYSDBA
> SELECT * FROM V$INSTANCE; 
(note : run this again after dot-running vars-for-grid.sh if you wish)
And here you are :

""

Wrap up :

In our Solaris 10 8/11, we have now a running Oracle Database instance, installed on an ASM Diskgroup, from which the Oracle Distribution has been installed on an ACFS filesystem.

Sincerely yours,

Eric Bezille & Dominic Kay

Comments:

Hello Eric,
Great job Eric as you mentioned this part of ASM will be important with arrival of Supercluster. The goal now is to determine the best way to use on Solaris ACFS or ZFS except for the RAC Environement.

Florent.

Posted by Florent Thibault on octobre 03, 2011 at 04:38 AM CEST #

It will depend on the skills you have in house and if you plan to scale-out your architecture. In any case, you will always see ZFS in the system boot environment.

Posted by Eric & Dominic on octobre 05, 2011 at 12:39 PM CEST #

Hi,

This was my exam question, still not sure about the answer.
Any help is much appreciatedl

Which three fragments will complete this statement correctly ?
In a cluster environmet, an acfs volume

a-)will be automatically mounted by a node on rebooy by default
b-) must be manually mounted after a node reboot
c-) will be automatically mounted by a node if it is defined as cluster stack startup if it is included in the ACFS mount registry.
d-)will be automatically mounted to all node if it is defined as cluster resource when dependent cluster resource requires access
e-)will be automatically mounted to all node in the cluster when the file system is registered
f-)must be mounted before it can be registered.

Posted by guest on novembre 15, 2011 at 03:20 PM CET #

Sorry in the delay in answering your question. But here it is eventually. You will find the detailed answer here : http://www.oracle.com/technetwork/database/cloud-storage/acfs-technical-overview-514457.pdf in section "CRS Managed ACFS filesystems" on page 14. I invite you to look at the previous pages as well which covers "Creating General Purpose ACFS filesystem" starting page 12, which will help you understand the do and don't when setting up ACFS in one case (General Purpose) or in the other (in a cluster environment = CRS managed).

Posted by Eric Bezille on janvier 12, 2012 at 01:18 PM CET #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Eric Bezille

Search

Archives
« avril 2014
lun.mar.mer.jeu.ven.sam.dim.
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today