Wednesday Jan 13, 2010

FC Storage Arrays and Common Array Manager(CAM) 101


In this entry I'll talk about some of the terms that we'll here while working with Storage Arrays, and on the roles that they play in configuring a Storage Array.

Storage Profile: The first thing that we need to set while configuring an array is the Storage Profile.

In the profile we tell the kind of RAID level , the Segment size, if Read ahead should be enabled, etc. that we would like to use.


Some of the questions that we would ask before configuring the Storage profile are:

          a) What kind of RAID level we would like to use, do we need mirroring or should we go with

    striping, just striping + mirroring, etc.

    b) It would help to know the size of the I/O that the application is making, this will help with selecting the appropriate segment size.

    c) Knowing if the I/O is random or sequential, or is it a mixed bag. This will help us decide whether to enable read ahead or not.



Storage Pool: We can now take all or part of the disks in the array and attach a profile to it. This would mean that all the disks in this pool would have the RAID level, the segment size, read ahead property, etc of the storage profile that was attached to it.


Virtual Disks: We can think of virtual disks as groups of physical disks on which the volumes are created.


Volumes: From the Storage Pool that we created we could create one or more volumes. The volume that we create will be the ones that the servers see as disks. We can then mount filesystems on them, or use them as raw devices from the server side.


Mappings: The volumes that we create are now mapped onto controllers. A volume can only be mapped/attached to one controller. It's through the ports of the controller that the volume talks to the server.


Controller Ports: Controller usually have multiple ports, where each port could have a speed of say 2Gbps or 4 Gbps or something. If our controller has 4 x 4 Gbps ports, and you have connected Links from the 4 ports to the server, then you'll have 4 different paths leading to the same LUN.We  could use MpxIO to fold the 4 paths into a single path on the server side.

Note that if we are using only 1 x 4 Gbps port of your controller, and converting Gbps to MB/sec,  that you can get a throughput of around 500 MB/sec.


Here are the steps to create a Volume using Sun StorageTek Common Array Manager(CAMS):

Start with creating a Profile Storage Pool Create a new Volume Map the Volume to a controller.

Now when you do a format from the server(Solaris) side you'll see the LUNs that you have created. 


Thursday Dec 10, 2009

Mounting etc3 filesystems on X2270 server running Oracle Enterprise Linux

For a Hyperion Essbase benchmark on Oracle Enterprise Linux I had 2 x 6140 Fiber Channel Arrays connected to a X2270 server running Oracle Enterprise Linux 5.3 Update 3 64-bit

We needed 2 x filesystems of 2 TB each, so I configured each 6140 array to have a (RAID 1+0) LUN of 2TB. The LUNS were then mapped to a 4Gbps controller port.

The X2270 has 1 PCI-E slot, and the dual ported HBA was connected to the 2 x 6140 arrays.

In Linux fdisk -l will list the partition tables of all the disks attached to the server(/proc/partitions).

We see that the 2 x 2 TB LUNs that were created can be seen as /dev/sdc and /dev/sdd on the server side. BTW /dev/sda and /dev/sfb are the two local disks on the system, /dev/sda1 has the root / filesystem, and /deb/sdb1 is the /test filesystem.


[root@base02 ~]# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sda1 \* 1 58712 471604108+ 83 Linux

/dev/sda2 58713 60801 16779892+ 82 Linux swap / Solaris


Disk /dev/sdb: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdb1 1 60801 488384001 83 Linux


Disk /dev/sdc: 2096.2 GB, 2096239738880 bytes

255 heads, 63 sectors/track, 254853 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdc1 1 254853 2047106691 83 Linux


Disk /dev/sdd: 2395.7 GB, 2395702558720 bytes

255 heads, 63 sectors/track, 291260 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Disk /dev/sdd doesn't contain a valid partition table


Disk /dev/sde: 20 MB, 20971520 bytes

1 heads, 40 sectors/track, 1024 cylinders

Units = cylinders of 40 \* 512 = 20480 bytes


Device Boot Start End Blocks Id System

/dev/sde1 2 1024 20460 83 Linux


We can create a slice on the device using the fdisk command as shown below.

[root@base02 ~]# fdisk /dev/sdd

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.



The number of cylinders for this disk is set to 291260.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

(e.g., DOS FDISK, OS/2 FDISK)


WARNING: The size of this disk is 2.4 TB (2395702558720 bytes).

DOS partition table format can not be used on drives for volumes

larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID

partition table format (GPT).


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


Command (m for help): m

Command action

a toggle a bootable flag

b edit bsd disklabel

c toggle the dos compatibility flag

d delete a partition

l list known partition types

m print this menu

n add a new partition

o create a new empty DOS partition table

p print the partition table

q quit without saving changes

s create a new empty Sun disklabel

t change a partition's system id

u change display/entry units

v verify the partition table

w write table to disk and exit

x extra functionality (experts only)


Command (m for help): p


Disk /dev/sdd: 2395.7 GB, 2395702558720 bytes

255 heads, 63 sectors/track, 291260 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System


Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-291260, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-267349, default 267349):

Using default value 267349


Command (m for help): t

Selected partition 1

Hex code (type L to list codes): L


0 Empty 1e Hidden W95 FAT1 80 Old Minix bf Solaris

1 FAT12 24 NEC DOS 81 Minix / old Lin c1 DRDOS/sec (FAT-


0 Empty 1e Hidden W95 FAT1 80 Old Minix bf Solaris

1 FAT12 24 NEC DOS 81 Minix / old Lin c1 DRDOS/sec (FAT-

2 XENIX root 39 Plan 9 82 Linux swap / So c4 DRDOS/sec (FAT-

3 XENIX usr 3c PartitionMagic 83 Linux c6 DRDOS/sec (FAT-

4 FAT16 <32M 40 Venix 80286 84 OS/2 hidden C: c7 Syrinx

5 Extended 41 PPC PReP Boot 85 Linux extended da Non-FS data

6 FAT16 42 SFS 86 NTFS volume set db CP/M / CTOS / .

7 HPFS/NTFS 4d QNX4.x 87 NTFS volume set de Dell Utility

8 AIX 4e QNX4.x 2nd part 88 Linux plaintext df BootIt

9 AIX bootable 4f QNX4.x 3rd part 8e Linux LVM e1 DOS access

a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e3 DOS R/O

b W95 FAT32 51 OnTrack DM6 Aux 94 Amoeba BBT e4 SpeedStor

c W95 FAT32 (LBA) 52 CP/M 9f BSD/OS eb BeOS fs

e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi ee EFI GPT

f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ef EFI (FAT-12/16/

10 OPUS 55 EZ-Drive a6 OpenBSD f0 Linux/PA-RISC b

11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f1 SpeedStor

12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f4 SpeedStor

14 Hidden FAT16 <3 61 SpeedStor a9 NetBSD f2 DOS secondary

16 Hidden FAT16 63 GNU HURD or Sys ab Darwin boot fb VMware VMFS

17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE

18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto

1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep

1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT

Hex code (type L to list codes): 83


Command (m for help): m

Command action

a toggle a bootable flag

b edit bsd disklabel

c toggle the dos compatibility flag

d delete a partition

l list known partition types

m print this menu

n add a new partition

o create a new empty DOS partition table

p print the partition table

q quit without saving changes

s create a new empty Sun disklabel

t change a partition's system id

u change display/entry units

v verify the partition table

w write table to disk and exit

x extra functionality (experts only)


Command (m for help): w

The partition table has been altered!


Calling ioctl() to re-read partition table.

Syncing disks.

[root@base02 ~]#



We can now see the partition that we created using fdisk command.


[root@base02 ~]# fdisk -l


Disk /dev/sda: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sda1 \* 1 58712 471604108+ 83 Linux

/dev/sda2 58713 60801 16779892+ 82 Linux swap / Solaris


Disk /dev/sdb: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdb1 1 60801 488384001 83 Linux


Disk /dev/sdc: 2096.2 GB, 2096239738880 bytes

255 heads, 63 sectors/track, 254853 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdc1 1 254853 2047106691 83 Linux


WARNING: The size of this disk is 2.4 TB (2395702558720 bytes).

DOS partition table format can not be used on drives for volumes

larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID

partition table format (GPT).



Disk /dev/sdd: 2395.7 GB, 2395702558720 bytes

255 heads, 63 sectors/track, 291260 cylinders

Units = cylinders of 16065 \* 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdd1 1 267349 2147480811 83 Linux


Disk /dev/sde: 20 MB, 20971520 bytes

1 heads, 40 sectors/track, 1024 cylinders

Units = cylinders of 40 \* 512 = 20480 bytes


Device Boot Start End Blocks Id System

/dev/sde1 2 1024 20460 83 Linux

[root@base02 ~]#

[root@base02 ~]#

[root@base02 ~]# cat /etc/fstab

LABEL=/ / ext3 defaults 1 1


[root@base02 ~]# cat /etc/fstab

LABEL=/ / ext3 defaults 1 1

tmpfs /dev/shm tmpfs defaults 0 0

devpts /dev/pts devpts gid=5,mode=620 0 0

sysfs /sys sysfs defaults 0 0

proc /proc proc defaults 0 0

LABEL=SWAP-sda2 swap swap defaults 0 0

/dev/sdb1 /test ext3 defaults 0 0

/dev/sdc1 /array ext3 defaults 0 0



We create the mount point here

[root@base02 ~]# mkdir /array2


Then we mkfs the new partition that we created on the device.

[root@base02 ~]# mkfs.ext3 /dev/sdd1

mke2fs 1.39 (29-May-2006)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

268435456 inodes, 536870202 blocks

26843510 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=4294967296

16384 block groups

32768 blocks per group, 32768 fragments per group

16384 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,

102400000, 214990848, 512000000


Writing inode tables: done

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done


This filesystem will be automatically checked every 30 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

[root@base02 ~]#



We then make an entry of the mount point and the device in the /etc/fstab,

the would make the filesystem persistent across reboots.

[root@base02 ~]# vi /etc/fstab

[root@base02 ~]# cat /etc/fstab

LABEL=/ / ext3 defaults 1 1

tmpfs /dev/shm tmpfs defaults 0 0

devpts /dev/pts devpts gid=5,mode=620 0 0

sysfs /sys sysfs defaults 0 0

proc /proc proc defaults 0 0

LABEL=SWAP-sda2 swap swap defaults 0 0

/dev/sdb1 /test ext3 defaults 0 0

/dev/sdc1 /array ext3 defaults 0 0

/dev/sdd1 /array2 ext3 defaults 0 0

[root@base02 ~]#


Mount the filesystem

[root@base02 ~]# mount /array2


Voila!!!

[root@base02 ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 436G 45G 369G 11% /

tmpfs 12G 0 12G 0% /dev/shm

/dev/sdb1 459G 58G 378G 14% /test

/dev/sdc1 1.9T 57G 1.8T 4% /array

none 12G 104K 12G 1% /var/lib/xenstored

/dev/sdd1 2.0T 199M 1.9T 1% /array2

[root@base02 ~]#


We have filesystems /array and /array2 on the partition's that we created.

Tuesday Oct 06, 2009

Enabling Jumbo Frames and increasing MTU on a M5000

Below are the steps that I followed to enable Jumbo Frames and to increase the MTU from 1500 to
9000 for bge0 on a SunFire M5000 running Oracle Hyperion Essbase


­bash­3.00# ifconfig ­-a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index
1
        inet 127.0.0.1 netmask ff000000
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU>
mtu 1500 index 3
        inet 199.199.224.2 ­­> 199.199.224.1 netmask ffffff00
        ether 0
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 8
        inet 10.1.15.106 netmask ffffff00 broadcast 10.1.15.255
        ether 0:21:28:1a:8b:28
­

bash­3.00# grep bge /etc/path_to_inst
"/pci@0,600000/pci@0/pci@8/pci@0/network@2" 0 "bge"
"/pci@0,600000/pci@0/pci@8/pci@0/network@2,1" 1 "bge"
­

bash­3.00# cat /etc/system
                set bge:bge_jumbo_enable = 1
­

 bash­3.00# cat /platform/sun4u/kernel/drv/bge.conf
default_mtu=9000;
name="bge" parent="/pci@0,600000" unitaddress="2" default_mtu=9000;
­

bash­3.00#reboot
­

bash­3.00# ifconfig ­-a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index
1
        inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9000 index 2
        inet 10.1.15.106 netmask ffffff00 broadcast 10.1.15.255
        ether 0:21:28:1a:8b:28
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU>
mtu 1500 index 3
        inet 199.199.224.2 ­­> 199.199.224.1 netmask ffffff00
        ether 0

Tuesday Jun 10, 2008

How to setup console access through the Network management port of a T2000

Ok, your team mate in India has just asked for console access for the Netra T2000
server that is sitting in the lab in Menlo Park, California.  Fret not, the answer to it
the NET MGT port can be used for exactly that purpore.

How to to set the IP address for the NET MGT port.

To connect through the network management port, use the telnet or ssh (based on
the value you provided below)

For telnet:
sc> setsc if_connection telnet

For ssh:
sc> setsc if_connection ssh

The Net management port can be activated by assigning it an IP address either by
using a static IP address, or through DHCP.

Commands to set up using a static IP address

sc> setsc netsc_dhcp false

sc> setsc netsc_ipaddr ip-address

sc> setsc netsc_ipnetmask ip-netmask

sc> setsc netsc_ipgateway ip-address

sc> resetrc

sc> shownetwork

sc> showsc

If you intend to use DHCP:
sc> setsc netsc_dhcp true

sc> resetrc

You are now ready to remotely access the consoles of your machine using telnet or ssh.
eg.

# telnet ip-address
Trying ip-address
Connected to ip-address (ip-address).
Escape character is '\^]'.

Copyright 2007 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.

Sun(tm) Advanced Lights Out Manager CMT v1.6

Please login: admin
Please Enter password: \*\*\*\*\*\*\*

sc>

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript"> </script> <script type="text/javascript"> _uacct = "UA-4767980-1"; urchinTracker(); </script>

How to Upgrade System Firmware of a Netra T2000 Server:

For one of my projects I was trying to setup LDoms on a T2000 server. To setup LDOMS we need to meet install

Solaris 10 Update 3, Build 8 or later. Also, the System Firmware needs to be 6.4 or later. The T2000 I had was from

an older lot, so the System Firmware was not up to the mark and needed to be upgraded.

Below are steps that need to followed to upgrade the System Firmware to the required level

Where can we find System Firmwares:

The best place to get firmware is from the system handbook:
http://sunsolve.sun.com/handbook_pub/Systems/

Off of the front page for each of the systems is a link to the latest firmware. For the Netra T2000:
http://sunsolve.sun.com/handbook_pub/validateUser.do?target=Systems/Netra_T2000/Netra_T2000

 Internally within Sun, System Firmwares can also be found at:

http://gates.west/biweekly/SystemFirmware/sysfw-6.6/

How to Upgrade System Firmware of a Netra T2000 Server:

Download Sun_System_Firmware-6_6_0_build_04-Netra_T2000.bin and the sysfwdownload
tool from the above location.

The below steps from 1 to 11 are from the README that comes along with the Firmware in
the Tools directory.

Assuming you have verified that your system supports this utility, follow
these steps for updating your Sun System Firmware using the sysfwdownload
utility:

1. Login to the system as root via the network using telnet
or rlogin or via the System Controller SERIAL MGT port.

2. Change directory to /tmp

# cd /tmp/

3. If there is not a subdirectory named 'images', then create it:

# mkdir images

4. Change directory to ./images

# cd images

5. Copy the 'sysfwdownload' binary and Sun System Firmware image
(e.g. Sun_System_Firmware---.bin) from the SunSolve
site to the /tmp/images directory.

6. Use the sysfwdownload utility to download the Sun System Firmware image
to the System Controller:
(In this example, we use Sun_System_Firmware-6_2_0-Sun_Fire_T2000.bin)

# /tmp/images/sysfwdownload Sun_System_Firmware-6_2_0-Sun_Fire_T2000.bin
.......... (10%).......... (20%).......... (30%).......... (41%).......... (51%).......... (61%).......... (71%).......... (82%).......... (92%)........ (100%)
Download completed successfully.

7. Wait until the download completes successfully. This should take 10-15
minutes.

8. Power off the system. (i.e. to standby mode).

a) As root, exit the OS such that the system returns to the PROM's "ok"
prompt:

# shutdown -i0
{0} ok

b) Access the System Controller command line interface (CLI). This is
accomplished using the console escape characters. (normally "#.")

{0} ok #.
sc>

c) From the System Controller CLI, issue the poweroff command
(this sequence will take about 60 seconds to completely power the
server off):

sc> poweroff
Are you sure you want to power off the system [y/n]? y
sc>
SC Alert: SC Request to Power Off Host.

SC Alert: Host system has shut down.
sc>

9. Make sure that your virtual keyswitch setting is not in the LOCKED position.
You can check the setting from the System Controller CLI with the following
command:

sc> showkeyswitch

If the virtual key switch is in LOCKED position you can change that
with the following command:

sc> setkeyswitch -y normal

10. Flash update the downloaded Sun System Firmware image:

sc> flashupdate -s 127.0.0.1

'127.0.0.1' is the default address for the local host.

As the download process progresses, a series of periods appear across your
screen.

.......................

When the download process is finished, ALOM displays the message:

Update complete. Reset device to use new software.

11. The Sun System Firmware has now been updated. For the system to use
the new firmware you must reset the System Controller.

Type the resetsc command to reset ALOM.

sc> resetsc
User Requested SC Shutdown

12. Power on the system controller.
sc> poweron

At the ok prompt type boot.

Check the current version of the System Firmware:
-----------------------------------------------------------------------------------
sc> showhost
System Firmware 6.6.0_build_04 Netra[TM] T2000 2007/12/11 20:49

Host flash versions:
OBP 4.x.build_128\*\*\*PROTOTYPE BUILD\*\*\* 2007/12/11 18:29
Hypervisor 1.6.0.build_04\*\*PROTOTYPE\*\* 2007/12/11 18:56
Netra[TM] T2000 POST 4.x.build_128\*\*\*PROTOTYPE BUILD\*\*\* 2007/12/11 18:58

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript"> </script> <script type="text/javascript"> _uacct = "UA-4767980-1"; urchinTracker(); </script>
About

Mayur Shetty

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today