Wednesday May 21, 2008

Using mirror mounts to get a better /net

One problem with the automounter is that when you use the /net mount points to mount a server if the admin on that server adds a share then you client won't see that share until the automounter timesout the mount. This obviously requires that the mounts are unused which for a large nfs server could never happen.

So given an NFS server host called sa64-zfs-gmp03.eu which is sharing a directory /newpool/cjg on a client you can do:

#  ls /net/sa64-zfs-gmp03.eu/newpool
cjg
#  ls /net/sa64-zfs-gmp03.eu/newpool/cjg
SPImage         ipmiLog         ppcenv          sel.bin         tmp
SPValueAdd      mcCpu0Core0Log  processLog      summaryLog
evLog           mcCpu1Core0Log  prsLog          swLog
hwLog           mcCpu2Core0Log  pstore          tdulog.tar
# cd  /net/sa64-zfs-gmp03.eu/newpool/cjg
# ls
SPImage         ipmiLog         ppcenv          sel.bin         tmp
SPValueAdd      mcCpu0Core0Log  processLog      summaryLog
evLog           mcCpu1Core0Log  prsLog          swLog
hwLog           mcCpu2Core0Log  pstore          tdulog.tar

However if at this point on the server you create and share a new file system:

# zfs create -o sharenfs=rw newpool/cjg2
# share
-@newpool/cjg   /newpool/cjg   rw   ""  
-@newpool/cjg2  /newpool/cjg2   rw   ""  
# echo foo > /newpool/cjg2/file
# 

You can't now directly access it on the client:

# ls /net/sa64-zfs-gmp03.eu/newpool/cjg2
/net/sa64-zfs-gmp03.eu/newpool/cjg2: No such file or directory
#

Now we all know you can work around this by using aliases for the server or even different capitalization:

# ls /net/SA64-zfs-gmp03.eu/newpool/cjg2
file
# 

however lots of users just won't buy that and I don't blame them.

With the advent or mirror mounts to NFSv4 you can do a lot better and there is an RFE (4107375) for the automounter to do this for you, which looks like it would be simple on a client that can do mirror mounts but until that is done here is a work-around. Create a file “/etc/auto_mirror “that contains this line:

\* &:/

Then add this line to auto_master:

/mirror auto_mirror  -nosuid,nobrowse,vers=4

or add a new key to an existing automount table:

: s4u-nv-gmp03.eu TS 50 $; nismatch mirror auto_share
mirror / -fstype=autofs,nosuid,nobrowse auto_mirror.org_dir.cte.sun.com.
: s4u-nv-gmp03.eu TS 51 $; 

Now if we do the same test this time replacing the “/net” path with the “/mirror” path you get:

# ls /mirror/sa64-zfs-gmp03.eu/newpool/
cjg
# ls /mirror/sa64-zfs-gmp03.eu/newpool/cjg
SPImage         ipmiLog         ppcenv          sel.bin         tmp
SPValueAdd      mcCpu0Core0Log  processLog      summaryLog
evLog           mcCpu1Core0Log  prsLog          swLog
hwLog           mcCpu2Core0Log  pstore          tdulog.tar
# (cd /mirror/sa64-zfs-gmp03.eu/newpool/cjg ; sleep 1000000) &
[1]     10455
# ls /mirror/sa64-zfs-gmp03.eu/newpool/cjg2
/mirror/sa64-zfs-gmp03.eu/newpool/cjg2: No such file or directory

Here I created the new file system on the server and put the file in.

# ls /mirror/sa64-zfs-gmp03.eu/newpool/cjg2
file
# 

If you are an entirely NFSv4 shop then you could change the “/net” mount point to use this.

Tuesday Mar 18, 2008

When is a good idea to modify an underlying mirror?

Following on from “When to run fsck” and “When to run quotacheck” here is another:

When to modify the individual sub mirrors that make up a mirrored volume?

Answer: Never.

With the Logical volume manger in Solaris you can build a mirror from two sub mirrors:

# metastat d0
d0: Mirror
    Submirror 0: d10
      State: Okay         
    Submirror 1: d11
      State: Okay         
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 20482875 blocks (9.8 GB)

d10: Submirror of d0
    State: Okay         
    Size: 20482875 blocks (9.8 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c1d0s0          0     No            Okay   Yes 


d11: Submirror of d0
    State: Okay         
    Size: 20482875 blocks (9.8 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c5d0s0          0     No            Okay   Yes 


Device Relocation Information:
Device   Reloc  Device ID
c1d0   Yes      id1,cmdk@AST3320620AS=____________3QF09GL1
c5d0   Yes      id1,cmdk@AST3320620AS=____________3QF0A1QD
# 

So here we have the mirror “d0” made up of devices “d10” and “d11”. Each of these devices can be addressed in the file system as /dev/md/rdsk/d0 /dev/md/rdsk/d10 and /dev/md/rdsk/d11 respectively. The block devices are also available if you so desire. While being able to address the underlying disk devices that make up a mirror is interesting and potentially useful it is only useful if you really know what you are doing.

Reading from the mirrors is o.k. Writing and that includes just mounting the file system is not. So if the device is idle you can do:


# cmp /dev/md/rdsk/d10 /dev/md/rdsk/d11


#

Which if it returns 01 gives you a feeling of confidence, although if you are this paranoid, and I am, then ZFS is a much better bet.


For example if the mirror contains a file system then mounting one side of the mirror and making modifications is a really really bad idea, even if the mirror is unmounted. Once you have made such a modification you would have to make sure the other side of the mirror had exactly the same change at the block level propagated to it. Realistically the only way to achieve that is for you to detach the other mirror and then reattach it so allow it to resync. If you really know what you are doing there are tricks you could do but I suspect those that really know what they are doing would not get into this mess in the first place.



1 If it does not then you have to look at how the mirror was constructed before you start to worry. If you did “metainit d0 –m d10 d11” or have grown the metadevice then the mirrors will never have been brought into sync. So only the blocks that have been written to since the operation will correctly comapare. Hence this is nothing to worry about. See I told you you do really have to know what you are doing.

About

This is the old blog of Chris Gerhard. It has mostly moved to http://chrisgerhard.wordpress.com

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today