NAS storage server in OpenSolaris JeOS Prototype

OpenSolaris 200906 JeOS Prototype is PURE CLI enviroment, so its small true command line only enviroment, as such is ideal install BASE for experimenting with system features like ZFS and SERVER network storage protocols like (CIFS, NFS,iSCSI).

In this example I will demonstrate a creation of simple STORAGE multi-protocol NAS using ZFS and network storage protocols (CIFS, NFS, iSCSI) integration.

I will also demonstrate some ZFS native features together with storage oriented ones.

General Information - JeOS Prototype OpenSolaris project

Download OpenSolaris 200906 JeOS Prototype VM for one 10+ most popular Virtualization Formats:

JeOS: Just enough OS Project Homepage

OpenSolaris 2009.06 JeOS Prototype VM Images Download

Discussion jeos-discuss@opensolaris.org (Main page)

Discussion jeos-discuss@opensolaris.org (Archives)

Part 1: Check JeOS readiness as user root

1. Switch to root

pfexec su - root

2a. Check, if PKG will clean download cache on SUCCESS

root@osol-jeos:~# pkg property flush-content-cache-on-success
PROPERTY                       VALUE
flush-content-cache-on-success True

2b. Check, then we increase PKG download patience for slowest networks

root@osol-jeos:~# set | grep PKG
PKG_CLIENT_TIMEOUT=120
PKG_TIMEOUT_MAX=16

2c. Check, then we use right main repository for opensolaris.org

root@osol-jeos:~# pkg publisher
PUBLISHER                             TYPE     STATUS   URI
opensolaris.org          (preferred)  origin   online   http://pkg.opensolaris.org/release/
contrib                               origin   online   http://pkg.opensolaris.org/contrib/

2d. Check, then we can access these repositories

root@osol-jeos:~# pkg refresh --full

2e. Check, opensolaris.org download speed of whole repository catalog

root@osol-jeos:~# curl pkg.opensolaris.org/release/catalog/0 >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1333k  100 1333k    0     0  93491      0  0:00:14  0:00:14 --:--:--  126k

2f. Check, if we have enough memory and activated swap (I recommend if possible to have VM with 768MB-1GB RAM configured)

root@osol-jeos:~# top | grep Memory
Memory: 1024M phys mem, 525M free mem, 512M total swap, 512M free swap

2g. Check, if you have enough disk space in host, in case then you use a growing Virtual disk Format

At least 2GB of disk space is needed.

2h. Add Virtual Disks, if you want more deep tests, like iSCSI

For example, add 3x 6GB disks for ZFS RADIZ based OpenSolaris installation test

Part2: Creating NAS server on OpenSolaris JeOS Prototype

2.Option 1: Install STORAGE-SERVER with all servers/protocol

Note: This one is too BIG for small CLI JeOS Prototype, it install a GUI X/Gnome dependencies and large driver set, see bellow:

root@osol-jeos:~#  pkg install -nv storage-server
Creating Plan | Before evaluation:
UNEVALUATED:
+pkg:/storage-server@0.1,5.11-0.111:20090508T165041Z

After evaluation:
None -> pkg:/storage-server@0.1,5.11-0.111:20090508T165041Z
None -> pkg:/SUNWnfss@0.5.11,5.11-0.111:20090508T162505Z
None -> pkg:/SUNWvscan@0.5.11,5.11-0.111:20090508T164122Z
None -> pkg:/SUNWmda@0.5.11,5.11-0.111:20090508T162120Z
None -> pkg:/SUNWvscankr@0.5.11,5.11-0.111:20090508T164123Z
None -> pkg:/SUNWlsimega@0.5.11,5.11-0.111:20090508T161903Z
None -> pkg:/SUNWadpu320@0.5.11,5.11-0.111:20090508T153040Z
None -> pkg:/SUNWamr@0.5.11,5.11-0.111:20090508T153048Z
None -> pkg:/SUNWndmp@0.5.11,5.11-0.111:20090508T162452Z
None -> pkg:/SUNWfcmdb@0.5.11,5.11-0.111:20090508T154319Z
None -> pkg:/SUNWfctl@0.5.11,5.11-0.111:20090508T154322Z
None -> pkg:/SUNWfcp@0.5.11,5.11-0.111:20090508T154320Z
None -> pkg:/SUNWemlxs@0.5.11,5.11-0.111:20090508T154002Z
None -> pkg:/SUNWstmf@0.5.11,5.11-0.111:20090508T163712Z
None -> pkg:/SUNWii@0.5.11,5.11-0.111:20090508T160911Z
None -> pkg:/SUNWscm@0.5.11,5.11-0.111:20090508T163449Z
None -> pkg:/SUNWspsv@0.5.11,5.11-0.111:20090508T163647Z
None -> pkg:/SUNWsmba@3.0.34,5.11-0.111:20090508T163557Z
None -> pkg:/SUNWgamin@0.5.11,5.11-0.111:20090508T154714Z
None -> pkg:/SUNWgnome-base-libs@0.5.11,5.11-0.111:20090508T154912Z
None -> pkg:/SUNWTiff@0.5.11,5.11-0.111:20090508T153019Z
None -> pkg:/SUNWxwplt@0.5.11,5.11-0.111:20090508T164551Z
None -> pkg:/SUNWxorg-clientlibs@0.5.11,5.11-0.111:20090508T164427Z
None -> pkg:/SUNWxwxft@2.1.12,5.11-0.111:20090508T164617Z
None -> pkg:/SUNWfontconfig@2.5.0,5.11-0.111:20090508T154526Z
None -> pkg:/SUNWfreetype2@2.3.7,5.11-0.111:20090508T154657Z
None -> pkg:/SUNWxwice@0.5.11,5.11-0.111:20090508T164538Z
None -> pkg:/SUNWxwfsw@0.5.11,5.11-0.111:20090508T164536Z
None -> pkg:/SUNWxorg-mesa@7.2,5.11-0.111:20090508T164436Z
None -> pkg:/SUNWxorg-server@1.5.3,5.11-0.111:20090508T164440Z
None -> pkg:/SUNWxorg-xkb@0.5.11,5.11-0.111:20090508T164455Z
None -> pkg:/SUNWdbus-libs@0.5.11,5.11-0.111:20090508T153752Z
None -> pkg:/SUNWpciaccess@0.10.5,5.11-0.111:20090508T162626Z
None -> pkg:/SUNWpixman@0.12.0,5.11-0.111:20090508T162835Z
None -> pkg:/SUNWhal@0.5.11,5.11-0.111:20090508T160732Z
None -> pkg:/SUNWdbus-bindings@0.5.11,5.11-0.111:20090508T153749Z
None -> pkg:/SUNWpolkit@0.5.11,5.11-0.111:20090508T162850Z
None -> pkg:/SUNWxsvc@0.5.11,5.11-0.111:20090508T164518Z
None -> pkg:/SUNWxwplr@0.5.11,5.11-0.111:20090508T164549Z
None -> pkg:/SUNWpapi@0.5.11,5.11-0.111:20090508T162622Z
None -> pkg:/SUNWjpg@0.5.11,5.11-0.111:20090508T161234Z
None -> pkg:/SUNWmlib@0.5.11,5.11-0.111:20090508T162151Z
None -> pkg:/SUNWcups-libs@1.3.9,5.11-0.111:20090508T153738Z
None -> pkg:/SUNWmpapi@0.5.11,5.11-0.111:20090508T162211Z
None -> pkg:/SUNWmpsvplr@0.5.11,5.11-0.111:20090508T162213Z
None -> pkg:/SUNWsmpd@0.5.11,5.11-0.111:20090508T163623Z
None -> pkg:/SUNWdmgt@0.5.11,5.11-0.111:20090508T153928Z
None -> pkg:/SUNWimac@0.5.11,5.11-0.111:20090508T160942Z
None -> pkg:/SUNWima@0.5.11,5.11-0.111:20090508T160941Z
None -> pkg:/SUNWisns@0.5.11,5.11-0.111:20090508T161051Z
None -> pkg:/SUNWrdc@0.5.11,5.11-0.111:20090508T163217Z
None -> pkg:/SUNWmegasas@0.5.11,5.11-0.111:20090508T162131Z
None -> pkg:/SUNWmv88sx@0.5.11,5.11-0.111:20090508T162220Z
None -> pkg:/SUNWqlcu@0.5.11,5.11-0.111:20090508T163207Z
None -> pkg:/SUNWluxop@0.5.11,5.11-0.111:20090508T161908Z
None -> pkg:/SUNWiscsitgt@0.5.11,5.11-0.111:20090508T161048Z
None -> pkg:/SUNWfcip@0.5.11,5.11-0.111:20090508T154318Z
None -> pkg:/SUNWfcsm@0.5.11,5.11-0.111:20090508T154322Z
None -> pkg:/SUNWaac@0.5.11,5.11-0.111:20090508T153030Z
None -> pkg:/SUNWqlc@0.5.11,5.11-0.111:20090508T163206Z
None -> pkg:/SUNWmms@0.5.11,5.11-0.111:20090508T162204Z
None -> pkg:/SUNWpostgr-83-libs@8.3.7,5.11-0.111:20090508T163014Z
None -> pkg:/SUNWmpathadm@0.5.11,5.11-0.111:20090508T162212Z
None -> pkg:/SUNWfcprt@0.5.11,5.11-0.111:20090508T154321Z
None -> pkg:/SUNWsmbs@0.5.11,5.11-0.111:20090508T163612Z
None -> pkg:/SUNWsmbskr@0.5.11,5.11-0.111:20090508T163614Z
None -> pkg:/SUNWsmedia@0.5.11,5.11-0.111:20090508T163620Z
None -> pkg:/SUNWfilebench@0.5.11,5.11-0.111:20090508T154334Z
None -> pkg:/SUNWiscsi@0.5.11,5.11-0.111:20090508T161040Z
Actuators:
      restart_fmri: svc:/system/manifest-import:default
      restart_fmri: svc:/application/desktop-cache/input-method-cache:default
      restart_fmri: svc:/application/desktop-cache/pixbuf-loaders-installer:default
None

root@osol-jeos:~# pkg install -nv storage-server | grep "None -" | sed 's/\\// /g' | sed 's/@/ /g' | awk '{print $4}' | xargs -n1 pkg list -saH
storage-server                 Multi Protocol Storage Server cluster
SUNWnfss                       Network File System (NFS) server support
SUNWvscan                      Virus Scan Service Kernel
SUNWmda                        Solaris Volume Manager Assistant
SUNWvscankr
SUNWlsimega                    LSI Hardware RAID HBA Driver
SUNWadpu320                    Adaptec Ultra320 Driver
SUNWamr                        LSI MegaRAID SCSI HBA Driver
SUNWndmp                       NDMP
SUNWfcmdb                      Fibre Channel adb macros and mdb modules
SUNWfctl                       Sun Fibre Channel Transport layer
SUNWfcp                        Sun FCP SCSI Device Driver
SUNWemlxs                      Emulex-Sun LightPulse Fibre Channel Adapter driver
SUNWstmf                       Sun Common Multiprotocol SCSI Target
SUNWii                         Sun StorageTek Availability Suite Point-In-Time Copy
SUNWscm                        Sun StorageTek Availability Suite Cache Management
SUNWspsv                       Sun StorageTek Availability Suite Volume Driver
SUNWsmba                       samba - A Windows SMB/CIFS fileserver for UNIX
SUNWgamin                      Library providing the FAM File Alteration Monitor API
SUNWgnome-base-libs            GNOME base GUI libraries
SUNWTiff                       libtiff - library for reading and writing TIFF
SUNWxwplt                      X Window System platform software
SUNWxorg-clientlibs            X.Org Foundation X Client Libraries
SUNWxwxft                      Xft (X Freetype) Library
SUNWfontconfig                 Fontconfig library
SUNWfreetype2                  FreeType2 Font library
SUNWxwice                      X Window System Inter-Client Exchange (ICE) Components
SUNWxwfsw                      X Window System platform required font software
SUNWxorg-mesa                  Xorg Server Mesa Modules and Libraries
SUNWxorg-server                X.Org Foundation Xserver
SUNWxorg-xkb                   X.Org Foundation Xserver keyboard layouts
SUNWdbus-libs                  Simple IPC library based on messages - client libraries
SUNWpciaccess                  PCI bus access library and tools
SUNWpixman                     Pixman library
SUNWhal                        Hardware Abstraction Layer
SUNWdbus-bindings              Simple IPC library based on messages
SUNWpolkit                     PolicyKit
SUNWxsvc                       Sun Xserver pseudo driver
SUNWxwplr                      X Window System platform software configuration
SUNWpapi                       Free Standards Group Open Printing API
SUNWjpg                        jpeg - The Independent JPEG Groups JPEG software
SUNWmlib                       mediaLib - Shared Libraries
SUNWcups-libs                  CUPS (Runtime Libraries)
SUNWmpapi                      SNIA Multipath Management API Common Library
SUNWmpsvplr                    Sun MP API library for the scsi_vhci driver
SUNWsmpd                       Target Driver for Serial SCSI Management Protocol (SMP) Compliant Devices
SUNWdmgt                       Solaris Data Management WBEM/CIM API
SUNWimac                       Solaris IMA Common Library
SUNWima                        Solaris IMA Software Initiator Plugin
SUNWisns                       Solaris iSNS Server
SUNWrdc                        Sun StorageTek Availability Suite Remote Mirror
SUNWmegasas                    LSI MegaSAS RAID Controller HBA Driver
SUNWmv88sx                     Marvell 88sx sata driver
SUNWqlcu                       Qlogic Fibre Channel Adapter Utilities
SUNWluxop                      Sun Enterprise Network Array firmware and utilities
SUNWiscsitgt                   Sun iSCSI Target
SUNWfcip                       Sun FCIP IP/ARP over FibreChannel Device Driver
SUNWfcsm                       FCSM driver
SUNWaac                        Adaptec AdvanceRaid Controller SCSI HBA Driver
SUNWqlc                        Qlogic ISP 2200/2202 Fibre Channel Device Driver
SUNWmms                        MMS - Media Management Software
SUNWpostgr-83-libs             PostgreSQL client libraries
SUNWmpathadm                   Solaris Multipathing CLI
SUNWfcprt                      Fibre Channel HBA Port utility
SUNWsmbs                       SMB Server
SUNWsmbskr                     SMB Server (Kernel)
SUNWsmedia                     Storage media management library
SUNWfilebench                  FileBench
SUNWiscsi                      Sun iSCSI Management Utilities
root@osol-jeos:~# pkg install storage-server
DOWNLOAD                                    PKGS       FILES     XFER (MB)
SUNWnfss                                    1/69      0/4846   0.00/118.36\^C

Check which packages makes a GUI (X/Gnome) deps, looks like old SAMBA will do this ?

root@osol-jeos:~# pkg install -nv SUNWsmba
Creating Plan | Before evaluation:
UNEVALUATED:
+pkg:/SUNWsmba@3.0.34,5.11-0.111:20090508T163557Z

After evaluation:
None -> pkg:/SUNWsmba@3.0.34,5.11-0.111:20090508T163557Z
None -> pkg:/SUNWgamin@0.5.11,5.11-0.111:20090508T154714Z
None -> pkg:/SUNWgnome-base-libs@0.5.11,5.11-0.111:20090508T154912Z
None -> pkg:/SUNWTiff@0.5.11,5.11-0.111:20090508T153019Z
None -> pkg:/SUNWxwplt@0.5.11,5.11-0.111:20090508T164551Z
None -> pkg:/SUNWxorg-clientlibs@0.5.11,5.11-0.111:20090508T164427Z
None -> pkg:/SUNWxwxft@2.1.12,5.11-0.111:20090508T164617Z
None -> pkg:/SUNWfontconfig@2.5.0,5.11-0.111:20090508T154526Z
None -> pkg:/SUNWfreetype2@2.3.7,5.11-0.111:20090508T154657Z
None -> pkg:/SUNWxwice@0.5.11,5.11-0.111:20090508T164538Z
None -> pkg:/SUNWxwfsw@0.5.11,5.11-0.111:20090508T164536Z
None -> pkg:/SUNWxorg-mesa@7.2,5.11-0.111:20090508T164436Z
None -> pkg:/SUNWxorg-server@1.5.3,5.11-0.111:20090508T164440Z
None -> pkg:/SUNWxorg-xkb@0.5.11,5.11-0.111:20090508T164455Z
None -> pkg:/SUNWdbus-libs@0.5.11,5.11-0.111:20090508T153752Z
None -> pkg:/SUNWpciaccess@0.10.5,5.11-0.111:20090508T162626Z
None -> pkg:/SUNWpixman@0.12.0,5.11-0.111:20090508T162835Z
None -> pkg:/SUNWhal@0.5.11,5.11-0.111:20090508T160732Z
None -> pkg:/SUNWdbus-bindings@0.5.11,5.11-0.111:20090508T153749Z
None -> pkg:/SUNWpolkit@0.5.11,5.11-0.111:20090508T162850Z
None -> pkg:/SUNWxsvc@0.5.11,5.11-0.111:20090508T164518Z
None -> pkg:/SUNWxwplr@0.5.11,5.11-0.111:20090508T164549Z
None -> pkg:/SUNWpapi@0.5.11,5.11-0.111:20090508T162622Z
None -> pkg:/SUNWjpg@0.5.11,5.11-0.111:20090508T161234Z
None -> pkg:/SUNWmlib@0.5.11,5.11-0.111:20090508T162151Z
None -> pkg:/SUNWcups-libs@1.3.9,5.11-0.111:20090508T153738Z
Actuators:
      restart_fmri: svc:/system/manifest-import:default
      restart_fmri: svc:/application/desktop-cache/input-method-cache:default
      restart_fmri: svc:/application/desktop-cache/pixbuf-loaders-installer:default
None

root@osol-jeos:~# pkg install SUNWsmba
DOWNLOAD                                    PKGS       FILES     XFER (MB)
SUNWsmba                                    0/26      0/4290    0.00/99.76\^C

1. Option ONE: Install smaller STORAGE-NAS cluster

Network storage servers included here are (CIFS, NFS, old iSCSI target):

root@osol-jeos:~#  pkg install -nv storage-nas | grep "None -" | sed 's/\\// /g' | sed 's/@/ /g' | awk '{print $4}' | xargs -n1 pkg list -saH
storage-nas                    Network Attached Storage cluster
SUNWndmp                       NDMP
SUNWiscsi                      Sun iSCSI Management Utilities
SUNWimac                       Solaris IMA Common Library
SUNWima                        Solaris IMA Software Initiator Plugin
SUNWsmbs                       SMB Server
SUNWsmbskr                     SMB Server (Kernel)
SUNWvscan                      Virus Scan Service Kernel
SUNWnfss                       Network File System (NFS) server support
SUNWiscsitgt                   Sun iSCSI Target
root@osol-jeos:~# pkg install storage-nas
DOWNLOAD                                    PKGS       FILES     XFER (MB)
SUNWndmp                                    1/10       0/108     0.00/3.57\^C

Looks storage-server install a lot of drivers and also a BIG part of GUI enviroment.

Check if we can install new COMSTAR iSCSI target or new CIFS implementation without GUI (X/GNOME) dependencies:

Check new COMSTAR iSCSI target can be easily added here:

root@osol-jeos:~# pkg install -nv SUNWiscsit
Creating Plan | Before evaluation:
UNEVALUATED:
+pkg:/SUNWiscsit@0.5.11,5.11-0.111:20090508T161047Z

After evaluation:
None -> pkg:/SUNWiscsit@0.5.11,5.11-0.111:20090508T161047Z
None -> pkg:/SUNWstmf@0.5.11,5.11-0.111:20090508T163712Z
None -> pkg:/SUNWiscsidm@0.5.11,5.11-0.111:20090508T161041Z
Actuators:
      restart_fmri: svc:/system/manifest-import:default
None

root@osol-jeos:~# pkg install -nv SUNWiscsit | grep "None -" | sed 's/\\// /g' | sed 's/@/ /g' | awk '{print $4}' | xargs -n1 pkg list -saH
SUNWiscsit                     Sun iSCSI COMSTAR Port Provider
SUNWstmf                       Sun Common Multiprotocol SCSI Target
SUNWiscsidm                    Sun iSCSI Data Mover

root@osol-jeos:~# pkg install SUNWiscsit
DOWNLOAD                                    PKGS       FILES     XFER (MB)
SUNWiscsit                                   0/3        0/59     0.00/1.69\^C

2. Install storage-nas and COMSTAR

root@osol-jeos:~# pkg install storage-nas SUNWiscsit
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                  13/13     167/167     5.26/5.26

PHASE                                        ACTIONS
Install Phase                                501/501
Deleting content cache

3. Check then both iSCSI targets are disabled

disabled       16:23:59 svc:/network/iscsi/target:default
disabled       16:24:00 svc:/network/iscsi_initiator:default
disabled       16:24:01 svc:/system/iscsitgt:default
disabled       16:24:02 svc:/system/stmf:default

4. Reboot JeOS instance

PS: At least in 200805 it was needed to reboot OS to get drivers loaded

init 6

Part 3: Create a ZFS based NAS exported shares

Export NFS share

1. Create a ZFS share for NFS experiment

root@osol-jeos:~# zfs create rpool/export/home/nfsshare

2. Enable NFS server

  • Some packages used in OpenSolaris installation in NFS server setup (SUNWgssc SUNWsmedia) are not installed by default in JeOS prototype
  • Bunch of NFS/RPC related services are disabled by security setup applying to JeOS ISC framework Immutable Service Container
  • Kernel NFS part is not install by dependencies on OS Core, see discussion zfs not sharing nfs shares on OSOl 2009.06 core?
root@osol-jeos:~# pkg install SUNWgssc SUNWsmedia
root@osol-jeos:~# pkg list -s | grep nfs
  SUNWnfsc                       Network File System (NFS) client support
  SUNWnfsckr                     Network File System (NFS) client kernel support
  SUNWnfss                       Network File System (NFS) server support

root@osol-jeos:~# pkg list -sa SUNWnfsskr
  NAME (PUBLISHER)               SUMMARY
  SUNWnfsskr                     Network File System (NFS) server kernel support

root@osol-jeos:~# pkg install SUNWnfsskr

root@osol-jeos:~# svcadm enable  network/inetd 

root@osol-jeos:~# svcadm enable nfs/server  nfs/nlockmgr rpc/bind rpc/gss rpc/smserver nfs/status nfs/mapid nfs/rquota nfs/cbd  

root@osol-jeos:~# svcs -a | grep rpc
disabled       17:50:13 svc:/network/rpc/nisplus:default
disabled       17:50:13 svc:/network/rpc/keyserv:default
disabled       17:56:29 svc:/network/rpc/metamed:default
disabled       17:56:29 svc:/network/rpc/mdcomm:default
disabled       17:56:29 svc:/network/rpc/meta:default
disabled       17:56:29 svc:/network/rpc/rex:default
disabled       17:56:29 svc:/network/rpc/metamh:default
online         17:50:23 svc:/network/rpc/bind:default
online         17:56:29 svc:/network/rpc/gss:default
online         18:14:44 svc:/network/rpc/smserver:default

root@osol-jeos:~# svcs -a | grep nfs
disabled       17:50:13 svc:/network/nfs/client:default
disabled       17:56:29 svc:/network/nfs/server:default
online         17:50:23 svc:/network/nfs/cbd:default
online         17:50:23 svc:/network/nfs/status:default
online         17:50:24 svc:/network/nfs/nlockmgr:default
online         17:50:24 svc:/network/nfs/mapid:default
online         17:56:29 svc:/network/nfs/rquota:default
root@osol-jeos:~# svcs -l  nfs/server
fmri         svc:/network/nfs/server:default
name         NFS server
enabled      false (temporary)
state        disabled
next_state   none
state_time   Mon Dec 14 16:43:04 2009
logfile      /var/svc/log/network-nfs-server:default.log
restarter    svc:/system/svc/restarter:default
contract_id
dependency   require_any/error svc:/milestone/network (online)
dependency   require_all/error svc:/network/nfs/nlockmgr (online)
dependency   optional_all/error svc:/network/nfs/mapid (online)
dependency   require_all/restart svc:/network/rpc/bind (online)
dependency   optional_all/none svc:/network/shares/group (multiple)
dependency   require_all/error svc:/system/filesystem/local (online)

cat  /var/svc/log/network-nfs-server:default.log
  [ Dec 14 16:42:51 Enabled. ]
  [ Dec 14 16:42:51 Executing start method ("/lib/svc/method/nfs-server start"). ]
  No NFS filesystems are shared
  [ Dec 14 16:42:51 Method "start" exited with status 0. ]
  [ Dec 14 16:42:51 Stopping because service disabled. ]

Enable NFS server start even when no NFS shares are defined:

root@osol-jeos:~# svcprop -p application/auto_enable nfs/server
 true
root@osol-jeos:~# svccfg -s nfs/server setprop application/auto_enable=false
root@osol-jeos:~# svcadm refresh nfs/server
root@osol-jeos:~# svcprop -p application/auto_enable nfs/server
 false
root@osol-jeos:~# svcadm disable nfs/server; sleep 3; svcadm enable nfs/server
root@osol-jeos:~# svcs nfs/server
STATE          STIME    FMRI
online         11:20:26 svc:/network/nfs/server:default

Check then all RPC services are running (Edited to make smaller output)

root@osol-jeos:~# rpcinfo | more

  program version netid     address             service    owner
    100000    4    ticots    osol-jeos.rpc       rpcbind    superuser
...
    100024    1    udp       0.0.0.0.221.173     status     superuser
...
    100021    1    udp6      ::.15.205           nlockmgr   1
...
    100234    1    ticotsord Y\\000\\000\\000       gssd       superuser
...
    100155    1    tcp6      ::.186.216          smserverd  superuser
...
    100011    1    udp6      ::.133.106          rquotad    superuser
...
   00005    1    tcp       0.0.0.0.250.118     mountd     superuser
... 
   100003    4    tcp       0.0.0.0.8.1         nfs        1
   100227    2    tcp       0.0.0.0.8.1         nfs_acl    1

3. Enable NFS share

root@osol-jeos:~#  zfs set sharenfs=on rpool/export/home/nfsshare

root@osol-jeos:~# dfshares
RESOURCE                                  SERVER ACCESS    TRANSPORT
 osol-jeos:/export/home/nfsshare       osol-jeos  -         -
root@osol-jeos:~# sharemgr show -vp
default nfs=()
zfs
    zfs/rpool/export/home/nfsshare nfs=()
          /export/home/nfsshare

PS: In case then you will be not able to share, add public IP to /etc/hosts

4. Play with ZFS and NFS share

root@osol-jeos:~# mkdir /export/home/nfsshare/osol
root@osol-jeos:~# chown osol /export/home/nfsshare/osol
root@osol-jeos:~# zfs set quota=5M rpool/export/home/nfsshare 
root@osol-jeos:~# svcadm enable autofs
root@osol-jeos:~# su - osol
osol@osol-jeos:~$ touch /export/home/nfsshare/osol/yes
root@osol-jeos:~#  ls  /net/osol-jeos/export/home/nfsshare/osol/
yes
osol@osol-jeos:~$ dd if=/dev/zero of=/net/osol-jeos/export/home/nfsshare/osol/zero.zer bs=100k
dd: writing `/net/osol-jeos/export/home/nfsshare/osol/zero.zer': Disc quota exceeded
52+0 records in
51+0 records out
5222400 bytes (5.2 MB) copied, 1.57468 s, 3.3 MB/s

osol@osol-jeos:~$ exit

root@osol-jeos:~# zfs list  rpool/export/home/nfsshare
NAME                         USED  AVAIL  REFER  MOUNTPOINT
rpool/export/home/nfsshare  5.02M      0  5.02M  /export/home/nfsshare

root@osol-jeos:~#  zfs set quota=30M rpool/export/home/nfsshare
root@osol-jeos:~#  zfs set reservation=10M rpool/export/home/nfsshare
root@osol-jeos:~# zfs list  rpool/export/home/nfsshare
NAME                         USED  AVAIL  REFER  MOUNTPOINT
rpool/export/home/nfsshare  5.02M  25.0M  5.02M  /export/home/nfsshare

root@osol-jeos:~#  zfs set quota=none rpool/export/home/nfsshare
root@osol-jeos:~#  zfs set reservation=none rpool/export/home/nfsshare
root@osol-jeos:~# zfs list  rpool/export/home/nfsshare
NAME                         USED  AVAIL  REFER  MOUNTPOINT
rpool/export/home/nfsshare  5.02M  6.25G  5.02M  /export/home/nfsshare

Export CIFS share

1. Create a ZFS share for NFS experimentas started by default

root@osol-jeos:~# zfs create  -o casesensitivity=mixed  rpool/export/home/cifsshare

2. Enable CIFS server

root@osol-jeos:~# svcadm enable smb/server idmap

root@osol-jeos:~# svcs -l smb/server
fmri         svc:/network/smb/server:default
name         smbd daemon
enabled      true
state        online
next_state   none
state_time   Mon Dec 14 17:03:03 2009
logfile      /var/svc/log/network-smb-server:default.log
restarter    svc:/system/svc/restarter:default
contract_id  66
dependency   require_any/error svc:/milestone/network (online)
dependency   require_all/error svc:/system/filesystem/local (online)
dependency   require_all/error svc:/system/idmap:default (online)

3. Enable CIFS share

root@osol-jeos:~# zfs  set sharesmb=on rpool/export/home/cifsshare

root@osol-jeos:~# zfs get sharesmb rpool/export/home/cifsshare
NAME                         PROPERTY  VALUE     SOURCE
rpool/export/home/cifsshare  sharesmb  on        local

root@osol-jeos:~# sharemgr show -vp
default nfs=()
zfs
    zfs/rpool/export/home/cifsshare smb=()
          rpool_export_home_cifsshare=/export/home/cifsshare

4. Setup encrypted password for smb services for test user OSOL

root@osol-jeos:~# mkdir /export/home/cifsshare/osol
root@osol-jeos:~# chown osol /export/home/cifsshare/osol
echo "other password required pam_smb_passwd.so.1 nowarn">>/etc/pam.conf

root@osol-jeos:~# passwd osol
New Password: justone1
Re-enter new Password: justone1
passwd: password successfully changed for osol

5. Check CIFS share on windows machine

Start "My computer"
On CMD line \\\\ip_of_JeOS_server\\
Login with User "OSOL" / password "justone1"
Map share with letter with name "rpool_export_home_cifsshare" 

6. Play now for example with ZFS compression

root@osol-jeos:~# zfs get compressratio rpool/export/home/cifsshare
  NAME                         PROPERTY       VALUE  SOURCE
  rpool/export/home/cifsshare  compressratio  1.00x  -

root@osol-jeos:~# zfs set compression=on rpool/export/home/cifsshare

Copy there some data which can be well compressed like logs:

root@osol-jeos:~# zfs get compressratio rpool/export/home/cifsshare
NAME                         PROPERTY       VALUE  SOURCE
rpool/export/home/cifsshare  compressratio  1.89x  -

Export ZVOL as iSCSI decive

1. First we need to activate COMSTAR

root@osol-jeos:~# svcadm disable iscsitgt
root@osol-jeos:~#  svcadm enable stm
root@osol-jeos:~#  svcs stmf
STATE          STIME    FMRI
online         12:16:51 svc:/system/stmf:default

root@osol-jeos:~#  svcadm enable iscsi/target
root@osol-jeos:~#  svcs iscsi/target
STATE          STIME    FMRI
online         12:18:05 svc:/network/iscsi/target:default

Create a zpool RIADZ 'ipool' from pre-prepared disks

Note: From preprepared 3x 6Gb disks size of 2 disks will be avaiable for ZVOL and snapshost

root@osol-jeos:~# pkg install SUNWinstall-test
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1         7/7     0.25/0.25

PHASE                                        ACTIONS
Install Phase                                  23/23
Deleting content cache

root@osol-jeos:~# /opt/install-test/bin/test_td -d
Disk discovery
Total number of disks: 6
---------------------------------
 num |    name|  ctype|size [MB]|
---------------------------------
   1 |\* c1t0d0|   scsi|     8192|
   2 |  c1t1d0|   scsi|       80|
   3 |  c1t2d0|   scsi|     6144|
   4 |  c1t3d0|   scsi|     6144|
   5 |  c1t4d0|   scsi|     6144|
   6 |    dump|unknown|      384|
---------------------------------

root@osol-jeos:~# zpool create ipool raidz c1t2d0 c1t3d0 c1t4d0
root@osol-jeos:~# zpool list ipool
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
ipool  17.9G   740K  17.9G     0%  ONLINE  -
root@osol-jeos:~# zfs list ipool
NAME    USED  AVAIL  REFER  MOUNTPOINT
ipool  91.2K  11.7G  25.3K  /ipool

I will create a 8G zpool so we have space for snapshots

root@osol-jeos:~# zfs list ipool/iscsishare
NAME               USED  AVAIL  REFER  MOUNTPOINT
ipool/iscsishare     8G  11.7G  21.3K  -

Share this iscsipool by iSCSI (with COMSTAR)

root@osol-jeos:~# ls /dev/zvol/rdsk/ipool/iscsishare
/dev/zvol/rdsk/ipool/iscsishare

root@osol-jeos:~# sbdadm create-lu  /dev/zvol/rdsk/ipool/iscsishare
Created the following LU:
              GUID                    DATA SIZE           SOURCE
--------------------------------  -------------------  ----------------
600144f0b9a2c80000004b2782400001      8589869056       /dev/zvol/rdsk/ipool/iscsishare

root@osol-jeos:~# stmfadm add-view 
root@osol-jeos:~# stmfadm list-view -l 600144f0b9a2c80000004b2782400001
View Entry: 0
    Host group   : All
    Target group : All
    LUN          : 0

root@osol-jeos:~#  itadm create-target
Target iqn.1986-03.com.sun:02:b2229ed6-abb8-c3ce-af87-d5924432c8e7 successfully created

Now you can use iSCSI Initiator (Client) in your OS to access share, interesting option is to use a build in iSCSI initiator build in VirtualBox Virtualization, see more blog:

Using Solaris iSCSI target in VirtualBox iSCSI Initiator

RAIDZ Hint: You can use a disk corrupting scrip to test a ZFS RAIDZ functionality, like Ronny's DD random harmer in:

Corrupting an online and active ASM disk by writing chunks of random data to the disk randomly

Comments:

Excellent guide! I really wish we had a pure server version of OpenSolaris (much like Ubuntu-Server) that has these JeOS packages and no Gnome at install time. I've tried a few things (baking an ISO using the JeOS/Milax distro) in the past but gave up in the end because of dependency woes.

Posted by Thushan Fernando on December 14, 2009 at 08:24 PM PST #

So... can we get a cd ISO instead of a VM image?

Posted by TimC on December 16, 2009 at 10:51 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Hands-on experience with Virtualization

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today