A tour of a kernel zone

In my earlier post, I showed how to configure and install a kernel zone.  In this post, we'll take a look at this kernel zone.

The kernel zone was installed within an LDom on a T5-4.

root@vzl-212:~# prtdiag -v | head -2
System Configuration:  Oracle Corporation  sun4v SPARC T5-4
Memory size: 65536 Megabytes
root@vzl-212:~# psrinfo | wc -l
      32

The kernel zone was configured with:

 root@vzl-212:~# zonecfg -z myfirstkz create -t SYSsolaris-kz

Let's take a look at the resulting configuration.

root@vzl-212:~# zonecfg -z myfirstkz info | cat -n
     1    zonename: myfirstkz
     2    brand: solaris-kz
     3    autoboot: false
     4    autoshutdown: shutdown
     5    bootargs: 
     6    pool: 
     7    scheduling-class: 
     8    hostid: 0x2b2044c5
     9    tenant: 
    10    anet:
    11        lower-link: auto
    12        allowed-address not specified
    13        configure-allowed-address: true
    14        defrouter not specified
    15        allowed-dhcp-cids not specified
    16        link-protection: mac-nospoof
    17        mac-address: auto
    18        mac-prefix not specified
    19        mac-slot not specified
    20        vlan-id not specified
    21        priority not specified
    22        rxrings not specified
    23        txrings not specified
    24        mtu not specified
    25        maxbw not specified
    26        rxfanout not specified
    27        vsi-typeid not specified
    28        vsi-vers not specified
    29        vsi-mgrid not specified
    30        etsbw-lcl not specified
    31        cos not specified
    32        evs not specified
    33        vport not specified
    34        id: 0
    35    device:
    36        match not specified
    37        storage: dev:/dev/zvol/dsk/rpool/VARSHARE/zones/myfirstkz/disk0
    38        id: 0
    39        bootpri: 0
    40    capped-memory:
    41        physical: 2G
    42    suspend:
    43        path: /system/zones/myfirstkz/suspend
    44        storage not specified
    45    keysource:
    46        raw redacted

There are a number of things to notice in this configuration.

  • No zonepath.  Kernel zones install into a real or virtual disks - quite like the way that logical domains install into real or virtual disks.  The virtual disk(s) that contain the root zfs pool are specified by one or more device resources that contain a bootpri property (line 39).  By default, a kernel zone's root disk is a 16 GB zfs volume in the global zone's root zfs pool.  There's more about this in the solaris-kz(5) man page.  It's never been a good idea to directly copy things into a zone's zonepath.  With kernel zones that just doesn't work.
  • The device resource accepts storage URI's (line 37).  See suri(5).  Storage URI's were introduced in Solaris 11.1 in support of Zones on Shared Storage (rootzpool and zpool resources).  This comes in really handy when a kernel zone is installed on external storage and may be migrated between hosts from time to time.
  • The device resource has an id property (line 38).  This means that this disk will be instance 0 of zvblk - which will translate into it being c1d0.  We'll see more of that in a bit.
  • The anet resource has an id property (line 34).  This means that this anet will be instance 0 of zvnet - which will normally be seen as net0.  Again, more of that in a bit.
  • A memory resource control, capped-memory, is set by default (lines 40 - 41).  In the solaris or solaris10 brand, this would mean that rcapd is used to soft limit the amount of physical memory a zone can use.  Kernel zones are different.  Not only is this a hard limit on the amount of physical memory that the kernel zone can use - the memory is immediately allocated and reserved as the zone boots.
  • A suspend resource is present, which defines a location for to write out a suspend file when zoneadm -z zonename suspend is invoked.
  • The keysource resource is used for an encryption key that is used to encrypt suspend images and host data.  solaris-kz(5) has more info on this.

There are several things not shown here that may also be of interest:

  • Previously, autoshutdown (line 4) allowed halt and shutdown as values.  It now also supports suspend for kernel zones only.  As you may recall, autoshutdown is used by svc:/system/zones:default when it is transitioning from online to offline.  If set to halt, the zone (kernel or otherwise) is brought down abruptly.  If set to shutdown, a graceful shutdown is performed.  Now, if a kernel zone has it set to suspend, the kernel zone will be suspended as svc:/system/zones:default goes offline.  When zoneadm boot is issued for a suspended zone, the zone is resumed.
  • If there are multiple device resources that have bootpri set (i.e. bootable devices), zoneadm install will add all of the boot devices to a mirrored root zpool.

From the earlier blog entry, this kernel zone was booted and sysconfig was performed.  Let's look inside.

To get into the zone, you can use zlogin just like you do with any other zone.

root@vzl-212:~# zlogin myfirstkz
[Connected to zone 'myfirstkz' pts/3]
Oracle Corporation      SunOS 5.11      11.2    April 2014
root@myfirstkz:~# 

As I alluded to above, a kernel zone gets a fixed amount of memory.  The value shown above matches the value shown in the capped-memory resource in the zone configuration.

root@myfirstkz:~# prtconf | grep ^Memory
Memory size: 2048 Megabytes

By default, a kernel zone gets one virtual cpu.  You can adjust this with the virtual-cpu or dedicated-cpu zonecfg resources.  See solaris-kz(5).

root@myfirstkz:~# psrinfo
0       on-line   since 04/18/2014 22:39:22

Because a kernel zone runs its own kernel, it does not require that packages are in sync between the global zone and the kernel zone.  Notice that the pkg publisher output does not say (syspub) - the kernel zone and the global zone can even use different publishers for the solaris repository.  As SRU's and updates start to roll out you will see that you can independently update the global zone and the kernel zones on it.

root@myfirstkz:~# pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online F http://internal-ips-repo.example.com/

Because a kernel zone runs its own kernel, it considers itself to be a global zone.

root@myfirstkz:~# zonename
global

The root disk that I mentioned above shows up at c1d0.

root@myfirstkz:~# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1d0 <kz-vDisk-ZVOL-16.00GB>
          /kz-devices@ff/disk@0
Specify disk (enter its number): ^D

And the anet shows up as net0 using physical device zvnet0.

root@myfirstkz:~# dladm show-phys
LINK              MEDIA                STATE      SPEED  DUPLEX    DEVICE
net0              Ethernet             up         1000   full      zvnet0

Let's jump on the console and see what happens when bad things happen...

root@myfirstkz:~# logout

[Connection to zone 'myfirstkz' pts/3 closed]

root@vzl-212:~# zlogin -C myfirstkz
[Connected to zone 'myfirstkz' console]

myfirstkz console login: root
Password: 
Apr 18 23:47:06 myfirstkz login: ROOT LOGIN /dev/console
Last login: Fri Apr 18 23:32:28 on kz/term
Oracle Corporation      SunOS 5.11      11.2    April 2014
root@myfirstkz:~# dtrace -wn 'BEGIN { panic() }'
dtrace: description 'BEGIN ' matched 1 probe

panic[cpu0]/thread=c4001afbd720: dtrace: panic action at probe dtrace:::BEGIN (ecb c400123381e0)

000002a10282acd0 dtrace:dtrace_probe+c54 (252acb8f029b3, 0, 0, 33fe, c4001b75e000, 103215b2)
  %l0-3: 0000c400123381e0 0000c40019b82340 00000000000013fc 0000c40016889740
  %l4-7: 0000c4001bc00000 0000c40019b82370 0000000000000003 000000000000ff00
000002a10282af10 dtrace:dtrace_state_go+4ac (c40019b82340, 100, 0, c40019b82370, 16, 702a7040)
  %l0-3: 0000000000030000 0000000010351580 0000c4001b75e000 00000000702a7000
  %l4-7: 0000000000000000 0000000df8475800 0000000000030d40 00000000702a6c00
000002a10282aff0 dtrace:dtrace_ioctl+ad8 (2c, 612164be40, 2a10282bacc, 202003, c400162fcdc0, 64747201)
  %l0-3: 000000006474720c 0000c40019b82340 000002a10282b1a4 00000000ffffffff
  %l4-7: 00000000702a6ee8 00000000702a7100 0000000000000b18 0000000000000180
000002a10282b8a0 genunix:fop_ioctl+d0 (c40019647a40, 0, 612164be40, 202003, c400162fcdc0, 2a10282bacc)
  %l0-3: 000000006474720c 0000000000202003 0000000001374f2c 0000c40010d84180
  %l4-7: 0000000000000000 0000000000000000 00000000000000c0 0000000000000000
000002a10282b970 genunix:ioctl+16c (3, 6474720c, 612164be40, 3, 1fa5ac, 0)
  %l0-3: 0000c4001a5ea958 0000000010010000 0000000000002003 0000000000000000
  %l4-7: 0000000000000003 0000000000000004 0000000000000000 0000000000000000

syncing file systems... done
dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel sections: zfs
 0:04  90% done (kernel)
 0:05 100% done (zfs)
100% done: 127783 (kernel) + 12950 (zfs) pages dumped, dump succeeded
rebooting...
Resetting...

[NOTICE: Zone rebooting]
NOTICE: Entering OpenBoot.
NOTICE: Fetching Guest MD from HV.
NOTICE: Starting additional cpus.
NOTICE: Initializing LDC services.
NOTICE: Probing PCI devices.
NOTICE: Finished PCI probing.


SPARC T5-4, No Keyboard
Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.36.0, 2.0000 GB memory available, Serial #723535045.
Ethernet address 0:0:0:0:0:0, Host ID: 2b2044c5.



Boot device: disk0  File and args: 
SunOS Release 5.11 Version 11.2 64-bit
Copyright (c) 1983, 2014, Oracle and/or its affiliates. All rights reserved.
Hostname: myfirstkz
Apr 18 23:48:44 myfirstkz savecore: System dump time: Fri Apr 18 23:47:42 2014
Apr 18 23:48:44 myfirstkz savecore: Saving compressed system crash dump files in directory /var/crash

myfirstkz console login: Apr 18 23:49:02 myfirstkz savecore: Decompress all crash dump files with '(cd /var/crash && savecore -v 0)' or individual files with 'savecore -vf /var/crash/vmdump{,-<secname>}.0'

SUNW-MSG-ID: SUNOS-8000-KL, TYPE: Defect, VER: 1, SEVERITY: Major
EVENT-TIME: Fri Apr 18 23:49:07 CDT 2014
PLATFORM: SPARC-T5-4, CSN: unknown, HOSTNAME: myfirstkz
SOURCE: software-diagnosis, REV: 0.1
EVENT-ID: f4c0d684-da80-425f-e45c-97bd0239b154
DESC: The system has rebooted after a kernel panic.

After disconnecting from the console (~.) I was back at the global zone root prompt.  The global zone didn't panic - the kernel zone did.

root@vzl-212:~# uptime; zlogin myfirstkz uptime
  9:53pm  up  8:03,  2 users,  load average: 0.03, 0.12, 0.08
 11:52pm  up 5 min(s),  0 users,  load average: 0.04, 0.26, 0.15

That's the end of this tour.  Thanks for coming, and please come again!

Comments:

Do you know why this error is coming
root@solpoc2:~# zoneadm -z kernel verify
Platform does not support the kernel-zone brand.
zoneadm: zone kernel failed to verify
root@solpoc2:~#
-----------------------------------------------------------
root@solpoc2:~# zonecfg -z kernel info
zonename: kernel
brand: solaris-kz
autoboot: false
autoshutdown: shutdown
bootargs:
pool:
scheduling-class:
hostid: 0x167cdb9
tenant:
anet:
lower-link: auto
allowed-address not specified
configure-allowed-address: true
defrouter not specified
allowed-dhcp-cids not specified
link-protection: mac-nospoof
mac-address: auto
mac-prefix not specified
mac-slot not specified
vlan-id not specified
priority not specified
rxrings not specified
txrings not specified
mtu not specified
maxbw not specified
rxfanout not specified
vsi-typeid not specified
vsi-vers not specified
vsi-mgrid not specified
etsbw-lcl not specified
cos not specified
evs not specified
vport not specified
id: 0
device:
match not specified
storage: dev:/dev/zvol/dsk/rpool/VARSHARE/zones/kernel/disk0
id: 0
bootpri: 0
capped-memory:
physical: 2G
suspend:
path: /system/zones/kernel/suspend
storage not specified

Posted by guest on May 06, 2014 at 03:59 PM CDT #

In my earlier post, linked in the first paragraph:

https://blogs.oracle.com/zoneszone/entry/install_a_kernel_zone_in

I mentioned that there were some prerequisites:

http://docs.oracle.com/cd/E36784_01/html/E37629/gnwoi.html#scrolltoc

Unfortunately, the Intel and AMD prerequisites got too watered down in the beta docs. They should say something more like:

For x86, kernel zones require Nehalem+(Intel) or Barcelona+(AMD) based x86 machines with CPU virtualization (e.g., VT-x) enabled in the BIOS. Specifically, kernel zones on x86 requires Extended/Nested Page Table support, also referred to as EPT, NPT, or RVI (Rapid Virtualization Indexing).

So, for your situation, zoneadm verify is failing because your hardware does not support kernel zones. This could be because it does not have a new enough CPU, (x86) some features are disabled in the BIOS, or (sparc) you need a firmware update.

Posted by Mike Gerdts on May 06, 2014 at 04:15 PM CDT #

Hi,

thanks for this informative blog entry.

Do you know if there are plans to add also support for older SPARC models?

I would be interested in running this on T2/T2+ CPUs...

Thanks!

Kind regards,

Marco

Posted by guest on May 07, 2014 at 06:31 AM CDT #

Sorry, but kernel zones will not be supported on older SPARC hardware. The T4 CPU is the first one with the required hardware support.

Posted by Mike Gerdts on May 07, 2014 at 06:43 AM CDT #

How can you tell if your kernel zone has dedicated CPUs? I have a Sparc_T4-1 system with 64 CPUs and someone has set up a kernel zone on it. The information from zonecfg -z myzone info says:

1123 dedicated-cpu:
1124 ncpus: 8
1125 cpus not specified
1126 cores not specified
1127 sockets not specified

So, I'm not sure if this means:
A) The kernel zone gets 8 random CPUs, chosen as the firmware sees fit to distribute them or
B) This kernel zone has a block assigned to it, like CPUs 56-63. Since it says 'cpus not specified' I find this questionable.

Thanks for any help you can provide.

Posted by Adam on May 22, 2014 at 05:11 AM CDT #

Adam,

I think your question is answered at in the zonecfg(1M) man page. As I mentioned in another post, the online man pages are newer than those included in the beta.

http://docs.oracle.com/cd/E36784_01/html/E36871/zonecfg-1m.html#REFMAN1Mzonecfg-1m

See the paragraphs under "Automatically chosen cpus resources".

Posted by Mike Gerdts on May 22, 2014 at 09:36 AM CDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

I'm a Principal Software Engineer in the Solaris Zones team. In this blog, I'll talk about zones, how they interact with other parts of Solaris, and related topics.

Search

Categories
Archives
« February 2015
SunMonTueWedThuFriSat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
21
22
23
24
25
26
27
28
       
       
Today