Wednesday Apr 11, 2012

Allow JMX connection on JVM 1.6.x

While trying to monitor a JVM on a remote system using visualvm the activation of JMX gave me some challenges. Dr Google and my employers documentation quickly revealed some -D opts needed for JMX, but strangely it only worked for a Solaris 10 system (my setup: MacOS laptop monitoring SPARC Solaris based JVMs) On S11 with the same opts I saw that "my" JVM listening on port 3000 (which I chose for JMX), but visualvm was not able to get a connection. Finally I found out that at least my S11 installation needed an explicit setting of the RMI host name. This what finally worked:


        -Dcom.sun.management.jmxremote=true \
        -Dcom.sun.management.jmxremote.ssl=false \
        -Dcom.sun.management.jmxremote.authenticate=false \
        -Dcom.sun.management.jmxremote.port=3000 \
        -Djava.rmi.server.hostname=s11name.us.oracle.com \


Maybe this post saves someone else the time I spent on research 

Monday Mar 26, 2012

T4 Performance Counters explained

Now that T4 is out for a few months some people might have wondered what details of the new pipeline you can monitor. A "cpustat -h" lists a lot of events that can be monitored, and only very few are self-explanatory. I will try to give some insight on all of them, some of these "PIC events" require an in-depth knowledge of T4 pipeline. Over time I will try to explain these, for the time being these events should simply be ignored.

Notes:

  • some counters changed from tape-out 1.1 (*only* used in the T4 beta program) to tape-out 1.2 (used in the systems shipping today) The table only lists the tape-out 1.2 counters
  • I marked some counters "no good use known". That means there is no sensible application area (yet?) known. Some counters become useless while the overall design is being developed.  
  • check back often, I will every once in while edit this page. The use cases for these counters may change over time while we learn.

pic name (cpustat)

Prose Comment

Sel-pipe-drain-cycles,
Sel-0-[wait|ready],
Sel-[1,2]

Sel-0-wait counts cycles a strand waits to be selected. Some reasons can be counted in detail; these are:

  • Sel-0-ready: Cycles a strand was ready but not selected, that can signal pipeline oversubscription
  • Sel-pipe-drain-cycles: number of cycles a thread waits after a branch mispredict with correct path instruction at select for the mispredict to resolve
Sel-1,2 count how often one strand only selected one or two µop

Pick-any, Pick-[0|1|2|3]

Cycles one, two, three, no or at least one instruction or µop is picked

Instr_FGU_crypto, Instr_ld,
  Instr_st, SPR_ring_ops,
  Instr_other

Number of instructions executed on a strand, distinguished by type.

Instr_all

total number of instructions executed on that strand

Sw_count_intr

Nr of S/W count instructions on that vcpu (sethi %hi(fc000),%g0 (whatever that is)) 

Atomics

nr of atomic ops, which are LDSTUB/a, CASA/XA, and SWAP/A

SW_prefetch

Nr of PREFETCH or PREFETCHA instructions

Block_ld_st

Block loads or store on that vcpu

IC_miss_nospec,
IC_miss_[L2_or_L3|local|remote]\
_hit_nospec

Various I$ misses, distinguished by where they hit. All of these count per thread, but only primary events: T4 counts only the first occurence of an I$ miss on a core for a certain instruction. If one strand misses in I$ this miss is counted, but if a second strand on the same core misses while the first miss is being resolved, that second miss is not counted
This flavour of I$ misses counts only misses that are caused by instruction that really commit (note the "_nospec")

BTC_miss

Branch target cache miss

ITLB_miss

ITLB misses (synchronously counted)

ITLB_miss_asynch

dto. but asynchronously

[I|D]TLB_fill_\
[8KB|64KB|4MB|256MB|2GB|trap]

H/W tablewalk events that fill ITLB or DTLB with translation for the corresponding page size. The “_trap” event occurs if the HWTW was not able to fill the corresponding TLB

IC_mtag_miss,
IC_mtag_miss_\
[ptag_hit|ptag_miss|\
ptag_hit_way_mismatch]

I$ micro tag misses, with some options for drill down

Fetch-0, Fetch-0-all

fetch-0 counts nr of cycles nothing was fetched for this particular strand, fetch-0-all counts cycles nothing was fetched for all strands on a core

Instr_buffer_full

Cycles the instruction buffer for a strand was full, thereby preventing any fetch

BTC_targ_incorrect

Counts all occurences of wrongly predicted branch targets from the BTC

[PQ|ROB|LB|ROB_LB|SB|\
ROB_SB|LB_SB|RB_LB_SB|\
DTLB_miss]\
_tag_wait

These counters monitor pipeline behaviour therefore they are not strand specific:

  • PQ_...: cycles Rename stage waits for a Pick Queue tag (might signal memory bound workload for single thread mode)
  • ROB_...: cycles Select stage waits for a ROB (ReOrderBuffer) tag
  • LB_...: cycles Select stage waits for a Load Buffer tag
  • SB_...: cycles Select stage waits for Store Buffer tag
  • combinations of the above are allowed, although some of these events can overlap, the counter will only be incremented once per cycle if any of these occur
  • DTLB_...: cycles load or store instructions wait at Pick stage for a DTLB miss tag

[ID]TLB_HWTW_\
[L2_hit|L3_hit|L3_miss|all]

Counters for HWTW accesses caused by either DTLB or ITLB misses. Canbe further detailed by where they hit

IC_miss_L2_L3_hit,
IC_miss_local_remote_remL3_hit,
IC_miss

I$ prefetches that were dropped because they either miss in L2$ or L3$
This variant counts misses regardless if the causing instruction commits or not

DC_miss_nospec, DC_miss_[L2_L3|local|remote_L3]\
_hit_nospec

D$ misses either in general or detailed by where they hit
cf. the explanation for the IC_miss in two flavours for an explanation of _nospec and the reasoning for two DC_miss counters

DTLB_miss_asynch

counts all DTLB misses asynchronously, there is no way to count them synchronously

DC_pref_drop_DC_hit, SW_pref_drop_[DC_hit|buffer_full]

L1-D$ h/w prefetches that were dropped because of a D$ hit, counted per core. The others count software prefetches per strand

[Full|Partial]_RAW_hit_st_[buf|q]

Count events where a load wants to get data that has not yet been stored, i. e. it is still inside the pipeline. The data might be either still in the store buffer or in the store queue. If the load's data matches in the SB and in the store queue the data in buffer takes precedence of course since it is younger

[IC|DC]_evict_invalid,
[IC|DC|L1]_snoop_invalid,
[IC|DC|L1]_invalid_all

Counter for invalidated cache evictions per core

St_q_tag_wait

Number of cycles pipeline waits for a store queue tag, of course counted per core

Data_pref_[drop_L2|drop_L3|\
hit_L2|hit_L3|\
hit_local|hit_remote]

Data prefetches that can be further detailed by either why they were dropped or where they did hit

St_hit_[L2|L3],
St_L2_[local|remote]_C2C,
St_local, St_remote

Store events distinguished by where they hit or where they cause a L2 cache-to-cache transfer, i.e. either a transfer from another L2$ on the same die or from a different die

DC_miss, DC_miss_\
[L2_L3|local|remote]_hit

D$ misses either in general or detailed by where they hit
cf. the explanation for the IC_miss in two flavours for an explanation of _nospec and the reasoning for two DC_miss counters

L2_[clean|dirty]_evict

Per core clean or dirty L2$ evictions. L2_clean_evict can signal an instruction fetcher bottleneck: instruction can be considered clean because they don't change. If a high number of clean eviction occurs one might be evicting instructions from L2$ that have not been fetched fast enough

L2_fill_buf_full,
L2_wb_buf_full,
L2_miss_buf_full

Per core L2$ buffer events, all count number of cycles that this state was present

no good use known

L2_pipe_stall

Per core cycles pipeline stalled because of L2$

no good use known

Branches

Count branches (Tcc, DONE, RETRY, and SIT are not counted as branches)

Br_taken

Counts taken branches (Tcc, DONE, RETRY, and SIT are not counted as branches)

Br_mispred,
Br_dir_mispred,
Br_trg_mispred,
Br_trg_mispred_\
[far_tbl|indir_tbl|ret_stk]

Counter for various branch misprediction events. 

Cycles_user

counts cycles, attribute setting hpriv, nouser, sys controls addess space to count in

Commit-[0|1|2],
Commit-0-all,
Commit-1-or-2

Number of times either no, one, or two µops commit for a strand. Commit-0-all counts number of times no µop commits for the whole core

The various types of instructions that are monitored by the Instr_* counters are:

  • FGU operations are floating point and crypto operations 
  • Load operations: LDD, LDDA_0X14, LDDA_0X15, LDDA_0X1C, LDDA_0X1D, LDDA_0X22, LDDA_0X23, LDDA_0X26, LDDA_0X27,  LDDA_0X2A, LDDA_0X2B, LDDA_0X2E, LDDA_0X2F, LDDA_0X80, LDDA_0X82, LDDA_0X88, LDDA_0X8B, LDDA_0XE2, LDDA_0XE3, LDDA_0XEA, LDDA_0XEB, LDDF, LDDFA_0X16, LDDFA_0X1E, LDDFA_0X4, LDDFA_0X80, LDDFA_0X88, LDDFA_0X89, LDDFA_0X8A, LDDFA_0XC, LDDFA_0XD0, LDDFA_0XDA, LDDFA_0XF0, LDDFA_0XF8, LDF, LDFA_0X15,LDFA_0X1D, LDFA_0X4, LDFA_0X80, LDFA_0X81, LDFA_0X82, LDFA_0X83, LDFA_0X88, LDFA_0X89, LDFA_0X8B,LDFA_0XC, LDFSR, LDSB, LDSBA_0X14, LDSBA_0X15, LDSBA_0X1C, LDSBA_0X1D, LDSBA_0X31, LDSBA_0X4, LDSBA_0X80, LDSBA_0X81, LDSBA_0X88, LDSBA_0X8B, LDSBA_0XC, LDSH, LDSHA_0X14, LDSHA_0X1C, LDSHA_0X1D, LDSHA_0X36, LDSHA_0X38, LDSHA_0X39, LDSHA_0X3E, LDSHA_0X80, LDSHA_0X82, LDSHA_0X83, LDSHA_0X88, LDSHA_0X8A, LDSHA_0X8B, LDSW, LDSWA_0X14, LDSWA_0X15, LDSWA_0X1C, LDSWA_0X1D, LDSWA_0X36, LDSWA_0X80, LDSWA_0X83, LDSWA_0X88, LDSWA_0X89, LDSWA_0X8B, LDUBA_0X14, LDUBA_0X15, LDUBA_0X1C, LDUBA_0X1D, LDUBA_0X30, LDUBA_0X38, LDUBA_0X4, LDUBA_0X80, LDUBA_0X88, LDUBA_0X8B, LDUH, LDUHA_0X14, LDUHA_0X15, LDUHA_0X1C, LDUHA_0X1D, LDUHA_0X39, LDUHA_0X3E, LDUHA_0X4, LDUHA_0X80, LDUHA_0X83, LDUHA_0X88, LDUHA_0XC, LDUW, LDUWA_0X14, LDUWA_0X15, LDUWA_0X1C, LDUWA_0X1D, LDUWA_0X31, LDUWA_0X39, LDUWA_0X3E, LDUWA_0X4, LDUWA_0X80, LDUWA_0X81, LDUWA_0X82, LDUWA_0X83, LDUWA_0X88, LDUWA_0X89, LDUWA_0X8A, LDUWA_0X8B, LDUWA_0XC, LDX, LDXA_0X14, LDXA_0X15, LDXA_0X1C, LDXA_0X1D, LDXA_0X4, LDXA_0X41, LDXA_0X63, LDXA_0X80, LDXA_0X81, LDXA_0X82, LDXA_0X83, LDXA_0X88, LDXA_0X89, LDXA_0X8B, LDXA_0XC, RDASI, RDASR, RDCCR, RDFPRS, RDGSR, RDHPR, RDPR, RDTICK, RDY
  • Store operations: FLUSH, MEMBAR except MEMBAR #Lookaside and MEMBAR #LoadStore, STBA_0X14, STBA_0X15, STBA_0X1C, STBA_0X1D, STBA_0X30, STBA_0X31, STBA_0X36, STBA_0X38, STBA_0X39, STBA_0X3E, STBA_0X4, STBA_0X80, STBA_0X81, STBA_0X88, STBA_0X89, STBA_0XC, STBAR, STD, STDA_0X14, STDA_0X15, STDA_0X1C, STDA_0X1D, STDA_0X27, STDA_0X2F, STDA_0X4, STDA_0X80, STDA_0X88, STDA_0XEA, STDA_0XEB, STDF, STDFA_0X17,STDFA_0X4, STDFA_0X80, STDFA_0X88, STDFA_0X89, STDFA_0XC0, STDFA_0XC2, STDFA_0XC4, STDFA_0XC8,STDFA_0XCA, STDFA_0XCB, STDFA_0XCC, STDFA_0XD1, STDFA_0XD3, STDFA_0XD9, STDFA_0XE0, STDFA_0XF0,STDFA_0XF8, STDFA_0XF9, STF, STFA_0X14, STFA_0X15, STFA_0X1C, STFA_0X4, STFA_0X80, STFA_0X81, STFA_0X88,STFA_0X89, STFA_0XC, STFSR, STHA_0X14, STHA_0X15, STHA_0X1C, STHA_0X1D, STHA_0X30, STHA_0X31,STHA_0X4, STHA_0X80, STHA_0X88, STHA_0XC, STW, STWA_0X14, STWA_0X15, STWA_0X1C, STWA_0X1D, STWA_0X38, STWA_0X4, STWA_0X80, STWA_0X81, STWA_0X88, STWA_0XC, STX, STXA_0X14, STXA_0X15, STXA_0X1C, STXA_0X1D, STXA_0X22, STXA_0X23, STXA_0X26, STXA_0X27, STXA_0X2A, STXA_0X2B, STXA_0X2E, STXA_0X2F, STXA_0X41, STXA_0X73, STXA_0X80, STXA_0X81, STXA_0X88, STXA_0XC, STXA_0XE2, STXA_0XE3, STXA_0XEA, STXA_0XEB, STXA_0XF2, STXA_0XF3, STXA_0XFA, STXA_0XFB
  • SPR ring operations: LDXA_0X20, LDXA_0X21, LDXA_0X25, LDXA_0X45, LDXA_0X48, LDXA_0X49, LDXA_0X4C, LDXA_0X4E, LDXA_0X4F, LDXA_0X52, LDXA_0X54, LDXA_0X58, LDXA_0X64, LDXA_0X74, LDXA_0XB0, STXA_0X20, STXA_0X21,STXA_0X25, STXA_0X42, STXA_0X45, STXA_0X4C, STXA_0X4E, STXA_0X4F, STXA_0X50, STXA_0X52, STXA_0X54,STXA_0X57, STXA_0X58, STXA_0X5C, STXA_0X5F, STXA_0X64, STXA_0X72, STXA_0XB0, WRASI, WRASR, WRCCR,WRFPRS, WRGSR, WRHPR, WRPAUSE, WRPR, WRY, STICK_ENABLE
  • Other instructions: ADD, ADDCcc, ADDXCcc, ALLCLEAN, AND, ANDcc, ANDN, ANDNcc, CASA, CASXA, DONE, FLUSHW, HALT, INVALW, LDSTUB, LDSTUBA, MEMBAR #Lookaside, MEMBAR #LoadStore, MOVdTOX, MOVFN, MOVR, MOVxTOd, NOP, NORMALW, OR, ORcc, ORNcc, OTHERW, PREFETCH, PREFETCHA, RDCFR, RDPC, RESTORED, RETRY, SAVE, SAVED, SETHI_%G0, SIAM, SLLX, SRAX, SRLX, SUB, SUBcc, SUBCcc, SWAP, SWAPA, TADDcc, TADDccTV, TN, TSUBcc, TSUBccTV, WRCFR, XNOR, XNORcc, XOR, XORcc.

Tuesday Aug 25, 2009

Experiencing Linux on SPARC: LDOM@Gentoo revisited

The first (small) success!

Last time I worked on this I was left with close to nothing: neither debian nor the gentoo install CDs could boot (precisely: they did boot but couldn't mount the install image) Same problem on a native SPARC Niagara2 system (T5240). Only an old UltraSPARC-T1box was able to get further than booting a then useless CD, I could track it down to a squashfs mount issue and opened bug 279472 against gentoo install. After it has been resolved (it was caused by an incompatibility between squahfs and the kernel...) install-sparc64-minimal-20090817.iso does and the install works passed booting of the CD:

root@batoka-labor,linux-boot>ldm start gentoo
LDom gentoo started
root@batoka-labor,linux-boot>telnet 0 5000
Trying 0.0.0.0...
Connected to 0.
Escape character is '\^]'.

Connecting to console "gentoo" in group "gentoo" ....
Press ~? for control options ..


T5440, No Keyboard
Copyright 2008 Sun Microsystems, Inc.  All rights reserved.
OpenBoot 4.30.0, 4096 MB memory available, Serial #83541553.
Ethernet address 0:14:4f:fa:be:31, Host ID: 84fabe31.



{0} ok
{0} ok boot cdrom
Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args:
SILO Version 1.4.14
\\

----------------------------------------------------
Welcome to the Gentoo/Linux SPARC64 weekly InstallCD
----------------------------------------------------

boot: gentoo cdroot=/dev/vdiskb1
Allocated 64 Megs of memory at 0x40000000 for kernel
Loaded kernel version 2.6.28
Loading initial ramdisk (1847651 bytes at 0x6800000 phys, 0x40C00000 virt)...
\\

...a lot of Linux kernel humble-mumble omitted....

>> Activating mdev
>> Making tmpfs for /newroot
>> Looking for the cdrom
>> Attempting to mount media:- /dev/vdiskb1
>> Media found on /dev/vdiskb1
>> Determining root device...
>> Determining looptype ...
>> Mounting squashfs filesystem
>> Copying read-write image contents to tmpfs
>> No cdupdate.sh script found, skipping...
>> Booting (initramfs)..
INIT: version 2.86 booting

Gentoo Linux; http://www.gentoo.org/
 Copyright 1999-2007 Gentoo Foundation; Distributed under the GPLv2

 \* Mounting proc at /proc ...                                             [ ok ] \* Mounting sysfs at /sys ...                                             [ ok ] \* Mounting /dev ...                                                      [ ok ] \* Starting udevd ...                                                     [ ok ] \* Populating /dev with existing devices through uevents ...              [ ok ] \* Waiting for uevents to be processed ...                                [ !! ] \* Mounting devpts at /dev/pts ...                                        [ ok ] \* Mounting local filesystems ...                                         [ ok ] \* Mounting USB device filesystem (usbfs) ...                             [ ok ] \* Activating (possible) swap ...                                         [ ok ] \* Setting system clock using the hardware clock [UTC] ...                [ ok ] \* Configuring kernel parameters ...                                      [ ok ] \* Updating environment ...                                               [ ok ] \* Cleaning /var/lock, /var/run ...                                       [ ok ] \* Updating inittab ...                                                   [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Setting hostname to livecd ...                                         [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Starting lo
 \*   Bringing up lo
 \*     127.0.0.1/8
                                                                          [ ok ] \*   Adding routes
 \*     127.0.0.0/8 ...                                                    [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Initializing random number generator ...                               [ ok ]INIT: Entering runlevel: 3
 \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Starting syslog-ng ...                                                 [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Hardware detection started ...
 \* Detected 8 active T2 (Niagara2) CPU(s) of 8 total
                                                                          [ ok ] \* Unpacking firmware ...tar: ./ql2400_fw.bin.4.02.02-MID: time stamp 2009-08-17 16:32:14 is 294342652.02588504 s in the future
tar: ./ql2100_fw.bin.1.19.38-TP: time stamp 2009-08-17 16:32:14 is 294342652.020644702 s in the future
tar: ./ql2400_fw.bin.4.00.26-IP: time stamp 2009-08-17 16:32:14 is 294342652.002011365 s in the future
tar: ./whiteheat_loader.fw: time stamp 2009-08-17 18:18:59 is 294349057.001625306 s in the future
tar: ./edgeport/boot2.fw: time stamp 2009-08-17 18:18:59 is 294349057.001033736 s in the future
tar: ./edgeport/boot.fw: time stamp 2009-08-17 18:18:59 is 294349057.000764341 s in the future
tar: ./edgeport/down2.fw: time stamp 2009-08-17 18:18:59 is 294349056.999340397 s in the future
tar: ./edgeport/down3.bin: time stamp 2009-08-17 18:18:59 is 294349056.998421595 s in the future
tar: ./edgeport/down.fw: time stamp 2009-08-17 18:18:59 is 294349056.997250881 s in the future
tar: ./edgeport: time stamp 2009-08-17 18:18:59 is 294349056.997046251 s in the future
tar: ./ql2400_fw.bin.4.00.27-IP: time stamp 2009-08-17 16:32:14 is 294342651.978127465 s in the future
tar: ./ql2400_fw.bin.4.00.22-IP: time stamp 2009-08-17 16:32:14 is 294342651.619182933 s in the future
tar: ./emi26/loader.fw: time stamp 2009-08-17 18:18:59 is 294349056.618648762 s in the future
tar: ./emi26/firmware.fw: time stamp 2009-08-17 18:18:59 is 294349056.61824269 s in the future
tar: ./emi26/bitstream.fw: time stamp 2009-08-17 18:18:59 is 294349056.608059889 s in the future
tar: ./emi26: time stamp 2009-08-17 18:18:59 is 294349056.607868344 s in the future
tar: ./ql2322_fw.bin.3.03.18: time stamp 2009-08-17 16:32:14 is 294342651.596939813 s in the future
tar: ./ql2322_fw.bin.3.03.20-IPX: time stamp 2009-08-17 16:32:14 is 294342651.585373532 s in the future
tar: ./keyspan_pda/xircom_pgs.fw: time stamp 2009-08-17 18:18:59 is 294349056.585084344 s in the future
tar: ./keyspan_pda/keyspan_pda.fw: time stamp 2009-08-17 18:18:59 is 294349056.584870918 s in the future
tar: ./keyspan_pda: time stamp 2009-08-17 18:18:59 is 294349056.584716098 s in the future
tar: ./whiteheat.fw: time stamp 2009-08-17 18:18:59 is 294349056.582405129 s in the future
tar: ./ql6312_fw.bin.3.03.18: time stamp 2009-08-17 16:32:15 is 294342652.572845455 s in the future
tar: ./ql2100_fw.bin.1.17.38: time stamp 2009-08-17 16:32:14 is 294342651.566923053 s in the future
tar: ./emi62/loader.fw: time stamp 2009-08-17 18:18:59 is 294349056.566665534 s in the future
tar: ./emi62/spdif.fw: time stamp 2009-08-17 18:18:59 is 294349056.564197985 s in the future
tar: ./emi62/midi.fw: time stamp 2009-08-17 18:18:59 is 294349056.561709436 s in the future
tar: ./emi62/bitstream.fw: time stamp 2009-08-17 18:18:59 is 294349056.551455273 s in the future
tar: ./emi62: time stamp 2009-08-17 18:18:59 is 294349056.551267465 s in the future
tar: ./ql2400_fw.bin.4.00.16: time stamp 2009-08-17 16:32:14 is 294342651.206974394 s in the future
tar: ./ql2400_fw.bin.4.00.18-IP: time stamp 2009-08-17 16:32:14 is 294342651.189752357 s in the future
tar: ./sun/cassini.bin: time stamp 2009-08-17 18:18:59 is 294349056.189298893 s in the future
tar: ./sun: time stamp 2009-08-17 18:18:59 is 294349056.189123841 s in the future
tar: ./ti_5052.fw: time stamp 2009-08-17 18:18:59 is 294349056.187469867 s in the future
tar: ./ql2300_fw.bin.3.03.20-IPX: time stamp 2009-08-17 16:32:14 is 294342651.176112833 s in the future
tar: ./ql2300_fw.bin.3.03.18: time stamp 2009-08-17 16:32:14 is 294342651.166202945 s in the future
tar: ./ql2500_fw.bin.4.02.02-MID: time stamp 2009-08-17 16:32:14 is 294342651.146737233 s in the future
tar: ./ti_3410.fw: time stamp 2009-08-17 18:18:59 is 294349056.145086666 s in the future
tar: ./ql2200_fw.bin.2.02.08-TP: time stamp 2009-08-17 16:32:14 is 294342650.990960402 s in the future
tar: ./kaweth/new_code_fix.bin: time stamp 2009-08-17 18:18:59 is 294349055.990153646 s in the future
tar: ./kaweth/trigger_code.bin: time stamp 2009-08-17 18:18:59 is 294349055.989936921 s in the future
tar: ./kaweth/trigger_code_fix.bin: time stamp 2009-08-17 18:18:59 is 294349055.989728222 s in the future
tar: ./kaweth/new_code.bin: time stamp 2009-08-17 18:18:59 is 294349055.989515785 s in the future
tar: ./kaweth: time stamp 2009-08-17 18:18:59 is 294349055.989361296 s in the future
tar: ./ql2500_fw.bin.4.02.02: time stamp 2009-08-17 16:32:14 is 294342650.974604057 s in the future
tar: ./ql2400_fw.bin.4.00.23-IP: time stamp 2009-08-17 16:32:14 is 294342650.958950448 s in the future
tar: .: time stamp 2009-08-17 18:18:59 is 294349055.958335788 s in the future
                                                 [ ok ]
l\* Not Loading APM Bios support ...
 \* Not Loading ACPI support ...
 \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Network device eth0 detected, DHCP broadcasting for IP ...
 \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Starting portmap ...                                                   [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* ERROR:  cannot start nfsmount as rpc.statd could not start
 \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
 \* Caching service dependencies ...                                       [ ok ] \* Auto-scrambling root password for security ...                         [ ok ] \* One of the files in /etc/{conf.d,init.d} or /etc/rc.conf
 \* has a modification time in the future!
F\* Caching service dependencies ...                                       [ ok ] \* Starting local ...                                                     [ ok ]
Welcome to the Gentoo Linux Minimal Installation CD!

The root password on this system has been auto-scrambled for security.

If any ethernet adapters were detected at boot, they should be auto-configured
if DHCP is available on your network.  Type "net-setup eth0" to specify eth0 IP
address settings by hand.

Check /etc/kernels/kernel-config-\* for kernel configuration(s).
The latest version of the Handbook is always available from the Gentoo web
site by typing "links http://www.gentoo.org/doc/en/handbook/handbook.xml".

To start an ssh server on this system, type "/etc/init.d/sshd start".  If you
need to log in remotely as root, type "passwd root" to reset root's password
to a known value.

Please report any bugs you find to http://bugs.gentoo.org. Be sure to include
detailed information about how to reproduce the bug you are reporting.
Thank you for using Gentoo Linux!

livecd ~ #

Ladies and Gentlemen: Gentoo install environment is up and running.

Next I'll follow the gentoo docs to the letter and install to the local (virtual) disk.

Wednesday Jul 29, 2009

Experiencing Linux on SPARC: LDOM@Gentoo

On it goes, I'll begin with gentoo, latest Minimal CD I could put my hands on (install-sparc64-minimal-20090727.iso). After binding/starting the LDOM one boots it nearly as usual:

root@batoka-labor,linux-boot>telnet 0 5000
Trying 0.0.0.0...
Connected to 0.
Escape character is '\^]'.

Connecting to console "gentoo" in group "gentoo" ....
Press ~? for control options ..

{0} ok boot cdrom
Boot device: /virtual-devices@100/channel-devices@200/disk@1  File and args:
SILO Version 1.4.14
\\

----------------------------------------------------
Welcome to the Gentoo/Linux SPARC64 weekly InstallCD
----------------------------------------------------

boot: gentoo cdroot=/dev/vdiskb1

It took me half a day to find out about the "cdroot" option and its usage. It became necessary because the gentoo kernel only checks "normal" CDROM drive locations.

After a lot of Linux kernel boot messages, it runs in an error:

>> Making tmpfs for /newroot
>> Looking for the cdrom
>> Attempting to mount media:- /dev/vdiskb1
>> Media found on /dev/vdiskb1
>> Determining root device...
>> Determining looptype ...
>> Mounting squashfs filesystem
mount: mounting /dev/loop0 on /newroot/mnt/livecd failed: Invalid argument
!! Failed to $1; failing back to the shell...

The kernel has only little more to say:

# dmesg|tail
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
fuse init (API version 7.10)
Intel(R) PRO/1000 Network Driver - version 7.3.21-k3-NAPI
Copyright (c) 1999-2006 Intel Corporation.
EXT4-fs: Update your userspace programs to mount using ext4
EXT4-fs: ext4dev backwards compatibility will go away by 2.6.31
ISO 9660 Extensions: Microsoft Joliet Level 3
ISO 9660 Extensions: RRIP_1991A
SQUASHFS: Mounting a different endian SQUASHFS filesystem on loop0
#

Trying the mount command from the command line fails too. Next try: same setup but use 2008.0 Gentoo minimal CD image:

>> Making tmpfs for /newroot
>> Looking for the cdrom
!! Media not found
>> No bootable medium found. Waiting for new devices...
>> Looking for the cdrom
!! Media not found
!! Could not find CD to boot, something else needed!
>> Determining root device...
!! Could not find the root block device in .
   Please specify another value or: press Enter for the same, type "shell" for a shell, or "q" to skip...
root block device() :: shell
To leave and try again just press <Ctrl>+D


BusyBox v1.7.4 (2008-06-12 10:01:44 UTC) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/bin/ash: can't access tty; job control turned off

Well, again no luck. This time the failure has any easy explanation: Looks like this CD image does not a vdisk driver in it, the  "/dev" directory does not list any vdisk devices...

I'll go check with Dr. Google once more, although seemingly no one has tried before.

 

Experiencing Linux on SPARC: The beginning...

Since Linux is an ongoing hype, and since I found some spare time I decided to explore an unknown land (at least to me): How can one install a whole Linux system on recent SPARC gear? And, to not reinvent the wheel, the platform shall be a Logical Domain.

The first research targeted the available distributions. The Linux Kernel itself does have LDOMs support since 2.6.2x (IIRC it was 2.6.23), but the Kernel by itself is of little to no use at all. (The support for the sun4v architecture has been in 2.6 for a longer time, but besides the sun4v support one has to have support for the vnet and vdisk drivers.) Ubuntu had dropped support for sun4v/SPARC systems, my employer does not maintain it's own distribution or extend the support of existing (commercial) distros, so the most important distributions (SuSE and Redhat) were no choice.

This left me with free distros, namely

  • Gentoo, which is a little freaky because one usually recomiles everything. But this recompilation makes it very "portable" and should provide optimal performance.
  • Debian, which is well known in Linux land for being able to run virtually everything that is based on silicon and is easy to administer in daily operation

(I will not explore LFS which is even too freaky for me. Well, maybe I'll try that later too...)

Testing ground was a Niagara 2 based system in Sun-internal lab, right now it's actually a T5440 hosting my playground LDOMs  (thanks to the simplicity of moving LDOMs around the actual system
doesn't matter much)

The only virtual services I used were: 

root@batoka-labor,linux-boot>ldm add-vsw net-dev=nxge0 primary-nxge0 primary
root@batoka-labor,linux-boot>ldm add-vds primary-vds primary

Both LDOMs had a straightforward setup (one core, 4G memory, virtual disk backed up by a ZFS volume):

root@batoka-labor,linux-boot>ldm create gentoo
root@batoka-labor,linux-boot>ldm set-vcpu 8 gentoo
root@batoka-labor,linux-boot>ldm set-mem 4G gentoo
root@batoka-labor,linux-boot>ldm add-vnet main primary-nxge0 gentoo
root@batoka-labor,linux-boot>ldm add-vdsdev /dev/zvol/rdsk/rpool/gentoo-root gentoo-root@primary-vds
root@batoka-labor,linux-boot>ldm add-vdisk rootdisk gentoo-root@primary gentoo

I switched back and forth between the different boot CDs by simply changing the physical device backing up the virtual device cdrom:

root@batoka-labor,linux-boot>ldm add-vdsdev /root/linux-boot/install-sparc64-minimal-20090713.iso gentoo-cd-20090713@primary-vds
root@batoka-labor,linux-boot>ldm add-vdisk cdrom gentoo-cd-20090713@primary gentoo

 (The debian guest had the same configuration)

About

Before Sun was acquired by Oracle I was about 12 yrs in pre-sales covering SPARC and Solaris. Today I work in a field role in Oracle Microelectronics and focus on SPARC performance, including working and presenting at customer sites all over EMEA

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today