Monday Mar 30, 2009

Sestinfo 2009 - Universidade Metodista

Last week the Sestinfo 2009 event happened. It is a set of presentations and mini-courses during the week at Metodista University, SP.

I had the opportunity to talk about OpenSolaris in a presentation on Thursday, and it was a four-hour course on Friday. Both were great. Thank you all for coming and congratulations for those who got a certification voucher!



[Read More]

Thursday Feb 12, 2009

OpenSolaris Zones Clusters VideoBlog

Here's a very interesting video talking about virtual clustering using Solaris Zones by Ellard Roush.

Enjoy It!

For more information on how to set up zones and Glassfish refer to:

http://blogs.sun.com/arbore/entry/glassfish_zone_zfs_isolation_over

[Read More]

Wednesday Feb 11, 2009

DTrace presentation at Campus-Party 2009!

On the last Friday of Campus-Party 2009, Pedro Reis and me (Marcelo Arbore) have performed a presentation together.
He talked about Zones, which is an O.S. abstraction for instantiating systems, allowing multiple applications to run isolated one from each other in the same system.

I showed some concepts and a brief demonstration on DTrace, which is a dynamic tracing framework for OpenSolaris that provides an infrastructure to allow administrators and developers to concisely answer arbitrary questions about the behavior of the operating systems and user programs (from the kernel to high level). The questions are written in a script C-like language called D.

Here you can see a blog post showing a DTrace demonstration:

http://blogs.sun.com/arbore/entry/basic_demonstration_of_dtrace_feature


[Read More]

Basic demonstration of DTrace feature of OpenSolaris

This is a demonstration, for a full material on DTrace please refer to the  Solaris DTrace  guide at: http://docs.sun.com/app/docs/doc/817-6223

D is a C-like event-oriented script that uses operational system instrumentation points as its events.
That points of instrumentation are called Probes.
You can list all the probes of your system using '-l' option.

# pfexec dtrace -l
ID      PROVIDER    MODULE                FUNCTION        NAME
1       dtrace                                            BEGIN
2       dtrace                                            END
3       dtrace                                            ERROR
4       python723   libpython2.4.so.1.0   PyEval_EvalFrame function-entry
5       python723   libpython2.4.so.1.0   dtrace_entry function-entry
[...]
71348   fbt         semsys                _info           return
71349   fbt         semsys                semexit         entry
71350   fbt         semsys                semexit         return
71351   syscall                           semsys          entry
71352   syscall                           semsys          return

You can see that I have 71352 places in my system that I can get statistics right now.

Every Probe in the system can be mapped by a 4-layer hierarchy:

provider : subsystem : function : probeName

More information on that can be found on chap 1 of the DTrace Guide: http://docs.sun.com/app/docs/doc/817-6223/chp-intro?a=view

What is important to understand now is that 'provider' is normally the name of module inside the kernel, 'subsystem' is a kernel subsystem or a user lib, 'function' is the function that we wanna inspect and 'probeName' is what point of that function we wanna get statistics. If I omit any member of this hierarchy the system will understand as '\*'(anything)

Provider 

The name of the DTrace provider that is publishing this probe. The provider name typically corresponds to the name of the DTrace kernel module that performs the instrumentation to enable the probe. 

Subsystem

If this probe corresponds to a specific program location, the name of the module in which the probe is located. This name is either the name of a kernel module or the name of a user library. 

Function 

If this probe corresponds to a specific program location, the name of the program function in which the probe is located. 

Name 

The final component of the probe name is a name that gives you some idea of the probe's semantic meaning, such as BEGIN or END.

So, let's say I wanna the entry point of every system call of the system:

syscall:::entry

The option '-n' is used to run the D script from the shell call, so this command should show the standard output for all syscall probes:

The standard output is to print the id of the CPU, the id of the process, the function name and the probe name:

# pfexec dtrace -n 'syscall:::entry{ }'
 CPU     ID               FUNCTION:NAME
  0      70861               ioctl:entry
  0      70861               ioctl:entry
  0      70999           sysconfig:entry
  0      70999           sysconfig:entry
  1      70767                read:entry
  1      70861               ioctl:entry
  1      70861               ioctl:entry
  [...]

This can be impressive, but just for a second. In the next moment you will want some way to aggregate this huge amount of information.

To aggregate it by syscall name and count how many times it was called:

# pfexec dtrace -n 'syscall:::entry{ @[probefunc]=count(); }'
\^C
  fstat                1
  getpid               1
  mmap                 1
  schedctl             1
  sigpending           1
  stat64               1
  sysconfig            3
  getgid               4
    [...]
  setcontext           24
  writev               39
  setitimer            47
  p_online             64
  clock_gettime        150
  read                 261
  write                301
  pollsys              335
  lwp_park             379
  ioctl                590

For more information on aggergations, refer to chap 9 of the Dtrace guide: http://docs.sun.com/app/docs/doc/817-6223/chp-aggs?a=view

You can short by execname too:


# pfexec dtrace -n 'syscall:::entry{ @[execname,probefunc]=count();}'
\^C
 dtrace               fstat                  1
  dtrace              lwp_park               1
  dtrace              lwp_sigmask            1
 [...]
  firefox-bin         read                   84
  Xorg                clock_gettime          98
  firefox-bin         ioctl                  132
  firefox-bin         pollsys                151
  rhythmbox           ioctl                  165
  rhythmbox           write                  169
  rhythmbox           lwp_park               230

One can say that so far, the only thing that we discover is that I'm listening to music right now, because rhythmbox is making syscalls like crazy.

So, let's say I wanna really understand what is going on when some process calls the syscall, let's say, ioctl. It wouldn't be perfect if we could just ask the process to say every function it enters?

Let's look at this script:

#!/sbin/dtrace -s

#pragma D option flowindent

syscall::ioctl:entry
{
self->follow = 1;
}
fbt:::
/self->follow/
{}
syscall::ioctl:return
/self->follow/
{
self->follow = 0;
exit(0);
}

Each probe description is an event.

Don't bother yourself with the first two lines. The first one is the interpreter, and the second is just a macro for spacing rightly the standard output.

The first will fire when the system enters in the system call ioctl. The second will fire every time the system changes the working function and the thread have a special thread variable follow. FBT is the function boundary provider, and it maps all the functions in the system. What is under the slashes is a filter, and it is called predicate. The last will fire every time a syscall ioctl returns with the thread variable.

Things are getting clearer now.

The flow gos like that:

The first process that calls ioctl will cause the system to put a thread variable named follow in that specific thread and print the standard output (probe details). The next functions that this thread calls will fire a probe, and the next, and the next, and all will print it details until we reach the return of the syscall. This probe will print itself, unset the thread variable and exit the dtrace program.

Without more delays, let's see the output:

# pfexec dtrace -s flw.d
dtrace: script 'flw.d' matched 60932 probes
CPU FUNCTION
 0  -> ioctl
 0    -> getf
 0      -> set_active_fd
 0      <- set_active_fd
 0    <- getf
 0    -> fop_ioctl
 0      -> crgetmapped
 0      <- crgetmapped
 0      -> spec_ioctl
 0        -> cdev_ioctl
 0          -> ddi_get_soft_state
 0          <- ddi_get_soft_state
 0          -> kmem_zalloc
 0            -> kmem_cache_alloc
 0              -> kmem_depot_alloc
 0              <- kmem_depot_alloc
 0              -> kmem_slab_alloc
 0              <- kmem_slab_alloc
 0            <- kmem_cache_alloc
 0          <- kmem_zalloc
 0          -> kmem_free
 0            -> kmem_cache_free
 0            <- kmem_cache_free
 0          <- kmem_free
 0        <- cdev_ioctl
 0      <- spec_ioctl
 0    <- fop_ioctl
 0    -> releasef
 0      -> clear_active_fd
 0      <- clear_active_fd
 0      -> cv_broadcast
 0      <- cv_broadcast
 0    <- releasef
 0  <- ioctl
 0  <= ioctl

  

Now you can search at http://src.opensolaris.org/source/ for each function and really understand what is happening when ioctl is called.

As we could imagine, ioctl deals with io control, some file descriptor configuration, in the middle we can see some memory allocation and of course, all of that are protected by mutual exclusions locks. All that can be seen in the output.

If you run this scripts some more times it won't be long until you get some result like that:

# pfexec dtrace -s flw.d
dtrace: script 'flw.d' matched 60932 probes
CPU FUNCTION
0  -> do_interrupt
 0    -> tlb_service
 0    <- tlb_service
 0    -> uppc_intr_enter
 0      -> uppc_setspl
 0      <- uppc_setspl
 0    <- uppc_intr_enter
 0    -> intr_thread_prolog
 0    <- intr_thread_epilog
 0  <- dispatch_hardint
 0  <- switch_sp_and_call
 0  <- do_interrupt
 0  -> sys_rtt_common
 0  <- sys_rtt_common
 0  -> do_interrupt
 0    -> tlb_service
 0    <- tlb_service
 0    -> uppc_intr_enter
 0      -> uppc_setspl
 0      <- uppc_setspl
 0    <- uppc_intr_enter
 0    -> intr_thread_prolog
 0    <- intr_thread_epilog
 0  <- dispatch_hardint
 0  <- switch_sp_and_call
 0  <- do_interrupt
 0  -> sys_rtt_common
 0  <- sys_rtt_common
 0  -> do_interrupt
 0    -> tlb_service
 0    <- tlb_service
 0    -> uppc_intr_enter
 0      -> uppc_setspl
 0      <- uppc_setspl
 0    <- uppc_intr_enter
 0    -> intr_thread_prolog
 0    <- intr_thread_epilog
 0  <- dispatch_hardint
 0  <- switch_sp_and_call
 0  <- do_interrupt
 0  -> sys_rtt_common
 0  <- sys_rtt_common
 0  -> do_interrupt
 0    -> tlb_service
 0    <- tlb_service
 0    -> uppc_intr_enter
 0      -> uppc_setspl
 0      <- uppc_setspl
 0    <- uppc_intr_enter
 0    -> intr_thread_prolog
 0    <- intr_thread_epilog
 0  <- dispatch_hardint
 0  <- switch_sp_and_call
 0  <- do_interrupt
 0  -> sys_rtt_common
 0  <- sys_rtt_common
 0  -> ioctl
 0    -> getf
 0      -> set_active_fd
 0      <- set_active_fd
 0    <- getf
 0    -> fop_ioctl
 0      -> crgetmapped
 0      <- crgetmapped
 0      -> nm_ioctl
 0        -> fop_ioctl
 0          -> crgetmapped
 0          <- crgetmapped
 0          -> fifo_ioctl
 0            -> fifo_strioctl
 0              -> strioctl
 0                -> job_control_type
 0                <- job_control_type
 0                -> strcopyout
 0                <- strcopyout
 0              <- strioctl
 0            <- fifo_strioctl
 0          <- fifo_ioctl
 0        <- fop_ioctl
 0      <- nm_ioctl
 0    <- fop_ioctl
 0    -> releasef
 0      -> clear_active_fd
 0      <- clear_active_fd
 0      -> cv_broadcast
 0      <- cv_broadcast
 0    <- releasef
 0  <- ioctl
 0  <= ioctl

Probably when ioctl was called, it needed to take some other thread out of the cpu. We can see some calls from the interrupt sequence. TLB is the table that maps virtual memory, and we can see some calls related to stack pointers and that cleary are evidencing context changes.

As we can see, Dtrace is a great way to find bugs, profile, do performance analyses/evaluation. It is also a great way to learn and unsderstand operational systems behavior.

So, Where we can go now?

Community at Opensolaris.org:
opensolaris.org/os/community/dtrace/

Adam Leventhal:
http://blogs.sun.com/ahl

BigAdmin resources:
www.sun.com/bigadmin/content/dtrace/

Sun's How To:
www.sun.com/software/solaris/howtoguides/dtracehowto.jsp

Solaris Internals:
http://www.solarisinternals.com

Please post a comment or send me a email if you have any doubts, suggestions or if you find something here that isn't that right. We're a community and your help is very welcome a encouraged. =)  [Read More]

Wednesday Feb 04, 2009

Glassfish over a Zone over OpenSolaris

Hi all!

Today I'll try to show you guys how to create a zone in OpenSolaris and install a glassfish application server on the top of it.

First of all, you need a native installation of Solaris10/OpenSololaris or a virtual machine with one of these installed. Here you can find how to set up a vm with OpenSolaris.

You can refer to the Zones Administration Guide too.

To see the zones on the system:

# zoneadm list -vi
ID  NAME     STATUS    PATH   BRAND    IP
0   global   running   /      native   shared

Let's configure one more called glasszone for us:

zonecfg -z glasszone
glasszone: No such zone configuredUse 'create' to begin configuring a new zone.
zonecfg:glasszone> create
zonecfg:glasszone> set zonepath=/export/home/glasszone
zonecfg:glasszone> set autoboot=true

Zonepath is the place that solaris will create the filesystem for the new zone. If you don't want your zone to start running automatically at the boot time, you can set autoboot=false or just leave it.

Now, let's add a virtual network interface, commit the zone, install it on the system, boot and login( -C option is used in the first time you start the zone to configure the hostname, dns server, root password and some other stuff).

zonecfg:glasszone> add net
zonecfg:glasszone:net> set address=10.0.2.20/24
zonecfg:glasszone:net> set physical=e1000g0
zonecfg:glasszone:net> set defrouter=10.0.2.2
zonecfg:glasszone:net> end
zonecfg:glasszone> commit
zonecfg:glasszone> exit
# zoneadm -z glasszone install
# zoneadm -z glasszone boot
# zlogin -C glasszone

The address is the ip of your subnet that the zone will use, and 24 is the number of 1's of the bitmask of the subnet mask that you have. More info on this here.

Remember that the interface will be shared, and the glasszone will be visible to the outsiders. This means that the glasszone and the globalzone will need different ip addresses of the net, and your interface will act like a router.

INFO: Zones doesn't support dhcp yet.

In my case, I'm using VirtualBox to virtualize the OpenSolaris system. It created a subnet for the system with the default router 10.0.2.2 and the default dns server 10.0.2.3 .

Physical is the interface that glasszone will share (must be a configured, plumbed, working interface).

You can see the interfaces information through the ifconfig command in the globalzone.

# ifconfig -a
[...]
e1000g0: flags=201004843 mtu 1500 index 2 inet 10.0.2.15 netmask ffffff00 broadcast 10.0.2.255 ether 8:0:27:31:8b:21
e1000g0:1: flags=201000843 mtu 1500 index 2 zone glasszone inet 10.0.2.20 netmask ffffff00 broadcast 10.0.2.255
[...]

Now the zone should be up and running:

# zoneadm list -vi
   ID    NAME    STATUS       PATH                BRAND    IP
   0    global    running       /                native    shared
   1    global    running       /export/home/glasszone    ipkg    shared

Login to it. We're going to install the glassfish app. Since a zone is an instance of the OpenSolaris operational system, if the fs is not mounted anywhere, the only way to communicate with it is though the network interface (remember that it is a virtual interface, so if you are communicating between zones, minimum delay is expected).

Basically you need two things: The jdk and the glassfish app.

My tip: Download both at once:
http://java.sun.com/javaee/downloads/index.jsp

Of course you can install it separately, but you will have to set some environment variables such as JAVA_HOME and the PATH. It isn't so hard, but it isn't so fast too, so I'll keep my advise.

You can download it logged to globalzone and then move it to the glasszone space. You can use wget to download it directly into your zone too.

TIP: Use -console option to install, because you have just a console in the zone.

You may have to set the execution permission for the downloaded file.

chmod +xxx runnable.bin

Now that you have installed it, you should be able to start a sample domain:

# asadmin start-domain domain1

Once it started, you should be able to see it on the hole subnet by typing in your browser:

http://10.0.2.20:8080
or
http://glasszone:8080

TIP: Probably you will have add the glasszone entry in the /etc/hosts in the globalzone.

#echo “10.0.2.20 glasszone” >> /etc/hosts
#cat /etc/hosts
[…]
127.0.0.1 localhost
10.0.2.20 glasszone

Now you have a "ready to use" web application server enviroment inside a zone inside the OpenSolaris OS.

HAVE FUN!

Please post a comment or send me a email if you have any doubts, suggestions or if you find something here that isn't that right. We're a community and your help is very welcome a encouraged. =)

References and related links:

http://blogs.sun.com/observatory/entry/screencast_installing_opensolaris_in_virtualbox

http://opensolaris.org/os/community/zones/

http://www.sun.com/bigadmin/content/zones/opensolaris_zones_preso.pdf

http://number9.hellooperator.net/articles/2007/03/29/glassfish-in-a-zone

http://docs.sun.com/app/docs/doc/817-1592
⁞[Read More]

Thursday Jan 22, 2009

OpenSolaris Course at Campus-Party 2009!

This week, here in São Paulo, is happening one of the most interesting events of the year, the Campus-Party.

The idea is simple:

Come with your computer and stay camping, immerse 24/4 in the computing world for a whole week!

A lot of good talks, mini-events, demos, video games, robots and, of course, open source software.

You can see more about the event in the blog:

http://blogs.sun.com/cparty

The osum community:

http://osum.sun.com/group/campusparty

The official website:

http://www.campus-party.com.br

Yesterday I had the opportunity to give a mini-course on OpenSolaris.

Vitório Sassi, Thiago Gonzaga, Ronaldo Prass and some other guys from SAMPAosug, POAosug and NORosug also showed the guys how to install the operational system, and how to use it with VirtualBox, a virtulization solution from Sun.

Here the slides:

Intro Open Solaris
View more presentations or upload your own. (tags: intro open)

It's all for now!

See ya and stay free!


[Read More]

Saturday Dec 13, 2008

OpenSolaris 2008.11

"OpenSolaris is the state of the art, cutting edge operating system that provides a rich, coherent platform for building and running applications.

OpenSolaris offers free, built-in features that help you build, debug and deploy new applications faster. It's is an operating system (OS), an open source project licensed under CDDL, and a community.

The project's goals are innovation, collaboration, and the extension of OpenSolaris technology." (www.opensolaris.com)

They just released the 2008.11 version, with some new cool tools like TimeSlider and much more hardware compatibility.

You can download it here: http://www.opensolaris.com/get/index.js

You can see a walk-through the installation here: http://blogs.sun.com/amitsaha/entry/open_solaris_2008_11_on

You can see videos and talks about the release and the new stuff here: http://www.opensolaris.com/learn/

You can get in touch with the community here: http://www.opensolaris.org

You can see the hardware compatibility list here: http://www.sun.com/bigadmin/hcl/

Download it! Try it! Enjoy it!

About

Hi, my name is Marcelo Arbore, I study electrical engineering at University of Sao Paulo, Brazil, and I work for Sun Microsystems. I also like to surf, I love the sea and I will live in California some day. marcelo.arbore@sun.com

Search

Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today