Dienstag Jan 06, 2009

A MySQL GlassFish Application

During my time as a teaching assistant, I used to say to the students, "Imagine, you visit your grandmother and she asks you, 'Now boy, what did you do at the university today?' It's hard, but you have to do your exercises in a way, as if you were talking to a six year old." On the other hand, students are often frustrated, if they don't understand a subject down to its bare bones. It's only natural to have such feelings.

Giving a tech talk, you have to find an interesting sample, but it also has to be simple, for you don't want your audience to fall asleep. I was dreaming of a database-web application without tables full of employees, departments and managers or flight schedules. It had to be something, where the design tools and the web stack stand more in the foreground.

I started a project -- www.mimesy.com, using the NetBeans IDE, JSP, the jMaki framework, the MySQL DBMS and the GlassFish application server. It's some kind of a blog site, but the user cannot actually blog, microblog or nanoblog or upload any media files. The only thing you can tell the world is a color. There is also no history of colors, just a color you have at the moment. Login data and the color information can easily be stored in only one database table.

Mittwoch Sep 24, 2008

A Kiss from the Lambda Calculus


This month, I went to the Berlin.jar conference (a Java conference in Berlin), a two-day event at the FHTW Berlin (university of applied sciences). This year, it was the first Berlin.jar and it's planned to do it again next year.

One of the many presentations was about the Ruby programming language. Like some other languages, the language contains the lambda keyword. Historically it comes from the λ-calculus, a theory intended to be a foundation for mathematics (like Frege's logicism, this attempt failed). The keyword lambda introduces an (anonymous) function (object). In particular a function can be returned by a function and can be a parameter to a function, including itself. The lambda example, given in the presentation, was an anonymous function which takes no arguments and returns a random number.

This wouldn't be possible in the λ-calculus, nor in a pure functional programming language (e.g. Haskell). There the purpose of λ is to bind variable names, just like quantifiers do it in predicate logic, and a λ without variables is syntactically incorrect. It would also be impossible to have a function that doesn't always return the same value for the same arguments, a function with no formal parameters is simply a constant. In contrast, an imperative language allows side effects (the result of a function does not only depend on its arguments), therefore the meaning of the Ruby lambda is not the same.

An imperative λ. How ironic. Walking around after the conference, I made this picture of a remaining segment of the Berlin Wall near Ostbahnhof (Brezhnev kisses Honecker, above "Lord, help me to survive" is written in Russian).


Dienstag Sep 02, 2008

Sun Studio 12 IDE on Debian GNU/Linux

I have installed the Sun Studio 12 IDE on Debian Etch recently. With a slow internet connection, you don't want to always download everything, and I have some of the SDN All Access Kits (the 10/2007 version) for giving away to students. The kit consists of two DVDs, with Sun Studio 12 (20070730) and a lot of other software on them. On Debian GNU/Linux, we need some magic if we want to get the install wizard to work (there are other versions, I know).

SDN All Access Kit

Type su in a terminal window and become root. Change to the Sun Studio directory on the DVD and Copy the Linux directory to root's home: cp -r Linux ~ (do not use GNOME's file manager).

Install the Java Platform SE 6, I used the Java SE Development Kit 6 from one of the DVDs (copy it to root's home, extract, copy the extracted directory to /usr/java). Install additional Debian packages:

   apt-get install gawk alien libc6-dev

The alien package will give us the rpm command (RPM Package Manager), rpm is needed during the installation. The installer assumes it to be in the /bin directory, therefore create a link:

   ln -s /usr/bin/rpm /bin/rpm

Start the install wizard: cd ~/Linux && ./installer (now it should work). In the components window, uncheck Sun Studio 12 Performance Library ML (otherwise the installation will not complete, any ideas?). Leave everything else as it is and click '->' until you see a 'Close' button. Click 'Close'.

Donnerstag Jul 24, 2008

The Number of Rows

Last month, I held a presentation where I showed how to create a rich client application for a MySQL 5 database using the NetBeans 6.1 IDE. For a single table we can use the Database Application template (New Project → Java → Java Desktop Application → Database Application). Assuming that the table exists and we have created a connection to it, we get a ready to use GUI by just clicking 'Next' and 'Finish'. One of the funny things we can add in the GUI editor is a slider, whose value is bound to a data field (Properties → Binding tab). Then the data in the form gets updated when the slider is moved.



Suppose we want a label to reflect the number of rows in the table, especially we want the label to change when the number changes. Unfortunately, the bind trick doesn't work in all situations. A bound label doesn't get updated when we add or delete rows. During a demo, you don't want to write to much code and you want the solution to be simple. Without going to much into detail, we get the update functionality by adding an event handler for main frame's propertyChange events.

   private void formPropertyChange(java.beans.PropertyChangeEvent evt) {

Montag Jul 21, 2008

Sequential ZFS Mirrors

Last month, an ambassador colleague shared the idea of having ZFS mirrors on only one hard drive. One of the immediate questions is: how fast is such a zpool? I did some tests on a 1.7 GHz Pentium M laptop with a 120 GB IDE drive (WDC WD1200BEVE) and 1 GB of main memory, running the OpenSolaris 2008.05 distribution. Leaving some space behind the root file system during the installation, I used the remaining three primary partitions for a zpool (“m3”) where the data was replicated twice (two mirrors).


Testhomedir (within rpool)m3
Creating a 1 GB file (with dd)33 s, 32 MB/s134 s, 7.8 MB/s
Reading the 1 GB file (with dd)22 s, 46.7 MB/s41 s, 25.6 MB/s
Compiling the wxWidgets library9 m 18 s9 m 41 s

Dienstag Jun 17, 2008

The Many Faces of a Sequence Comprehension

Last month I held a presentation about the JavaFX Script programming language. It was easy to impress the audience by showing how fast a graphical user interface can be done. Last year I worked as a teaching assistant for a Java course, where I had to use the Eclipse IDE. Due to not having a visual GUI editor, I spent a whole day preparing a simple Swing source code example (the students then used it for an exercise). In the JavaFX Script programming language the more intuitive way of writing (and reading) Swing applications is a great relief.

In the functional part of my presentation I used a sequence comprehension, similar to that I found in the draft language reference [1], in order to decide if a number is a prime number or not.

for (i in [2..n/2] where n % i == 0) i       (1)

It returns a sequence of factors of a natural number n, not including 1 and n, so that n is prime iff the sequence of factors is empty. In mathematics one could say, there is a corresponding set of factors of n denoted by

{i | i ∈ [2..n/2], n mod i = 0}.

A sequence comprehension consists of three parts: a list of input sequences, a filter and an expression, which is "applied" to the filtered input. The expression in (1) is i, so that just the identity function is applied to the filtered input. We get the same result by only filtering [2..n/2].

   function filter (  lst : Integer [] 
                    , f   : function(:Integer):Boolean ) : Integer [] { 

       var res : Integer [] = [];

       for (i in lst) {
           if (f(i)) {
               insert i into res;

       return res;

   function factors (n : Integer) : Integer [] {
       return filter(  [2..n/2]
                     , function(i) { return n % i == 0; } );

Instead of defining a filter function, that takes a sequence and a predicate and returns a new sequence, we could use the built-in selection syntax.

[2..n/2][i | n % i == 0]

On the other hand, the filter function demonstrates the possibility of passing functions as arguments. The predicate passed by factors is an anonymous function.

There is also an equivalent sequence comprehension for a sequence syntax without filtering. The filter has just to be always true.


Donnerstag Apr 24, 2008

Too Many Return Types

(inspired by a discussion on one of my mailing lists)

The compiler says 'No'

Thank to the overloading feature in the Java programming language, method names within a class have not to be distinct (as long as the signatures differ). Can we overload a method by changing only the return type, i.e. if there is a method, can we define another one with the same name and parameter types and order, but with a different return type? The compiler says 'No'.

   class A {
       S f( p ) { ... }
       T f( p ) { ... }   // ST, p is a parameter list

This behavior is defined in the language specification [1] and, with the return type not being part of the signature, it is not a violation of our definition of overloading. Of course, the language-specification argument is only for the compiler a strong one for treating the situation in this way. For those, who sit on the other side of the code, it's more a fact than a reason.

Who you gonna call

Although we don't hold methods to be objects in the sense that they are instances of OOP classes, in our minds a method still has a type, and the most interesting properties are possessed by its input and output. Thus, if we imagine that a method is something like a function, we can write

P1 × P2 × … × PnS

as the type of the first f from the example above; the type of the second f looks similar, S is replaced by T. Let's now suppose, there is a call to f somewhere in the source, so that it's
  1. a call where the return value is thrown away, or
  2. a call where the return value is passed to a method or an operator (as an argument).
At this point we run into difficulties, for it's not always clear which f should be called (we'll see, that this holds for both cases). Since the difference lies in the method type, we must specify the type of the method we want to be called. The types behind the arrow can be derived from the call itself. This is how an overloaded method is normally chosen, but here we have to specify the return type as well.

The obvious solution is at the same time the behavior we want to have claiming return-type overloading. A call is always within a certain context, and it's the context that shall specify the return type. For a (non-overloaded) method g of the type SR (Svoid), putting a call to f into the argument place, we want the first f to be chosen. If S or T is void, case 1. specifies an f too, otherwise we don't know which f is meant.

However, there remain more situations where the context does not do the job. First of all, return-type overloading collides with parameter-type overloading (and with generic programming). Suppose there are now two methods g, one taking an S and one taking a T as an argument. Then in g(f(...)) we want the call to f to specify the context, on the other hand we want the context to specify an f.

Implicit type conversion can be treated as a kind of parameter-type overloading (e.g. a method that takes a java.lang.Object as an argument does not distinguish between all the subtypes). Sometimes we want to cast a return value. Again, we can think of a cast operator as an overloaded method.

From the existence of such cases it is apparent, that we need a special syntax that resolves ambiguity, and it's not possible to stay within the grammar of the language.


Montag Apr 21, 2008

Demonstrating ZFS

Last week I held a presentation about ZFS. Simplicity is one of the features of ZFS, but there are some things beyond ZFS, and the parts all together are everything but trivial. This text is a step-by-step instruction on how such a presentation can be done.

Preparatory steps

Get some USB flash drives of the same size. I used eight 1 GB USB flash drives with LED lights (so the activity on the storage pool was visible) and two 4-port USB hubs. Don't worry about the uselessness of owning eight memory sticks, at the end of the presentation you'll give them away to the people.

Install an OpenSolaris distribution. I used the SXDE 1/08 distribution on a 1.7 GHz Pentium M laptop with 1 GB of main memory and two USB ports. During the presentation you'll use a video file. Download a Chaplin movie from www.archive.org. Install mplayer from www.blastwave.org and tell the file browser to use it as the default application for the video file. In Preferences → Removable Drives and Media disable Mount removable drives when hot-plugged.

Creating a zpool

Type su in a terminal window and become root. Connect one of the hubs to the laptop and the second hub to the first (on a laptop with two USB ports the free port can be used for a mouse). Connect two sticks to the first hub and four sticks to the second.

Find out the device names of the flash drives, rmformat lists all removable devices when started without parameters. The names are like /dev/rdsk/c2t0d0p0 or /dev/rdsk/c18t0d0p0, where c = controller, t = target, d = disk, and p0 means "whole disk". On all six sticks, delete the data at the beginning of the space. Write a shell script for that, use

   dd if=/dev/zero of=/dev/rdsk/DeviceName count=2048

to fill the first 2048 sectors with zeroes (delete this data on the remaining two sticks too). Create a zpool that contains three mirrors and call it usbpool, the command may look like

   zpool create -f usbpool mirror c2t0d0p0 c4t0d0p0 \\
      mirror c18t0d0p0 c19t0d0p0 mirror c20t0d0p0 c21t0d0p0

Run the zpool list command to see that the pool is actually there, zpool status shows a tree structure of the pool. The pool is automatically mounted and is accessible through the /usbpool directory. Add rights, so that you can write to it as a normal user: chmod a+rw /usbpool.

Simulating a disk failure

Open the file browser, copy the Chaplin movie to /usbpool and open it from there. Depending on your laptop, the video may appear only on the laptop, so that the mplayer window remains blank on the projector output, try to use a different video output driver (mplayer -vo help prints a list of drivers). While playing the movie copy something else to the pool, you can copy the /opt/csw directory, which has 232 MB after installing mplayer. To simulate a disk failure, disconnect one stick from the first hub and one from the second. If every mirror is completely on one hub, no data is lost. In order to have some activity on the pool, delete a directory within /usbpool/csw. Now the zpool status command says, that two sticks have been removed. The video keeps playing. Connect the sticks back to the hubs, the ports do matter, which stick you connect to a port does not. Delete one more directory within the pool. The changes are automatically distributed across all sticks. The zpool status command now says, that all sticks are online. Stop the video and close the file browser.

Export, import, add, replace, destroy

Unmount the pool with the zpool export usbpool command. Disconnect all sticks and put them back in some random order. This step shows, that you could take the data with you and reinstall the pool somewhere else. The zpool import usbpool command brings everything back. You can now open the video again. Connect a new stick to the first hub and add it to the pool.

   zpool add -f usbpool c5t0d0p0

With the zpool list command you can see that the size of the pool has grown. Disconnect one of the mirror sticks and replace it with a new one. The zpool status command says, that the state of one of the sticks is FAULTED.

   zpool replace usbpool c21t0d0p0

Now the state of all sticks should be ONLINE. Cancel all activity and destroy the pool.

   zpool destroy usbpool

Before giving the sticks away, you should first create an FAT32 file system on each of them. Use the fdisk utility and manually create an FAT32 primary partition on one of the sticks. Save the partition table to a file.

   fdisk -W ptable c2t0d0p0

The partition table is now stored in a file called ptable. Write a shell script, that creates the file system on all connected sticks at once.


   # Create partition tables, ptable is a file created with fdisk -W ...,
   # ptable describes the partition table to be written to the sticks,
   # do not use this method, unless the sticks have the same capacity/geometry.

   # WARNING: make sure there are only USB sticks in this range
   for i in `seq 2 21`; do
      fdisk -F ptable c"$i"t0d0p0

   # Create FAT32 file systems, label=SUNTECHDEMO

   for i in `seq 2 21`; do
      echo "Creating FAT32 on $i . . ."
      echo y | mkfs -F pcfs -o fat=32,b=SUNTECHDEMO /dev/rdsk/c"$i"t0d0p0:c

Coincidentally, there were eight people in my presentation, so I was able to give a USB stick to each of them. If there are more people you might want to use random numbers. A Python interpreter is preinstalled in the SXDE 1/08 distribution, so you could draw the numbers there, for instance. If one of the numbers repeats, it would be a bad idea to give the stick to a person sitting next to a person that already has a stick, because for the former the probability of getting a stick is greater than for the people who are not sitting next to an already drawn number.

Try out RAID-Z.

More material

  • There is a video (in German) on YouTube, where a similar presentation is done (update: there is also an English version).
  • http://www.opensolaris.org/os/community/zfs/demos/zfsdemo.pdf
  • About

    Sun Campus Ambassador,
    University of Frankfurt (Germany)


    « April 2014