Friday Jun 25, 2010

TOTD #142: GlassFish 3.1 - SSH Provisioning and Start/Stop instance/cluster on local/remote machines

GlassFish 3.1 Milestone 2 enables SSH provisioning that allows you to create, start, stop, and delete a cluster spanning multiple instances on local and remote machines from the Domain Administration Server (DAS). This Tip Of The Day (TOTD) builds upon TOTD #141 and explains how you can create such a cluster on Amazon EC2 with Ubuntu 10.04. Carla also blogged about a similar scenario here.

The cluster topology created is shown below:

The key points shown in the topology are:

  • It consists of DAS and a remote machine "fruits" (shown in green color)
  • There is one cluster "food" spanning these two machines (shown in yellow color)
  • DAS has "broccoli" and "spinach" instances (shown in red color)
  • "fruits" has "apple", "banana", and "orange" instances (shown in red color)

Amazon EC2 assigned the public IP address of "ec2-184-72-12-163.us-west-1.compute.amazonaws.com" to DAS and "ec2-184-72-17-228.us-west-1.compute.amazonaws.com" to the remote machine. These IP addresses are used in the command invocations below.

Lets get started!

  1. Configure SSH between DAS and the remote machine - More details about SSH key setup here.
    1. Copy the keypair  generated in TOTD #141 (ec2-keypair.pem) to DAS as:
      ~/.ec2 >scp -i /Users/arungupta/.ec2/ec2-keypair.pem /Users/arungupta/.ec2/ec2-keypair.pem \\\\
      ubuntu@ec2-184-72-12-163.us-west-1.compute.amazonaws.com
      ec2-keypair.pem 100% 1751 1.7KB/s 00:00
      
      
      Notice, the public IP address of DAS is specified here. This key will be used to copy the private keys generated in next step to the remote machine.
    2. Generate a private/public key pair on DAS as:
      ubuntu@ip-10-160-47-196:~$ ssh-keygen -t dsa
      Generating public/private dsa key pair.
      Enter file in which to save the key (/home/ubuntu/.ssh/id_dsa): 
      Enter passphrase (empty for no passphrase): 
      Enter same passphrase again: 
      Your identification has been saved in /home/ubuntu/.ssh/id_dsa.
      Your public key has been saved in /home/ubuntu/.ssh/id_dsa.pub.
      The key fingerprint is:
      0a:b8:cd:8c:a0:7f:3d:00:9e:ec:ac:06:a1:f1:2f:cb ubuntu@ip-10-160-47-196
      The key's randomart image is:
      +--[ DSA 1024]----+
      | |
      | |
      | |
      |o .. |
      |o\*.o. S |
      |+.=\*.. . |
      |ooo.+o. |
      | ++ o o |
      |o.E+ . |
      +-----------------+
      
      
    3. Copy the generated public to ".ssh" directory of the remote machine as:
      ubuntu@ip-10-160-142-175:~/.ssh$ scp -i ec2-keypair.pem id_dsa.pub \\\\
      ubuntu@ec2-184-72-17-228.us-west-1.compute.amazonaws.com:.ssh/authorized_keys2
      
    4. Make sure the ssh connection works between DAS and remote machine without specifying any key or passphrase as shown below:
      ssh ubuntu@ec2-184-72-17-228.us-west-1.compute.amazonaws.com
      
  2. Install "sun-java6-jdk" and "unzip" package and GlassFish on DAS and remote machine as explained in TOTD #141. In short:
    ssh -i /Users/arungupta/.ssh/ec2-keypair.pem ubuntu@ec2-XX-XX-XX-XX.us-west-1.compute.amazonaws.com
    sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
    sudo apt-get update
    sudo apt-get install sun-java6-bin sun-java6-jre sun-java6-jdk
    sudo update-java-alternatives -s java-6-sun
    sudo apt-get install unzip
    wget http://dlc.sun.com.edgesuite.net/glassfish/3.1/promoted/glassfish-3.1-b06.zip
    unzip glassfish-3.1-b06.zip
    
  3. Start GlassFish on DAS and remote machine as:
    export ENABLE_REPLICATION=true
    export PATH=~/glassfishv3/bin:$PATH
    asadmin start-domain --verbose &
    
  4. Create the cluster and instances by issuing the following commands on the DAS
    1. Create the cluster as:
      ubuntu@ip-10-160-142-175:~$ asadmin create-cluster food
      [#|2010-06-25T22:11:27.604+0000|INFO|glassfish3.1|org.hibernate.validator.util.Version|
      _ThreadID=23;_ThreadName=http-thread-pool-4848(2);|Hibernate Validator bean-validator-3.0-JBoss-4.0.2_03|#]
      
      [#|2010-06-25T22:11:27.638+0000|INFO|glassfish3.1|org.hibernate.validator.engine.
      resolver.DefaultTraversableResolver|_ThreadID=23;_ThreadName=http-thread-pool-4848(2);|
      Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.|#]
      
      
      Command create-cluster executed successfully.
      
    2. Create a node on the remote machine as:
      ubuntu@ip-10-160-142-175:~3$ asadmin create-node-ssh --nodehost \\\\
      ec2-184-72-17-228.us-west-1.compute.amazonaws.com --nodehome /home/ubuntu/glassfishv3 fruits
      
      Command create-node-ssh executed successfully.
      
    3. List all the nodes as:
      ubuntu@ip-10-160-142-175:~$ asadmin list-nodes
      localhost
      fruits
      
      Command list-nodes executed successfully.
      
    4. Create two instances ("broccoli" and "spinach") on DAS as:
      ubuntu@ip-10-160-142-175:~$ asadmin create-instance --cluster=food \\\\
      --systemproperties AJP_INSTANCE_NAME=broccoli:AJP_INSTANCE_PORT=19090 broccoli
      [#|2010-06-25T23:22:02.891+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin
      .com.sun.enterprise.v3.admin.cluster|_ThreadID=103;_ThreadName=http-thread-pool-4848(2);|
      Creating instance broccoli on localhost|#]
      
      [#|2010-06-25T23:22:03.750+0000|INFO|glassfish3.1|null|_ThreadID=30;_ThreadName=stdout;|
      Using DAS host localhost and port 4848 from existing das.properties for nodeagent 
      ip-10-160-142-175. To use a different DAS, create a new nodeagent by specifying a
      new --nodeagent name with the correct values for --host and --port.|#]
      
      [#|2010-06-25T23:22:03.785+0000|INFO|glassfish3.1|null|_ThreadID=30;_ThreadName=stdout;|
      Command _create-instance-filesystem executed successfully.|#]
      
      
      Command create-instance executed successfully.
      ubuntu@ip-10-160-142-175:~$ asadmin create-instance --cluster=food \\\\
      --systemproperties AJP_INSTANCE_NAME=spinach:AJP_INSTANCE_PORT=19091 spinach
      [#|2010-06-25T23:22:24.813+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=106;_ThreadName=http-thread-pool-4848(5);|
      Creating instance spinach on localhost|#]
      
      [#|2010-06-25T23:22:25.636+0000|INFO|glassfish3.1|null|_ThreadID=32;_ThreadName=stdout;|
      Using DAS host localhost and port 4848 from existing das.properties for nodeagent 
      ip-10-160-142-175. To use a different DAS, create a new nodeagent by specifying a
      new --nodeagent name with the correct values for --host and --port.|#]
      
      [#|2010-06-25T23:22:25.672+0000|INFO|glassfish3.1|null|_ThreadID=32;_ThreadName=stdout;|
      Command _create-instance-filesystem executed successfully.|#]
      
      
      Command create-instance executed successfully.
      
      The AJP_INSTANCE_NAME and AJP_INSTANCE_PORT properties will be used by mod_jk in a subsequent blog.
    5. Create three instances ("apple", "banana", and "orange") on the remote machine as:
      ubuntu@ip-10-160-142-175:~$ asadmin create-instance --cluster=food --node=fruits \\\\ 
      --systemproperties AJP_INSTANCE_NAME=apple:AJP_INSTANCE_PORT=19090 apple
      [#|2010-06-25T23:23:33.208+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=104;_ThreadName=http-thread-pool-4848(3);|
      Creating instance apple on fruits|#]
      
      [#|2010-06-25T23:23:35.682+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=104;_ThreadName=http-thread-pool-4848(3);|
      Command _create-instance-filesystem executed successfully.
      |#]
      
      
      Command create-instance executed successfully.
      ubuntu@ip-10-160-142-175:~$ asadmin create-instance --cluster=food --node=fruits \\\\
      --systemproperties AJP_INSTANCE_NAME=banana:AJP_INSTANCE_PORT=19091 banana
      [#|2010-06-25T23:23:59.697+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=102;_ThreadName=http-thread-pool-4848(1);|
      Creating instance banana on fruits|#]
      
      [#|2010-06-25T23:24:01.500+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=102;_ThreadName=http-thread-pool-4848(1);|
      Using DAS host ip-10-160-142-175.us-west-1.compute.internal and port 4848 from 
      existing das.properties for nodeagent ip-10-160-142-20. To use a different DAS,
      create a new nodeagent by specifying a new --nodeagent name with the correct
      values for --host and --port.
      Command _create-instance-filesystem executed successfully.
      |#]
      
      
      Command create-instance executed successfully.
      ubuntu@ip-10-160-142-175:~$ asadmin create-instance --cluster=food --node=fruits \\\\
      --systemproperties AJP_INSTANCE_NAME=orange:AJP_INSTANCE_PORT=19092 orange
      [#|2010-06-25T23:24:13.286+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=105;_ThreadName=http-thread-pool-4848(4);|
      Creating instance orange on fruits|#]
      
      [#|2010-06-25T23:24:15.089+0000|INFO|glassfish3.1|javax.enterprise.system.tools.admin.
      com.sun.enterprise.v3.admin.cluster|_ThreadID=105;_ThreadName=http-thread-pool-4848(4);|
      Using DAS host ip-10-160-142-175.us-west-1.compute.internal and port 4848 from 
      existing das.properties for nodeagent ip-10-160-142-20. To use a different DAS,
      create a new nodeagent by specifying a new --nodeagent name with the correct
      values for --host and --port.
      Command _create-instance-filesystem executed successfully.
      |#]
      
      
      Command create-instance executed successfully.
      
  5. Start the cluster
    1. List all instances as:
      ubuntu@ip-10-160-142-175:~3$ asadmin list-instances
      broccoli not running
      spinach not running
      apple not running
      banana not running
      orange not running
      
      Command list-instances executed successfully.
      
    2. Start the cluster as:
      ubuntu@ip-10-160-142-175:~$ asadmin start-cluster food
      
      . . .
      
      Command start-cluster executed successfully.
      
      
    3. List all the instances again as:
      ubuntu@ip-10-160-142-175:~$ asadmin list-instances
      
      . . .
      
      broccoli running
      spinach running
      apple running
      banana running
      orange running
      

      The HTTP ports of each instance can be grepped from DAS's "domain.xml". Here are the ports for each created instance:
      broccoli 28080
      spinach 28081
      apple 28080
      banana 28081
      orange 28082

      On Amazon, you may have to poke holes in the firewall as:
      ec2-authorize default -p 28080
      ec2-authorize default -p 28081
      ec2-authorize default -p 28082
      
      And now "http://ec2-184-72-12-163.us-west-1.compute.amazonaws.com:28080/" ("broccoli" instance on DAS) will show the default index page. Similarly other host and port combinations will show this page as well.

This blog showed how to create a GlassFish 3.1 cluster spanning multiple instances on Amazon EC2 with Ubuntu 10.04.

Subsequent blogs will show:

  • How to deploy an app to this cluster and some variations ?
  • How to front-end this cluster with mod_jk for load-balancing ?

Technorati: totd glassfish clustering ssh instance amazon ec2 ubuntu

Thursday Jun 24, 2010

TOTD #141: Running GlassFish 3.1 on Ubuntu 10.04 AMI on Amazon EC2

GlassFish 3.1 Milestone 2 was released this week, download the zip file.

TOTD #138 provide simple instructions to get you started with Milestone 1 and you can follow similar steps to get started with Milestone 2 as well. A more detailed blog on the new functionality (SSH Provisioning and Dynamic App Deployment) is coming as well.

In order to test the newly added clustering capabilities of GlassFish 3.1, I decided to run Milestone 2 build on a Ubuntu 10.04 instance on Amazon EC2. My host operating system is Mac OS X and even though the steps are defined at multiple locations (mentioned below) but complete set of steps were missing. This Tip Of The Day (TOTD) shows how get started with running a GlassFish 3.1 build on Ubuntu 10.04 instance on Amazon EC2.

This TOTD consulted the following blogs:

Twitterverse was very helpful and Divyen helped smoothen the rough edges!

Lets get started!

  1. In your home directory, create a new ".ec2" directory.
  2. Generate SSH Key & X.509 certificates
    1. Generate X.509 certificates key
      1. Go to your AWS account and click on "Security Credentials".
      2. In "Access Credentials" section, click on "X.509 Certificates" and click on "Create a new Certificate". Make sure to download the private key file and X.509 certificate in ".ec2" directory as "cert-xxxxxx.pem" and "pk-xxxxxxx.pem".
    2. SSH key - Public AMIs can be accessed using an ssh key. Give the following command in ".ec2" directory:
      ec2-add-keypair ec2-keypair > ec2-keypair.pem
      

      This will generate the SSH key in the file "ec2-keypair.pem".
  3. EC2 API Tools
    1. Download Amazon EC2 API Tools from here and unzip them in ".ec2" directory.
    2. Move "lib" and "bin" directory from the extracted directory to ".ec2" directory.
  4. Create ".ec2.profile" file in ".ec2" directory and add the contents:
    export EC2_HOME=~/.ec2
    export PATH=$PATH:$EC2_HOME/bin
    export EC2_PRIVATE_KEY=`ls $EC2_HOME/pk-\*.pem`
    export EC2_CERT=`ls $EC2_HOME/cert-\*.pem`
    export EC2_URL=https://ec2.us-west-1.amazonaws.com
    

    The last line sets the default zone to US-West-1. The AMI id used later is from this region. Source this file by giving the following command in ".ec2" directory:
    source ".ec2.profile"
    
  5. On your firewall, authorize port 22 for SSH and 8080 for HTTP access for GlassFish on your firewall as:
    ec2-authorize default -p 22
    ec2-authorize default -p 8080
    
  6. Run the instance as:
    ec2-run-instances ami-c597c680 -k ec2-keypair
    
  7. Obtain the public IP address of Ubuntu instance as:
    ec2-describe-instances
    RESERVATION     r-XXXXXXXX      XXXXXXXXXXXX    default
    INSTANCE        i-XXXXXXXX      ami-c597c680    ec2-XX-XX-XX-XX.us-west-1.compute.amazonaws.com       ip-XX-XX-XX-XX.us-west-1.compute.internal       running ec2-keypair     0               m1.small        2010-06-24T16:23:44+0000        us-west-1a      aki-XXXXXXXX    monitoring-disabled      XX-XX-XX-XX   XX-XX-XX-XX
    

    The address "ec2-XX-XX-XX-XX.us-west-1.compute.amazonws.com" is the public IP address and will be used for ssh next.
  8. SSH to the ready Ubuntu instance as:
    ssh -i ec2-keypair.pem ubuntu@ec2-XX-XX-XX-XX.us-west-1.compute.amazonaws.com
    

  9. Install JDK 6 on Ubuntu
    1. Add the Ubuntu partner repository as:
      sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
      
    2. Update the list of packages as:
      sudo apt-get update
      
    3. Install JDK 6 as:
      sudo apt-get install sun-java6-bin sun-java6-jre sun-java6-jdk
      
    4. Make sure the recently added JDK is at the top of JVM search order by giving the command:
      sudo update-java-alternatives -s java-6-sun
      
      This command adds "/usr/lib/jvm/java-6-sun" to the top of "/etc/jvm" file.
  10. Install "unzip" package as:
    sudo apt-get install unzip
    
  11. Download & start GlassFish 3.1
    1. Download GlassFish 3.1 Milestone 2 as:
      wget http://dlc.sun.com.edgesuite.net/glassfish/3.1/promoted/glassfish-3.1-b06.zip
      
    2. Unzip the downloaded zip file as:
      unzip glassfish-3.1-b06.zip
      
    3. Start GlassFish as:
      ./glassfishv3/glassfish/bin/asadmin start-domain --verbose
      
    4. And now your default web page is accessible at "http://ec2-XX-XX-XX-XX.us-west-1.compute.amazonws.com:8080".
  12. Finally terminate the instance as:
    ec2-terminate-instances  i-XXXXXXXX
    

How are you using GlassFish in the cloud ?

Technorati: totd glassfish ubuntu amazon ec2 cloud osxtips

Thursday Jun 03, 2010

Jazoon 2010 Day 3 - Java EE 6 on Cloud, HTML 5 with JSF 2 and another run on Üetliberg

The Day 3 of Jazoon (Day 1 & 2) started with a great presentation on Gaia satellite and Data Processing by William O'Mullane. Gaia is a European Space Agency space mission that will be launched in Summer of 2012 to compile a catalog of approximately 1 billion stars. Most of their software is Java-based and have been fairly happy with the decision taken 10 years ago. They are very happy with the performance of Java and in certain cases its even 10x faster than C.

I gave a talk on "Running Java EE 6 applications in the Cloud". The talk explained flexibility, light-weight, extensibility, and ease-of-use of Java EE 6. It demonstrated a simple sample development and rapid deployment feature using NetBeans IDE. And then quickly jumped into how this application can be deployed on Amazon EC2, RightScale, Elastra, and Joyent infrastructure. I learned about the Simple Cloud API (Storage only) and Apache libcloud from Doug Tidwell's talk yesterday and so will try them as well.

I gave a tee-shirt from the Bay Area JUG Roundup to Rainer Grau - moderator of the conference and he was kind enough to change it for rest of the day. See his picture below.

The slides from my talk are available below:

More detailed steps to deploy your Java EE 6 application on each cloud management / provider mentioned above will be available in a later blog. Instant feedback via twitter is always much appreciated as shown in the case below:

The Spring 3.0 Themes and Trends talk was quite a dejavu because it was mostly a rehash of what is available in the Java EE 6 today. And I could not understand why would anybody use Spring (instead of Java EE) to use the goodness of the JSF and JPA technologies ?

Roger's talk on "Exploring HTML 5 with JSF 2" was very interesting and had a packed room, small but packed. He showed lots of HTML 5 samples and how Java Server Faces 2 can be used to include new tags like audio / video in a JSF composite component, Web Sockets and even Web Workers with JSF 2. His slides are available at:

Roger & I will work on publishing detailed steps on these demos in the coming days.

Spring Roo was a good one but it seems like that its basically trying to bring Ruby-on-Rails concepts like Dynamic Functions and Scaffold with some usual tooling tricks and heavy code generation to the Java programming language. One more framework, one more convention, one more programming style ... not sure if it fills any gap!

Dan & Aslak presented on Real Java EE Testing with Arquillian and this was mostly a demo-driven talk showing how different Java EE technologies can be easily tested using ShrinkWrap and Arquillian. And the cool thing is that they support deployment to GlassFish Embedded as well.

Here are some pictures from earlier today:

And the day concluded with yet another great run up Üetliberg, just a simple out & back but 1404 ft elevation gain in 4.59 miles. The elevation map is shown below:

More details about the run are given below:

Here are some pictures captured during the run:


It is just beautiful - definitely worth going at least once!

Last but not the least, Roger & I had a great dinner at au gratin - a great restaurant + adjoining bar on the first floor on Bahnhofplatz. They've got great food variety at a great location, reasonably priced (as per the Swiss standards ;-), and an American Football aficionado server by the name David. If you meet him, say our hello to him :-)

And here is the complete photo album:

This was my first Jazoon conference in the beautiful country of Switzerland and I thoroughly enjoyed it. I had some useful conversations and made some good contacts. I'd love to come back here again next year as well!

Next step Über Conf ...

Technorati: conf jazoon zurich javaee glassfish cloud amazon rightscale elastra joyent html5 jsf2 spring

About

profile image
Arun Gupta is a technology enthusiast, a passionate runner, author, and a community guy who works for Oracle Corp.


Java EE 7 Samples

Stay Connected

Search

Archives
« August 2015
SunMonTueWedThuFriSat
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
     
Today