Wednesday May 06, 2009

Importing OpenSolaris EC2 Image on Local Xen Environment


In this blog entry, I am going to walk you through the steps for importing OpenSolaris Amazon EC2 AMI on local Xen environment running on OpenSolaris 2008.11.

I assume
- you are running OpenSolaris 2008.11 build rc2 dom0.
- ec2 ami/api tools installed at /opt/ec2
- your ec2 certificate and private key files available at /opt/ec2/keys
- your ec2 keypair file located at /opt/ec2



[ Step-1 ]
Launch fresh OpenSolaris 2008.11 AMI Instance
ec2-run-instances ami-7db75014 -k your-keypair

Login in with your keypair file
ssh -i /opt/ec2/your-keypair root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

[ Step-2 ]
Remove EC2 specific stuff from the instance.

Open “/etc/ssh/sshd_config” and replace "PermitRootLogin without-password" with "PermitRootLogin yes" and save the file.

Set the root password
passwd
passwd: Changing password for root
New Password:
Re-enter new Password:
passwd: password successfully changed for root

Disable and remove ec2 specific services

svcadm disable svc:/ec2/sshkey:default
svcadm disable svc:/ec2/fixes:default
svcadm disable svc:/ec2/mount:default

svccfg delete svc:/ec2/sshkey:default
svccfg delete svc:/ec2/fixes:default
svccfg delete svc:/ec2/mount:default

[ Step-3 ]
Log out of the ec2 instance and transfer your keys at /mnt/keys to begin re-bundling image.
scp -i /opt/ec2/id-your-keypair -r /opt/ec2/keys/ root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:/mnt

[ Step-4 ]
Login back to ec2 instance.
ssh -i /opt/ec2/your-keypair root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

[ Step-5 ]
Start bundling the image.

# cd /mnt
# /opt/ec2/sbin/rebundle.sh -v opensolaris_2008_11_ami.img
Creating image file /mnt/opensolaris_2008_11_ami.img
Preparing clone pool on /dev/lofi/1
Cloning
#

rebundle.sh will take about 10-15 minutes to create raw opensolaris image.

[ Step-6 ]
Bundle image and create parts.

export BUCKET=my-bucket
export JAVA_HOME=/usr/jdk/latest
export EC2_HOME=/opt/ec2
export PATH=$PATH:$EC2_HOME/bin
export RUBYLIB=$EC2_HOME/lib
export EC2_URL=https://ec2.amazonaws.com

export EC2_PRIVATE_KEY=/mnt/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem
export EC2_CERT=/mnt/keys/cert-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem

export EC2_KEYID=xxxxxxxxxxxxxxxxxxxxxxx
export EC2_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

export DIRECTORY=/mnt
export IMAGE=opensolaris_2008_11_ami.img

# ec2-bundle-image -c $EC2_CERT -k $EC2_PRIVATE_KEY \\
--kernel aki-6552b60c --ramdisk ari-6452b60d \\
--block-device-mapping "root=rpool/52@0,ami=0,ephemeral0=1" \\
--user <user-id> --arch i386 \\
-i $DIRECTORY/$IMAGE -d $DIRECTORY/parts

[ Step-7 ]
Upload image parts on s3

# cd $DIRECTORY/parts

# ec2-upload-bundle -b $BUCKET -m opensolaris_2008_11_ami.img.manifest.xml \\
--url http://s3.amazonaws.com \\
--retry -a $EC2_KEYID -s $EC2_KEY

[ Step-8 ]
Log out of the instance.
Download image parts and unbundle it to re-create the raw image locally.

# mkdir /osol-ec2-image
# mkdir /osol-ec2-image/parts
# cd /osol-ec2-image/parts

# ec2-download-bundle -b my-bucket -m opensolaris_2008_11_ami.img.manifest.xml \\
-a <your-access-key-id> -s <your-secret-key-id> \\
--privatekey /opt/ec2/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx .pem

# ec2-unbundle -m opensolaris_2008_11_ami.img.manifest.xml \\
-k /opt/ec2/keys/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx .pem

# mv opensolaris_2008_11_ami.img
# cd /osol-ec2-image/
[ Step-9 ]
Create “domu.py” at /osol-ec2-image with following lines

name = "opensolaris-ec2-image"
vcpus = 1
memory = "1024"

disk = ['file://osol-ec2-image/opensolaris_2008_11_ami.img,0,w']

vif = ['']

on_shutdown = "destroy"
on_reboot = "restart"
on_crash = "destroy"


[ Step-10 ]
Launch domu image and login with root user and the password you set during step-2

xm create domu.py

Using config file "./domu.py".
Started domain opensolaris-ec2-image

xm console opensolaris-ec2-image
v3.1.4-xvm chgset 'Mon Oct 13 22:14:51 2008 -0700 15904:94bd0a643efe'

SunOS Release 5.11 Version snv_101b 32-bit

Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.
Hostname: dhcp-umpk16-85-23
Reading ZFS config: done.
Mounting ZFS filesystems: (5/5)

dhcp-umpk16-85-23 console login:

Sunday May 03, 2009

Connecting Cisco VPN using vpnc on latest ubuntu 9.04

Install vpnc package with following command.

divyen@divyen-laptop:~$ sudo apt-get install vpnc

Create a file default.conf at /etc/vpnc/ with following lines and
fill in the required parameters replacing strings enclosed within “<” and “>" without keeping "<" and ">”.

divyen@divyen-laptop:~$ sudo vi /etc/vpnc/default.conf

IPSec gateway <gateway>
IPSec ID <group-id>
IPSec secret <group-pass-key>
Xauth username <user-name>
NAT Traversal Mode cisco-udp

Note: Make sure you don't append extra space at the end of parameter value.

Connect using "vpnc-connect" command.
divyen@divyen-laptop:~$ sudo vpnc-connect
[sudo] password for divyen:
Enter password for user-name@company-vpn-gateway:


To disconnect type in following command.
divyen@divyen-laptop:~$ sudo vpnc-disconnect
Terminating vpnc daemon (pid: 7894)


Tuesday Apr 28, 2009

Create a fully virtualized ubuntu-9.04 DomU on Sun xVM Hypervisor

Share/Save/Bookmark<script type="text/javascript">a2a_linkname="Create a fully virtualized ubuntu-9.04 DomU on Sun xVM Hypervisor";a2a_linkurl="http://blogs.sun.com/divyen/entry/create_a_fully_virtualized_ubuntu";</script><script type="text/javascript" src="http://static.addtoany.com/menu/page.js"></script>

Lets begin with the installation of xVM Hypervisor on OpenSolaris 2008.11.

You can follow exact steps mentioned on Dileep Kumar's blog - "How to create domU in OpenSolaris 2008.05". These steps are also valid for OpenSolaris 2008.11.

Once dom0 is up and running, walk through the below given steps to create a fully virtualized ubuntu 9.04 DomU. Fully virtualized DomU is also known as HVM Domain (Hardware-assisted Virtual Machine Domain), with that you can run unmodified guest operating system without the knowledge of underlying hypervisor.

Step-1: Download and save ubuntu 9.04 ISO file at /ubuntu-hvm

mkdir /ubuntu-hvm
cd /ubuntu-hvm
wget http://ftp.ucsb.edu/pub/mirrors/linux/ubuntu/jaunty/ubuntu-9.04-desktop-i386.iso


Step-2: Create a virtual Disk for the guest domain.

dd if=/dev/zero of=/ubuntu-hvm/ubuntu-9.04-desktop-i386.img bs=1k seek=10000k count=1


Step-3: Create guest configuration file install.py with following script at /ubuntu-hvm

Detailed explanation on each lines used in the script is given here.


disk = ['file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.img,hda,w','file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.iso,hdc:cdrom,r']

memory = 1024
vcpus=1
name = "ubuntu-904-hvm-install"
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
vif = [ '' ]

on_poweroff = 'destroy'
on_reboot   = 'destroy'
on_crash    = 'destroy'

boot='dc'
vnc=1
sdl=0

vncdisplay=5

import os, re
arch = os.uname()[4]
if re.search('64', arch):
     arch_libdir = 'lib64'
else:
     arch_libdir = 'lib'
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'





Launch the guest domain to begin the installation.

cd /ubuntu-hvm/
xm create install.py


Bring up VNC Session to proceed with the installation. In the install.py script we specified to bind VNC on port 127.0.0.1:5

vncviewer 127.0.0.1:5


If HVM guest is unsupported on your system, you may see following error message.
Error: HVM guest support is unavailable: is VT/AMD-V supported by your CPU and enabled in your BIOS?

Step-4: Install guest domain.

Snap-01
ubuntu_hvm_snap_01.gif

Snap-02
ubuntu_hvm_snap_02.gif

Snap-03
ubuntu_hvm_snap_03.gif

Snap-04
ubuntu_hvm_snap_04.gif

Snap-05
ubuntu_hvm_snap_05.gif
Do not click on the restart. Follow step-5 instead.

Step-5: Destroy the guest domain and create new configuration file to boot the domU from the disk image.

xm destroy ubuntu-904-hvm-install
cd /ubuntu-hvm/


Create new configuration file "domu.py" with following script at /ubuntu-hvm


disk = ['file:/ubuntu-hvm/ubuntu-9.04-desktop-i386.img,hda,w']

memory = 1024
vcpus=1
name = "ubuntu-904-hvm-guest"
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
vif = [ '' ]

on_poweroff = 'destroy'
on_reboot   = 'destroy'
on_crash    = 'destroy'

boot='c'
vnc=1
sdl=0

vncdisplay=5

import os, re
arch = os.uname()[4]
if re.search('64', arch):
     arch_libdir = 'lib64'
else:
     arch_libdir = 'lib'
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'



Step-6: Finally launch the HVM guest domain

cd /ubuntu-hvm/
xm create domu.py


Bring up VNC Session

vncviewer 127.0.0.1:5


Snap-05
ubuntu_hvm_snap_06.gif

Snap-06
ubuntu_hvm_snap_07.gif

Wednesday Mar 25, 2009

Consuming PHP Zend Rest API in Java

Share/Save/Bookmark<script type="text/javascript">a2a_linkname="Consuming PHP Zend Rest API in Java";a2a_linkurl="http://blogs.sun.com/divyen/entry/consuming_php_zend_rest_api";</script><script type="text/javascript" src="http://static.addtoany.com/menu/page.js"></script>

In my last Weblog entry, I described the steps on developing Rest API in PHP using Zend Framework.
Now we have Rest API ready to be consumed by the client. Let consume it using Java Client.

Before beginning the coding part, we need to download following library.

The Jakarta Commons HttpClient 3.1 Library
http://mirror.olnevhost.net/pub/apache/httpcomponents/commons-httpclient/binary/commons-httpclient-3.1.tar.gz

Apache Commons Logging Library
http://mirrors.kahuki.com/apache/commons/logging/binaries/commons-logging-1.1.1-bin.tar.gz

Apache Commons Codec Library
http://download.nextag.com/apache/commons/codec/binaries/commons-codec-1.3.tar.gz

First of all lets create the new NetBeans project. Select “java” from the project categories and “java Application” from the projects list. Click Next, name your project as “zend_rest_client” and finally click Finish button.

Snap-1
Snap-1
Snap-2
Snap-2

Right click on the Library and Select the Property. Add downloaded Library files using “Add Jar/Folder”.

Snap-3
Snap-3

Click “OK” and then replace the content of Main.Java file with the following code.

package zend_rest_client;

import com.sun.org.apache.xerces.internal.parsers.DOMParser;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StringReader;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;
import org.w3c.dom.\*;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
/\* @author divyen patel \*/

public class Main {

    public static void main(String[] args) throws IOException, SAXException {
        String str_REST_SERVER = "http://localhost";
        String str_Port = "80"; //Default Apache Port

        String str_rest_API_page = "weather_rest.php";
        String str_rest_method = "get_temperature";
        String str_rest_method_argument = "zipcode=95035";

        String str_rest_url = str_REST_SERVER + ":" + str_Port + "/" + str_rest_API_page + "?method=" + str_rest_method + "&" + str_rest_method_argument;
        System.out.println("Request URL:" + str_rest_url + "\\n");


        // Send GET request
        HttpClient client = new HttpClient();
        GetMethod method = new GetMethod(str_rest_url);


        int statusCode = client.executeMethod(method);

        InputStream rstream = null;
        rstream = method.getResponseBodyAsStream();
        BufferedReader br = new BufferedReader(new InputStreamReader(rstream));
        String line;
        String xmlString = "";

        while ((line = br.readLine()) != null) {
            xmlString = xmlString + line;
        }

        System.out.println("Request Response:" + xmlString + "\\n");

        //Parsing XML Response

        DOMParser parser = new DOMParser();

        parser.parse(new InputSource(new StringReader(xmlString)));

        Document document = parser.getDocument();
        System.out.println("Parsing XML Response");
        if (document.getElementsByTagName("status").item(0).getTextContent().equals("passed")) {
            System.out.println("Status: Passed");
            System.out.println("zipcode:" + document.getElementsByTagName("zipcode").item(0).getTextContent());
            System.out.println("tempHigh:" + document.getElementsByTagName("tempHigh").item(0).getTextContent());
            System.out.println("templow:" + document.getElementsByTagName("templow").item(0).getTextContent());
        } else {
            System.out.println("Status: Failed");
        }
    }
}


Finally right click on the project and Build it. You may see some warnings stating “DOMParser is Sun proprietary API and may be removed in a future release”. You can ignore these warnings.

Run the project and see the output in the IDE
Snap-4
Snap-4

You can also test it in the terminal console, using "java -jar <location of the jar file>"
Snap-5
Snap-5

If you have any better solution for consuming REST API in Java, please share with me. I am egar to learn it.

Saturday Feb 28, 2009

Developing Rest APIs in PHP using the Zend Framework

Share/Save/Bookmark<script type="text/javascript">a2a_linkname="Developing Rest APIs in PHP using the Zend Framework";a2a_linkurl="http://blogs.sun.com/divyen/entry/developing_a_rest_apis_in";</script><script type="text/javascript" src="http://static.addtoany.com/menu/page.js"></script>

Developing Rest APIs in PHP using the Zend Framework is as quick as a wink.
Lets begin with the installation and environment setup. I assume you are running OpenSolaris.

Step-1: Installation and Configuration of AMP [Note: skip this if you have already setup and configured AMP]
bash# su - root
bash# pkg refresh
bash# pkg install amp
bash# /usr/sbin/svccfg import /var/svc/manifest/network/http-apache22.xml
bash# /usr/sbin/svcadm enable http:apache22
bash# /usr/sbin/svccfg import /var/svc/manifest/application/database/mysql.xml
bash# /usr/sbin/svcadm enable mysql:version_50


Step-2: Installation and Configuration of Zend Framework.
bash# cd /usr/php/lib
bash# wget http://framework.zend.com/releases/ZendFramework-1.7.5/ZendFramework-1.7.5.tar.gz
bash# tar -xzf ZendFramework-1.7.5.tar.gz


Edit php.ini file and set the include_path variable to include Zend framework library path.
bash# vi /etc/php/5.2/php.ini

Add “include_path”, if it does not exists. If it is already set, just add path to the zend library at the end. Make sure to use “:” path separator.

include_path = ".:/usr/php/lib/ZendFramework-1.7.5/library"

bash# svcadm restart svc:/network/http:apache22


Step-3: Create Sample Database for Weather REST API.
Lets create the sample database for weather REST API.
bash# /usr/mysql/5.0/bin/mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \\g.
Your MySQL connection id is 1
Server version: 5.0.67 Source distribution
Type 'help;' or '\\h' for help. Type '\\c' to clear the buffer.

mysql> create database weatherdb;
Query OK, 1 row affected (0.00 sec)

mysql>use weatherdb;
Database changed


create tbl_zip_temperature table using following query.
create table tbl_zip_temperature (zip_code char(5), day_highest_temp int, day_lowest_temp int, PRIMARY KEY (zip_code));

mysql> create table tbl_zip_temperature (
-> zip_code char(5),
-> day_highest_temp int,
-> day_lowest_temp int,
-> PRIMARY KEY (zip_code)
-> );
Query OK, 0 rows affected (0.00 sec)



Lets dump in some dummy records with following queries.

insert into tbl_zip_temperature values(95035,67,52);
insert into tbl_zip_temperature values(95112,70,54);
insert into tbl_zip_temperature values(95032,70,50);

mysql> insert into tbl_zip_temperature values(95035,67,52);
Query OK, 1 row affected (0.00 sec)

mysql> insert into tbl_zip_temperature values(95112,70,54);
Query OK, 1 row affected (0.00 sec)

mysql> insert into tbl_zip_temperature values(95032,70,50);
Query OK, 1 row affected (0.00 sec)



Step-4: Create Weather REST API using Zend Framework Library.
Here we are all set to begin the coding.
Lets create the weather_rest.php file under the document root (/var/apache2/2.2/htdocs) with following code.
 
<?php
include_once ('Zend/Rest/Server.php');

function get_temperature($zipcode) {
    //error_code=0 for Sucesss / error_code=1 for Failure
    $error_code = 0;

    $doc = new DOMDocument();
    $doc->formatOutput = true;

    // Creating Responce Element
    $root_element = $doc->createElement("response");
    $doc->appendChild($root_element);

    //Create MYSQL Connection
	
    $MYSQL_DB_IP='localhost';
    $MYSQL_DB_USER='root';
    $MYSQL_DB_PASS='';
    $MYSQL_DB_NAME='weatherdb';
	
    $conn = new mysqli($MYSQL_DB_IP, $MYSQL_DB_USER, $MYSQL_DB_PASS , $MYSQL_DB_NAME);
    $sql = "select day_highest_temp, day_lowest_temp from tbl_zip_temperature where zip_code='".$zipcode."';";
    $result = $conn->query($sql);
    
    if ($result === false) {
        $error_code = 1;
    }
    else {
        if ($conn->affected_rows > 0) {        
		$row = $result->fetch_assoc();	
                $zip_element = $doc->createElement("zipcode");
                $zip_element->appendChild($doc->createTextNode($zipcode));
                $temp_high_element = $doc->createElement("tempHigh");
                $temp_high_element->appendChild($doc->createTextNode($row["day_highest_temp"]));
                $temp_low_element = $doc->createElement("templow");
                $temp_low_element->appendChild($doc->createTextNode($row["day_lowest_temp"]));
                $root_element->appendChild($zip_element);
                $root_element->appendChild($temp_high_element);
                $root_element->appendChild($temp_low_element);
        }
        else {
            $error_code = 1;
        }
    }
    
   //Return status failed, if any error found.
    if ($error_code == 1) {
        $statusElement = $doc->createElement("status");
        $statusElement->appendChild($doc->createTextNode("failed"));
        $root_element->appendChild($statusElement);
    }
    // Return status passed, if no error found
    else if ($error_code == 0) {
        $statusElement = $doc->createElement("status");
        $statusElement->appendChild($doc->createTextNode("passed"));
        $root_element->appendChild($statusElement);
    }
    return simplexml_load_string($doc->saveXML());
}

$server = new Zend_Rest_Server();
$server->addFunction('get_temperature');
$server->handle();

?>


That's all, you are done with creating REST API. Lets open the firefox, to test this API.
Type in following URL in the address bar and hit enter.
http://localhost/weather_rest.php?method=get_temperature&zipcode=95035

Output of the REST API call with the status passed
Snapshot1:http://blogs.sun.com/divyen/resource/rest_api_php_01.png

if you hit http://localhost/weather_rest.php?method=get_temperature&zipcode=95055, you will get the output as given below. There is no record for this zip code in the database.

Output of the REST API call with the status failed
Snapshot1:http://blogs.sun.com/divyen/resource/rest_api_php_02.png

if you hit http://localhost/weather_rest.php?method=get_temperature, without any argument, you will get the output as given below.

Output of the REST API call without proper argument
Snapshot1:http://blogs.sun.com/divyen/resource/rest_api_php_03.png

You can also consume this RESTfull API in your Java Application. I will blog about it in my next post.

For Detailed explanation on “Zend_Rest_Server”, visit Zend Framework - Programmer Reference Guide

Don't forget to leave your feedback :)
About

Divyen Patel

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today