Friday Feb 20, 2009

Bidding adieu to my FJ Cruiser

Finally I parted ways with my 07 FJ. Don't get me wrong -- I've been quite happy with my fav. SUV so far. However, I don't think it is infant-friendly. Opening the front doors while taking the car seat out was a pain in the neck. This is something I didn't realize while buying this head-turner. I am back to a sedan now. I am gonna miss you my FJ. For the curious minds, the new owner is a hunter in Holland.


 

Monday Jul 28, 2008

Google killer or just another search engine?

Former Google engineer Tom Costello claims to offer the biggest web index with 120 billion web pages -
three times more than other search engines. There is no official figure of google's web index so you may want to believe what Costello says.
The search looks quite impressive with a time almost equal to google. A search of "solaris" on both the sites return the following:


www.google.com


65,600,000 for solaris. (0.22 seconds)


www.cuil.com

21,607,122 results for solaris ( nearly same or lesser time as google)

Cuil looks impressive because it loads images too within this search time (unlike google)



Wednesday Jan 30, 2008

A day with IBM DB2

It was fun working with DB2 after having already experienced Oracle and mySQL. It started with the installation of DB2 on a Solaris 10 SPARC machine. I was interested in running a command-line installation but sadly it doesn't configure the database for you. Installation is quite easy and I easily sailed past that. I desperately needed some reference to configure the db2. Nevertheless, a search on google brought me to http://www.ldas.ligo-wa.caltech.edu/doc/db2/doc/html/installDB2.html. It is a good site which explains the necessary steps like user creation etc to configure the db2 (You might want to be little cautious as it discusses an old release of db2). It also lets u know how to create a sample database.

Once the configuration was done, I wanted to test the sample database access using my java code. I set  the db2jcc4.jar ( found in the /opt/IBM/db2/V9.5/java) in the CLASSPATH and a sample application below worked.

 public static void main(String[] args) throws Exception
{

 String url = "jdbc:db2://HOST_NAME:50002/sample";
 Class.forName("com.ibm.db2.jcc.DB2Driver";);
 Connection conn = DriverManager.getConnection(url, "ldasdb", "ldasdb";);
 Statement stmt = conn.createStatement() ;

  // Execute the query
  ResultSet rs = stmt.executeQuery( "SELECT \* FROM staff" ) ;

  // Loop through the result set
  while( rs.next() )
       System.out.println( rs.getString(1) ) ;

   // Close the result set, statement and the connection
   rs.close() ;
   stmt.close() ;
   conn.close() ;
}

Nice! I am almost done and I need to use the existing SQLs to create a database and some tables. See how it goes below:

 1. go to the installation path and run db2 for a db2  prompt ( use ldasdb for this)
 2. create a database or use sample database mentioned in the page
 db2 => create database vdtest
 3. Connect to the database
 db2=> connect to vdtest

4. Create a table

Now my old SQL fails because  you need to mention in the primary key that it is not null. Well, I don't know if it was obvious or not :) but I did change the SQL to "UID       VARCHAR(15)     NOT  NULL PRIMARY KEY" from "UID       VARCHAR(15)      PRIMARY KEY" and the table is created.

5. Inserting records

Once again I hit a roadblock here. But it turned out to be the column names following the table name in the INSERT statement. Workaround was to take the column names from the INSERT.

INSERT INTO USER_HOBBY  VALUES ('TEST000','Art',1)

Didn't have the time to investigate how to insert only for a few select columns...Saving it for some other time :)


 

Thursday Jan 10, 2008

मेरा पहला हिंदी Blog

हिंदी में blog करने का ये मेरा पहला अनुभव है. उम्मीद है की ये शौक जारी रहेगा.

Tata unveils NANO ( Cheaptest car ever)

Finally, we get to see the cheapest car. Priced at USD $2,500 , now everybody can own a car. Have a look yourself.

Let us look at the specification ( shamelessly copied from www.timesofindia.com ):

Looks: The snub-nosed car keeps in the tradition of the Fiat 500, Nissan Micra and the Smart.

Dimensions: 3.1 metres (10.23 feet) long, 1.5 metres wide and 1.6 metres high. Can seat four to five people.

Engine: A two cylinder 623 cc, 33 horsepower rear mounted, all aluminium, multi-point fuel injection petrol engine can power the car to top speeds of 105 kilometres per hour (65 miles per hour).

Fuel Efficiency: 20 kilometres per litre, or 50 miles per gallon is claimed.

Pollution: Exceeds Indian regulatory requirements and can meet strict Euro IV emission standards. In terms of overall pollutants, Tata says the car is better than two-wheelers manufactured in India currently.

Safety: Car exceeds current regulatory requirements with a strong passenger compartment, crumple zones, intrusion resistant doors, seat belts, strong seats and anchorage.

Initial Annual Production Target: 250,000 units to rise later to 350,000. PRICE: Basic model price 100,000 rupees (2,500 dollars) plus tax and transport costs, which will bring on the road price to at least 120,000 rupees. The price of two deluxe models that will include air-conditioning and other features to be announced later.

Nearest Domestic Car Rival: Maruti 800, part of Japanese-owned Suzuki Maruti stable whose base model sells for about 4,800 dollars -- nearly double the price of the Nano.

Nearest International Rival: China's Chery QQ which retails for 3,600 dollars.

Sales: Tata will focus on selling the car in India for the next two to three years, before eyeing Latin American and Southeast Asian markets.

Market: India's car market is a huge draw because car penetration is just seven per 1,000 people, compared to 550 per 1,000 in such countries as Germany or 476 in France, according to the Society of Indian Automobiles.

Company Details: Tata Motors is India's largest vehicle company with revenues of 7.2 billion dollars in 2006-2007. It is the leader in commercial vehicles, such as trucks and buses, and the second largest in passenger vehicles. There are over four million Tata vehicles on Indian roads.

Interestingly, Mr. Tata thought of this when he saw a family getting wet in the rain. What a feat -- especially when Tata Motors is going to buy out Jaguar!!

Wednesday Dec 12, 2007

Checking configuration in DPS

A quick recap of the commands to verify the configuration:

bash-3.00# dpconf info -p 5390
Enter "cn=Proxy Manager" password:
Instance Path : /space/dps_tiko
Host Name : void
Port : 5390
Secure port : 5636
SSL server certificate : defaultServerCert
Server version : Directory Proxy Server 6.2 (More recent than "dpconf" version)

bash-3.00# dpconf get-server-prop -p 5390
Enter "cn=Proxy Manager" password:
allow-cert-based-auth : deny
allow-ldapv2-clients : true

bash-3.00# dpconf list-ldap-data-sources -p 5390
Enter "cn=Proxy Manager" password:
dsmmuc05

bash-3.00# dpconf get-ldap-data-source-prop -p 5390 dsmmuc05
Enter "cn=Proxy Manager" password:
bind-dn : none
bind-pwd : none
client-cred-mode : use-client-identity
connect-timeout : 10s

bash-3.00# dpconf list-ldap-data-source-pools -p 5390
Enter "cn=Proxy Manager" password:
MasterPool
defaultDataSourcePool

bash-3.00# dpconf get-ldap-data-source-pool-prop -p 5390 MasterPool
Enter "cn=Proxy Manager" password:
client-affinity-policy : read-write-affinity-after-any
client-affinity-timeout : 20s

bash-3.00# dpconf list-attached-ldap-data-sources -p 5390 MasterPool
Enter "cn=Proxy Manager" password:
dsmmuc05

bash-3.00# dpconf get-attached-ldap-data-source-prop -p 5390 MasterPool dsmmuc05
Enter "cn=Proxy Manager" password:
add-weight : 1
bind-weight : 1
compare-weight : 1

bash-3.00# dpconf list-ldap-data-views -p 5390
Enter "cn=Proxy Manager" password:
MasterView
root data view

Wednesday Jul 18, 2007

iPhones flood wireless LAN at Duke University

See http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9027279&source=NLT_PM&nlid=8.

Looks like Apple should stop selling it now ;).

Tuesday Jul 17, 2007

Maximize performance during bulk deletes in Opends

This article explains you how to gain performance improvement using the ds-cfg-backend-subtree-delete-batch-size configuration attribute. The default recommended value for this is 5000. However, this may vary from machine to machine. The default value seems to be yielding a reasonable output with a Java Heap Size of 512MB.

There is no defined formulae to achieve this value for the best performance gain. Rather, it should be mostly based on the the sleepycat utilities which help determining the DB Cache size, Java Heap Size Settings , number of records stored in different databases.

As per the default Opends setting , the assigned DB Cache Size is 10% of the JVM heap size. So if the JVM heap size for Opends is 512MB then the DB Cache may grow up to 53 MB.

In my setup, I imported nearly 100,000 entries to the opends similar to the follwing one:

dn:cn=user#0,dc=example,dc=com

objectclass: inetorgperson

cn: user#0

sn: common

After the import, following databases were populated:

dc_example_dc_com_dn2id : Record Count is 100001

dc_example_dc_com_id2entry:Record Count is 100001

dc_example_dc_com_referral: Record Count is 0

dc_example_dc_com_id2children: Record Count is 1

dc_example_dc_com_id2subtree:Record Count is 1

dc_example_dc_com_givenName.equality:Record Count is 0

dc_example_dc_com_givenName.presence:Record Count is 0

dc_example_dc_com_givenName.substring:Record Count is 0

dc_example_dc_com_objectClass.equality: Record Count is 2

dc_example_dc_com_member.equality:Record Count is 0

dc_example_dc_com_uid.equality:Record Count is 0

dc_example_dc_com_cn.equality:Record Count is 100000

dc_example_dc_com_cn.presence :Record Count is 1

dc_example_dc_com_cn.substring: Record Count is 212220

dc_example_dc_com_telephoneNumber.equality:Record Count is 0

dc_example_dc_com_telephoneNumber.presence:Record Count is 0 dc_example_dc_com_telephoneNumber.substring:Record Count is 0

dc_example_dc_com_sn.equality:Record Count is 1

dc_example_dc_com_sn.presence:Record Count is 1

dc_example_dc_com_sn.substring:Record Count is 6

dc_example_dc_com_ds-sync-hist.ordering:Record Count is 0

dc_example_dc_com_mail.equality:Record Count is 0

dc_example_dc_com_mail.presence:Record Count is 0

dc_example_dc_com_mail.substring:Record Count is 0

dc_example_dc_com_entryUUID.equality: Record Count is 100001

dc_example_dc_com_aci.presence:Record Count is 0

Total number of records in the databases is 612236 and the memory needed by locks is 612236\*142( Currenlty JE uses a 142 byte lock size for a record) =86937512 bytes = 82 MB. But it doesn't require 82MB of memory in one Go because Not all database locks are acquired together.

I won't go into the details of the lifespan of a database lock. I would rather do the performance comparions between two different batch sizes selected on the basis of looking at the cache misses and various other debug outputs from the berkeley db ( Thanks to Linda for taking so much of pain). The two tests were run with a batch size of 5K and 20K respectively. Whilst we saw almost marginal time difference between the two ( 5K was better), the 5K batch size exhibited a better memory footprint as most of the memory utilized in the deletes ( like lock information) gets released after a while. This provides more headroom compared to the other one for the future operations.

 

Fig 1: Graph of the cacheTotal, lockBytes, and adminBytes fields from the stats, which gives us a picture of how the cache is composed over time

Fig. 2: Graph of Cache Miss

The test run shows that both 5K and 20K runs have the same elapsed time. But JE's overall requirements for memory are falling in the 5K case. Probably  at the beginning of the deleteLeaf operations, JE has to pull a fair bit of the database into cache, in order to do the deletes. As the deletes progress, the btrees are pruned, and the overall size of the database is falling, so JE actually needs less cache for data. But the 20K batch case uses a bigger lump of memory for the locks, and that lump keeps JE's high water memory usage close to the limit of the 53Mb cache.

In the 5K batch case, the lock lumps are smaller, so JE's high water memory usage points are falling, which gives more margin. The 20K batch case has bigger cache miss spikes. The 5K batch case stays nice and even. (Though not really sure why the 5K batch has a bigger spike at the very beginning).

So since the elapsed time is the same, I'd go with the 5K batch.

My sincere thanks goes to Linda Lee because most of the statistics are based on the analysis of my test results by her.

Monday Jul 16, 2007

limitation of file descriptors in solaris

If fopen doesn't work for you because of the file descriptors limit, use PR_Open calls to fix the issue.

Thursday Jul 12, 2007

Flipping through albums

Another blog on iphone mp3 albums :). It rocks!

Setting artwork for mp3 albums in iphone.

If you are using ipod in your iphone and you don't have any album artwork, you are surely missing the fun. Generally if you buy an album from iTunes, or your rip a CD, you would get the artwork done for you. However, if you have your old mp3 songs you want to find an easy way for that, just see these instructions ;)

1. Add the songs to the iTune library and get the Artwork by doing a right click on a particular song or all the songs.

2. If the above doesn't work , it means that iTunes couldn't find an album artwork for you. In this case, you gotta burn little energy by opening images.google.com. Search for the image, copy it and paste it at the artwork box shown in the picture. You can select all the songs of an album/artist to apply the same image.

Monday Jul 02, 2007

Implementing Opends Back-end using Sleepycat

Like Previous versions of directory servers, opends backend is built upon sleepycat database. However, unlike previous versions, the sleepycat database is purely Java-based. Sleepycat db provides the Database class which appears like a HashMap where both the keys and the values are represented using the DatabaseEntry objects. An Environment encompasses the databases and it's the unit for cache and transactions among the databases.

Fig. A Database has keys and values. A DatabaseEntry object represents both key and value. You may want to note that a DatbaseEntry object takes only the byte array.

 

 

 

 

 

 

 

  The three main classes of the Opends backend are:

  RootContainer - Represents a backend id. It encapsulates the JE Environment class.

  EntryContainer - Represents a base DN for the RootContainer.

  DatabaseContainer - Represents a particular database.

 As evident from the description, RootContainer contains 1..n EntryContainer objects.

  

  Opends creates a number of databases related to the actual data and the necessary indexes. DBDump utility from Sleepycat can be used to dump the content of a database. For ex,

  bash-2.03$ java com.sleepycat.je.util.DbDump -h ../db -p -s dc_example_dc_com_dn2id

 dc=example,dc=com
 \\00\\00\\00\\00\\00\\00\\00\\01
 cn=user#0,dc=example,dc=com
 \\00\\00\\00\\00\\00\\00\\00\\02
 cn=user#10,dc=example,dc=com

 

 bash-2.03$ java com.sleepycat.je.util.DbDump -h ../db -p -s dc_example_dc_com_id2entry

 \\00\\00\\00\\00\\00\\00\\00\\01
 \\01`h\\02\\01\\00\\04c\\01\\11dc=example,dc=com\\12organizationalUnit\\01ou\\00\\01\\06people\\01entryUUID\\00\\01$54c89c45-327b-3d38-8a2e-b7814de651a4
 \\00\\00\\00\\00\\00\\00\\00\\02
 \\01`x\\02\\01\\00\\04s\\01\\1bcn=user#0,dc=example,dc=com\\0dinetorgperson\\02sn\\00\\01\\06commoncn\\00\\01\\06user#0\\01entryUUID\\00\\01$f1ae0607-4fcf-3878-813b-fff5cc871b37

 Opends Uses a 8-byte AtomicLong to create the entryId.

 An algorithm for the LDAP ADD operation ( Small details ignored)

 Connection Handler and then LDAPRequest Handler thread receives the request from the Client.
 The Add request is passed on to the pool of worker threads
 A worker thread performs the following operations in sequence ( some operations are omitted):
 Grab a READ Lock on the parent entry to ensure that the parent entry doesn't undergo any changes which this operation is in progress.
 Grab a WRITE Lock on the entry to be created.
 Search for the entry in the backend to make sure that it doesn't exist.
 Get the Backend instance for that entry dn. It is found by searching into the TreeMap maintained by the DirectoryServer class.
 Find the EntryContainer associated with the entry DN.
 Create a new Java DB Transaction Object
 Open dn2id database and verify that the key with the dn doesn't exist already.
 Read the EntryID of the parent from dn2id database.
 Get a new EntryID from the RootContainer for the new Entry.
 Insert the new Entry into dn2id database.
 Insert into Id2Entry database.
 Insert into Indexes
 Insert into id2children and id2subtree databases
 Commit/Rollback the transacation
 Return

 

You can find more information onSleepycat andOpends integration in this presentation

Friday Jun 22, 2007

Some information on the Class Loaders

It's just a recap of the class loaders from my experience.


Class Loaders use a delegation model to delegate the loading of a class to its parent class loaders recursively before handling it.

Arrays don't have any class loaders. The class loader of an array is same as of the element type. Therefore, an array of primitive values doesn't have any class loader.


a. Bootstrap class loader is a native implementation in the JVM and it doesn't have any parent. If you do a getClassLoader() on the class loaded by this  loader, it would return NULL.

 ex. String.class.getClassLoader() = null.< /FONT >

b. ClassLoader for extension (JRE/lib/ext) - sun.misc.Launcher$ExtClassLoader is used for loading the above classes. In some implementations, it is
                combined with bootstrap class loader.
 
c. Delegation model
 bootstrap <- ExtClassLoader <- AppClassLoader

d. BootstrapClass loader loads the classes from the local file system in a platform-dependent manner. It loads  classes of java.\* ,javax.\* and org.omg.\*, and if the search is made on those loaded classes to find out the class loaders, it would return NULL.

How does a ClassLoader provide security ?

Class loaders play a critical role in providing security because the class loader is responsible for locating and fetching the class file, consulting the security policy and defining the class object with the appropriate permissions.

What is the algorithm for loading a class ?

   a. Check if the class is loaded in the cache.
   b. If the current class loader has a delegation parent , redirect the loading request to that. Otherwise, redirect it to the bootstrap class loader.
   c.  If not found, call a customized method to find the class from the current classloader.

   In pseudo code
   findLoadedClass()
   parent.loadClass()
   findClass()

What are the rules for loading classes in terms of individual class loaders?

    a. First class of an application: AppClassLoader
    b. First class of an applet: AppletClassLoader
    c. class.forName("someclass") : bootstrap classLoader
   d. A reference inside a loaded class: current class loader

 a delegation model

   URLClassLoader::findClass()
   AppClassLoader::loadClass()
   classLoader::loadClass()
   ClassLoader::loadClassInternal() -> Called by JVM

What all does a classloader do?

     a.  produces a stream of binary data.
     b. parse the binary data into internal structures in the method area of JVM.
     c. creates an instance of Class to represent the type.

 

Austin downtown tour

Well..Austin isn't safe. A bunch of robbers robbing the Opends team in the broad daylight ;).

Wednesday Jan 31, 2007

Water for OSCAR

It's an irony that an Indian movie was contending another classic Indian movie for the oscars. Rang De Basanti ( known as RDB because of it's length) was an official entry from India. Whereas, Water is ( yeah..still) an official entry from Canada.

Some people do take a pride in the fact that it's the first Hindi movie being nomiated as Canada's official entry into the Oscars, but I don't. I feel pity for those stupid fundamentalists who bereft India of a classic movie made on the real data and facts.

About

This is the blog of a software engineer, specialized in identity management. Kunal Sinha works in Directory Services Engineering (OpenDS) team from Austin,Texas.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks