The steps for using Berkeley DB Java Edition and Android have changed with the changes in Android 0.9. Here is the revised HOWTO. Thanks to Tao for putting this together.
A Simple HOWTO for Using Berkeley DB Java Edition on the Android Platform
3.3.62, June 04, 2008
Overview
JE can be used with the Google Android platform. This document discusses how to create a simple program and UI which will allow a user to do trivial JE "gets" and "puts" from within Android.
Installation
* Start by downloading and installing the Android SDK. Put the Android "tools" directory in your path so that you pick up the command line tools.
Steps With Android Eclipse Plugin
* Follow the installation instructions to install and configure the Android Eclipse plugin.
* Go to "File->New->Other Project->Android->Android Project". Then provide a Project name, application name , package name (com.sleepycat.je), and activity name (JEExample).
* Unpack the JE distribution (referred to as in this document), copy the contents of the /src directory to the directory where you want to store this Eclipse project's source code (referred to as in this document).
* Go to /src and delete the following directories: com/sleepycat/je/jmx and com/sleepycat/je/jca. Also delete these two source files: com/sleepycat/persist/model/ClassEnhancer.java and com/sleepycat/persist/model/ClassEnhancerTask.java.
* Make a new directory com/sleepycat/je/xa in /src and copy XAException.java, XAResource.java and Xid.java (shown at the bottom of this page) into this new directory.
JEExample.java to /src/com/sleepycat/je,
main.xml to res/layout/main.xml and
strings.xml to res/values/strings.xml
These files are shown at the bottom of this page.
* Build a new Configure Launch for this project, then run it as an Android project.
Steps Without Android Eclipse Plugin
* Create a work directory (referred to as in this document).
* Make a new directory /je, copy /src to /je.
* Go to /je/src and delete the following directories: com/sleepycat/je/jmx and com/sleepycat/je/jca. Also delete these two source files: com/sleepycat/persist/model/ClassEnhancer.java and com/sleepycat/persist/model/ClassEnhancerTask.java.
* Make a new directory com/sleepycat/je/xa in /je/src and copy XAException.java, XAResource.java and Xid.java (shown at the bottom of this page) into this new directory.
* Review /docs/intro/hello-android.html for instructions on how to create a project. Do
activityCreator --out com.sleepycat.je.JEExample
This creates /src/com/sleepycat/je/JEExample.java
* Replace the generated contents of /src/com/sleepycat/je/JEExample.java, /res/layout/main.xml, /res/values/strings.xml with the source code for JEExample.java, main.xml and strings.xml shown at the bottom of this page.
* Go to the /tools, open and edit the dx.bat file, change the last line from "call java -Djava.ext.dirs=%frameworkdir% -jar %jarpath% %*" to "call java -Xms256M -Xmx512M -Djava.ext.dirs=%frameworkdir% -jar %jarpath% %*".
* Run the Android emulator.
* In , do
ant install
This will compile JEExample.java as well as convert the resulting class files to dex files. This also creates /bin/JEExample-debug.apk and starts this application on the emulator.
Run the test
* Create the JE environment directory by doing:
adb shell mkdir /data/je
You can remove the JE Environment files by doing:
adb shell rm /data/je/*
* Run the JEExample application in the emulator by clicking on the Home icon, then the Applications folder, then the JEExample icon. A screen titled "JEExample" should appear with a TextEdit box, a "Put Data" button, and a "Get Data" button.
* Put some key/data pairs into the JE database by entering (e.g.)
k1/data1
and pressing the "Put Data" button.
* Retrieve ("get") key/data pairs from the JE database by entering (e.g.)
k1
and pressing the "Get Data" button.
Source Code
* com.sleepycat.je.xa package.
XAException.java
package com.sleepycat.je.xa;
public class XAException extends Exception {
public static final int XA_RBBASE = 0;
public static final int XA_RBROLLBACK = 0;
public static final int XAER_INVAL = 0;
public static final int XAER_NOTA = 0;
public static final int XAER_DUPID = 0;
public static final int XA_RBPROTO = 0;
public static final int XAER_PROTO = 0;
public XAException() {}
public XAException(String s) {}
public XAException(int e) {}
}
XAResource.java
package com.sleepycat.je.xa;
public interface XAResource {
public static final int TMENDRSCAN = 0;
public static final int TMFAIL = 0;
public static final int TMJOIN = 0;
public static final int TMNOFLAGS = 0;
public static final int TMONEPHASE = 0;
public static final int TMRESUME = 0;
public static final int TMSTARTRSCAN = 0;
public static final int TMSUCCESS = 0;
public static final int TMSUSPEND = 0;
public static final int XA_RDONLY = 0;
public static final int XA_OK = 0;
boolean setTransactionTimeout(int a)
throws XAException;
void start(Xid a, int b)
throws XAException;
void end(Xid a, int b)
throws XAException;
int prepare(Xid a)
throws XAException;
void commit(Xid a, boolean b)
throws XAException;
void forget(Xid a)
throws XAException;
Xid[] recover(int a)
throws XAException;
void rollback(Xid a)
throws XAException;
int getTransactionTimeout()
throws XAException;
boolean isSameRM(XAResource a)
throws XAException;
}
Xid.java
package com.sleepycat.je.xa;
public interface Xid {
int getFormatId();
byte[] getGlobalTransactionId();
byte[] getBranchQualifier();
}
@Override
public void onCreate(Bundle icicle) {
super.onCreate(icicle);
try {
final File envDir = new File("/data/je");
final EnvironmentConfig envConfig = new EnvironmentConfig();
envConfig.setTransactional(true);
envConfig.setAllowCreate(true);
final Environment env = new Environment(envDir, envConfig);
final DatabaseConfig dbConfig = new DatabaseConfig();
dbConfig.setTransactional(true);
dbConfig.setAllowCreate(true);
dbConfig.setSortedDuplicates(true);
final Database db = env.openDatabase(null, "exampledb", dbConfig);
setContentView(R.layout.main);
final Button button1 = (Button) findViewById(R.id.do_put);
button1.setOnClickListener(new Button.OnClickListener() {
public void onClick(View v) {
final EditText editText =
(EditText) findViewById(R.id.entry);
final String keyData = editText.getText().toString();
final int idx = keyData.indexOf("/");
String key = null;
String data = null;
String result = null;
if (idx < 0) {
result = "enter key/data to put";
} else {
key = keyData.substring(0, idx);
data = keyData.substring(idx + 1);
result = key + "/" + data;
final DatabaseEntry keyEntry =
new DatabaseEntry(key.getBytes());
final DatabaseEntry dataEntry =
new DatabaseEntry(data.getBytes());
try {
final Transaction txn =
env.beginTransaction(null, null);
final OperationStatus res =
db.put(txn, keyEntry, dataEntry);
if (res != OperationStatus.SUCCESS) {
result = "Error: " + res.toString();
}
txn.commit();
} catch (DatabaseException DE) {
result = "Caught exception: " + DE.toString();
}
}
Log.d("JE", "did put of: " + result);
if (result.contains("Caught exception:")) {
new AlertDialog.Builder(JEExample.this).
setTitle("Put Data").setMessage(result).
setPositiveButton("Quit", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int whichButton) {
}
}).show();
} else {
new AlertDialog.Builder(JEExample.this).
setTitle("Put Data").setMessage("You put the key/data pair: " + result).
setPositiveButton("Confirm", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int whichButton) {
}
}).show();
}
}
});
final Button button2 = (Button) findViewById(R.id.do_get);
button2.setOnClickListener(new Button.OnClickListener() {
public void onClick(View v) {
final EditText editText =
(EditText) findViewById(R.id.entry);
final String key = editText.getText().toString();
final DatabaseEntry keyEntry =
new DatabaseEntry(key.getBytes());
final DatabaseEntry dataEntry = new DatabaseEntry();
String result = null;
try {
final Transaction txn =
env.beginTransaction(null, null);
final OperationStatus res =
db.get(txn, keyEntry, dataEntry, null);
if (res != OperationStatus.SUCCESS) {
result = "Error: " + res.toString();
} else {
result = new String(dataEntry.getData());
}
txn.commit();
} catch (DatabaseException DE) {
result = "Caught exception: " + DE.toString();
}
Log.d("JE", "did get of: " + result);
if (result.contains("Caught exception:")) {
new AlertDialog.Builder(JEExample.this).
setTitle("Get Data").setMessage(result).
setPositiveButton("Quit", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int whichButton) {
}
}).show();
} else {
new AlertDialog.Builder(JEExample.this).
setTitle("Get Data").setMessage("Get result: " + result).
setPositiveButton("Confirm", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int whichButton) {
}
}).show();
}
}
});
} catch (Exception DE) {
TextView tv = new TextView(this);
tv.setText("blew chunks " + DE);
setContentView(tv);
}
}
}
This story started out with a user telling us that Berkeley DB Java Edition's group commit feature didn't work very well on ext3. The reason is that we only group fsync's and not writes & fsyncs. And ext3 takes an exclusive mutex on the inode for any IO operation, whereas Windows and Solaris don't (i.e. they don't block concurrent seek/read/write operations while an fsync is happening). So I set off to fix this problem for some upcoming release.
While debugging my new group-commit code in JE, I was seeing occasional DbChecksumExceptions. The odd thing was that when I examined the log file, the last set of records in the log file were fine except that the "prevOffset"pointers were all pointing later, rather than earlier. JE's log file records all have a prevOffset pointer which allows it to walk backwards in the log file. So it would be just plain wrong if one of those prevOffset pointers actually pointed forward instead of behind. It was if there were two writes going on, but one of the buffers overlaid the other.
The DbChecksumException was fairly reproducible in my test so I set out to put some debugging code in. For instance, I put in a check which made sure that just prior to the write() call to the log that the first record in the buffer being written had a prevOffset that was earlier rather than later. That check never fired and yet the results were there on disk: records with prevOffsets later rather than earlier.
So then I put in a check which read the buffer right back immediately after writing it to disk. This check would occasionally fire. But why did it fire when the check just prior to write() didn't?
I went so far as to repeat the read-after-write if it failed the first time. It would always pass on a subsequent read.
So the sequence was something like this:
seek(); write();
repeat until success { seek; read(); }
It appeared that something else was changing the underlying file pointer between the seek() and the read() so that the read was reading from the wrong place.
I sprinkled lightweight event tracing code throughout the relevant methods, catching all seek, read, write, and fsync events. But they all looked correct.
Finally I ran my code with "truss" so that I could see all the system calls. The output showed something like this:
So sure enough, there was another thread (Thread 2) messing with the underlying file descriptor (13) right between my seek() and my read(). But all of our seeks() had been covered with event tracing. All of our uses of the RandomAccessFile for seek, read, write, and fsync were synchronized so it seemed "impossible" that some other thread could actually be doing a seek call on the RAF.
I finally narrowed it down to a call to RandomAccessFile.length() in an assert statement (my test was running with -ea). Our assumption in writing the code had been that RAF.length() would not side-effect the file descriptor. But looking at the source code, it is clear that it is doing a SEEK_CUR, SEEK_END, SEEK_SET sequence which is not thread-safe. That's certainly not what one would expect.
A quick search for this reveals that others have been bitten by this. We'll accept responsibility for this one since we should have made all method calls on RandomAccessFile be synchronized, but I have to confess that this was subtle.
