Derby 10.5 preview: In-memory backend

The upcoming Derby 10.5 release will contain a long-awaited feature: an in-memory storage backend. With this backend, your entire database will be stored in main memory instead of on disk.

But isn't the whole point of using a database that the data should be stored safely on disk or some other kind of persistent storage? Normally, yes, but there are cases where you don't really care if you lose the database when the application crashes.

For instance, if you are running unit tests against your application, it's probably more important that the tests run fast and that it's easy to clean up after the tests. With the in-memory storage backend, you'll notice that many database operations (like database creation, inserts and updates) are a lot faster because they don't need to access the disk. Also, there's no need to clean up and delete the database files after the tests have completed, since the database goes away when the application terminates.

So how is the in-memory backend enabled? That's simple, you just add the memory subprotocol to the JDBC connection URL, and no other changes should be needed to your application. If you normally connect to the database with the URL jdbc:derby:MyDB you should instead use jdbc:derby:memory:MyDB (and of course add any of the connection attributes needed, like create=true to create the database). Here's an example in IJ, Derby's command line client:

$ java -jar derbyrun.jar ij
ij version 10.5
ij> connect 'jdbc:derby:memory:MyDB;create=true';
ij> create table my_table(x int);
0 rows inserted/updated/deleted
ij> insert into my_table values 1, 2, 3;
3 rows inserted/updated/deleted
ij> exit;

After exiting, you can verify that no database directory was created:

$ ls MyDB
MyDB: No such file or directory

More or less everything you can do with an ordinary database should be possible to do with an in-memory database, including taking a backup and restoring it. This can be useful, as it allows you to dump the database to disk before you shut down your application, and reload it into memory the next time you start the application. Looking again at the example above, you could issue this command before typing exit in IJ:

ij> call syscs_util.syscs_backup_database('/var/backups');
0 rows inserted/updated/deleted

Later, when you restart the application, you can load the database back into the in-memory store by using the createFrom connection attribute:

$ java -jar derbyrun.jar ij
ij version 10.5
ij> connect 'jdbc:derby:memory:MyDB;createFrom=/var/backups/MyDB';
ij> select \* from my_table;
X          
-----------
1          
2          
3          

3 rows selected
Comments:

Awesome!

Posted by Ashwin Jayaprakash on April 22, 2009 at 12:51 AM CEST #

This looks great! Is it possible to shadow/copy a table in a persistent database into an in-memory DB (use the in-memory DB as a read-only cache)?

Posted by F Stokke on April 22, 2009 at 05:41 AM CEST #

F Stokke: The createFrom connection attribute could be used to load a database, the same way as you create a database from a backup:

connect 'jdbc:derby:memory:cachedb;createFrom=/path/to/persistent/db';

But that would shadow the entire database, not just a single table. If you just need a single table, you probably need to import it manually (using the export/import procedures, or perhaps a table function that accesses the other database).

Posted by Knut Anders Hatlen on April 22, 2009 at 06:05 AM CEST #

Very cool. Has anyone tried to use this with Terracotta? It seems their "network attached memory" system would be perfect for a large in memory database.

Posted by Collin on April 22, 2009 at 08:52 AM CEST #

Post a Comment:
Comments are closed for this entry.
About

kah

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today