Monday Jul 14, 2014

Berkeley DB 12cR1 ( released

New Releases of Oracle Berkeley DB

Available For Download Now
Release Update

The newest release of Berkeley DB 12cR1 ( is available now. Here is a summary of the new features:

      upgrades to in-memory OLTP throughput & performance
HA improvement to identify a single master in a 2 site replication group
HA useability improvements
new Blob support added into replication
removed need for fail check monitor process
reduced the time for a database backup
and a lot more!

Berkeley DB continues to enable the most powerful embedded database solutions

      Handle TBs of data with a 1MB library
Flexible, lightweight storage engine, small footprint
Runs on low power ARM devices to cluster of high-end servers
Over 50 open source software projects embed BDB -- check them out on Wikipedia
Completely customizable, choose from 5 different access methods
Industrial quality and battle tested with over 200 million deployments

BDB is hands-down the best edge, mobile, and embedded database available to developers.  With the flexibility to place log files and/or database in any directory, applications can easily take advantage of the IO performance of flash caches, flash disks or SSDs.

Top notch performance

  • Berkeley DB performs over 5 million operations a second on a 100GB database running on a 1/8 rack Exadata V2 Database Machine configured with 256GB RAM and 12 cores.
  • Berkeley DB can insert 100 thousand records in 72 milliseconds and read those records in 30 milliseconds, running on a 8 core XEON-based commodity server.  The records contain a 4 byte key and a 48 byte data value.  This was run using the benchmark described on pages 34-36 in an ebook on SQL Server 2014 from Microsoft Press.
  • To put this into perspective, we compared Berkeley DB to SQL Server 2014's In-Memory OLTP feature (code name Hekaton)  which has similar technology.  Berkeley DB, an open source product, is about 20% faster than SQL Server 2014 which takes 94 milliseconds for the same 100k insert operations on an 8 core X64 Intel commodity box with 14 GB memory.

    We are making available a benchmark program in C that can be configured to validate Berkeley DB throughput for the 100K insert test here.

What folks are saying:

Open source Fedora package maintainer, Lubomir Rintel, says "Berkeley DB has quietly served behind the scenes as the database for the RPM Package Manager.   It has proven itself time and time again as a robust and efficient storage engine.   It stores the meta information of the installed rpms.  Under heavy workloads, BDB proves itself reliable. Countless people that use popular Linux distributions have used BDB through RPM and never knew it.  With this new release,   BDB continues its tradition of being a solid storage engine"

Oracle Tape Product Manager, Dan Deppen, says "Berkeley DB is integral to Oracle StorageTek Storage Archive Manager (SAM-QFS).  We have been embedding Berkeley DB in our product for over a decade and it is vital to our disk archiving feature which is used to send files to remote data centers to enable disaster recovery.  Performance and scalability are critical because SAM-QFS supports some of the largest archive customers in the world.   HPC sites, research centers, national libraries and other customers requiring massive scalability and high reliability depend on SAM-QFS and Berkeley DB to maintain availability of their critical data."

  Oracle Identity Management Vice President, Shirish Puranik, says "Berkeley DB is a critical component of Oracle Unified Directory (OUD) and Oracle Directory Server Enterprise Edition (ODSEE).  We have been using Berkeley DB in these products as a high performance, transaction embedded database repository for several years.  Berkeley DB has exceed our expectations for performance and stability. Our Berkeley DB based products are widely deployed in production at largest telcos and financial institutions all over the world.   The improvements for BLOB support and high availability in the 6.1 release are welcome."

Software Downloads


Please direct product questions to our Product Help Mail list for Berkeley DB. You can also email the Berkeley DB Product Management team directly.

Berkeley DB Product Management
Internal -

Thursday Jun 12, 2014

Data management in unexpected places

Data management in unexpected places

When you think of network switches, routers, firewall appliances, etc., it may not be obvious that at the heart of these kinds of solutions is an engine that can manage huge amounts of data at very high throughput with low latencies and high availability.

Consider a network router that is processing tens (or hundreds) of thousands of network packets per second. So what really happens inside a router? Packets are streaming in at the rate of tens of thousands per second. Each packet has multiple attributes, for example, a destination, associated SLAs etc. For each packet, the router has to determine the address of the next “hop” to the destination; it has to determine how to prioritize this packet. If it’s a high priority packet, then it has to be sent on its way before lower priority packets. As a consequence of prioritizing high priority packets, lower priority data packets may need to be temporarily stored (held back), but addressed fairly. If there are security or privacy requirements associated with the data packet, those have to be enforced. You probably need to keep track of statistics related to the packets processed (someone’s sure to ask). You have to do all this (and more) while preserving high availability i.e. if one of the processors in the router goes down, you have to have a way to continue processing without interruption (the customer won’t be happy with a “choppy” VoIP conversation, right?). And all this has to be achieved without ANY intervention from a human operator – the router is most likely to be in a remote location – it must JUST CONTINUE TO WORK CORRECTLY, even when bad things happen.

How is this implemented? As soon as a packet arrives, it is interpreted by the receiving software. The software decodes the packet headers in order to determine the destination, kind of packet (e.g. voice vs. data), SLAs associated with the “owner” of the packet etc. It looks up the internal database of “rules” of how to process this packet and handles the packet accordingly. The software might choose to hold on to the packet safely for some period of time, if it’s a low priority packet.

Ah – this sounds very much like a database problem. For each packet, you have to minimally

· Look up the most efficient next “hop” towards the destination. The “most efficient” next hop can change, depending on latency, availability etc.

· Look up the SLA and determine the priority of this packet (e.g. voice calls get priority over data ftp)

· Look up security information associated with this data packet. It may be necessary to retrieve the context for this network packet since a network packet is a small “slice” of a session. The context for the “header” packet needs to be stored in the router, in order to make this work.

· If the priority of the packet is low, then “store” the packet temporarily in the router until it is time to forward the packet to the next hop.

· Update various statistics about the packet.

In most cases, you have to do all this in the context of a single transaction. For example, you want to look up the forwarding address and perform the “send” in a single transaction so that the forwarding address doesn’t change while you’re sending the packet. So, how do you do all this?

Berkeley DB is a proven, reliable, high performance, highly available embeddable database, designed for exactly these kinds of usage scenarios. Berkeley DB is a robust, reliable, proven solution that is currently being used in these scenarios.

First and foremost, Berkeley DB (or BDB for short) is very very fast. It can process tens or hundreds of thousands of transactions per second. It can be used as a pure in-memory database, or as a disk-persistent database. BDB provides high availability – if one board in the router fails, the system can automatically failover to another board – no manual intervention required. BDB is self-administering – there’s no need for manual intervention in order to maintain a BDB application. No need to send a technician to a remote site in the middle of nowhere on a freezing winter day to perform maintenance operations.

BDB is used in over 200 million deployments worldwide for the past two decades for mission-critical applications such as the one described here. You have a choice of spending valuable resources to implement similar functionality, or, you could simply embed BDB in your application and off you go! I know what I’d do – choose BDB, so I can focus on my business problem. What will you do?

Wednesday Jun 27, 2012

Yammer, Berkeley DB, and the 3rd Platform

If you read the news, you know that the latest high-profile social media acquisition was just confirmed. Microsoft has agreed to acquire Yammer for 1.2 billion. Personally, I believe that Yammer’s amazing success can be mainly attributed to their wise decision to use Berkeley DB Java Edition as their backend data store. :-)

I’m only kidding, of course. However, as Ryan Kennedy points out in the video I recently blogged about, BDB JE did provide the right feature set that allowed them to reliably grow their business. Which in turn allowed them to focus on their core value add. As it turns out, their ‘add’ is quite valuable!

This actually makes sense to me, a lot more sense than certain other recent social acquisitions, and here’s why. Last year, IDC declared that we are entering a new computing era, the era of the “3rd Platform.” In case you’re curious, the first 2 were terminal computing and client/server computing, IIRC. Anyway, this 3rd one is more complicated. This year, IDC refined the concept further. It now involves 4 distinct buzzwords: cloud, social, mobile, and big data.

Yammer is a social media platform that runs in the cloud, designed to be used from mobile devices. Their approach, using Berkeley DB Java Edition with High Availability, qualifies as big data. This means that Yammer is sitting right smack in the center if IDC’s new computing era. Another way to put it is: the folks at Yammer were prescient enough to predict where things were headed, and get there first.

They chose Berkeley DB to handle their data. Maybe you should too!

Tuesday Jun 05, 2012

Highlights from recent Yammer video

A few weeks back, Ryan Kennedy of Yammer gave a talk about Berkeley DB Java Edition. You can find it posted here on Alex Popescu's Blog, or go directly to the video post itself. It was full of useful nuggets of information, such as why they chose to use BDB JE, performance, and some tips & tricks at the end. At over 40 minutes, the video is quite long. Ryan is an entertaining speaker, so I suggest you watch all of it. But if you only have time for the highlights, here are some times you can sync to:

 06:18 hear the Berkeley DB JE features that caused Yammer select it, including:

  • replication
  • auto leader election, failover
  • configurable durability and consistency guarantees

23:10 System performance characteristics

35:08 Check out the tips and tricks for using Berkeley DB JE

I know the Berkeley DB development team is very pleased that BDB JE is working out well for Yammer. We definitely encourage others out there to take note of this success, especially if your requirements are similar to Yammer's (which Ryan outlines at the beginning of his talk)

Monday Feb 27, 2012

Now you can build Berkeley DB into your Android apps

I want to make everyone aware of a small change we made in the last release that promises to make a big difference for our customers using Android.

We added support for the Android platform some time back. However, there was a caveat: Berkeley DB had to be integrated at the OS level, replacing the SQLite routines that usually ship with Android. Having Berkeley DB built into the OS provided some advantages. But it also meant that customers were unable to build a BDB-enabled app that could be deployed to any generic Android device. Also some customers were hesitant to make OS modifications themselves.

In our latest release, Berkeley DB 5.3, we have added the capability to use BDB on a per-application basis. This means you can build your application to use Berkeley DB, and the library routines will be bundled in when you package everything up. The result is an application that can take advantage of Berkeley DB’s strengths on any Android device. We've even included instructions on how to achieve this in our online documentation.

Berkeley DB supports a wide array of mobile platforms. Starting with Oracle’s own embedded Java, which many forget still dominates the feature phone market. BDB also supports Android and iOS, the two platforms that continue to own the smartphone marketplace. This latest enhancement will make it even easier for Android developers to use BDB. Finally, we offer the option to choose between a SQLite-compatible SQL API, and our traditional key/value API (for you NoSQL fans out there). Our combination of features and platform support make Berkeley DB the best choice for anyone who needs an enterprise-grade data store on a mobile platform.

Friday Feb 17, 2012

2011 Was a Banner Year for Berkeley DB!

I’ve been wanting to write this post since December, and only just now getting around to it. Fortunately, the contents of the post itself will serve as a pretty good explanation for the delay. :-)

I realize we’re already in February here. But I still think it’s worth mentioning what an amazing year 2011 was for the Berkeley DB team. Therefore I’d like to take a moment and look back at the major events of 2011, product by product.

Berkeley DB:

We set the bar pretty high, with TWO major releases in one year: 5.2 and 5.3. Both releases added exciting new features, too many to list here. Some of the main areas to receive enhancements were the SQL API, High Availability, HEAP, and XA. Expanded platform support now includes more Java platforms and per-application Android support.

Berkeley DB Java Edition:

Serving as the basis for a major new Oracle product (NoSQL Database) might have been enough for some products. But not Berkeley DB JE! In addition to that accomplishment, the team put out a major release as well, 5.0. This release included the much anticipated DiskOrderedCursor class, a plethora of performance improvements, and some High Availability enhancements to boot.

Revamped product launch: Database Mobile Server 11g

Database Mobile Server 11g is Oracle’s newly re-architected sync and device management solution for mobile, embedded and M2M platforms. It replaces Database Lite, and is designed to use Berkeley DB as the remote data store. DMS, as we call it, is also compatible with the very popular open-source SQLite database. There has been a lot of focus on mobile and embedded/M2M as they relate to enterprise recently, and every indicator points to these topics becoming even more important. This one is definitely a product to keep an eye on.

New Product Launch: NoSQL Database

I’m not prepared to say I saved the “best” for last, since all these products are truly great. But certainly NoSQL Database product got a huge amount of attention last year, and deservedly so. Oracle’s entry into this red-hot space promises to be exactly what many of our customers have been waiting for. The data management solution to their Big Data problem from the company they already trust to store the rest of their data.

To recap, that’s 3 major product releases, and 2 product launches. All from one team. That is amazing, by any standard. Congratulations to the development team, your work output last year was the stuff of legends! Of course, these folks are not ones to rest on their laurels. I know for a fact they are already hard at work on future versions of all the products listed above, and Berkeley DB XML too. The best is yet to come!

Tuesday Nov 29, 2011

Announcing Berkeley DB Java Edition Major Release

Berkeley DB Java Edition 5.0 was just released. There are a number of new features, enhancements, and options in there that our users have been asking for. Chief among them is a new class called DiskOrderedCursor, which greatly increases performance of systems using spinning platter magnetic hard drives. A number of users expressed interest in this feature, including Alex Feinberg of LinkedIn. Berkeley DB Java Edition is part of Project Voldemort, a distributed key/value database used by LinkedIn.

There have been many other improvements and optimizations. Concurrency is significantly improved, as is the performance of update and delete operations. New and interesting methods include Environment.preload, which allows multiple databases to be preloaded simultaneously. New Cursor methods enable for more effective searching through the database.

We continue to enhance Berkeley DB Java Edition’s High Availability as well. One new feature is the ability to open a replicated node read-only when the master is unavailable. This can allow critical systems to continue offering some functionality, even during a network or master node failure.

There’s a lot more in release 5.0. I encourage you to take a look at the extensive changelog yourself. As always, you can download the new release and try it out here:

Thursday Sep 22, 2011

Berkeley DB at OpenWorld/JavaOne 2011

It’s the question on everyone’s mind: what is Berkeley DB bringing to OpenWorld this year? Even if you’re more preoccupied with the latest iPhone 5 rumors, (I hear the front facing camera can look into your eyes and tell you what you had for breakfast) Berkeley DB users (or fans) who are planning to attend Oracle OpenWorld in San Francisco about 10 days from now will want to read this post.

As always, we will have a session where you can learn about the cool and interesting things our customers are doing with a key/value data store. But this year, we have two additional general areas of focus: Embedded, and Mobile Applications.

Let’s take a closer look at the embedded front first. As the cost of components like network hardware, flash memory, and microcontrollers continues to fall, OEMs in many verticals are choosing to enhance their product lineup by adding new applications and internet-enabled features. Networked cash registers? That was only the beginning. This was the year cars started getting on the internet, and I already blogged about that. Who knows what the next years will bring?

Whatever our customers are planning, we’ve got solutions that can help. Our first session with an embedded focus is 15185, BDB and Embedded Java presentation. There you’ll learn how we are extending Java’s “write once, deploy anywhere” mantra. With Berkeley DB and Database Mobile Server, you get worry free data management and sync capabilities after you’ve deployed, as well.

Next, over on the JavaOne side we have 25143, Telemetry and Synchronization with Embedded Java and Berkeley DB. This session will feature Java Architect Greg Bollella talking about how these same technologies are enabling telemetry solutions to plug into the enterprise right alongside your existing data and apps. The ability to manage networks of embedded devices using existing enterprise frameworks could prove to be quite revolutionary. The embedded Java platform, when coupled with Berkeley DB and Database Mobile Server, has the ability to do just that. We’re excited about this, and we think our customers will be too.

On the mobile applications side of things Tata Consulting Services (TCS) will be joining us for session 15178, Achieve Ubiquitous Data Access, from Device Databases to Enterprise Repositories. There you’ll learn how TCS is helping their customers deploy mobile applications that maximize the ROI of their existing enterprise infrastructure.

Rounding out the list is our key/value customer highlight session. You thought I forgot, right? No chance! Session 15167 is entitled Transactional Key-Value Storage: Super Simple, Super Fast, Super Flexible. Raghunath Nambiar, an Architect at Cisco, will co-present with us. The topic will be super cool things you can accomplish using the key-value paradigm for data management.

In addition to the sessions, we will have a number of exciting demos for you to check out, both at OpenWorld and JavaOne.

Finally, be on the lookout for an exciting product announcement building on the inherent strengths of the Berkeley DB product family!

Hope to see you there!

Tuesday Jul 12, 2011

Customer Highlight: AdaptiveMobile

Last week AdaptiveMobile announced they signed a 3 year agreement to embed Oracle Berkeley DB in their Network Protection Platform, a key component of their product portfolio.

AdaptiveMobile is the world leader in mobile security, enabling trusted networks for the world’s largest operator groups and protecting one in six subscribers globally. AdaptiveMobile provides operators with the most comprehensive network-based security solutions enabling them to protect their consumer and enterprise customers against the growing threat of mobile abuse. From the announcement: “AdaptiveMobile selected Oracle Berkeley DB for its outstanding data retrieval speeds, reliability, scalability and availability and its integration with the company’s existing application environment.”

Here is a great quote from Gareth Maclachlan, AdaptiveMobile Chief Operating Officer: “Oracle’s Berkeley DB has enabled us to develop a solution that resolves the content and access control challenges of both network operators and their customers without impacting the user experience. “Working with Oracle has allowed us to reduce our development time and cut our total cost of ownership for network operators by helping us reduce our hardware and administration costs.”

For more information, you can see the announcement, or learn more about AdaptiveMobile’s products.

Wednesday Jun 22, 2011

New Release of Oracle Berkeley DB

We are pleased to announce that a new release of Oracle Berkeley DB, version, is available today.

Our latest release includes yet more value added features for SQLite users, as well as several performance enhancements and new customer-requested features to the key-value pair API.  We continue to provide technology leadership, features and performance for SQLite applications.  This release introduces additional features that are not available in native SQLite, and adds functionality allowing customers to create richer, more scalable, more concurrent applications using the Berkeley DB SQL API.

This release is compelling to Oracle’s customers and partners because it:

  • delivers a complete, embeddable SQL92 database
  • as a library under 1MB size
  • drop-in API compatible with SQLite version 3
  • no-oversight, zero-touch database administration
  • industrial quality, battle tested Berkeley DB B-TREE for concurrent transactional data storage

New Features Include:

  • MVCC support for even higher concurrency
  • direct SQL support for HA/replication
  • transactionally protected Sequence number generation functions
    • lower memory requirements, shared memory regions and faster/smaller memory on startup
  • easier B-TREE page size configuration with new ''db_tuner" utility

New Key-Value API Features Include:

  • HEAP access method for constrained disk-space applications (key-value API)
  • faster QUEUE access method operations for highly concurrent applications -- up 2-3X faster! (key-value API)
  • new X/open compliant XA resource manager, easily integrated with Oracle Tuxedo (key-value API)
    • additional HA/replication management and communication options (key-value API)

and a lot more!

BDB is hands-down the best edge, mobile, and embedded database available to developers.

Downloads available today on the Berkeley DB download page

Product Documentation


Information about Berkeley DB products directly from the people who build them.


« March 2015