X

Learn all about Oracle BerkeleyDB family of databases here. Scalable and high-performance data management services.

Recent Posts

Oracle Berkeley DB Java Edition - New Release

Oracle Berkeley DB Java Edition (JE) – Version 7.5 Oracle Berkeley DB Java Edition 7.5 is now available for download. Download Page – Click Here Documentation Page – Click Here Oracle Berkeley DB Java Edition Version 7.5 – New Features New configuration parameters have been added for limiting disk usage and read operations are now allowed when these thresholds are exceeded.  Benefit: Applications continue to run even when the disk usage > disk usage limit. The data verifier has been enhanced to perform Btree verification in  order to detect internal corruption. Benefit: Corruption can be detected early and can be checked on-demand. Berkeley DB JE continues to enable the most powerful embedded database solutions Pure Java library Handle TBs of data with a small, lightweight, embedded library Flexible, transactional data management engine Runs on a variety of operating systems and platforms. Over 50 open source software projects embed BDB -- check them out on Wikipedia Various options for APIs Industrial quality and battle tested Supporting Assets and Resources: Collateral and sales assets continue to be updated. Oracle Berkeley DB Web Site Oracle Berkeley DB blog Oracle Berkeley DB Java Edition Datasheet Oracle Berkeley DB Family Datasheet LinkedIn Group – https://www.linkedin.com/groups/2131985 Twitter – @berkeleydb Community Berkeley DB – https://community.oracle.com/community/database/high_availability/berkeley_db_family/berkeley_db Community for Berkeley DB Java Edition - https://community.oracle.com/community/database/high_availability/berkeley_db_family/berkeley_db_java_edition Blogs - https://blogs.oracle.com/berkeleydb/ Questions & Help Questions: External Email Alias – bdb@oss.oracle.com

Oracle Berkeley DB Java Edition (JE) – Version 7.5 Oracle Berkeley DB Java Edition 7.5 is now available for download. Download Page – Click Here Documentation Page – Click Here Oracle Berkeley DB Java...

Berkeley DB

Oracle Berkeley DB Java Edition 7.0 Announced

Oracle Berkeley DB JavaEdition 7.0 is now available for download. DownloadPage – Click Here Documentation Page –Click Here Oracle Berkeley DB JavaEdition Version 7.0 – New Feature · Time-To-Live –efficient aging out of “expired” data – This is a common IoT requirement. Berkeley DB JE continues to enable the mostpowerful embedded database solutions · Pure Java library · Handle TBs of data with a small, lightweight,embedded library · Flexible, transactional data managementengine · Runs on a variety of operating systems andplatforms. · Over 50 open source software projects embedBDB -- check them out on Wikipedia · Various options for APIs · Industrial quality and battle tested Supporting Assets and Resources: Collateral and sales assetscontinue to be updated. · OracleBerkeley DB Web Site · Oracle Berkeley DB blog · OracleBerkeley DB Java Edition Datasheet · OracleBerkeley DB Family Datasheet LinkedIn Group – https://www.linkedin.com/groups/2131985 Twitter – @berkeleydb Community Berkeley DB – https://community.oracle.com/community/database/high_availability/berkeley_db_family/berkeley_db Community for Berkeley DBJava Edition - https://community.oracle.com/community/database/high_availability/berkeley_db_family/berkeley_db_java_edition Blogs - https://blogs.oracle.com/berkeleydb/

Oracle Berkeley DB Java Edition 7.0 is now available for download. Download Page – Click Here Documentation Page – Click Here Oracle Berkeley DB Java Edition Version 7.0 – New Feature · Time-To-Live –effici...

Berkeley DB

BDB User Authentication

BDB User Authentication(courtesy of  Brayden Zhang)Introduction In Berkeley DB 6.2, user authentication extension is added into the SQL interface. The new user authentication layer is both secure and easy to use. In this article, we will introduce how to use the user authentication, and also explain some details. The SQLite user authentication User authentication was first included in SQLite 3.8.7, and was described well in the following page: http://www.sqlite.org/src/doc/trunk/ext/userauth/user-auth.txt  There are four user authentication APIs: sqlite3_user_authenticate(conn, username, pwd, nPwd); sqlite3_user_add(conn, username, pwd, nPwd, isAdmin); sqlite3_user_change(conn, username, pwd, nPwd, isAdmin); sqlite3_user_delete(conn, username);  A database becomes a user authentication required database once the first user is added with a sqlite3_user_add() call. To visit an authentication-required SQLite database, an authenticated user must be logged into the database connection first; otherwise attempts to read or write from the database will fail with errors. From the implementation view, SQLite stores user/password-hash in an internal sqlite_user table and verifies user login with that table. This SQLite userauth will be activated in Berkeley DB SQL layer with - DBDBSQL_USER_AUTHENTICATION compile option. Security considerations about the SQLite userauth Let’s rethink about user authentication. When we are talking about user login, we mean we are putting the sensitive data into a safe zone. If a user need to access the data, we will ask the user for his name/password. Once we are sure the user is authenticated, we will let him into the safe zone. Thinking about a client/server Database, it provides the safe zone as the data is physically stored in a backend host. Next let’s go back into the SQLite userauth, where is the safe zone? I guess the answer is: “it does not exist (yet).” It will be SQLite application’s resposibilly to provide it. In some cases it should be not so easy, such as an Android application. Anyone who has the access to this Android device could read the database file in binary editor; also if you built a SQLite without the userauth compile option, you could access an authentication-required database freely(no need for a login)  Encryption should be a common(and neccesary) way to provide a safe zone for SQLite user authentication. In SQLite, to visit an encrypted authentication-required database, we need a call sequence as: sqlite3_open_v2(); sqlite3_key_v2(); sqlite3_user_authenticate(); /* Database is now usable */  The next question is: who will provide the key? The application could keep the key in somewhere and call sqlite3_key_v2() with the key. This means the application needs to implement some mechanism to protect the key; Or the key will be remembered by every user, and he needs to provide it as well as his username/password. This sounds less attractive.  Keystore-based userauth in Berkeley DB It will be much better if an authentication-required database is encrypted, but a user could access the database with only his user name/password. We provide such an enhancement with a keystore file. Encryption is mandatory for a keystore based authentication-required database and we take the responsibly to protect the database encryption key. This is totally transparent to the end user. A user just does the login with a call sequence as: sqlite3_open_v2(); sqlite3_user_authenticate(); /* Database is now usable */  In our enhanced sqlite3_user_authenticate() , we actually do things as below:  1. fetch the database encryption key from keystore file using user name and password; 2. apply the encryption key to the database(that is, calling sqlite3_key_v2() ); 3. verify user/password via sqlite_user table, as the orignal userauth does.  This enhanced keystore based userauth will be activated with - DBDBSQL_USER_AUTHENTICATION_KEYSTORE compile option. How the keystore file works We store keystore entries in the keystore file. Every entry is the database encryption key which is encrypted with a salted hash value of user’s password. Say we have items below: DEK - database encryptio key EDEK - encrypted database encryption key PWD - user's password SALT - a randomly generated byte string USERNAME - user's name encrypt - Berkeley DB's encryption function; decrypt - Berkeley DB's decryption function; hash - Berkeley DB's SHA1 Secure Hash Algorithm function When a user was added, we get the database encryption key from memory cache, and we compute  EDEK = encrypt(DEK, hash(SALT, PWD))  then we save a keystore entry as: (USERNAME, SALT, EDEK) . During a user login, to get the database encryption key back, first we find the keystore entry with user’s name, then we compute the database encryption key with: DEK = decrypt(EDEK, hash(SALT, PWD));  Thus, by using hash/encryption, We ensure that: The database encryption key in the keystore file is encrypted and could never be leaked from the keystore file; Only user’s password hash is stored, so user’s password could never be leaked from the keystore file. Make the Berkeley DB keystore userauth robust If keystore file get corrupted, some of the end users will not be able to access the database. We try to make our keystore authentication as robust as possible. Checksum and backup/restore We enable checksum for the keystore file and have the checksum verified every time when keystore file was used. We also implement a simple backup/restore mechanism of the keystore file. Locking file for the keystore We expect the database be visited by multiple processes. To avoid keystore file from messed by multiple process writes, we introduce a lock file. Only when a process could have the .lck file created(exclusively), it could access the keystore file; when it finished, it will remove the .lck file. In rare cases, if user application fails when accessing the keystore file,  the locking file may not be cleaned. In this case, user needs to clean the .lck file under the database environment. Treat updates of sqlite_user table and keystore file as an atomic operation When adding/deleting/updating a user, we will first update the sqlite_user database table then the keystore file. We need to ensure that: Updates to sqlite_user table and keystore file will both succeed or both fail. This is guaranteed by database transactions. We put the updates of sqlite_user table and keystore file in a transaction and only commit when we succeed to update the keystore file; if we failed to update the keystore file, we will rollback the database updates. Please note as we use transactions internally in userauth API, you should not call the userauth API within an outside transaction. If you do that, userauth API will return an error message.

BDB User Authentication (courtesy of  Brayden Zhang) Introduction In Berkeley DB 6.2, user authentication extension is added into the SQL interface. The new user authentication layer is both secure and...

Berkeley DB

Slices in BDB

Slices in Berkeley DB(courtesy of Charles Koester) With the new Berkeley DB 6.2 release, there are a number of new features.  One of the more exciting ones is the use of Slices.It is possible to improve the multiprocessor scaling of your DB databases by implementing slices. Slices cause your database records to be spread evenly across sub-environments and sub-databases using an internal hashing algorithm. Because your records are stored in separate physical environments and databases, this allows for better parallel processing support by spreading read and write workloads across separate, discrete processor cores. The new “slices” feature can improve the multi threaded performance of IOT and many other workloads. Throughput in multi threaded applications can improve by 3x or more. Transactions which access a single key — e.g., solo DB->get(), DB->put, or DB->del() calls, or DB->get() followed by an update call — are ideal for the use of the slices feature.. Workloads in which related sets of keys are accessed inside a single transaction can also use slices. By using the new DB->set_slice_callback() API call to designate the slice-relevant portion of the key, a transaction can access many different keys as long as they share the slice callback value.Read more about the new Slices feature in Berkeley DB at:  http://docs.oracle.com/cd/E17076_05/html/programmer_reference/bdb_slices.html

Slices in Berkeley DB (courtesy of Charles Koester)  With the new Berkeley DB 6.2 release, there are a number of new features.  One of the more exciting ones is the use of Slices. It is possible to...

Berkeley DB

BDB Write Forwarding

(Courtesy of Paula Bingham)  Simpler High Availability with Write Forwarding Do you have a simple Berkeley DB application that would benefit from higher availability or improved read scalability? These are well-known benefits of using Berkeley DB High Availability (BDB-HA).  You can use BDB-HA to turn your application into a distributed application using two or more nodes.  One node is the master node on which you can perform read and write operations; the other nodes are replicas which only support read operations.  When one node is down, the other nodes keep your application available to users. If you have many read operations, they can be distributed among different nodes.  You might ask "But wait, what if I don't want to restrict write operations to the master node?" and "Isn't it complicated to add BDB-HA to an existing application?" If your application performs its write operations one-per-transaction without using cursors, it is a candidate for the new Replication Manager write forwarding feature.  When write forwarding is enabled, these simple write operations can be performed on a replica. The code changes to add Replication Manager write forwarding to your application are relatively simple and fall into the following major categories: Configure and start Replication Manager:  You need to explicitly enable write forwarding, configure your nodes, and start Replication Manager. Manage database creation: You only need to create your databases on the initial master node.  BDB-HA replicates them to the replica nodes. Error handling: Minor code changes to handle a few additional BDB-HA errors. See "Configuring for Write Forwarding"  in the Berkeley DB documentation for more information and sample code illustrating these points.

(Courtesy of Paula Bingham)  Simpler High Availability with Write Forwarding Do you have a simple Berkeley DB application that would benefit from higher availability or improved read scalability? These...

Berkeley DB

Information on a new feature in Berkeley DB 6.2 - BDB Server

Information on a new feature in Berkeley DB 6.2 (courtesy of Yong Li)BDB Server (aka. BDB as standalone server or thrift) Beyond being an industry-leading embedded key-valuedatabase, BDB Server turns BDB into a key-value database server which supportsa client-server style of access. User applications can now use BDB Serverclient drivers to access remote BDB Servers. Currently, the following BDB features are supported byBDB Server: * B-Tree, Hash and Recno access methods.* Transactions. (BDB Server can be run in TransactionData Store mode only) * Secondary indices and foreign constraints * Join cursors * Sequences The following BDB features are not supported by BDBServer now: * Queue and Heap access methods * Replication * Most call-back functions The following features are specific to BDB Server: * Support for SSL over TCP * Handle timeout. If a handle has not been access for along time, it can be closedautomatically by the server. System requirement for BDB Server: * JDK 8 or above Currently BDB Server client drivers are only available inJava and require JDK 8 or above. The Java client driver API is very similar tothe BDB Java API with a few functions to open and close server connections. Get Berkeley DB version 6.2 at:  http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index.html

Information on a new feature in Berkeley DB 6.2  (courtesy of Yong Li) BDB Server (aka. BDB as standalone server or thrift) Beyond being an industry-leading embedded key-valuedatabase, BDB Server turns...

Berkeley DB

Oracle Berkeley Database (BDB) 6.2 Announced

OracleBerkeley Database Version 6.2 (12.1.6.2.23) is now available for download. DownloadPage – Click Here DocumentationPage – Click Here OracleBerkeley Database Version 6.2 – New Features · Slices – provides significantly better scalabilityon large SMP machines as compared to the previous release. · Write Forwarding – No longer are application writesrestricted to the Master node when this feature is enabled.  Simple writeoperations can now be performed on the replica and BDB-HA will forward thewrites to the Master automatically. · BDB as a Stand-AloneServer– Givesuser an out-of-the-box solution to a key-value database in a client-serverarchitecture. · SQL Performance Enhancements – Index-based SQLquery performance has been improved by as much as 40% out of the box whencompared to BDB 6.1. · Authentication – User authentication has been added to the BDB SQL API.  The database cannow be set up to authenticate users prior to them accessing data in thedatabase.  The new authentication capability, coupled with existingencryption, can be used to create secure, high performance embedded databaseapplications. BerkeleyDB continues to enable the most powerful embedded database solutions · HandleTBs of data with a 1MB library · Flexible,lightweight, transactional data management engine · Runson a wide variety of operating systems and platforms ranging from low power ARMdevices to clusters of high-end servers · Over50 open source software projects embed BDB -- check them out on Wikipedia · Completelycustomizable, with choice of 5 different access methods · Industrialquality and battle tested SupportingAssets and Resources Collateraland sales assets continue to be updated. · Oracle BDB Web Site · Oracle BDB Blog website – continue to check for new blogs · Oracle BDB DatabaseProduct Family Datasheet · Oracle BDB DatabaseDatasheet LinkedIn Group - https://www.linkedin.com/groups/2131985/profile Twitter – berkeleydb What others aresaying: Opensource Fedora package maintainer, Lubomir Rintel, says "Berkeley DB hasquietly served behind the scenes as the database for the RPM PackageManager.   It has proven itself time and time again as a robust andefficient storage engine.   It stores the meta information of theinstalled rpms.  Under heavy workloads, BDB proves itself reliable. Countlesspeople that use popular Linux distributions have used BDB through RPM and neverknew it.  With this new release,   BDB continues its traditionof being a solid storage engine" Oracle Tape Product Manager, Dan Deppen, says "Berkeley DB is integralto  Oracle StorageTek Storage Archive Manager (SAM-QFS).  We havebeen embedding Berkeley DB in our product for over a decade and it is vital toour disk archiving feature which is used to send files to remote data centersto enable disaster recovery.  Performance and scalability are criticalbecause SAM-QFS supports some of the largest archive customers in theworld.   HPC sites, research centers, national libraries and othercustomers requiring massive scalability and high reliability depend on SAM-QFSand Berkeley DB to maintain availability of their critical data." Questions& Help External Email Alias – bdb@oss.oracle.com

Oracle Berkeley Database Version 6.2 (12.1.6.2.23) is now available for download. Download Page – Click Here Documentation Page – Click Here Oracle Berkeley Database Version 6.2 – New Features · Slices –...

Berkeley DB

Berkeley DB 12cR1 (12.1.6.1) released

New Releases of Oracle Berkeley DB Available For Download NowSubject: Release Update The newest release of Berkeley DB 12cR1 (12.1.6.1) isavailable now. Here is a summary of the new features:      upgrades to in-memory OLTP throughput &performancenew HA improvement to identify a single master in a 2 site replicationgroupnew HA useability improvements new Blob support added into replication removed need for fail check monitor process reduced the time for a database backup anda lot more! Berkeley DB continues to enable the most powerfulembedded database solutions      Handle TBs of data with a 1MB libraryFlexible, lightweight storage engine, small footprintRuns on low power ARM devices to cluster of high-end serversOver 50 open source software projects embed BDB -- check them out on WikipediaCompletely customizable, choose from 5 different access methodsIndustrial quality and battle tested with over 200 million deployments BDB is hands-down the best edge, mobile, and embeddeddatabase available to developers.  With the flexibility to place log filesand/or database in any directory, applications can easily take advantage of theIO performance of flash caches, flash disks or SSDs. Top notch performance: Berkeley DB performs over 5 million operations a second on a 100GB database running on a 1/8 rack Exadata V2 Database Machine configured with 256GB RAM and 12 cores. Berkeley DB can insert 100 thousand records in 72 milliseconds and read those records in 30 milliseconds, running on a 8 core XEON-based commodity server.  The records contain a 4 byte key and a 48 byte data value.  This was run using the benchmark described on pages 34-36 in an ebook on SQL Server 2014 from Microsoft Press. To put this into perspective, we compared Berkeley DB to SQL Server 2014's In-Memory OLTP feature (code name Hekaton)  which has similar technology.  Berkeley DB, an open source product, is about 20% faster than SQL Server 2014 which takes 94 milliseconds for the same 100k insert operations on an 8 core X64 Intel commodity box with 14 GB memory.    We are making available a benchmark program in C that can be configured tovalidate Berkeley DB throughput for the 100K insert test here. Whatfolks are saying: Opensource Fedora package maintainer, Lubomir Rintel, says "Berkeley DB has quietlyserved behind the scenes as the database for the RPM PackageManager.   It has proven itself time and time again as a robust andefficient storage engine.   It stores the meta information of theinstalled rpms.  Under heavy workloads, BDB proves itself reliable.Countless people that use popular Linux distributions have used BDB through RPMand never knew it.  With this new release,   BDB continues itstradition of being a solid storage engine" Oracle Tape Product Manager, Dan Deppen, says "Berkeley DB is integral toOracle StorageTek Storage Archive Manager (SAM-QFS).  We have beenembedding Berkeley DB in our product for over a decade and it is vital to ourdisk archiving feature which is used to send files to remote data centers toenable disaster recovery.  Performance and scalability are criticalbecause SAM-QFS supports some of the largest archive customers in theworld.   HPC sites, research centers, national libraries and othercustomers requiring massive scalability and high reliability depend on SAM-QFSand Berkeley DB to maintain availability of their critical data."   Oracle Identity Management Vice President, Shirish Puranik, says"Berkeley DB is a critical component of Oracle Unified Directory (OUD) andOracle Directory Server Enterprise Edition (ODSEE).  We have been usingBerkeley DB in these products as a high performance, transaction embeddeddatabase repository for several years.  Berkeley DB has exceed ourexpectations for performance and stability. Our Berkeley DB based products arewidely deployed in production at largest telcos and financial institutions allover the world.   The improvements for BLOB support and high availabilityin the 6.1 release are welcome." Software Downloads Downloads available today on the Berkeley DB download page. ProductDocumentation Questions? Please direct product questions to our Product Help Mail list for Berkeley DB. You can alsoemail the Berkeley DB Product Management team directly. ___________________________________________________Berkeley DB Product ManagementInternal - https://stbeehive.oracle.com/teamcollab/wiki/Berkeley+DB+Inside+ScoopOTN - http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html

New Releases of Oracle Berkeley DB Available For Download Now Subject: Release UpdateThe newest release of Berkeley DB 12cR1 (12.1.6.1) is available now. Here is a summary of the new features:      upgrad...

Berkeley DB

Data management in unexpected places

Data management in unexpectedplaces When you think of network switches, routers, firewallappliances, etc., it may not be obvious that at the heart of these kinds ofsolutions is an engine that can manage huge amounts of data at very highthroughput with low latencies and high availability. Consider a network router that is processing tens (orhundreds) of thousands of network packets per second. So what really happens inside a router? Packets are streaming in at the rate of tensof thousands per second. Each packet hasmultiple attributes, for example, a destination, associated SLAs etc. For each packet, the router has to determinethe address of the next “hop” to the destination; it has to determine how toprioritize this packet. If it’s a highpriority packet, then it has to be sent on its way before lower prioritypackets. As a consequence ofprioritizing high priority packets, lower priority data packets may need to betemporarily stored (held back), but addressed fairly. If there are security or privacy requirementsassociated with the data packet, those have to be enforced. You probably need to keep track of statisticsrelated to the packets processed (someone’s sure to ask). You have to do all this (and more) whilepreserving high availability i.e. if one of the processors in the router goesdown, you have to have a way to continue processing without interruption (thecustomer won’t be happy with a “choppy” VoIP conversation, right?). And all this has to be achieved without ANYintervention from a human operator – the router is most likely to be in aremote location – it must JUST CONTINUE TO WORK CORRECTLY, even when bad thingshappen. How is this implemented? As soon as a packet arrives, it is interpreted by the receiving software. The software decodes the packet headers inorder to determine the destination, kind of packet (e.g. voice vs. data), SLAsassociated with the “owner” of the packet etc. It looks up the internal database of “rules” of how to process thispacket and handles the packet accordingly. The software might choose to hold on to the packet safely for someperiod of time, if it’s a low priority packet. Ah – this sounds very much likea database problem. For each packet, youhave to minimally · Look up the most efficient next “hop” towardsthe destination. The “most efficient”next hop can change, depending on latency, availability etc. · Look up the SLA and determine the priority ofthis packet (e.g. voice calls get priority over data ftp) · Look up security information associated withthis data packet. It may be necessary to retrieve the context for this networkpacket since a network packet is a small “slice” of a session. The context for the “header” packet needs tobe stored in the router, in order to make this work. · If the priority of the packet is low, then“store” the packet temporarily in the router until it is time to forward thepacket to the next hop. · Update various statistics about the packet. In most cases, you have to do all this in the context of asingle transaction. For example, youwant to look up the forwarding address and perform the “send” in a singletransaction so that the forwarding address doesn’t change while you’re sendingthe packet. So, how do you do all this? Berkeley DB is a proven, reliable, high performance, highlyavailable embeddable database, designed for exactly these kinds of usagescenarios. Berkeley DB is a robust, reliable, proven solution that is currentlybeing used in these scenarios. First and foremost, Berkeley DB (or BDB for short) is veryvery fast. It can process tens orhundreds of thousands of transactions per second. It can be used as a pure in-memory database,or as a disk-persistent database. BDBprovides high availability – if one board in the router fails, the system canautomatically failover to another board – no manual intervention required. BDB is self-administering – there’s no needfor manual intervention in order to maintain a BDB application. No need to send a technician to a remotesite in the middle of nowhere on a freezing winter day to perform maintenanceoperations. BDB is used in over200 million deployments worldwide for the past two decades for mission-criticalapplications such as the one described here. You have a choice of spending valuable resources to implement similarfunctionality, or, you could simply embed BDB in your application and off yougo! I know what I’d do – choose BDB, so I can focus on my businessproblem. What will you do?

Data management in unexpected places When you think of network switches, routers, firewall appliances, etc., it may not be obvious that at the heart of these kinds ofsolutions is an engine that can...

Berkeley DB

Yammer, Berkeley DB, and the 3rd Platform

If you read the news, you know that the latest high-profilesocial media acquisition was just confirmed. Microsoft has agreed to acquireYammer for 1.2 billion. Personally, I believe that Yammer’s amazing success canbe mainly attributed to their wise decision to use Berkeley DB Java Edition astheir backend data store. :-) I’m only kidding, of course. However, as Ryan Kennedy pointsout in the video I recently blogged about, BDB JE did provide the right featureset that allowed them to reliably grow their business. Which in turn allowedthem to focus on their core value add. As it turns out, their ‘add’ is quitevaluable! This actually makes sense to me, a lot more sense than certainother recent social acquisitions, and here’s why. Last year, IDCdeclared that we are entering a new computing era, the era of the “3rdPlatform.” In case you’re curious, the first 2 were terminal computing andclient/server computing, IIRC. Anyway, this 3rd one is morecomplicated. This year, IDC refined the concept further. It now involves 4 distinct buzzwords: cloud, social, mobile, and bigdata. Yammer is a social media platform that runs in the cloud,designed to be used from mobile devices. Their approach, using Berkeley DB JavaEdition with High Availability, qualifies as big data. This means that Yammeris sitting right smack in the center if IDC’s new computing era. Another way toput it is: the folks at Yammer were prescient enough to predict where thingswere headed, and get there first. They chose Berkeley DB to handle their data. Maybe youshould too!

If you read the news, you know that the latest high-profile social media acquisition was just confirmed. Microsoft has agreed to acquire Yammer for 1.2 billion. Personally, I believe that...

Berkeley DB

Now you can build Berkeley DB into your Android apps

I want to make everyone aware of a small change we made inthe last release that promises to make a big difference for our customers usingAndroid. We added support for the Android platform some time back.However, there was a caveat: Berkeley DB had to be integrated at the OS level,replacing the SQLite routines that usually ship with Android. Having BerkeleyDB built into the OS provided some advantages. But it also meant that customerswere unable to build a BDB-enabled app that could be deployed to any genericAndroid device. Also some customers were hesitant to make OS modificationsthemselves. In our latest release, Berkeley DB 5.3, we have added thecapability to use BDB on a per-application basis. This means you can build yourapplication to use Berkeley DB, and the library routines will be bundled inwhen you package everything up. The result is an application that can takeadvantage of Berkeley DB’s strengths on any Android device. We've even included instructions on how to achieve this in our online documentation. Berkeley DB supports a wide array of mobile platforms.Starting with Oracle’s own embedded Java, which many forget still dominates thefeature phone market. BDB also supports Android and iOS, the two platforms that continue toown the smartphone marketplace. This latest enhancement will make it eveneasier for Android developers to use BDB. Finally, we offer the option to choose between aSQLite-compatible SQL API, and our traditional key/value API (for you NoSQLfans out there). Our combination of features and platform support make BerkeleyDB the best choice for anyone who needs an enterprise-grade data store on a mobileplatform.

I want to make everyone aware of a small change we made in the last release that promises to make a big difference for our customers using Android. We added support for the Android platform some time...

2011 Was a Banner Year for Berkeley DB!

I’ve been wanting to write this post since December, andonly just now getting around to it. Fortunately, the contents of the postitself will serve as a pretty good explanation for the delay. :-) I realize we’re already in February here. But I still thinkit’s worth mentioning what an amazing year 2011 was for the Berkeley DB team.Therefore I’d like to take a moment and look back at the major events of 2011,product by product. Berkeley DB: We set the bar pretty high, with TWO major releases in oneyear: 5.2 and 5.3. Both releases added exciting new features, too many to listhere. Some of the main areas to receive enhancements were the SQL API, HighAvailability, HEAP, and XA. Expanded platform support now includes more Javaplatforms and per-application Android support. Berkeley DB Java Edition: Serving as the basis for a major new Oracle product (NoSQLDatabase) might have been enough for some products. But not Berkeley DB JE! Inaddition to that accomplishment, the team put out a major release as well, 5.0.This release included the much anticipated DiskOrderedCursor class, aplethora of performance improvements, and some High Availability enhancementsto boot. Revamped product launch: Database Mobile Server 11g Database Mobile Server 11g is Oracle’s newly re-architectedsync and device management solution for mobile, embedded and M2M platforms. Itreplaces Database Lite, and is designed to use Berkeley DB as the remote datastore. DMS, as we call it, is also compatible with the very popular open-sourceSQLite database. There has been a lot of focus on mobile and embedded/M2M asthey relate to enterprise recently, and every indicator points to these topicsbecoming even more important. This one is definitely a product to keep an eyeon. New Product Launch: NoSQL Database I’m not prepared to say I saved the “best” for last, sinceall these products are truly great. But certainly NoSQL Database product got ahuge amount of attention last year, and deservedly so. Oracle’s entry into thisred-hot space promises to be exactly what many of our customers have beenwaiting for. The data management solution to their Big Data problem fromthe company they already trust to store the rest of their data. To recap, that’s 3 major product releases, and 2 productlaunches. All from one team. That is amazing, by any standard. Congratulationsto the development team, your work output last year was the stuff of legends!Of course, these folks are not ones to rest on their laurels. I know for a factthey are already hard at work on future versions of all the products listedabove, and Berkeley DB XML too. The best is yet to come!

I’ve been wanting to write this post since December, and only just now getting around to it. Fortunately, the contents of the post itself will serve as a pretty good explanation for the delay. :-) I...

Berkeley DB Java Edition

Announcing Berkeley DB Java Edition Major Release

Berkeley DB Java Edition 5.0 was just released. There are anumber of new features, enhancements, and options in there that our users havebeen asking for. Chief among them is a new class called DiskOrderedCursor, which greatly increases performance ofsystems using spinning platter magnetic hard drives. A number of users expressed interestin this feature, including Alex Feinberg of LinkedIn. Berkeley DB Java Editionis part of Project Voldemort, a distributed key/value database used byLinkedIn. There have been many other improvements andoptimizations. Concurrency is significantly improved, as is the performance of updateand delete operations. New and interesting methods include Environment.preload, whichallows multiple databases to be preloaded simultaneously. New Cursor methods enable for more effectivesearching through the database. We continue to enhance Berkeley DB Java Edition’s High Availability as well. One new feature is the ability to open areplicated node read-only when the master is unavailable. This can allowcritical systems to continue offering some functionality, even during a networkor master node failure. There’s a lot more in release 5.0. I encourage you to take alook at the extensive changelog yourself. As always, you can download the newrelease and try it out here: http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html

Berkeley DB Java Edition 5.0 was just released. There are a number of new features, enhancements, and options in there that our users have been asking for. Chief among them is a new class called DiskOr...

Berkeley DB at OpenWorld/JavaOne 2011

It’s the question on everyone’s mind: what is Berkeley DBbringing to OpenWorld this year? Even if you’re more preoccupied with thelatest iPhone 5 rumors, (I hear the front facing camera can look into your eyesand tell you what you had for breakfast) Berkeley DB users (or fans) who areplanning to attend Oracle OpenWorld in San Francisco about 10 days from nowwill want to read this post. As always, we will have a session where you can learn aboutthe cool and interesting things our customers are doing with a key/value datastore. But this year, we have two additional general areas of focus: Embedded,and Mobile Applications. Let’s take a closer look at the embedded front first. As thecost of components like network hardware, flash memory, and microcontrollerscontinues to fall, OEMs in many verticals are choosing to enhance their productlineup by adding new applications and internet-enabled features. Networked cashregisters? That was only the beginning. This was the year cars started gettingon the internet, and I already blogged about that. Who knows what the nextyears will bring? Whatever our customers are planning, we’ve got solutionsthat can help. Our first session with an embedded focus is 15185, BDB andEmbedded Java presentation. There you’ll learn how we are extending Java’s“write once, deploy anywhere” mantra. With Berkeley DB and Database MobileServer, you get worry free data management and sync capabilities after you’vedeployed, as well. Next, over on the JavaOne side we have 25143, Telemetry andSynchronization with Embedded Java and Berkeley DB. This session will featureJava Architect Greg Bollella talking about how these same technologies areenabling telemetry solutions to plug into the enterprise right alongside yourexisting data and apps. The ability to manage networks of embedded devicesusing existing enterprise frameworks could prove to be quite revolutionary. Theembedded Java platform, when coupled with Berkeley DB and Database MobileServer, has the ability to do just that. We’re excited about this, and we thinkour customers will be too. On the mobile applications side of things Tata ConsultingServices (TCS) will be joining us for session 15178, Achieve Ubiquitous DataAccess, from Device Databases to Enterprise Repositories. There you’ll learn howTCS is helping their customers deploy mobile applications that maximize the ROIof their existing enterprise infrastructure. Rounding out the list is our key/value customer highlightsession. You thought I forgot, right? No chance! Session 15167 is entitledTransactional Key-Value Storage: Super Simple, Super Fast, Super Flexible. RaghunathNambiar, an Architect at Cisco, will co-present with us. The topic will be supercool things you can accomplish using the key-value paradigm for datamanagement. In addition to the sessions, we will have a number of excitingdemos for you to check out, both at OpenWorld and JavaOne. Finally, be on the lookout for an exciting productannouncement building on the inherent strengths of the Berkeley DB productfamily! Hope to see you there!

It’s the question on everyone’s mind: what is Berkeley DB bringing to OpenWorld this year? Even if you’re more preoccupied with thelatest iPhone 5 rumors, (I hear the front facing camera can look into...

Berkeley DB

Customer Highlight: AdaptiveMobile

Last week AdaptiveMobile announced they signed a 3 yearagreement to embed Oracle Berkeley DB in their Network Protection Platform, akey component of their product portfolio. AdaptiveMobile is the world leader in mobile security, enabling trusted networks for the world’s largest operator groups and protecting one in six subscribers globally. AdaptiveMobile provides operators with the most comprehensive network-based security solutions enabling them to protect their consumer and enterprise customers against the growing threat of mobile abuse. From the announcement: “AdaptiveMobile selected OracleBerkeley DB for its outstanding data retrieval speeds, reliability, scalabilityand availability and its integration with the company’s existing applicationenvironment.” Here is a great quote from Gareth Maclachlan, AdaptiveMobileChief Operating Officer: “Oracle’s Berkeley DB has enabled us to develop asolution that resolves the content and access control challenges of bothnetwork operators and their customers without impacting the user experience. “Working with Oracle has allowed us to reduce our development time andcut our total cost of ownership for network operators by helping us reduce ourhardware and administration costs.” For more information, you can see the announcement, orlearn more about AdaptiveMobile’s products.

Last week AdaptiveMobile announced they signed a 3 year agreement to embed Oracle Berkeley DB in their Network Protection Platform, a key component of their product portfolio. AdaptiveMobile is the...

Berkeley DB

New Release of Oracle Berkeley DB

We are pleased to announce that a new release of Oracle Berkeley DB, version 11.2.5.2.28, is available today. Our latest release includes yet more value added features for SQLite users, as well as several performance enhancements and new customer-requested features to the key-value pair API.  We continue to provide technology leadership, features and performance for SQLite applications.  This release introduces additional features that are not available in native SQLite, and adds functionality allowing customers to create richer, more scalable, more concurrent applications using the Berkeley DB SQL API. This release is compelling to Oracle’s customers and partners because it: delivers a complete, embeddable SQL92 database as a library under 1MB size drop-in API compatible with SQLite version 3 no-oversight, zero-touch database administration industrial quality, battle tested Berkeley DB B-TREE for concurrent transactional data storage New Features Include: MVCC support for even higher concurrency direct SQL support for HA/replication transactionally protected Sequence number generation functions lower memory requirements, shared memory regions and faster/smaller memory on startup easier B-TREE page size configuration with new ''db_tuner" utility New Key-Value API Features Include: HEAP access method for constrained disk-space applications (key-value API) faster QUEUE access method operations for highly concurrent applications -- up 2-3X faster! (key-value API) new X/open compliant XA resource manager, easily integrated with Oracle Tuxedo (key-value API) additional HA/replication management and communication options (key-value API) and a lot more! BDB is hands-down the best edge, mobile, and embedded database available to developers. Downloads available today on the Berkeley DB download pageProduct Documentation

We are pleased to announce that a new release of Oracle Berkeley DB, version 11.2.5.2.28, is available today. Our latest release includes yet more value added features for SQLite users, as well as...

Mobile Enterprise

Embedded Systems Conference San Jose

The annual Embedded Systems Conference in San Jose was held at the beginning of May. This was Oracle’s first year at the conference, and we did get visitors who were surprised to see us there. However some people, myself included, think we're going to see increased convergence between enterprise and embedded in the coming years. I know we're not alone, because a certain other big name in enterprise systems had a booth right next to ours! Since embedded is not a topic everyone is familiar with, I want to give a little insight into this conference and the embedded space in general, as I think it is potentially an important growth area for products like Berkeley DB. Since I was an embedded developer myself in a past life, this is familiar territory for me. Wikipedia defines embedded systems as "a computer system designed to do one or a few dedicated and/or specific functions." It is important to note that embedded is generally considered to be distinct from mobile. Mobile platforms are typically derived in some way from desktop platforms such as Linux, Windows, OSX, and are often more general purpose devices. Before the advent of cheap, high-res, general purpose LCD displays, having a display in your device meant a Cathode Ray Tube (CRT). Embedded devices were traditionally ‘headless,’ meaning they had no display and no generic input device such as a keyboard. Because of this, in the early days developers would commonly develop on desktop machines using a cross toolchain. A cross toolchain is a set of tools designed to build software on a target embedded platform, which was a completely different hardware architecture and OS from the desktop development platform. Nowadays, many embedded platforms are powerful enough that they can run their own toolchains. Such was the case with our demo, more on that below. Another common aspect of an embedded system is “real time” requirements. A simplified definition would be if a given operation does not complete by a certain time, it’s just as bad as not finishing at all. Real Time Operating Systems, or RTOSes, can provide guarantees about when operations will finish. Real time embedded devices are still quite prevalent in some industries, including military, aviation, industrial manufacturing, and networking. The embedded space has certainly been encroached on by the rise of mobile, but as long as we have mission critical devices there will continue to be a requirement for embedded devices. Now back to the Embedded Systems Conference. Booth traffic was high, we were averaging about 1 visitor per minute the nearly the whole time I was there. I attribute this partly to curiosity, but mostly to our great giveaways! They did their job, I talked to a number of people who ended up having a genuine interest in what we were showing, and were initially attracted to the booth by our swag. Also we held a drawing for an iPad, which brought a ton of people to register. Our demo was a temperature sensor attached to a small device called a SheevaPlug, which is a general purpose embedded development device from Marvell. By embedded standards, the SheevaPlug is a very powerful device, and we were able to develop directly on it. The idea behind the demo was that the device represented one of many nodes in a sensor network. Some real world examples of this include weather stations, or monitoring conditions inside laboratories or industrial facilities. Our demo showed the system collecting temperature data, which was then uploaded to Oracle Database. All of this was running on top of Java SE Embedded. The demo was well received. Nearly everyone who listened to me present agreed that the sync functionality would be useful to them, or useful in general if they didn’t need it themselves. The main purpose of our presence at ESC was to showcase the power, ease of use, and versatility of Java Embedded. When you combine that Berkeley DB and Oracle Database Lite Mobile Server, you get a system that has out of the box capability to move data to and from enterprise storage systems. After a few simple configuration steps, the data stored on the local Berkeley DB or SQLite data store is connected to the enterprise backend. This is a potent combination of features, and one that we feel will be in high demand in the coming years, as M2M and embedded solutions continue to proliferate.

The annual Embedded Systems Conference in San Jose was held at the beginning of May. This was Oracle’s first year at the conference, and we did get visitors who were surprised to see us there. However...

Berkeley DB

Ubiquitous Mobile Applications

It goes without saying that smart phones are already very popular and enjoying rapid growth. But a few things finally happened in the last year that people have been predicting for a long time, things that could make smartphones and mobile apps an even bigger part of our lives.First off, in what many consider to be a bold move from a normally conservative company, GM has already released mobile apps that allow many of their cars to be remotely monitored, unlocked, and even started. So as long as you have a high degree of confidence in your smartphone's battery life, you don't need your car keys anymore.Next, we have Starbucks, who have quietly sidestepped the ongoing mobile payments dispute by launching a barcode based payments app that doesn't require any special technology. Other large retailers are probably watching this closely, and if the big cell phone companies don't get their acts together, they might just end up missing the boat again. In any case, many industry observers feel that the Starbucks system could be the spark that sets off an explosion of mobile payment systems, from McDonald's to Neiman Marcus, and everywhere in between. Here is some more detail on the Starbucks solution, and for an industry analysis click here. But I saved the biggest one for last. Are you ready? Take a deep breath, and then read this: Smartphones outsold PCs for the first time, Q4 2010. This milestone happened much sooner than anyone expected, thanks mostly to new Android activation numbers so high they're almost hard to believe. At the start of 2011 people were talking about activations in excess of 125k units per day. Currently the number being tossed around is 300k units per day. If Android can keep it up, that platform alone will outsell PCs in 2011. Folks, this could be the end of an era. When we combine these milestones, the smartphone appears to poised to become a sort of ultimate swiss army knife; a single device that does everything. The thought of it is exciting; imagine the convenience! Alas, as with all great things in life, there is a downside. Money, privacy, communication, and even physical security could be compromised if the software or data on the device is not secure.The average person has at least 3 things with them when they leave the house: wallet, keys, and cell phone. Today, if one of the three things were to be lost, that would be bad. It would be a huge inconvenience, in the best case. However, when a cell phone is lost today, the unfortunate person is not normally forced to cancel their credit cards and reprogram their car locks. Worse than that, if we envision the future that the stories listed above point to, there is a potential for any or all of the three to be stolen electronically, by a thief far away, even in a different country. That should be a sobering prospect, both for consumers and the various companies providing services to them. It is easy to tout the benefits of technology convergence; it's a fun and exciting topic. But as technology providers, it is our duty to also consider the potential drawbacks. In order to keep everyone safe, sensitive mobile data needs to be securely stored on the local device, and transmitted reliably to each company's data vaults. Those same companies need to be able to determine, with 100% accuracy, whether a certain request is coming from an authorized device or an imposter. This could be a mobile purchase, a request to unlock your car, or even your home.Two critical components of a safe, trustworthy solution for storing and syncing sensitive mobile data are Oracle's Berkeley DB and Database Mobile Server. Most of the world's large enterprises already store their critical data in Oracle Database. Berkeley DB is a stable, mature product with over 10 years proven history. It is ideally positioned to be the mobile data store to complement Oracle Database. It is secure in the traditional sense, meaning it can protect mobile data from malicious intent, and in the database architecture sense as well, with full transactional guarantees. Database Mobile Server provides the final piece to form a complete solution: the capability to synchronize your data between the mobile devices and your Oracle backend, and manage the mobile application, data, and even the device itself if desired.If you're considering a mobile application that will access sensitive data, data that you already trust Oracle to store in your backend infrastructure, Berkeley DB and Database Mobile Server are the best choices to handle it.

It goes without saying that smart phones are already very popular and enjoying rapid growth. But a few things finally happened in the last year that people have been predicting for a long time,...

Berkeley DB

Is Berkeley DB a NoSQL solution?

Berkeley DB is a library. To use it to store data you must link the library into your application. You can use most programming languages to access the API, the calls across these APIs generally mimic the Berkeley DB C-API which makes perfect sense because Berkeley DB is written in C. The inspiration for Berkeley DB was the DBM library, a part of the earliest versions of UNIX written by AT&T's Ken Thompson in 1979. DBM was a simple key/value hashtable-based storage library. In the early 1990s as BSD UNIX was transitioning from version 4.3 to 4.4 and retrofitting commercial code owned by AT&T with unencumbered code, it was the future founders of Sleepycat Software who wrote libdb (aka Berkeley DB) as the replacement for DBM. The problem it addressed was fast, reliable local key/value storage.At that time databases almost always lived on a single node, even the most sophisticated databases only had simple fail-over two node solutions. If you had a lot of data to store you would choose between the few commercial RDBMS solutions or to write your own custom solution. Berkeley DB took the headache out of the custom approach. These basic market forces inspired other DBM implementations. There was the "New DBM" (ndbm) and the "GNU DBM" (GDBM) and a few others, but the theme was the same. Even today TokyoCabinet calls itself "a modern implementation of DBM" mimicking, and improving on, something first created over thirty years ago. In the mid-1990s, DBM was the name for what you needed if you were looking for fast, reliable local storage.Fast forward to today. What's changed? Systems are connected over fast, very reliable networks. Disks are cheap, fast, and capable of storing huge amounts of data. CPUs continued to follow Moore's Law, processing power that filled a room in 1990 now fits in your pocket. PCs, servers, and other computers proliferated both in business and the personal markets. In addition to the new hardware entire markets, social systems, and new modes of interpersonal communication moved onto the web and started evolving rapidly. These changes cause a massive explosion of data and a need to analyze and understand that data. Taken together this resulted in an entirely different landscape for database storage, new solutions were needed.A number of novel solutions stepped up and eventually a category called NoSQL emerged. The new market forces inspired the CAP theorem and the heated debate of BASE vs. ACID. But in essence this was simply the market looking at what to trade off to meet these new demands. These new database systems shared many qualities. They were designed to address massive amounts of data, millions of requests per second, and scale out across multiple systems.The first large-scale and successful solution was Dynamo, Amazon's distributed key/value database. Dynamo essentially took the next logical step and added a twist. Dynamo was to be the database of record, it would be distributed, data would be partitioned across many nodes, and it would tolerate failure by avoiding single points of failure. Amazon did this because they recognized that the majority of the dynamic content they provided to customers visiting their web store front didn't require the services of an RDBMS. The queries were simple, key/value look-ups or simple range queries with only a few queries that required more complex joins. They set about to use relational technology only in places where it was the best solution for the task, places like accounting and order fulfillment, but not in the myriad of other situations.The success of Dynamo, and it's design, inspired the next generation of Non-SQL, distributed database solutions including Cassandra, Riak and Voldemort. The problem their designers set out to solve was, "reliability at massive scale" so the first focal point was distributed database algorithms. Underneath Dynamo there is a local transactional database; either Berkeley DB, Berkeley DB Java Edition, MySQL or an in-memory key/value data structure. Dynamo was an evolution of local key/value storage onto networks. Cassandra, Riak, and Voldemort all faced similar design decisions and one, Voldemort, choose Berkeley DB Java Edition for it's node-local storage. Riak at first was entirely in-memory, but has recently added write-once, append-only log-based on-disk storage similar type of storage as Berkeley DB except that it is based on a hash table which must reside entirely in-memory rather than a btree which can live in-memory or on disk.Berkeley DB evolved too, we added high availability (HA) and a replication manager that makes it easy to setup replica groups. Berkeley DB's replication doesn't partition the data, every node keeps an entire copy of the database. For consistency, there is a single node where writes are committed first - a master - then those changes are delivered to the replica nodes as log records. Applications can choose to wait until all nodes are consistent, or fire and forget allowing Berkeley DB to eventually become consistent. Berkeley DB's HA scales-out quite well for read-intensive applications and also effectively eliminates the central point of failure by allowing replica nodes to be elected (using a PAXOS algorithm) to mastership if the master should fail. This implementation covers a wide variety of use cases. MemcacheDB is a server that implements the Memcache network protocol but uses Berkeley DB for storage and HA to replicate the cache state across all the nodes in the cache group. Google Accounts, the user authentication layer for all Google properties, was until recently running Berkeley DB HA. That scaled to a globally distributed system. That said, most NoSQL solutions try to partition (shard) data across nodes in the replication group and some allow writes as well as reads at any node, Berkeley DB HA does not.So, is Berkeley DB a "NoSQL" solution? Not really, but it certainly is a component of many of the existing NoSQL solutions out there. Forgetting all the noise about how NoSQL solutions are complex distributed databases when you boil them down to a single node you still have to store the data to some form of stable local storage. DBMs solved that problem a long time ago. NoSQL has more to do with the layers on top of the DBM; the distributed, sometimes-consistent, partitioned, scale-out storage that manage key/value or document sets and generally have some form of simple HTTP/REST-style network API. Does Berkeley DB do that? Not really.Is Berkeley DB a "NoSQL" solution today? Nope, but it's the most robust solution on which to build such a system. Re-inventing the node-local data storage isn't easy. A lot of people are starting to come to appreciate the sophisticated features found in Berkeley DB, even mimic them in some cases. Could Berkeley DB grow into a NoSQL solution? Absolutely. Our key/value API could be extended over the net using any of a number of existing network protocols such as memcache or HTTP/REST. We could adapt our node-local data partitioning out over replicated nodes. We even have a nice query language and cost-based query optimizer in our BDB XML product that we could reuse were we to build out a document-based NoSQL-style product. XML and JSON are not so different that we couldn't adapt one to work with the other interchangeably. Without too much effort we could add what's missing, we could jump into this No SQL market withing a single product development cycle.Why isn't Berkeley DB already a NoSQL solution? Why aren't we working on it? Why indeed...

Berkeley DB is a library. To use it to store data you must link the library into your application. You can use most programming languages to access the API, the calls across these APIs generally mimic...

Berkeley DB

The C++ Standard Template Library as a BDB Database (part 3)

In the second entry I showed that the Berkeley DB's C++ STL API can substitute in for in-memory data structures like vectors. Now in this final installment we'll see Berkeley DB at work storing object instance data referenced by STL structures.Berkeley DB's C++ STL API, the dbstl, will automatically store the STL container in an on-disk database, but what about the data referenced by the elements within the STL container? You must choose how to store the values pointed to within the STL data structures so that they too can be reconstituted from the database. This means storing the actual data that the pointers point to, which isn't so hard. Let's get started.Suppose we have a simple class like a "Car" (as below) that has an "Engine" member (a pointer) referencing an instance of the class "Engine". class Car {  size_t length, width, height;  Engine *engine;  string model;  // Member functions follow ...};class Engine {  size_t horse_power;  char num_cylinder;  float displacement;  // Member functions follow ...}; view raw stl-ex-2a.cpp This Gist brought to you by GitHub. Now we want to store the "owner name" of the Car and we do that using a map container like std::map container, here is the pseudo code: typedef std::map<const char *, Car> owner_car_map_t;owner_car_map_t ocmap;while (has more owner-car pairs to input) {  // Accept input data for owner name and car, create car.  cin>>owner>>len>>width>>height>>hp>>ncyl>>displ;  Engine *pEngine = new Engine(hp, ncyl, displ);  Car car(len, width, height, pEngine);  ocmap[owner.c_str()] = car;} view raw stl-ex-2b.cpp This Gist brought to you by GitHub. Note how the preceding code stores all information in memory, so it can't be automatically persisted by Berkeley DB. When using dbstl, we can persist it, as follows. typedef dbstl::db_map<const char *, Car> owner_car_map_t;// (1)owner_car_map_t ocmap;while (has more owner-car pairs to input) {  // Accept input data for owner name and car, create car.  cin>>owner>>len>>width>>height>>hp>>ncyl>>displ;  Engine *pEngine = new Engine(hp, ncyl, displ);  Car car(len, width, height, pEngine);  ocmap[owner.c_str()] = car;// (2)} view raw stl-ex-2c.cpp This Gist brought to you by GitHub. Now the code will store the ower name strings and Car objects into underlying Berkeley DB database as a key/data pair in (2). In (1), though we are using a "const char*" type as key, they key string characters can be stored into underlying Berkeley DB database. This is possible because we are using a class Car rather than primitive types like int, double, etc, we don't need the ElementHolder template here. If we are storing a char* string pair in the map, we should use this type: dbstl::db_map>.However, the Engine objects referenced by each "car.engine" member pointer are not yet properly stored into database. At this point only their memory address is stored, which is meaningless for persistence. This is because by default the Berkeley DB STL API (dbstl) will simply copy the object to be stored using memcpy, (i.e. we are doing a shallow copy of objects) by default. Any pointer members in the object is shallow copied, the objects they refer to are not copied. This is sufficient for classes with no member pointers, i.e. when each of its instance locates on a single continous chunk of memory, but not sufficient to be completely persisted if they have pointer members.In order to store the Engine object of each car object, we need to do deep copy of such objects, we should register the following callback functions to do so. They cooperate closely together to enable an elegant deep copy of any complex objects. u_int32_t CarSize(const Car &car)// (3){  return sizeof(length) * 3 + model.length() + 1 +     sizeof(*engine); // (4)}void CopyCar(void *dest, const Car&car)// (5){  // Copy all bytes of the car object into memory chunk M   // referenced by dest.  // M is preallocated and just big enough because our CarSize is used  // to measure the object size. Note the bytes  // of the Engine object E referenced by car.engine should be copied  // into this M, rather than the car.engine pointer value,  // in order to deep copy E.  char *p = (char *)dest;  memcpy(p, car.length, sizeof(size_t));  p += sizeof(size_t);  memcpy(p, car.width, sizeof(size_t));  p += sizeof(size_t);  memcpy(p, car.height, sizeof(size_t));  p += sizeof(size_t);  memcpy(p, car.model.c_str(), car.model.length() + 1); // (6)  p += car.model.length() + 1;// (6)  memcpy(p, car.engine, sizeof(*car.engine);// (7)}void RestoreCar(Car &car, const void* src)// (8){  // src references the memory chunk M which contains the bytes of a Car  // object previously marshalled by CopyCar function, so we know  // the data structure and composition of M. Thus here we can  // un-marshal bytes stored in M to assign to each member of car.  // Since we have data of the Engine member in M, we should create an  // Engine object E using 'new' operator, and assign to its members   // using bytes in M, and assign E's pointer to car.engine.  char *p = src;  memcpy(car.length, p, sizeof(size_t));  p += sizeof(size_t);  memcpy(car.width, p, sizeof(size_t));  p += sizeof(size_t);  memcpy(car.height, p, sizeof(size_t));  p += sizeof(size_t);  car.model = p; // (9)  p += car.model.length() + 1;// (9)  memcpy(car.engine, p, sizeof(*car.engine);  }dbstl::DbstlElemTraits<Car> *pCarTraits =   dbstl::DbstlElemTraits<Car>::instance();// (10)pCarTraits->set_size_function(CarSize);// (11)pCarTraits->set_copy_function(CopyCar);// (12)pCarTraits->set_restore_function(RestoreCar);// (13) view raw stl-ex-2d.cpp This Gist brought to you by GitHub. In (3), this function measures a Car's size in bytes, dbstl uses it to allocate just enough space to store an object's bytes. We should consider space needed for all members we want to store.In (4) Since we want to deep copy the engine object "E" referenced by "car.engine", we should consider its size too using sizeof(*car.engine) since Engine is a simple class each of whose instance locates on a continuous memory; And we want to store the model string's trailing '\0' character to unmarshal it easily, so we add 1 to the model.length().In this function we return a size just big enough so that all bytes can be placed into M with no extra trash bytes left. This depends on what bytes we want to store, i.e. the CopyCar function.In (5), this function does the marshalling work, see the comments in the function body for more information.In (6), note how the string is copied --- we only copy its characters, ignoring all other members. And we want to copy the tailing '\0' for easier handling later.In (7), note we are copying the Engine object rather than the car.engine pointer, in order to completely persist the car object. Since in CarSize we have allowed for the space for the Engine object, we have just enough space here.In (8), this function does the unmarshalling work, see the comments in the function body for more information.In (9), we are safe to do so here because we copied the trailing '\0'.In (10) This is the global singleton for class Car where we register callback functions for dbstl to use internally to store/retrieve Car objects.(11)(12)(13) We register callback functions like this. This way, we can do deep copy for instances of the Car class. And note that we should do (10), (11), (12), (13) actions before (1).There are some other features in dbstl that are beyond standard C++ STL classes, which enable you to make use of Berkeley DB's advanced features such as secondary indexes, bulk retrieval (to speed up reading chunks of consecutive data items), transactions, replication, etc. Please refer to the dbstl API documentation and reference manual for details.The full code for this example can be found here.There are quite a lot of example code in the download package $(db)/examples_stl and $(db)/test_stl/base demonstrating these advanced features; And in $(db)/test_stl/ms_examples and $(db)/test_stl/stlport there are dbstl standard usage example code. These example code is good resource to start with.

In the second entry I showed that the Berkeley DB's C++ STL API can substitute in for in-memory data structures like vectors. Now in this final installment we'll see Berkeley DB at work storing...

Berkeley DB

The C++ Standard Template Library as a BDB Database (part 2)

In the first entry I touched on some of the features available when using Berkeley DB's C++ STL API. Now it's time to dive into more detail.Every copy of Berkeley DB comes with this API, there is nothing extra to download or add-on. Simply download the latest version and configure it to build the C++ and STL APIs and you're ready to go.We've worked to make it very easy to use dbstl if you already have C++ STL programming experience. It is especially easy to map your C++ STL containers existing code to dbstl. In the following sections I will list several pieces of C++ code which uses C++ STL, and show you how to convert them to use dbstl instead. If you want to use dbstl to manage purely in-memory data structures there is very little conversion is needed from standard STL, but most of the time you will choose Berkeley DB for its persistence, concurrency and transactional recovery features. To use those features you will need to use some of the basic features of dbstl.I. Basic Features 1. Suppose we have the following code which uses C++ STL std::vector container: int vector_test(int, char**){  typedef std::vector<double> dbl_vec_t; // (1)  dbl_vec_t v1; // Empty vector of doubles. (1)  v1.push_back(32.1);  v1.push_back(40.5);  // The rest follows ... view raw stl-ex-1a.cpp This Gist brought to you by GitHub. This is the code to use dbstl instead, note that only lines 3 and 4 are modified. int vector_test(int, char**){  typedef dbstl::db_vector <double, ElementHolder<double> > dbl_vec_t; // (1)  dbl_vec_t v1; // Empty vector of doubles. (1)  v1.push_back(32.1);  v1.push_back(40.5);  // The rest follows ... view raw stl-ex-1b.cpp This Gist brought to you by GitHub. The reason for the change is:a. All dbstl classes and global functions are defined inside "dbstl" namespace.b. For all dbstl container class templates, we must add one more type parameter ElementHolder if T is a primitive data type like int, double, char*, etc; If T is a class type, this type parameter is not needed.c. Here we used default constructor for v1, in the dbstl case, this means an anonymous in-memory database will be created and used only by the v1 vector. You can only share the v1 vector or its database handle to share the underlying database in current process only. Alternatively, you can create a database environment and open a database inside it explicitly and use the opened handles to create a container like the following: int vector_test(int, char**){  typedef dbstl::db_vector <double, ElementHolder<double> > dbl_vec_t;  DbEnv *penv = new DbEnv(DB_CXX_NO_EXCEPTIONS);// (2)  penv->open("dbenv",     flags | DB_CREATE | DB_INIT_MPOOL | DB_PRIVATE, 0777); // (3)  pdb = dbstl::open_db(penv, "vector2.db",     DB_RECNO, DB_CREATE | dboflags, 0);// (4)  dbstl::register_db_env(penv);// (5)  dbstl::register_db(pdb);// (6)  dbl_vec_t v1(penv, pdb);// (7)  v1.push_back(32.1);  v1.push_back(40.5);  // The rest follows ... view raw stl-ex-1c.cpp This Gist brought to you by GitHub. This snippets contains the majority of code you need to add to convert your C++ STL application into a dbstl-enabled application.In line 3, we must create the DbEnv object using "new" operator, this is also a requirement to create a Db object; In line 4, the "flags" variable can be set to open a transactional environment, a concurrent data store environment, or simply a data store environment. Actually any valid use of Berkeley DB via its C/C++ API can be used here.Between lines 3 and 4, you can set various flags or callback functions to configure the environment, in the same way you used Berkeley DB C++ API. There is a helper function dbstl::open_env to open an environment in one call, but with less configurations to do.In line 5, we used a helper function dbstl::open_db to open the database, and optionally set various flags to the database. It helps you to open a database easier, though there are something you can't do with it, for example setting callback functions. So use it if it is sufficient for you, or simply open a database in the same way you use Berkeley DB C++ API. We need to pay attention that different type of containers have different requirement to its database handles, see dbstl API documentation for details. Here db_vector requires the database to be of DB_RECNO type.In lines 6 and 7, we must register the created database and environment handles into dbstl in each thread using the handles, see the documentation for the two register functions for details.In line 8, we pass the database and environment handles to v1, so that v1 is backed by pdb database. Other thread of control can also open the database and access it concurrently, using dbstl, or simply using DB C/C++ API.2. Apart from the above code to construct a dbstl container, the rest of the code does not need any modification. The following snippet can be appended to the above three snippets to be three complete functions doing basically the same thing:   for(dbl_vec_t::iterator itr = v1.begin(); itr != v1.end(); ++itr)    *itr *= 2;  for (int i = 0; i < v1.size(); i++)    v1[i] += 3;  v1.swap(v2); // Swap the vector's contents.  v2 = v1; // Assign one vector to another.  assert(v1 == v2);  std::reverse(v1.begin(), v1.end());    // More standard features follow ...  for(dbl_vec_t::reverse_iterator ritr = v1.begin(); ritr != v1.rend(); ++ritr)    *ritr /= 2;  v1.pop_back();  v1.insert(v1.begin(), 34);  assert(v1.front() == 34);  v2.assign(v1.begin(), v1.end());  return 0;} view raw stl-ex-1d.cpp This Gist brought to you by GitHub. Here in line 1, there are some more parameters in v1.begin() to control the behavior of the created iterator in dbstl. Refer to the dbstl API documentation for more information. In 2, after this call, the new value for this element is stored into database, also true for line 5. In line 4, you can make the v1.size() call to compute faster but not precisely. This is helpful when the database contains millions of key/data pairs. Like size(), there are some more similar methods in all container classes which have default parameters to work like C++ STL, but can be configured to work better with Berkeley DB in special situations. In 7 the key/data pairs in v2's backing database is written to v1's backing database, after the data in v1 is truncated. Also true for line 8, where v1's key/data pairs written into v2 after data in v2 is truncated.In line 10, almost all algorithm functions in C++ STL library can work with dbstl, because dbstl has standard iterators and containers, the default behaviors of dbstl containers and iterators follow C++ STL specifications. The exception to this fact is the "inplace_merge", "find_end" and "stable_sort" in the STL library of GCC compiler, these three functions don't work with dbstl correctly. Apart from them, all C++ STL algorithms are always applicable to dbstl.Starting at line 14, dbstl containers and iterators have all methods that each corresponding C++ STL containers have, and each with identical default behaviors. So you can use dbstl just the same way you use C++ STL containers/iterators to access Berkeley DB.In our next post...Next up we'll dive even deeper into more advanced features of the dbstl API. For now if you'd like to read ahead the code is here.

In the first entry I touched on some of the features available when using Berkeley DB's C++ STL API. Now it's time to dive into more detail. Every copy of Berkeley DB comes with this API, there is...

Berkeley DB

Berkeley DB 11gR2 (11.2.5.1.19) Released!

The first release of Oracle Berkeley DB with support for SQL came out roughly six months ago and was on par or better in terms of performance with SQLite while keeping 100% drop-in API compatible with the SQLite3 APIs. We tested it in Firefox, Chrome and other popular and demanding SQLite-based open source applications to make sure it worked out of the box and it did! The advantages of BDB's storage engine were clear to everyone who tried it for themselves; BDB is faster, more scalable, more concurrent and offers greater operational efficiency for the same workload. This release delivers even more functionality, more speed, and much more BDB goodness to users of the BDB SQL storage API.The Berkeley DB SQL API provides a very easy to use, well understood interface for Berkeley DB users -- SQL, JDBC, ODBC and ADO.NET. Building applications with Berkeley DB just got a whole lot simpler. With the addition of the SQL API, Berkeley DB clearly has pulled into the lead in the embedded/mobile/edge SQL database race. Our plan is simple: Ubiquity by supporting SQL, JDBC, ODBC and ADO.NET, Flexibility by allowing the application developer to chose the API that works best for them, Provide best-in-class storage engine features, performance, concurrency and scalability,Stay fully compatible with the SQLite API, With this release, we continue to widen the gap between Berkeley DB and the other embedded storage solutions that are available. We are committed to sustaining a competitive advantage for customers who choose Berkeley DB's SQL storage APIs. Our value is in our ability to put your products ahead of your competition without adding complexity or forcing unnecessary compromises. We want you to have an obvious advantage over your competition simply by choosing Berkeley DB. The fact is that Berkeley DB users now have two streams of technical innovation working for them. This release:delivers a complete, embeddable SQL92 databaseas a library under 1MB sizedrop-in API compatible with SQLite version 3no-oversight, zero-touch database administration>15 year battle tested Berkeley DB BTREE for concurrent transactional data storagesupport for encrypted SQL database filessupport for Apple's iOS platform on iPhone/iPad/iPod mobile devicesreal binary file management with access using SQL (BFILEs)online database file space reclamation (VACUUM) that can be used while the database is operational without stopping or disrupting active transactionsfaster on many common operations than SQLite when single threaded, much faster under concurrent loadwe've reduced database file size, a more optimized index layoutadded an option to split databases into multiple files, one per-table, rather than all in one filewe've added an option to share cache, locks, and transaction logs across multiple separate databases to reduce overhead when dealing with dozens (or more) databases in use at onceeasier performance monitoring/measurement using DTrace or Systemtap probes on Solariswe've added a new command line tool called "db_replicate" which turns any SQL or non-SQL database environment into a replicated HA cluster in one simple step, no code required (use it to test HA, keep a hot/live backup, or even in production!)and a lot more!Berkeley DB is hands-down the best edge, mobile, and embedded database available to developers. We're moving more and more of the core BDB features into the SQL API with every release. We hope you're excited because we certainly are!Software DownloadsDownloads available at (direct links: .tar.gz -- .zip -- .msi)Documentation

The first release of Oracle Berkeley DB with support for SQL came out roughly six months ago and was on par or better in terms of performance with SQLite while keeping 100% drop-in API compatible with...

Berkeley DB

Oracle join the SQLite Consortium

Yesterday Oracle became the latest member of the SQLite Consortium.Over the past few months Oracle has been launching various products with new support for the SQLite3 API. Oracle Database Lite Mobile Server now supports synchronization of data within SQLite databases to Oracle Database. Oracle JDeveloper is adding support for SQLite. And most recently Oracle Berkeley DB 11gR2 (11.2.5.0.21) was released with a new SQL API which is in fact a layering of SQLite query processing code on top of Berkeley DB storage code. Combining the two products brings SQL processing to Berkeley DB users and a more robust, concurrent, and scalable storage layer (btree) to the SQLite community of users.To show our commitment to the community which built SQLite and demonstrate our sincere desire to be a good citizen and partner in that community we've joined the SQLite Consortium. In so doing we gain access to more complete test suites which we will use to validate our combined BDB SQL(ite) API. When we find and fix bugs or make improvements to the SQLite code it is our intention to provide that code to the SQLite team for integration into their product. We don't want to see SQLite fork, we will work to continually integrate and cooperate with the SQLite developers.Finally, we fundamentally believe that the SQLite approach to small device SQL storage is the right one. They are making the right trade-offs, hard decisions not to include infrequently used features or over optimize code prematurely. The SQLite SQL engine is an amazing example of the 80/20 rule and due to that focus of purpose it is now a dominant, defacto solution in mobile and embedded devices as well as small to medium sized ISV applications.As a new emergent standard, the SQLite3 ANSI C API, is now supported by at least two vendors and used by millions of open source developers around the globe. Ask your vendor, "Do you support the SQLite3 API?" because that's the new "ODBC" for in-device and in-application relational database access.The Oracle mobile platform database strategy is evolving and the Berkeley DB storage product is the way we expect everyone will build solutions on the edge or in consumer devices. Berkeley DB's new SQLite-based SQL API is the new standard by which all solutions will be measured.Is your solution based on the SQLite API or Berkeley DB's SQL?

Yesterday Oracle became the latest member of the SQLite Consortium. Over the past few months Oracle has been launching various products with new support for the SQLite3 API. Oracle Database Lite Mobile...

Berkeley DB Java Edition

A LocusTree to search genomic loci using Berkeley DB Java Edition

Pierre Lindenbaum has adapted Jan Aerts' LocusTree implementation written in Ruby to work on top of Berkeley DB Java Edition. In his blog post he writes about using this combination to build a genome browser.Genomic research is rapidly growing. There is an explosion of new data and along with that an explosion of new questions to be asked of and answered by that data. George Church is behind the Personal Genome Project working to collect huge amounts of new data, but gathering this data is only the first step. Storing and analyzing it is where the discovery process needs technological advancements. To compound the issue even more, today we examine base pairs within genes by understanding their proximity to other sections of the genome, there is new work to examine the proximity of base pairs which are near each other in the folds of a DNA strand rather than their proximity along the length of the strand. Could it be important to understand how two segments overlap? For both problems the solution is a sturdy technical foundation which allows you to search quickly and find interesting features within the data.One area where there is always a need for more work is that of drug discovery. The central question of this article in the Economist is that "a toxic mix of science and economics" is preventing research and discovery. A large piece of the economics half has to do with the cost associated with making discoveries. Using new techniques which are faster and manage more data can help to change that by lowering the cost of research.In this case, searching genomic data sets, the use of the Berkeley DB Java Edition database make perfect sense. It is a non-relational dataset, this data isn't tabular or relational at all rather it's commonly organized in key/value pairs. BDB JE is a spectacular database for indexing large data sets without the overhead of other more complex systems and a perfect fit for Pierre's visualization project.

Pierre Lindenbaum has adapted Jan Aerts' LocusTree implementation written in Ruby to work on top of Berkeley DB Java Edition. In his blog post he writes about using this combination to build...

Berkeley DB

The C++ Standard Template Library as a BDB Database (part 1)

If you've used C++ you undoubtedly have used the Standard Template Libraries. Designed for in-memory management of data and collections of data this is a core aspect of all C++ programs.Berkeley DB is a database library with a variety of APIs designed to ease development, one of those APIs extends and makes use of the STL for persistent, transactional data storage. dbstl is an STL standard compatible API for Berkeley DB. You can make use of Berkeley DB via this API as if you are using C++ STL classes, and still make full use of Berkeley DB features.Being an STL library backed by a database, there are some important and useful features that dbstl can provide, while the C++ STL library can't. The following are a few typical use cases to use the dbstl extensions to the C++ STL for data storage.When data exceeds available physical memory.Berkeley DB dbstl can vastly improve performance when managing a dataset which is larger than available memory. Performance suffers when the data can't reside in memory because the OS is forced to use virtual memory and swap pages of memory to disk. Switching to BDB's dbstl improves performance while allowing you to keep using STL containers.When you need concurrent access to C++ STL containers.Few existing C++ STL implementations support concurrent access (create/read/update/delete) within a container, at best you'll find support for accessing different containers of the same type concurrently. With the Berkeley DB dbstl implementation you can concurrently access your data from multiple threads or processes with confidence in the outcome.When your objects are your database.You want to have object persistence in your application, and store objects in a database, and use the objects across different runs of your application without having to translate them to/from SQL. The dbstl is capable of storing complicated objects, even those not located on a continous chunk of memory space, directly to disk without any unnecessary overhead.These are a few reasons why you should consider using Berkeley DB's C++ STL support for your embedded database application. In the next few blog posts I'll show you a few examples of this approach, it's easy to use and easy to learn.

If you've used C++ you undoubtedly have used the Standard Template Libraries. Designed for in-memory management of data and collections of data this is a core aspect of all C++ programs. Berkeley DB is...

Berkeley DB at Oracle Open World 2009

Where to find Berkeley DB people & sessions at Oracle Open WorldIf you are interested in hearing about Berkeley DB, learning more about how the Berkeley DB products can be integrated into your applications/appliances/devices, seeing exciting Berkeley DB customer use cases or speaking directly with one of the Berkeley DB product development engineers, you can find us here: Session ID#: S311365, Title: Oracle Berkeley DB: Lightning-Fast Key Value Storage Just Got Faster, Date: Sun, Oct. 11th, Time: 15:45 - 16:45, Venue: Hilton Hotel, Room: Golden Gate 1, Track: Oracle Develop: Database, Speaker: David Segleau (Director Product Management), including customer presentations from Lucas Vogel, Managing Partner at EndPoint Systems and Madhu Bhimaraju, Database Architecture Engineer at Verizon Wireless. We will be covering several of the new features in Berkeley DB 4.8 and discussing how Verizon Wireless uses Berkeley DB to provide services to an ever-growing customer base. Session ID#: S311364, Title: Oracle Berkeley DB Java Edition High Availability: Java Persistence at Network Speeds, Date: Sun, Oct 11th, Time: 14:30 - 15:30, Venue: Hilton Hotel, Room: Golden Gate 1, Track: Oracle Develop: Database, Speaker: Sam Haradhvala (Senior Engineer on the Berkeley DB Java Edition product). We will be discussing the new High Availability/Replication functionality that is now available in Berkeley DB Java Edition -- how it works, how it can improve your application performance, reliability and throughput, as well as common use cases and configurations. Berkeley DB Applications Demo/Lunch. Date: Mon, Oct. 12th, Meeting Time: 11:00 - 13:00, Venue: Hilton Hotel - Union Square Room 6, 4th Fl. Greg Burd (Senior Product Manager) and several Berkeley DB engineers will be showing several Berkeley DB application demos, discussing how they were implemented and how similar functionality can be part of your application. Join us for lunch, some interesting demos and a open question and answer session. The Berkeley DB Product booth in the Exhibition Hall in Moscone West. We're workstation W-035, under the Database Track, in the Embedded Database sub-area just like last year. The booth is open Monday and Tuesday from 10:30-6:30 and Wednesday from 9:15-5:15. We're always delighted to talk with existing users, potential users and anyone who is curious about our embedded database libraries.We hope to see you all there!

Where to find Berkeley DB people & sessions at Oracle Open World If you are interested in hearing about Berkeley DB, learning more about how the Berkeley DB products can be integrated into...

Berkeley DB

Berkeley DB 4.8

Today we are very happy to announce the availability of an amazing new version of Berkeley DB. This release is full of new features, performance improvements, and much more.New in Berkeley DB 4.8:Performance has been enhanced for multi-threaded/process applications on CMP/SMP systems.New C# and .NET API providing more support for Microsoft programmers.New support for C++ Standard Template Library (STL) allows developers to use transactions, cache large datasets, and persist data using familiar STL data structures and APIs.New API to support referential integrity constraints including abort, cascade, nullify semantics.New B-tree compression APIs allowing you to reduce the amount of disk space required to store your data with your custom compression code (or our default implementation). Using compression will reduce disk I/O and allow more data to live in-cache. This space savings in turn improves performance (less I/O and more in-cache data) despite the overhead of compressing/decompressing data. Random reads is one case where performance could suffer when compressing data, be sure to understand your data access patterns and test your performance.New support for partitioning database files can improve application throughput and reduce lock contention when data is split across multiple I/O channels and/or disks.New bulk load/delete APIs can significantly improve application performance.New command line tool for bridging the gap between SQL and Berkeley DB databases called 'db_sql'. This utility will translate the majority of SQL92 DDL into Berkeley DB source code.

Today we are very happy to announce the availability of an amazing new version of Berkeley DB. This release is full of new features, performance improvements, and much more. New in Berkeley DB 4.8: Perf...

Welcome, let's DB->open() a conversation about data storage.

Here you will find articles about Berkeley DB products, interesting use cases, and fun factoids to keep you informed and hopefully interested in Berkeley DB products.The term "database" conjures up thoughts of SQL, tables, and client/server architecture. This hasn't always been true. Database is not just another way of saying "SQL" and it's not always a synonym for RDBMS (relational database management systems). In today's complex distributed and diverse software ecosystem there are many new and divergent requirements for data storage. A sizable portion of those cases don't have any need for SQL, but they do need transactions, recovery, concurrent access, replication, fail-over, hot-backup, and all the other core features of an RDBMS. There is a growing awareness that non-relational (aka NoSQL) storage solutions have an important role to play in systems of all shapes and sizes.Of course here within Oracle, and especially within the Berkeley DB group, we've known this for a long time. We've been working for over a decade on a database engine that has the qualities found at the core of most modern database products (relational, object, hierarchical, etc.) without any of the fluff or imposed structure layered on top. We've built a data storage component, a library to be incorporated into your application. And your product doesn't have to be a database. Most uses of Berkeley DB are within programs that never export storage services directly to their users, it can be a software application, a hardware controller, or a globally networked service.Berkeley DB products are designed to let you focus on building your application while leaving the complexity of data storage to us.For now DB->close().

Here you will find articles about Berkeley DB products, interesting use cases, and fun factoids to keep you informed and hopefully interested in Berkeley DB products. The term "database" conjures up...