Tuesday Sep 22, 2009

Videos to get acquainted with Apache Derby and Java DB

javadb.png

derbyLogo.png

Kristian Wagaan and Richard Hillegas, both Apache Derby committers who also work on Java DB, recently gave some great presentations about Apache Derby and Java DB.

Kristian Wagaan presented at the 2009 OpenSQLCamp, in conjunction with FrOSCon 2009 in Europe.

The presentation (slides) gives a brief overview of Apache Derby, its history and the community around it. Further, it goes into the details of features that have been added in the latest releases and show how they enable powerful ways to use a relational database.




Rick Hillegas gave an introduction of Java DB / Apache Derby at The San Francisco Java User Group's Sep 8th event in San Francisco.


Monday Sep 14, 2009

Java DB 10.5.3 is now available

Following the release of Apache Derby 10.5.3, Java DB which is Sun's supported version of this last one is also now available.

Java DB 10.5.3 is primarily a bug-fix release and inherits all the bug fixes and localized messages from Apache Derby 10.5.3.

Java DB offers installers for the following platforms:

  • Solaris
  • Linux
  • Windows
  • Mac OS X (newly added)

Here is a recap of all changes that are coming as part of Java DB 10.5.3:
  • SQL Roles - SQL roles (as described in SQL 2003 and errata) are useful for administering privileges for groups of users. Administering roles is generally less error-prone than administering privileges for individual users. SQL Roles are defined in Feature T331 "Basic roles" and Feature T332 "Extended roles". Java DB 10.5 implements a subset of T331, plus support for CURRENT_ROLE, which is a part of T332.

  • Generated Columns - Generated Columns is a feature which instructs Java DB to fill a column with an expression built out of other columns in the row. Java DB constructs these column values at INSERT and UPDATE time. The user declares indexes on these columns. This in turn improves SELECT performance since it lets users declare indexes on pre-computed pieces of the WHERE clause. This feature satisfies support for SQL Feature T175.

  • LOB Improvements - There were many performance and usability improvements for BLOBs and CLOBs.

  • Replication of encrypted databases - With 10.5 it is possible to replicate encrypted databases.

  • OFFSET/FETCH FIRST - SQL 2008 has added new syntax to support a direct way to limit the returned set of rows in a result set, through the fetch first and offset clauses.

  • In-memory back end - Initial implementation of a storage engine for Java DB where all data is kept in memory. There is no documentation for this feature. This functionality itself is not yet fully implemented, but users are welcome to experiment with it. For details, see the Primer for In-memory Back Ends.

  • Standard ALTER COLUMN syntax - Allow standard SQL "SET" keyword in ALTER COLUMN syntax, like so: "ALTER TABLE ALTER COLUMN columnname SET DEFAULT default-value"

  • SYSCS_UTIL.SYSCS_UPDATE_STATISTICS - New system procedure that updates cardinality statistics (or creates them if they do not exist) for a table's index or for all the indexes on a table, allowing a user to ensure that a query plan based on the most recent state of the table can be created.

  • Service tag - Introduction of the Java DB registration service (service tag).

Further details on new features, changes, and issues in this release can be found in the Release Notes.

Friday May 01, 2009

Apache Derby 10.5.1.1 Feature Release is now available!

The Apache Derby project is pleased to announce a new GA feature release
of Derby, 10.5.1.1.

Apache Derby is a sub-project of the Apache DB project. Derby is a pure
Java relational database engine which conforms to the ISO/ANSI SQL and
JDBC standards. Derby aims to be easy for developers and end-users to
work with.

Derby 10.5.1.1 can be obtained from the Derby download site:
http://db.apache.org/derby/derby_downloads.html.

It contains the following new features:
  • SQL Roles - SQL roles (as described in SQL 2003 and errata) are useful for administering privileges for groups of users. Administering roles is generally less error-prone than administering privileges for individual users. SQL Roles are defined in Feature T331 "Basic roles" and Feature T332 "Extended roles". Derby 10.5 implements a subset of T331, plus support for CURRENT_ROLE, which is a part of T332.
  • Generated Columns - Generated Columns is a feature which instructs Derby to fill a column with an expression built out of other columns in the row. Derby constructs these column values at INSERT and UPDATE time. The user declares indexes on these columns. This in turn improves SELECT performance since it lets users declare indexes on pre-computed pieces of the WHERE clause. This feature satisfies support for SQL Feature T175.
  • LOB Improvements - There were many performance and usability improvements for BLOBs and CLOBs.
  • Replication of encrypted databases - With 10.5 it is possible to replicate encrypted databases.
  • OFFSET/FETCH FIRST - SQL 2008 has added new syntax to support a direct way to limit the returned set of rows in a result set, through the fetch first and offset clauses.
  • In-Memory back end - Initial implementation of a storage engine for Derby where all data is kept in memory. There is no documentation for this feature. This functionality itself is not yet fully implemented, but users are welcome to experiment with it. For details, see the Primer for In-memory Back Ends.
  • Standard ALTER COLUMN syntax - Allow standard SQL "SET" keyword in ALTER COLUMN syntax, like so: "ALTER TABLE ALTER COLUMN columnname SET DEFAULT default-value"
  • SYSCS_UTIL.SYSCS_UPDATE_STATISTICS - New system procedure that updates cardinality statistics (or creates them if they do not exist) for a table's index or for all the indexes on a table, allowing a user to ensure that a query plan based on the most recent state of the table can be created.

In addition, Derby 10.5.1.1 contains many bug and documentation fixes.

Java DB, which is Sun's distribution of Apache Derby should be available as a 10.5.1.1 release level very soon.

Tuesday Dec 16, 2008

Java DB on Twitter - Become a follower

Java DB (based on Apache Derby) is now on Twitter. If you want to follow the latest about Java DB, then become a follower ;-)

Devoxx 08' - Java DB Quickie Presentations

I was at Devoxx (formely known as JavaPolis) 08' last week and like the previous years, the conference was packed with more than 3,000 attendees.

I had the opportunities to present 2 quickies and you can find the presentations below:

- How to JavaFX with Java DB in less than 10 minutes (pdf)

- Virtualization or How to query any dataset with Java DB (pdf)

Monday Aug 11, 2008

Apache Derby Case Study: Benefits of a Microkernel architecture

Software embracing a microkernel architecture has been happening for the last 20 years, yet the great aspects of this particular architecture are too often undermined and not considered when working on a new project. There are various reasons for this, history behind it with poor experiences, but there has also been a lot of lessons learned . So what can one benefit from a microkernel architecture?

Microkernel introduction

In the last two decades, a lot of changes happened in the world of software and Internet has revolutionized quite a few trends. Microkernel architecture used to be adopted for operating system type of implementations pretty much exclusively. I remember about ChorusOS, a Unix variant and first-generation microkernel OS developed in the early 90's which was targeted at high-availability applications in the telecommunications space. Chorus was truly innovative back then and like Mach, another first generation microkernel OS implementation, suffered performance issues due to message processing between the kernel and user spaces (no sharing of address spaces). Minix, another well-known unix-like OS based on a microkernel architecture was created by Andrew S. Tanenbaum in 1987 and influenced the development of Linux although Linus Torvalds went with a monolithic kernel architecture instead.

There has been quite a few debates regarding the benefits and tradeoffs of a microkernel architecture versus a monolithic one. Most Unix operating systems are based on the last one, with no messaging between parts of the system and an easier (and controlled) access between the kernel and user spaces. A microkernel-based implementation typically grants the software with added modularity and flexibility due to the abstraction and extra independency (clear boundaries) between the various modules / parts of a system.

I stated earlier that trends had changed in the last 20 years and although it may sound like stating the obvious, what I really meant to say is that software architectures do evolve due to other factors such as hardware, markets and  new requirements. Today one can see a lot more software implementations based on a microkernel architecture, due to an increase demand and new requirements motivating that choice. Hardware (e.g. memory, CPU) has gotten a lot faster and cheaper, there is a lot of hardware environments to port to, the internet and its fast moving pace has increased the demand for more dynamic and flexible software.

Multi-threaded server applications are a lot more popular these days and the overhead of a microkernel-based implementation is no longer an issue when it comes to interactions between various modules involved in a particular processing operation. It used to be true for the most part, when IPC mechanisms had to be used to relay messages between modules running as part of different processes.

Why Microkernel

A microkernel architecture could be summarized to a "highly modular collection of application-neutral abstractions upon which can be built robust, flexible and dynamic applications" and brings the following and non-exclusive benefits to software implementations:

  • Modularity
  • Flexibility
  • Extensibility
  • Better footprint management
  • Easier development, unit testing, maintenance and portability

The modularity, flexibility and extensibility aspects are what I would consider the most important ones of a microkernel-based implementation. A highly modular architecture  allows protocols to be separated from their implementations as well as maintaining a level of independence between the various modules and components / services (enforcement of boundaries) of the system. Flexibility grants you with dynamic loading and unloading of modules and extensibility gives the developers means to extend the current system with new protocol or schemes implementations.

Footprint management can be an important issue in certain environments and the ability to remove unnecessary components as part of your application can help reducing the footprint significantly.

Modules can be developed and unit tested independently of each others and portability is usually easier to handle as non portable logic can be placed at the kernel level directly and not mixed with non-kernel components.

As far as potential performance improvements, I think this is very debatable unless one is dealing with a single-threaded monolithic implementation and in this case it would be slower on SMP machines obviously. I'm a big fan of multi-threaded implementations versus multi-process ones and in this case both architectures would run better on multi-core systems.

Microkernel-based architecture has also reached the world of database systems and most recently, Drizzle, a fork of the MySQL core engine has been announced and is supposedly aimed at Web and Cloud applications. This confirms the need for more modular and flexible implementations at the database management system level.

Microkernels and the likes in Java land

The Java platform and more specifically the language, facilitates the development of microkernel-based implementations. The object-oriented aspect of Java allows abstractions via published interfaces to be defined for the various parts to be implemented. In other words, this enables a clear separation between the protocol various implementations. Java dynamic class loading also allows modules to be loaded on the fly as needed.

To name a few Java applications and frameworks which have adopted a microkernel-based architecture:

  • GlassFish V3 has adopted HK2, the Hundred Kilobytes Kernel, with a module subsystem and component model on top of OSGI
    https://hk2.dev.java.net/

For an example of a Spring DM application leveraging OSGI dynamic module system and separation of application logic into modules, I recommend reading this article.
Apache Derby which can also run as an OSGI bundle (since 2005) is used in this example as well.

There is also growing interest in the cloud computing arena with having a more modular, flexible and extensible Apache Hadoop. Being able to plug-in a query language of choice on top of Hadoop's huge datasets is a very attractive value-proposition indeed.

Apache Derby Case Study: How to benefit from a Microkernel architecture

Apache Derby / Java DB has been supporting a highly modular architecture since its inception in 1997 with its first release as part of Cloudscape. Derby's system is composed of a monitor, acting as a kernel of its own, booting and shutting down services as well as discovering appropriate modules to load for a particular service. As an example, different modules can be loaded during boot-up based on the environment such as the JVM version. Modules themselves can also require other modules to be loaded in the environment.

Derby already supports pluggable authentication schemes as well as pluggable raw storages via its module-based subsystem. It promotes a clean separation between the SQL and Store layers which favors the implementation and pluggability of different types of storages. For instance, one could envision a GRAPH storage for Derby to enable a graph-based data model to be accessed via Derby's relational engine. There is already some implementation of an in-memory storage and even a store-less storage type to test Derby's layers separation and submit queries against non-database objects.

The extra benefit of having a clear separation between a protocol and its implementation allows other projects to re-use parts of the system and contribute back to the open source community. For instance, several project have expressed their interest in integrating Derby's raw B-Tree storage as it has proven to be robust over 10 years and has an active open source community.

There are already on-going efforts and proposals to make Derby more flexible and reduce dependencies between modules of the store layer and have a cleaner pluggable storage implementation. When this modular architecture got implemented 10 years ago, there was little demand for specialized and pluggable storage. This has changed with new types of data models and requirements for web applications and cloud computing, translating into innovations at the data management level.

The good news for Derby is that it has already gotten the proper foundations in place to go the extra mile and have a clean microkernel architecture. This would not only benefit the store layer but also other parts of the engine, such as the SQL compiler (parsing and execution plan generation) and the access layer, residing above the store. For instance, one could imagine having Derby's SQL layer plugged-in on top of Hadoop, as an extension to be able to run SQL queries against MapReduce datasets.

In Conclusion

Software implementations which are supporting a microkernel architecture inherit highly modular and flexible characteristics. The performance gap between a monolithic and microkernel architecture is no longer much of an issue, especially in a multi-threaded environment where messaging is done via local interfaces without requiring the use of IPC mechanisms.

The dynamic nature of web applications and requirements to constantly adapt to new standards and protocols will continue to promote this architecture. OSGI has already showed some path and additional web frameworks have become compliant.

Derby with its highly-modular architecture excels in an embedded server mode configuration and is a natural database storage of choice for the Java platform and microkernel-based implementations in the middle-tier.

Thursday Apr 19, 2007

Ubuntu Users Gets Java DB Surprise and A Lot More Than That...


It has been announced today, Ubuntu Feisty Fawn (7.04) has been released and it includes Sun's complete Java stack.

This is the first time that an entire, production quality Java stack with tooling is being included in an open source GNU/Linux distribution - greatly simplifying the process for Linux developers and users to access Java technology.

The complete and fully supported components of Sun's Java stack which are packaged for Ubuntu are Java SE 6 JDK, GlassFish v1, NetBeans IDE 5.5 and Java DB 10.2 (based on Apache Derby). (FAQ)

You can also check Tom Marble's blog entry for additional information. This is very exciting indeed.

You can 'apt-get' install Java DB as an optional Ubuntu package such as:
# apt-get install sun-java6-javadb
and it should get installed under '/usr/lib/jvm/java-6-sun/db/'

Sunday Mar 04, 2007

Sun releases Java Enterprise System 5.0 with Java DB (based on Apache Derby)



"Java ES (Enterprise System) already includes Java DB, Sun's supported distribution of the open-source Apache Derby database. The middleware also supports the PostgreSQL open-source database.

Sun now lays claim to more than 1.3 million subscribers for Java ES compared to the nearly 1 million it declared in October 2005."

Monday Feb 26, 2007

Derby Embedded w/ Eclipse

To follow-up on the Java DB / Apache Derby with Netbeans blog entry that I recently posted, here is another user experience from PaweĊ‚ Piotrowski who used Eclipse (IDE) to access Derby in embedded mode.

Wednesday Feb 14, 2007

Connecting a GUI to a Java DB Database with NetBeans IDE


This NetBeans tutorial contributed by Talley Mulligan guides you through the process of connecting the GUI of an application to a Java DB database. It shows how to add data-aware JDBC components to the GUI that will enable the program to interact with the database.

If you have not seen it already, there is also a "Working with the Java DB (Derby) Database in Netbeans 5.5" tutorial contributed by Brian Leonard.

Wednesday Feb 07, 2007

ActiveMQ graduates from Apache Incubator and uses Apache Derby


Apache ActiveMQ Open Source Message Broker has now graduated to become a top level project (TLP) at Apache! Congratulations to the ActiveMQ community!

To achieve high performance of durable messaging, ActiveMQ embeds and uses Apache Derby (configured by default), to allow for some very fast persistence via JDBC, coupled along with ActiveMQ's high performance journal.

Thursday Feb 01, 2007

Projects & Products using Derby (Java DB)


If your product is using Apache Derby, I encourage you to make mention of it in the Projects & Products using Derby web page.

Java DB, which is based on Apache Derby and bundled with Sun JDK 6 is using the same database core engine.

Tuesday Jan 16, 2007

Java TV

Intriguing blog entry title eh? Well, no it's not a new product from Sun Microsystems, neither some TV that can run or deliver some Java...

It a is repository of video interviews around Java posted by Developpez.com, one of the biggest Java developers community in France.



Please note that a lot of the videos are in french but not all of them.

Most recently posted video interviews are of Marc Fleury (JBoss), Vincent Massol (XWiki) and myself (Java DB).

About

forsini

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks