Thursday Apr 18, 2013

New Derby feature release: 10.10.1.1

The Derby community voted to approve Apache Derby 10.10.1.1 earlier this week, and now it has been released. Get it from http://db.apache.org/derby/derby_downloads.html and try it out for yourself!

This release contains a number of new features, as well as bug fixes and other improvements. Here are the headlights from the release announcement:

  • JDBC 4.2 - Derby supports the Java 8 enhancements to JDBC.
  • Small device profile - Derby runs on the small CP2 profile of Java 8. See the corresponding JEP at http://openjdk.java.net/jeps/161
  • User-defined aggregates - Applications can create their own aggregate operators. See the "CREATE DERBY AGGREGATE" section in the Derby Reference Manual.
  • Varargs routines - SQL routines can be bound to user-written Java methods which have variable length argument lists.
  • Optional tools - Derby has new, optional tools to support data migration and metadata queries. See the "Optional tools" section in the Derby Tools and Utilities Guide.
  • SYSCS_UTIL.SYSCS_INVALIDATE_STORED_STATEMENTS - Derby has a new system procedure for invalidating stored prepared statements when you think that your metadata queries or triggers are misbehaving--for example if they throw a NoSuchMethodError on execution. See the section on this new system procedure in the Derby Reference Manual.
  • Faster query compilation - A number of changes have made SQL compilation faster.
  • Unencrypting databases - Encryption can be turned off on existing databases. See the section on the decryptDatabase attribute in the Derby Reference Manual.


Friday Oct 12, 2012

JavaOne talks available online

If you missed JavaOne last week (like me), you can now listen to the recordings and read the slides from the talks online.

Dag's introductory talk on Java DB can be found here, and the talk on how to use Java DB for storage on embedded devices, by Noel and Rick, is here.

Enjoy! :)

Friday Sep 21, 2012

Java DB talks at JavaOne 2012

It's soon time for JavaOne again in San Francisco, and Java DB is represented this year too.

Dag Wanvik will give an introductory talk on Java DB on Tuesday, October 2 at 10:00: CON5141 - Java DB in JDK 7: A Free, Feature-Rich, Embeddable SQL Database

Rick Hillegas and Noel Poore will discuss how to use Java DB on embedded devices in their talk on Thursday, October 4 at 14:00: CON6684 - Data Storage for Embedded Middleware

Mark your calendars! :)

Monday Jun 25, 2012

Derby 10.9.1.0 released

Earlier today, the release of Apache Derby 10.9.1.0 was announced.

In addition to the usual chunk of bug fixes, this release includes the following new features:

  • NATIVE authentication, a new authentication mechanism with better support for managing credentials. See this section of the developer's guide for an introduction.
  • JDBC 4.1 escape syntax completes Derby's support for JDBC 4.1.
  • Allow multi-column subqueries in EXISTS predicates (SQL:2003 Feature T501, Enhanced EXISTS predicate) to support auto-generated SQL from some persistence frameworks.

Download it now and try it out!

Wednesday Oct 26, 2011

Derby 10.8.2.2 released

Apache Derby 10.8.2.2 was released earlier today. From the release announcement:

The Apache Derby project is pleased to announce a new bug-fix release, 10.8.2.2. 

Apache Derby is a subproject of the Apache DB project. Derby is a pure
Java relational database engine which conforms to the ISO/ANSI SQL and
JDBC standards. Derby aims to be easy for developers and end-users to
work with.

Derby 10.8.2.2 can be obtained from the Derby download site:
 http://db.apache.org/derby/derby_downloads.html.

Please try out this new release.

Sunday Oct 02, 2011

Java DB and Derby at JavaOne 2011

Java DB poster

If you're at JavaOne in San Francisco this week, and you're interested in Java DB and Apache Derby, be sure not to miss these events:

  • Monday October 3 16:00 at Hilton San Francisco - Yosemite A/B/C: New Java DB Features in JDK 7 by Rick Hillegas
  • Tuesday October 4 19:30 at Hilton San Francisco - Golden Gate 6/7/8: BOF - Meet the Java DB Team
  • Wednesday October 5 12:00 in the atrium at the corner of Mission and 2nd Street: Apache Derby community lunch (open for everyone, no JavaOne pass needed)

Rick and I can also be found at the Java DEMOgrounds at Hilton, in the Core Java Platform corner, at the following hours:

  • Monday 10:15 - 15:30
  • Tuesday 09:45 - 12:45
  • Wednesday 09:15 - 12:00

You'll probably bump into one or both of us there at other times too. Hope to see many of you there!

Friday May 06, 2011

Derby 10.8 is out

Apache Derby 10.8.1.2 was released earlier this week. Read the full announcement here.

Derby 10.8.1.2 is a new feature release, the first one in the 10.8 series. The highlights include:

  • Automatic calculation of index statistics: In earlier releases, index cardinality statistics had to be updated manually (for example by calling the SYSCS_UTIL.SYSCS_UPDATE_STATISTICS system procedure) if they got out of date. Otherwise, the optimizer might choose suboptimal, poorly performing query execution plans because of the outdated information. This typically happened if one of the tables accessed in the query had grown or shrunk significantly after the index cardinality statistics were calculated initially. As of this release, Derby will recalculate the statistics if it detects that the size of the table has changed so much that the statistics are no longer usable.
  • JDBC 4.1: The early access builds of Java SE 7 (available for download here) have added new methods to many of the interfaces in the java.sql and javax.sql packages. This Derby release implements all of these methods so that you can try them out.
  • Improved interrupt handling: Earlier releases of Derby didn't handle interrupts very well. If a user thread was interrupted while performing a database operation, it could often end up crashing the database engine and require a reboot of the database. Starting with this release, interrupting a user thread shouldn't take down the entire database. The documentation has also been improved by clarifying which behaviour to expect if a thread is interrupted.
  • XML operators out of the box: Older releases of Derby required some work setting up your environment before you could use the XML operators (XMLEXISTS, XMLPARSE, XMLQUERY, XMLSERIALIZE). In Derby 10.8, the implementation of these operators has been made more portable and should work out of the box on most implementations of Java SE 6 and newer.

The release also contains many bug fixes and documentation fixes, as well as some performance improvements. As always, make sure that you read the release notes before upgrading to see if any of the changes could affect your application.

Wednesday Jan 05, 2011

OpenGrok 0.10

We're proud to announce that OpenGrok 0.10 has just been released. See the release notes below for details on how to get it and what's new in this release.

OPENGROK 0.10 RELEASE NOTES

OpenGrok is a fast and usable source code search and cross reference engine. It helps you search, cross-reference and navigate your source tree. It can understand various program file formats (C, C++, C#, Java, sh, Perl, Python, ...) and version control histories like (Mercurial, Git, SCCS, RCS, CVS, Subversion, Teamware, ClearCase, Perforce, Monotone and Bazaar). In other words it lets you grok (profoundly understand) the open source, hence the name OpenGrok. It is written in Java.

For more details go to http://www.opensolaris.org/os/project/opengrok/

To download the binaries, please go to: http://hub.opensolaris.org/bin/view/Project+opengrok/files#HBinarysourcereleases and look for 0.10 binary, source or a package file.

This is a major update. It is advised that you cleanup your old DATA_ROOT (ideally remove it, or move it aside) and run indexing from fresh. Also the web application should be replaced, since the new indexes are not compatible with the old web application. Note that this also applies to upgrades from the 0.10 release candidate, since the format of the xrefs changed between the release candidate and the release.

For install help refer to the install wiki page. The discussions page lists forums and mailing lists where you can ask for help or discuss bugs or new features. Bugs should be logged in Bugzilla in the Development/OpenGrok category.

New features

  • python analyzer, perl analyzer
  • listing of variables/methods for quicker navigation
  • sh analyzer xrefs improved
  • SQL analyzer updated to SQL:2008
  • UI usability improvements
  • updated Lucene to 3.0.2
  • introduced Maven as alternative build system to Ant

Bug fixes

  • Bug #5590: Unhighlighted difference in compare list
  • Bug #11264: Add a python specific language analyzer.
  • Bug #13184: provide quick links to annotate,history and download from search results view: diff link for history search
  • Bug #13184: provide quick links to annotate,history and download from search results view
  • Bug #13777: Indexer should print "Adding: ..." before it adds a file
  • Bug #14545: clicking on annotate doesn't preserve line number
  • Bug #14663: broken syntax highlight in ksh script when ` is used (and when " is escaped)
  • Bug #14924: Java method calls incorrectly classified as definitions
  • Bug #15468: Log useful info from indexer regardless of -v
  • Bug #15661: OpenGrok should support new ksh command substitution x=${ cmd ; }
  • Bug #15890: Missing anchor in xref
  • Bug #15938: Analyzers should only use JFlex's public API
  • Bug #15947: Detect GNU CSSC in addition to SCCS
  • Bug #15948: Fail if svnadmin is missing: tests depend on it
  • Bug #15954: teamware repository not detected anymore
  • Bug #15956: indexer fails on indexing
  • Bug #15961: Allow certain symlinks to be followed to help with using clearcase dynamic views
  • Bug #16006: Indexer's -P option removes customization of existing projects
  • Bug #16070: our ctags parser fails to parse a c signature
  • Bug #16076: problem of ELFAnalyzer in opengrok 0.9
  • Bug #16334: Indexer hangs when files contain 0x1A characters
  • Bug #16370: build.xml: Check for JFlex.jar and taskdef for jflex-task use different classpath
  • Bug #16451: Tray app doesn't work with other protocols than jmxmp
  • Bug #16454: Trayapp shuts down when a window is closed
  • Bug #16465: SVN history of filenames containing @ cannot be retrieved
  • Bug #16481: NumberFormatException when opening Connection Settings the second time
  • Bug #16611: svn repository with svn:// protocol as urn results in NPE when reading history
  • Bug #16660: Perforce repositories return months 1-12, not 0-11
  • Bug #16677: revamp option -v
  • Bug #16692: Index files left open after multiple project search
  • Bug #16760: Generating incorrect links for C/C++ "#include" statements if extension of a header file is not on the lexer's list
  • Bug #16761: Perforce history reader would not use per-project clients (defined by P4CONFIG files)
  • Bug #16764: Perforce source annotation: the date is one month ahead, time information is not available
  • Bug #16772: status.jsp should be less chatty in production environments
  • Bug #16783: more.jsp garbles matches on lines crossing internal JFlex buffer boundary
  • Bug #16786: Control for User Information Provider URL (userPage)
  • Bug #16833: Subversion repositories disabled by default
  • Bug #16848: Search context is displayed in lower case
  • Bug #16883: ShXref should reset stack on reuse
  • Bug #16891: Unneeded files included in opengrok.jar
  • Bug #16892: Maven build includes management client source in jar
  • Bug #16893: Maven build includes SQLXref.lex
  • Bug #16909: Move JFlex generated sources to default location (Maven)
  • Bug #16913: OpenGrok fails to use JDBC cache if -R option is specified
  • Bug #16961: Definitions should have serialVersionUID
  • Bug #16986: Relative path for data root and source root has stopped working
  • Bug #17074: Home link no longer works
  • Bug #17127: OpenGrok leaving too many open files
  • Bug #17128: add more syntax highlighting styles and list all definitions/functions
  • Bug #17169: enable wildcard prefix search in OpenGrok script by default
  • Bug #17294: show progress on demand
  • Bug #17349: path needs to be percent encoded
  • Bug #17373: breadcrumb path needs to be URI encoded
  • Bug #17374: history links need percent encoding
  • Bug #17375: annotate link needs percent encoding
  • Bug #17376: download link needs percent encoding
  • Bug #17378: Please remove email obfuscation
  • Bug #17550: Malformed search for query with national chars
  • Bug #17551: Double-click on some project in project list doesn't work in Internet Explorer
  • Bug #17554: Empty "Navigate" window in Perl code
  • Bug #17582: context shown in hits doesn't show symbol type for non lowercased symbols
  • Bug #17631: Cookie name "OpenGrok/project" is a reserved token

For bug details refer to Bugzilla.

The road ahead

A tentative roadmap for OpenGrok can be found here: http://www.opensolaris.org/os/project/opengrok/Roadmap/

Contributors

We'd like to thank all the people who report bugs, send us ideas for improvements or contribute code. Below are just the people who sent patches or pushed code, but that doesn't mean that we don't appreciate your work. Reports and feedback from the users of OpenGrok is essential to make the developers able to continue improving the product. Note that this is just a summary of how many changes each person made, which doesn't necessarily reflect how significant each change was. Inspect the history log with hg log -v -r 0.9:0.10 to get all details.

68 Knut Anders Hatlen <knut.hatlen@oracle.com>, <Knut.Hatlen@Sun.COM>
39 Lubos Kosco <Lubos.Kosco@Sun.COM>, <Lubos.Kosco@Oracle.COM>
10 Jorgen Austvik <jaustvik@acm.org>
9 Trond Norbye <trond.norbye@gmail.com>
5 Patrick Higgins <patrick.allen.higgins@gmail.com>
3 Yuriy Vasylchenko <yuriy.av@gmail.com>
2 Leo Davis <ldavis@fonix.com>
1 cicicici <cicicici@gmail.com>
1 Martin Walsh <martin.walsh@oracle.com>
1 seadog <seadog1234567890@gmail.com>

Friday Dec 17, 2010

What's new in Derby 10.7?

Apache Derby 10.7.1.1 was released earlier this week. You'll find the binaries and the release notes at the 10.7.1.1 download page. This is the first release from the 10.7 branch, and it contains some useful new features that were not part of the 10.6 releases. I will present some of these features here. There's also an interview with Rick Hillegas, the release manager for Derby 10.7.1, posted on the Java Spotlight Podcast, where the new features are discussed.

Boolean data type

Starting with 10.7, Derby supports the Boolean data type (yay, finally!). In earlier releases you would have to work around this limitation by using a CHAR(1) column or an SMALLINT column. Now you can declare the columns as BOOLEAN directly:

ij> create table t(b boolean);
0 rows inserted/updated/deleted
ij> insert into t values true, false, null;
3 rows inserted/updated/deleted
ij> select \* from t;
B    
-----
true 
false
NULL 

3 rows selected
ij> select \* from t where b;
B    
-----
true 

1 row selected

BOOLEAN may also be used for parameters and return values in stored procedures and functions, just like any other data type.

Plan exporter tool

Sometimes you want to know exactly how Derby executes a query. For example, you may have a big join that takes a very long time to complete, and you want to check if Derby uses the relevant indexes to get the best performance.

It has always been possible to get runtime statistics from Derby (see the Working with RunTimeStatistics chapter in Tuning Derby), but the statistics output has been hard to analyze (read: unstructured text format). The 10.6 release improved the situation somewhat by introducing XPLAIN style statistics, which puts the collected data into database tables that can be queried from SQL. Still, it can be difficult to dig up the information you want, especially for complex queries.

Derby 10.7 further improves this by adding a tool that processes the data in the XPLAIN tables for you, and produces an XML document representing the execution plan for a specific query. It can also do some simple rendering and output a graphical version of the plan in HTML format. This new tool is documented in the Derby Tools and Utilities Guide, in the PlanExporter chapter.

Let's take a look at an example. I wanted to inspect the query plan for a big and ugly join performed by OpenGrok to get the commit log for a version-controlled file. Enabling collection of statistics is easy. Just call one system procedure to enable it, and another one to say that you want XPLAIN style statistics and where to store them. I added this code in the method that executed the query I was interested in:

And for good measure I also disabled collection before returning from the method, so that I wouldn't get lots of statistics for queries I didn't care about:

Then I ran a JUnit test that I knew would exercise this code. After it had completed, I could connect to the database with the ij tool and verify that the statistics had been collected into the XPLAIN tables:

ij version 10.7
ij> connect 'jdbc:derby:/path/to/DB';
ij> select stmt_id from xplain.sysxplain_statements
>          where stmt_text like 'SELECT CS.REVISION%';
STMT_ID                             
------------------------------------
f93441e2-012c-e092-02e5-000003bb1070
8ef50298-012c-e092-02e5-000003bb1070
7bef4397-012c-e092-02e5-000003bb1070
0614c3de-012c-e092-02e5-000003bb1070
5623848d-012c-e092-02e5-000003bb1070

5 rows selected

The query I was interested in had been executed five times. I picked one of the STMT_ID values and passed it to the PlanExporter tool, asking it to produce both XML and HTML versions of the selected plan:

$ java org.apache.derby.tools.PlanExporter \\
    jdbc:derby:/path/to/DB XPLAIN \\
    f93441e2-012c-e092-02e5-000003bb1070 \\
    -xml plan.xml -html plan.html

The above command produced the following two files: plan.xml and plan.html. If you open the XML version, you'll see that most modern web browsers allow you to expand and collapse the nodes, which is neat when you analyze a large query and you want to focus on one specific part of it. For comparison, I'm also including plan.txt, which shows what the plan would look like in the old format (which is still supported, by the way).

Definer's rights for routines

A Derby stored function or stored procedure may execute SQL statements against the database using a nested connection. If you have SQL authorization enabled, the user that invokes the routine must have been granted privileges for all the SQL that is executed by the routine. So if the routine deletes rows from a table, you would have to grant the user delete privileges on that table, even if you don't really want that user to be allowed to delete rows except via the routine.

To address this concern, Derby 10.7 adds an optional clause (EXTERNAL SECURITY { DEFINER | INVOKER }) to the CREATE FUNCTION and CREATE PROCEDURE statements. This clause lets you specify whose privileges the routine should use when executing SQL statements on a nested connection, and could be used to delegate more fine-grained privileges to users.

For example, to allow a user to execute a procedure that performs some insert and select statements, you would previously have to give the user select and insert privileges on the tables involved, giving him carte blanche to insert whatever he wants into the tables. The code would look something like this, with multiple GRANT statements:

ij> create procedure add_item(id int, name varchar(10), category int)
>          external name 'MyProcedures.addItem'
>          language java parameter style java;
0 rows inserted/updated/deleted
ij> grant execute on procedure add_item to user1;
0 rows inserted/updated/deleted
ij> grant insert on items to user1;
0 rows inserted/updated/deleted
ij> grant select on categories to user1;
0 rows inserted/updated/deleted

If you use the new EXTERNAL SECURITY clause introduced in Derby 10.7 and specify that the procedure should execute with the definer's privileges, you only need to grant the user privileges to execute the procedure, and he will no longer have privileges to insert data directly into the table:

ij> create procedure add_item(id int, name varchar(10), category int)
>          external name 'MyProcedures.addItem'
>          language java parameter style java
>          external security definer;
0 rows inserted/updated/deleted
ij> grant execute on procedure add_item to user1;
0 rows inserted/updated/deleted

Less code, easier to maintain, and more secure.

TRUNCATE TABLE

New in 10.7 is also the TRUNCATE TABLE statement, which is handy if you want to delete all rows in a table. It is faster than DELETE FROM <TABLE>, especially on large tables, because it wipes the entire table and its indexes in one go instead of deleting one row at a time.

ij> select count(\*) from t;
1          
-----------
9          

1 row selected
ij> truncate table t;
0 rows inserted/updated/deleted
ij> select count(\*) from t;
1          
-----------
0          

1 row selected

Unicode database names

The embedded driver has always supported all characters in the Unicode character set in database names, as long as the underlying file system allows those characters in file names. But the network protocol has only worked with ASCII characters in the database names until now, so such databases couldn't be created or accessed from the client driver. In this release, the network protocol switched to UTF-8 encoding in the fields that hold the database name, so that you now, for example, can create and use a database with Japanese characters:

ij version 10.7
ij> connect 'jdbc:derby://localhost/データベース;create=true';
ij> 

This only affects the database names. The network protocol used UTF-8 encoding for SQL queries and results even before this improvement, so inserting non-ASCII text values into the database, or using non-ASCII characters in table and column names, is also supported, just like it was in all previous releases.

Wednesday Oct 13, 2010

Derby 10.6.2 is out

Apache Derby 10.6.2.1 has just been released. See the announcement from the release manager, Lily Wei, here. This is a maintenance release which contains mostly bug fixes, including the fix for one potential data corruption (DERBY-4677) and one regression in 10.6.1.0 (DERBY-4671). It also adds Chinese, French, German, Italian, Japanese, Korean and Spanish translations for the messages that were new in 10.6.1.0.

The downloads and the release notes can be found at http://db.apache.org/derby/releases/release-10.6.2.1.html.

Wednesday May 19, 2010

Derby 10.6.1 has been released

Apache Derby 10.6.1.0 has just been released. This is a new feature release, and the first release in the 10.6 series. See the full announcement here. The downloads and the release notes can be found at this URL: http://db.apache.org/derby/releases/release-10.6.1.0.cgi

Make sure that you read the release notes carefully before you upgrade. Pay special attention to the note about the security issue CVE-2009-4269, which details how to protect your database from a vulnerability in Derby's BUILTIN authentication and password substitution in previous releases. (The vulnerable code is still there to make it possible to access old databases, but the defaults have changed so that new databases will not have the vulnerability.)

The new features in this release include:

  • Sequence Generators - Named generators for allocating successive, evenly spaced numbers. See feature T176 of the SQL Standard.
  • User-defined types - Named types bound to serializable Java objects.
  • Restricted table functions - Limits on the columns and rows returned by table functions.
  • XPLAIN statistics collection - Query plan statistics stored in tables for later analysis.
  • GROUP BY ROLLUP - A subset of the SQL Standard ROLLUP functionality on the GROUP BY clause. See feature T431 of the SQL Standard.
  • CROSS JOIN - CROSS JOIN syntax. See feature F401-04 of the SQL Standard.
  • Named columns join - USING clauses in joins.
  • SHOW FUNCTIONS - IJ command that lists stored functions.
  • In-memory back end enhancements - Numerous improvements, including the ability to delete in-memory databases.
  • ORDER BY in subqueries - Syntax for explicitly ordering rows returned by subqueries. See features F851, F852, and F855 of the SQL Standard.
  • OFFSET, FETCH FIRST/NEXT in subqueries - Generalized syntax for retrieving row subsets. See features F856, F857, F858, F859, F861, F862, F863, and F864 of the SQL Standard.
  • NATURAL JOIN - Support for NATURAL JOIN. See feature T431 of the SQL Standard.
  • Qualified identifers in ij - Ability to reference cursors and prepared statements in other connections.
  • Configurable hash algorithm - Ability to customize the hash algorithm used by BUILTIN authentication.
  • Context-sniffing scripts - Ability of shipped scripts to locate Derby jars when DERBY_HOME isn't set.
  • Case-insensitive strings - Ability to ignore case in string comparisons and sorts.

See also my earlier preview entry, which describes some of these features in more detail. Most of them are also described in the Derby 10.6 manuals.


Tuesday May 11, 2010

OpenGrok 0.9

OpenGrok 0.9 has just been released. You can find the source and binary distributions here: http://hub.opensolaris.org/bin/view/Project+opengrok/files

The MD5 checksums are as follows:

4faccd0b27ce65603f123f9d948f9498  opengrok-0.9.tar.gz
9ea28e6182f11ade75510c053518e685  opengrok-0.9-src.tar.gz
c11189d426a05f107bf9f3ab1b72994a  OSOLopengrok-0.9.pkg

OpenGrok 0.9 Release Notes

OpenGrok is a wicked fast source code browser. http://hub.opensolaris.org/bin/view/Project+opengrok/

OpenGrok is a fast and usable source code search and cross reference engine. It helps you search, cross-reference and navigate your source tree. It can understand various program file formats (c, c++, c#, java, sh, ...) and version control histories like (Mercurial, Git, SCCS, RCS, CVS, Subversion, Teamware, ClearCase, Perforce, Monotone and Bazaar.) In other words it lets you grok (profoundly understand) the open source, hence the name OpenGrok. It is written in Java.

Since OpenGrok 0.8.1 was released December 8, 2009 many features and fixes have been incorporated into OpenGrok in almost 100 commits for the 0.9 release.

New Features

  • Faster Lucene engine (3.0.1) underneath - faster analyzers
  • New jQuery (1.4.2) underneath - UI improvements
  • Support for xrefs with Unicode characters (not indexed yet, but finally viewable)
  • New SMF service - just pass the source root, the rest is taken care of by the service (Solaris/OpenSolaris only)

Bug Fixes

  • Bug #437 (partial): OpenGrok should support source files with multibyte characters
  • Bug #3435: C functions are not considered as functions
  • Bug #3900: Unexpected result when clicking a symbol definition
  • Bug #5042: Author field empty in opengrok after a multiple changeset push
  • Bug #5848: trimmed diff link results in NULL pointer dereference
  • Bug #6641: .txx file extension should be included in the list for C++ analyzer
  • Bug #9407: set limit for directory traversal during search for repositories
  • Bug #13185: broken focus
  • Bug #13280: RFE: Configurable tab size for xref
  • Bug #13330: Wrong URL when search for camel-case identifier has only one match
  • Bug #13362: plainanalyzer xref writer can never end for a file with long lines
  • Bug #13364: regression : Jar analyzer doesn't analyze the jars it used to for 0.7
  • Bug #13524: Color coding in diffs is confusing
  • Bug #13529: listing of a file (in xref) might take a LONG time
  • Bug #13777: Indexer should print "Adding: ..." before it adds a file
  • Bug #13884: IndexOutOfBoundsException in List2TokenStream.next()
  • Bug #13949: Missing line numbers in xref for XML file
  • Bug #13951: Wrong revision in info.properties when not building from head of trunk
  • Bug #13952: size output not correct
  • Bug #13961: no quick search box shown on windows (path separator problem in project names/IDs)
  • Bug #13996: build.xml should provide a version info for generated src.tar.gz so we don't depend on hg
  • Bug #13997: main logo should take you HOME ;)
  • Bug #14030: Many lines with same line number (regression)
  • Bug #14724: Incremental update of Bazaar history broken by instability in revision numbers
  • Bug #14757: pressing Enter on project select list box takes you to xref instead of submit of the search form
  • Bug #14953: history index for CVS repos should only include default branch if CVS/Tag is set
  • Bug #14978: upgrade lucene to 3.0.1
  • Bug #15004: Improved handling of UTF Byte Order Marks
  • Bug #15097: make the search bar in listing stay on top of the page
  • Bug #15114: AnalyzerGuru:stripBOM should not use String to avoid encoding issues
  • Bug #15240: radio buttons in history could behave like e.g. on wikipedia
  • Bug #15391: Executor should log the commands executed
  • Bug #15394: improve IPS package and SMF for OpenSolaris
  • Bug #15608: Search field should not be active in file display
  • Bug #15609: self organizing buttons in history view are confusing
  • Bug #15632: Strange results when searching for dotted identifier
  • Bug #15636: wildcard search for definitions does not work
  • Bug #15637: query constructed only for AND operators
  • Bug #15643: Simplify CLASSPATH in ogindexd
  • Bug #15644: Matching lines not shown when using single-character wildcard (?)
  • Bug #15652: history page shows a stack trace if file doesn't have a history
  • Bug #15690: css browser caching breaks UI
  • Bug #15858: String index out of range: -8161 from Summarizer
  • Bug #15859: html analyzer doesn't escape forms

For bug details refer to this list.

Known Issues

The back button does not always work as expected in Firefox. This issue is tracked in Mozilla's bug tracker: https://bugzilla.mozilla.org/show_bug.cgi?id=565008

Contributors

We'd like to thank all the people who report bugs, and send us ideas and improvements. Below are just people who sent patches or pushed code. That doesn't mean that we don't appreciate your work. On the contrary, you are the people showing other people (who have the time to write code) what would be good to fix when you use this software and file bugs against it! Note that this is just a summary of how many changes each person made, which doesn't necessarily reflect how significant each change was. (Inspect the history log with hg log -v -r 0.8.1:0.9 to get all details.)

 45 Knut Anders Hatlen <Knut.Hatlen@sun.com>
 42 Lubos Kosco <Lubos.Kosco@sun.com>
  1 Chris Eldredge <chris.eldredge@gmail.com>
  1 Vitezslav Batrla <Vitezslav.Batrla@Sun.COM>

Saturday Feb 06, 2010

We're still here...

Some of you may be alarmed after last week's announcement and no mentioning of Java DB in the Oracle + Sun webcast. Don't be! The Java DB team is still here, working as hard as ever, in the Derby community and on Java DB. :)

See also Rick's message to derby-user.

Thursday Dec 10, 2009

An early look at Derby 10.6

The Derby community is currently working on the next feature release, Derby 10.6, and a number of new features have already been implemented. If you haven't already done so, now would be a good time to download the development sources and kick the tires on your favourite new feature! Early feedback to the community (either on the mailing lists or in the JIRA bug tracker) increases the likelihood of getting improvements and bug fixes implemented before the final release.

If you don't feel like building jar files from the development sources yourself, you can download the binaries used in the nightly regression testing from http://dbtg.foundry.sun.com/derby/bits/trunk/. Make sure that you understand all the warnings about the difference between a production build and a development build before you start using it!

The release page on the wiki lists some of the features that are planned for 10.6. Let's take a quick look at some of those that are mostly done and ready for testing.

Store Java objects in columns

DERBY-651 tracks the ongoing work that enables storing of Java objects directly in the columns of a table. The objects must belong to a class that implements the java.io.Serializable interface.

For example, if you want to store java.util.List objects in a table, you first need to declare a user-defined type that maps to the Java type, and create a column with that type:

stmt.execute("CREATE TYPE JAVA_UTIL_LIST " +
             "EXTERNAL NAME 'java.util.List' " +
             "LANGUAGE JAVA");
stmt.execute("CREATE TABLE T(C1 JAVA_UTIL_LIST)");

Next, you prepare a statement and use setObject() to insert the Java object that you want to store:

PreparedStatement ps = conn.prepareStatement(
        "INSERT INTO T(C1) VALUES (?)");

ArrayList lst = new ArrayList();
lst.add("First element");
lst.add("Second element");

ps.setObject(1, lst);
ps.execute();

Finally, you execute a SELECT query and use getObject() to restore the object from the database:

ResultSet rs = stmt.executeQuery("SELECT C1 FROM T");
while (rs.next()) {
    List list = (List) rs.getObject(1);
    System.out.println("Size of list: " + list.size());
}

You can also use your object types as arguments to user-defined functions or procedures:

stmt.execute("CREATE FUNCTION LIST_SIZE(LST JAVA_UTIL_LIST) " +
             "RETURNS INTEGER " +
             "EXTERNAL NAME 'MyListFunctions.listSize' " +
             "LANGUAGE JAVA PARAMETER STYLE JAVA");
ps = conn.prepareStatement("values list_size(?)");
ps.setObject(1, lst);

Multi-level grouping

Derby's GROUP BY syntax has been extended with the ROLLUP keyword, which allows for multi-level grouping. There's a fine write-up about the feature on this wiki page. The example below shows how you can ask for the total amount of sales of each product in each state, the total amount of sales in each state, the total amount of sales in each region, and the amount of sales in all regions, in one single query:

ij> SELECT REGION, STATE, PRODUCT, SUM(SALES) FROM SALES_HISTORY
        GROUP BY ROLLUP(REGION, STATE, PRODUCT)
        ORDER BY REGION, STATE, PRODUCT;
REGION    |STA&|PRODUCT   |4          
--------------------------------------
East      |MA  |Boats     |10     <-- all boat sales in Massachusetts
East      |MA  |Cars      |190        
East      |MA  |NULL      |200    <-- sum of all sales in Massachusetts
East      |NY  |Boats     |570        
East      |NY  |Cars      |10         
East      |NY  |NULL      |580        
East      |NULL|NULL      |780    <-- sum of all sales in region East
West      |AZ  |Boats     |40         
West      |AZ  |Cars      |300        
West      |AZ  |NULL      |340        
West      |CA  |Boats     |570        
West      |CA  |Cars      |750        
West      |CA  |NULL      |1320       
West      |NULL|NULL      |1660       
NULL      |NULL|NULL      |2440   <-- sum of all sales in all regions

15 rows selected

Richer JOIN syntax

Derby already supports INNER JOIN, LEFT OUTER JOIN and RIGHT OUTER JOIN. In 10.6, the syntax will be enhanced with support for

  • the CROSS JOIN operation
  • named column joins
  • sub-queries in ON clauses

CROSS JOIN is the simplest of the join operations, and SELECT \* FROM T1 CROSS JOIN T2 is just another way of writing SELECT \* FROM T1, T2. It may also be combined with other join operations:

ij> SELECT \* FROM T1 CROSS JOIN T2 LEFT OUTER JOIN T3 ON T1.A = T3.B;
A          |B          |A          |B          |A          |B          
-----------------------------------------------------------------------
5          |2          |2          |7          |NULL       |NULL       

1 row selected

Named columns join may save you some typing when the columns in the join key have the same name in the two tables being joined. In the example below, where two tables COUNTRIES and CITIES are joined, you no longer need to write the full join condition ON COUNTRIES.COUNTRY = CITIES.COUNTRY. Instead, you just say that the tables should be joined on the COUNTRY column with a USING clause:

ij> SELECT COUNTRY, COUNT(CITY_ID) FROM COUNTRIES JOIN CITIES
        USING (COUNTRY) GROUP BY COUNTRY;
COUNTRY                   |2          
--------------------------------------
Afghanistan               |1          
Argentina                 |1          
Australia                 |2          
.
.
.
United Kingdom            |1          
United States             |37         
Venezuela                 |1          

45 rows selected

Explain plan

The XPLAIN functionality makes it possible to access run-time statistics with SQL queries. See this section in the development version of Tuning Derby for details about how to enable it and use it. Here's one of the more advanced examples from the manual, which shows how to get all queries that performed a table scan on the COUNTRIES table, and the number of pages and rows visited in the scan:

ij> SELECT ST.STMT_TEXT,
           SP.NO_VISITED_PAGES AS PAGES,
           SP.NO_VISITED_ROWS AS "ROWS"
    FROM STATS.SYSXPLAIN_SCAN_PROPS SP, 
         STATS.SYSXPLAIN_RESULTSETS RS, 
         STATS.SYSXPLAIN_STATEMENTS ST 
    WHERE ST.STMT_ID = RS.STMT_ID AND 
          RS.SCAN_RS_ID = SP.SCAN_RS_ID AND 
          RS.OP_IDENTIFIER = 'TABLESCAN' AND 
          SP.SCAN_OBJECT_NAME = 'COUNTRIES';
STMT_TEXT                     |PAGES      |ROWS       
------------------------------------------------------
SELECT \* FROM COUNTRIES       |2          |114        

1 row selected

ORDER BY in sub-queries

Up till now, Derby has only accepted ORDER BY clauses in top-level SELECT queries. The ongoing work on DERBY-4397 will allow ORDER BY clauses in sub-queries as well. This is going to resolve the very old request for ordering of inserts (DERBY-4). There's also a request for OFFSET/FETCH in sub-queries (DERBY-4398), which would be very useful in combination with ORDER BY.

Dropping of in-memory databases

Derby 10.5 added support for in-memory databases, but there was no convenient API for deleting the in-memory databases and reclaiming the memory without taking down the Java process. In Derby 10.6, in-memory databases can be destroyed with an API similar to the one used for creating and shutting down databases. Simply add drop=true to the JDBC URL.

ij> connect 'jdbc:derby:memory:mydb;create=true';
ij> create table t (x int);
0 rows inserted/updated/deleted
ij> connect 'jdbc:derby:memory:mydb;drop=true';
ERROR 08006: Database 'memory:mydb' dropped.

Very nice if you're running JUnit tests and want to clear the database and free up the memory between each test case.

SHOW FUNCTIONS

Derby's interactive tool for running scripts or queries against a database, ij, lacked a command to show the functions that are stored in the database. Derby 10.6 adds SHOW FUNCTIONS to ij's list of SHOW commands, so now you can issue the following command to view all the functions in the APP schema:

ij> SHOW FUNCTIONS IN APP;
FUNCTION_SCHEM|FUNCTION_NAME               |REMARKS                            
----------------------------------------------------------
APP           |MY_FUNC                     |MyFunctions.f                      

1 row selected

Restricted table functions

In Derby 10.6 it will be possible to push predicates into table functions and process the restrictions before the function returns the results to the SQL engine. This can speed up execution of some table function queries significantly. Take this example:

SELECT \* FROM TABLE ( MY_FUNCTION() ) AS T WHERE X = 5

If MY_FUNCTION() returns thousands of rows, and only a handful of them match the restriction X = 5, a lot of work is wasted generating and scanning rows that are just thrown away. With restricted table functions, Derby's SQL engine will pass information about the restriction down to the table function, and the table function may use this information to produce and return only the rows that are actually needed.

Details about how to use restricted table functions can be found in the functional spec attached to DERBY-4357.

ROW_NUMBER improvements

Previous releases of Derby only allowed the ROW_NUMBER function to appear in the select list of a query, and it could not be used to build up more complex expressions. Now these limitations have been removed, and you can write queries like this one without getting syntax errors:

SELECT X / ROW_NUMBER() OVER ()
  FROM T
  ORDER BY ROW_NUMBER() OVER () DESC

Friday Nov 20, 2009

OpenGrok 0.8 is out

After a long wait, OpenGrok 0.8 has finally been released! To find out what's new, take a look at the release notes posted on Trond's blog. Or just go straight to the OpenGrok website to download it.

From the release notes:

OpenGrok is a wicked fast source code browser. http://www.opensolaris.org/os/project/opengrok/

OpenGrok is a fast and usable source code search and cross reference engine. It helps you search, cross-reference and navigate your source tree. It can understand various program file formats (c, c++, c#, java, sh, ...) and version control histories like (Mercurial, Git, SCCS, RCS, CVS, Subversion, Teamware, ClearCase, Perforce, Monotone and Bazaar.) In other words it lets you grok (profoundly understand) the open source, hence the name OpenGrok. It is written in Java.

Since OpenGrok 0.7 was released October 17, 2008 a lot of features and fixes has been incorporated into OpenGrok in almost 300 commits for the 0.8 release.

About

kah

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today