Apache Derby 10.6.1.0 Release

Use the links below to download a distribution of Apache Derby. You should always verify the integrity of distribution files downloaded from a mirror.

There are four different distributions:

• bin distribution - contains the documentation, javadoc, and jar files for Derby.
• lib distribution - contains only the jar files for Derby.
• lib-debug distribution - contains jar files for Derby with source line numbers.
• src distribution - contains the Derby source tree at the point which the binaries were built.

db-derby-10.6.1.0-bin.zip [PGP] [MD5]
db-derby-10.6.1.0-bin.tar.gz [PGP] [MD5]

db-derby-10.6.1.0-lib.zip [PGP] [MD5]
db-derby-10.6.1.0-lib.tar.gz [PGP] [MD5]

db-derby-10.6.1.0-lib-debug.zip [PGP] [MD5]
db-derby-10.6.1.0-lib-debug.tar.gz [PGP] [MD5]

db-derby-10.6.1.0-src.zip [PGP] [MD5]
db-derby-10.6.1.0-src.tar.gz [PGP] [MD5] (Note that, due to long filenames, you will need gnu tar to unravel this tarball.)

There are two separate Eclipse plugins for Derby:

• derby_core_plugin - provides the Derby jar files to other plugins in Eclipse.
• derby_ui_plugin - provides an Apache Derby Nature in Eclipse for easy database application development.

derby_core_plugin_10.6.1.zip [PGP] [MD5]
derby_ui_doc_plugin_1.1.2.zip [PGP] [MD5]

Please note: both plugins must be installed for full functionality. For information on installing and using the Derby plugins for Eclipse, please see the Using the 10 Core and 1.1 UI Derby plug-ins page.

These notes describe the difference between Derby release 10.6.1.0 and the preceding release 10.5.3.0.

In addition to many new features, 10.6.1.0 includes a fix for Security Bug CVE-2009-4269, affecting the BUILTIN authentication scheme and password hashing. For more information, see below.

Overview

Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs.

Derby functionality includes:

• Embedded engine with JDBC drivers
• Network Server
• Network client JDBC drivers
• Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)

New Features

This is a feature release. The following new features were added:

• Sequence Generators - Named generators for allocating successive, evenly spaced numbers. See feature T176 of the SQL Standard.
• User-defined types - Named types bound to serializable Java objects.
• Restricted table functions - Limits on the columns and rows returned by table functions.
• XPLAIN statistics collection - Query plan statistics stored in tables for later analysis.
• GROUP BY ROLLUP - A subset of the SQL Standard ROLLUP functionality on the GROUP BY clause. See feature T431 of the SQL Standard.
• CROSS JOIN - CROSS JOIN syntax. See feature F401-04 of the SQL Standard.
• Named columns join - USING clauses in joins.
• SHOW FUNCTIONS - IJ command that lists stored functions.
• In-memory back end enhancements - Numerous improvements, including the ability to delete in-memory databases.
• ORDER BY in subqueries - Syntax for explicitly ordering rows returned by subqueries. See features F851, F852, and F855 of the SQL Standard.
• OFFSET, FETCH FIRST/NEXT in subqueries - Generalized syntax for retrieving row subsets. See features F856, F857, F858, F859, F861, F862, F863, and F864 of the SQL Standard.
• NATURAL JOIN - Support for NATURAL JOIN. See feature T431 of the SQL Standard.
• Qualified identifers in ij - Ability to reference cursors and prepared statements in other connections.
• Configurable hash algorithm - Ability to customize the hash algorithm used by BUILTIN authentication.
• Context-sniffing scripts - Ability of shipped scripts to locate Derby jars when DERBY_HOME isn't set.
• Case-insensitive strings - Ability to ignore case in string comparisons and sorts.

Bug Fixes

The following issues are addressed by Derby release 10.6.1.0. These issues are not addressed in the preceding 10.5.3.0 release.

Fix for Security Bug CVE-2009-4269

Derby 10.6.1.0 also fixes a security flaw tracked by the Apache Common Vulnerabilities and Exposures id "CVE-2009-4269". This flaw made it easy to crack passwords managed by Derby's BUILTIN authentication logic. Originally, the BUILTIN logic was intended only for testing purposes. However, Derby's user documentation suggested that this scheme was production-ready and it appears that many users rely on BUILTIN authentication in production. Tracked by DERBY-4483, the flaw is addressed as follows:

1. The bug itself is corrected for newly created 10.6 databases.
2. Password substitution is not allowed when logging into a database where the bug is corrected and BUILTIN passwords are stored in the database. See the release note for DERBY-4483 below.
3. Derby's default password-hashing scheme is changed from SHA-1 to SHA-256, which is harder to crack.
4. The user guides are glossed with warnings against production use of the BUILTIN authentication mechanism.

Users are urged to

1. Migrate production systems off the BUILTIN mechanism onto Derby's LDAP and user-customized authentication schemes.
2. Or hard-upgrade to 10.6.1.0 immediately and perform the following additional steps:
   • Set derby.authentication.builtin.algorithm to a stronger authentication scheme like SHA-256 or SHA-512.
   • Reset all passwords stored in the database.
   • Stop using strong password substitution. Instead, encrypt all network traffic using SSL/TLS.

Issues

Compared with the previous release (10.5.3.0), Derby release 10.6.1.0 introduces the following new features and incompatibilities. These merit your special attention.

• Note for DERBY-4602: Default hash algorithm for BUILTIN authentication changed to SHA-256
• Note for DERBY-4483: Strong password substitution cannot be used with new defaults for BUILTIN authentication.
• Note for DERBY-4432: The in-memory back end will no longer create a database if the virtual database directory already exists.
• Note for DERBY-4380: Changed error code and message when referencing column not in scope in ON clause.
• Note for DERBY-4355: CROSS is now a reserved keyword and cannot be used as an unquoted identifier.
• Note for DERBY-4191: Some queries require additional SELECT privileges now.
• Note for DERBY-3844: Applications may no longer obtain a LOB object, or a Reader or an InputStream, from the same column more than once per row.
• Note for DERBY-2769: Comprehensive validity checks for the parameters of Clob.setString have been introduced.

Note for DERBY-4602

Summary of Change

Default hash algorithm for BUILTIN authentication changed to SHA-256

Symptoms Seen by Applications Affected by Change

If a database that uses BUILTIN authentication is opened on a platform that does not support the new default hash algorithm (SHA-256), the following exception may be seen when connecting to the database or when setting the password for a user:

ERROR XBCXW: The message digest algorithm 'SHA-256' is not supported
by any of the available cryptography providers. Please install a
cryptography provider that supports that algorithm, or specify another
algorithm in the derby.authentication.builtin.algorithm property.

The default algorithm is initialized to SHA-256 when the database is created. However, if SHA-256 is not available, it is initialized to the old default (SHA-1) instead. The error message above should therefore only be seen if the database was created on a platform that supports SHA-256 and opened on a platform that doesn't support SHA-256.

Incompatibilities with Previous Release

Databases created on a platform with support for the new default algorithm (SHA-256) may now require some changes before they can be used together with BUILTIN authentication on platforms that don't support the new algorithm. In previous releases, differences in the set of supported hash algorithms did not cause a need for changes when moving databases across platforms.

Rationale for Change

The default algorithm in previous releases (SHA-1) is not considered secure enough for most uses by U.S. government agencies. SHA-256 is widely recognized as more secure than SHA-1 and is therefore used as the default if the platform on which the database is created supports the algorithm.

Application Changes Required

If a database cannot be used on a platform because of this issue, one of the following steps must be taken:

Alternative 1

Recreate the database on the platform that doesn't support SHA-256. The new database will use the more widely available SHA-1 algorithm as default.

Alternative 2

Install a Java Cryptography Extension (JCE) Provider that supports the SHA-256 algorithm.

Alternative 3

On the platform on which the database was created, change the default algorithm to SHA-1 (or to some other algorithm known to be supported on the target platform) by executing the following SQL statement:

CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY(
        'derby.authentication.builtin.algorithm', 'SHA-1')

If there are any users defined at the database level before the algorithm is changed to SHA-1, their passwords will have to be set again to ensure that they are rehashed with SHA-1. For example, if there's a user called 'alice' with the password 'secret', the password must be updated with this SQL statement before it can be used on the platform with no support for SHA-256:

CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY(
        'derby.user.alice, 'secret)

Note for DERBY-4483

Summary of Change

Strong password substitution cannot be used with new defaults for BUILTIN authentication.

Symptoms Seen by Applications Affected by Change

In a database created with the new version of Derby, the BUILTIN authentication provider will by default store passwords in a way that's not compatible with the strong password substitution security mechanism. Applications that attempt to connect to the database using the Derby network client driver with securityMechanism=8 in the connection URL, will therefore fail to connect. The connection attempt will be refused with the following error message:

ERROR 08004: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08004, SQLERRMC:
Connection authentication failure occurred. Either the supplied
credentials were invalid, or the database uses a password encryption
scheme which is not compatible with the strong password substitution
security mechanism. If this error started after upgrade, refer to the
release note for DERBY-4483 for options.

Incompatibilities with Previous Release

Applications that use BUILTIN authentication and the strong password substitution security mechanism will not be able to establish connections to the database if the database uses the new defaults for BUILTIN authentication.

Only databases created with the new version of Derby will automatically use the new defaults. Databases upgraded from previous versions of Derby will continue to use the old defaults, and they will not be affected unless the settings for BUILTIN authentication are changed manually to enable the new behaviour.

Rationale for Change

The default BUILTIN authentication scheme used in previous releases has a weakness that makes it vulnerable to attacks. In the new release, an alternative BUILTIN authentication scheme without this vulnerability has been added. Despite this new scheme's incompatibility with strong password substitution, it was made the default for databases created with the new release of Derby in order to improve out-of-the-box security.

Application Changes Required

Applications that are affected by this incompatibility can be made to work by making one of the following changes:

Alternative 1: Use another security mechanism

You can switch to another security mechanism by changing the value of the securityMechanism connection attribute. Only the strong password substitution security mechanism is incompatible with the new BUILTIN authentication. Note that if you pick one of the security mechanisms that send your credentials unencrypted over the network, you may want to enable network encryption and authentication with SSL/TLS. Details about how to change security mechanisms and how to enable SSL/TLS can be found in the Derby Server and Administration Guide.

Alternative 2: Revert to the old BUILTIN authentication behaviour

It is possible to revert to the old behaviour for BUILTIN authentication, which will make it possible to connect when using the strong password substitution security mechanism. To revert to the old behaviour, set the database property derby.authentication.builtin.algorithm to NULL (or to an empty string) by executing this SQL statement:

CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY(
        'derby.authentication.builtin.algorithm', NULL)

If you have created any users before setting the above property to NULL, you will also need to set the passwords for all those users again to ensure that they are stored using the old format, since setting this property does not change how any existing passwords are stored. Users whose passwords are stored using the old format will be able to connect to the database with strong password substitution.

Note for DERBY-4432

Summary of Change

The in-memory back end will no longer create a database if the virtual database directory already exists.

Symptoms Seen by Applications Affected by Change

If database 'memory:/tmp/a/myDB' already exists, an attempt to create 'memory:/tmp/a' results in:

ERROR XJ041: Failed to create database 'memory:/tmp/a', see the next exception for details.
ERROR XBM0J: Directory memory:/tmp/a already exists.

Incompatibilities with Previous Release

If two (or more) in-memory databases are created, the application may fail to create the second database if the database paths are overlapping.

Rationale for Change

The fix makes the in-memory and the on-disk back ends consistent on this matter, and the change also fixes a memory leak when trying to boot a large number of non-existing in-memory databases (see DERBY-4432).

Application Changes Required

Adjust the paths of the in-memory databases if required. In some cases it may be sufficient to reorder the database creations, but this is not recommended as it would potentially have severe side-effects with the on-disk back end (a database nested within another database).

There is no way to revert to the old behavior.

Note for DERBY-4380

Summary of Change

Changed error code and message when referencing column not in scope in ON clause.

Symptoms Seen by Applications Affected by Change

In the previous releases, SQL statements that referenced columns that were not in scope in the ON clause of a JOIN, would under certain conditions fail with the following SQLState and message:

ERROR 42972: An ON clause associated with a JOIN operator is not valid.

Now, the statements that used to fail with that message will instead fail with the following SQLState and message:

ERROR 42X04: Column 'T1.X' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE  statement then 'T1.X' is not a column in the target table.

Rationale for Change

This change was necessary because the code that decided which of the two errors to raise, contained some logic that was not compatible with the introduction of sub-queries in ON clauses. Additionally, the new message makes it easier to see what is the problem with the statement, as it mentions both why the ON clause is invalid and the name of the column that is out of scope.

Application Changes Required

Applications that check for SQLState 42972 when SQLExceptions are raised, should now check for SQLState 42X04 instead.

Note for DERBY-4355

Summary of Change

CROSS is now a reserved keyword and cannot be used as an unquoted identifier.

Symptoms Seen by Applications Affected by Change

In previous releases of Derby, CROSS was not a reserved keyword, so applications could use CROSS as an identifier without quoting it. Applications that use CROSS as an identifier (for instance to name tables, views, columns, functions or procedures) and don't quote it, will now see errors similar to this one:

ERROR 42X01: Syntax error: Encountered "cross" at line 1, column 15.

Incompatibilities with Previous Release

Applications that use CROSS as an unquoted identifier will experience syntax errors.

Rationale for Change

CROSS was made a reserved keyword in order to support the CROSS JOIN operator. Also, the SQL:2003 standard defines CROSS as a reserved keyword, so applications that use it as an unquoted identifier are not portable.

Application Changes Required

SQL statements where CROSS is used as an unquoted identifier must be rewritten so that CROSS is enclosed in double quotes.

Examples:

The statement create table cross(x int) must be rewritten to create table "CROSS"(x int). Similarly, the statement select * from cross will have to be rewritten to select * from "CROSS".

Note for DERBY-4191

Summary of Change

Some queries require additional SELECT privileges now.

Symptoms Seen by Applications Affected by Change

In previous releases, certain queries incorrectly succeeded when the user lacked sufficient SELECT privileges. For instance, the following query succeeded even if the user did not have SELECT privilege on the table:

    select 1 from anotheruser.table1

And the following query succeeded if the user had UPDATE but not SELECT privilege on the column:

    update anotheruser.table1 set a = ( select max(a) + 2 from anotheruser.table1 ); 

Incompatibilities with Previous Release

Now Derby raises a SQLException for those situations. For the first query above, the user must now enjoy SELECT privilege on at least one column in the table. For the second query, the user must now enjoy SELECT as well as UPDATE privilege on the affected column.

Rationale for Change

The old behavior violated the SQL Standard. The new behavior is correct.

Application Changes Required

Database Administrators may need to grant users additional SELECT privileges.

Note for DERBY-3844

Summary of Change

Applications may no longer obtain a LOB object, or a Reader or an InputStream, from the same column more than once per row.

Symptoms Seen by Applications Affected by Change

Derby will throw an exception with SQLState XCL18 (carrying the message "Stream or LOB value cannot be retrieved more than once") when any of the following result set methods are invoked for the second time on a given column on a row:

• getBlob (new behavior)
• getClob (new behavior)
• getObject (new behavior)
• getAsciiStream (current behavior)
• getCharacterStream (current behavior)
• getBinaryStream (current behavior)

Incompatibilities with Previous Release

Applications which obtain two LOB objects from the same result set column on a row now fail.

Rationale for Change

Obtaining several LOB objects from the same column causes resource management problems and intermittent errors (see DERBY-3844 for a description). The change is in line with the maximum portability statement found in the JavaDoc for java.sql.ResultSet.

Application Changes Required

Users must recode applications which obtain multiple LOB objects (java.sql.Blob or java.sql.Clob) on the same column. Note that for instance getCharacterStream(1) followed by getClob(1) will also raise the exception.

Note for DERBY-2769

Summary of Change

Comprehensive validity checks for the parameters of Clob.setString have been introduced.

Symptoms Seen by Applications Affected by Change

There are three possible symptoms, all observable when invoking Clob.setString:

1. An exception may be raised at an earlier point in time than before.
2. For failing invocations, the SQL state of the raised exception may differ.
3. Invocations of Clob.setString with certain invalid parameters will now fail, whereas they succeeded before.

Incompatibilities with Previous Release

1. offset less than zero or more than or equal to the length of str was accepted, but is now rejected.
2. str with a value of null was accepted, but is now rejected.
3. len less than zero was accepted, but is now rejected.
4. offset + len more than the length of str was accepted, but is now rejected.

Rationale for Change

Make the parameter checking comply with the JDBC specification.

Application Changes Required

The following rules must be followed to avoid exceptions being raised when invoking Clob.setString:

1. offset must be smaller than the length of str, and cannot be negative.
2. str cannot be null.
3. len must be zero or greater.
4. offset + len must be smaller than the length of str.

Build Environment

Derby release 10.6.1.0 was built using the following environment:

• Branch - Source code came from the 10.6 branch.
• Machine - Mac OS X 10.5.8.
• Ant - Apache Ant version 1.7.1 compiled on June 27 2008.
• JDK 1.4 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_22-b02-329).
• Java 6 - Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125).
• Compiler - The 1.6.0_17-b04-248-9M3125 javac was used to compile all classes.
• JSR 169 - J2ME support was built using libraries from phoneME Advanced Milestone Release 2.

Verifying releases

It is essential that you verify the integrity of the downloaded files using the PGP and MD5 signatures. MD5 verification ensures the file was not corrupted during the download process. PGP verification ensures that the file came from a certain person.

The PGP signatures can be verified using PGP or GPG. First download the Apache Derby KEYS as well as the asc signature file for the particular distribution. It is important that you get these files from the ultimate trusted source - the main ASF distribution site, rather than from a mirror. Then verify the signatures using ...

% pgpk -a KEYS
% pgpv db-derby-X.Y.tar.gz.asc

or

% pgp -ka KEYS
% pgp db-derby-X.Y.tar.gz.asc

or

% gpg --import KEYS
% gpg --verify db-derby-X.Y.tar.gz.asc

To verify the MD5 signature on the files, you need to use a program called md5 or md5sum, which is included in many unix distributions. It is also available as part of GNU Textutils. Windows users can get binary md5 programs from here, here, or here.

We strongly recommend you verify your downloads with both PGP and MD5.