apache > db
Apache DB Project
 
Font size:      

Apache Derby 10.3.3.0 Release

IMPORTANT NOTICE

If you are currently using Derby 10.3.1.4 or Derby 10.3.2.1, it is strongly
recommended that you upgrade to Derby 10.4.1.3 or 10.3.3.0 to avoid
any chance of database corruption due to an issue with multiple threads
accessing a database that is documented in DERBY-3347.

This bug can cause unrecoverable database corruption during periods of
heavy, multi-thread I/O operations. The error produced in the test case
used to diagnose the problem was:

ERROR XSDB3: Container information cannot change once written: was 0, now 80.

It is felt that other errors might also be generated when this type of
corruption occurs. The corruption message will most likely refer to page 0
of the container. For example:

ERROR XSDG1: Page Page(0 ,Container(0, 5856)) could not be
written...

This bug corrupts the pages on disk and can go unnoticed. If you do not
run database consistency checks regularly it is recommended you begin doing
so as soon as possible after the upgrade. To insure that corruption has not
already occurred in existing databases, after upgrade run the database
consistency check at least once to validate all tables in the database. This
process is documented at:

http://wiki.apache.org/db-derby/DatabaseConsistencyCheck

If the corruption has already occurred there is no guaranteed recovery of data
other than to recover from the last good backup. When doing so one should
also check that the previous backup did not also have the corruption.

In some cases one may recover data from the existing
database, depending on the extent of the corruption, but will require
by hand data recovery. Depending on the type of corruption this may
be successful or not. one should consult the Derby list if attempting
this recovery - no automatic software solution to this recovery exists.

Version 10.3.3.0 can be downloaded from:
http://db.apache.org/derby/releases/release-10.3.3.0.cgi

Version 10.4.1.3 can be downloaded from:
http://db.apache.org/derby/releases/release-10.4.1.3.cgi

For help or questions, please post to the Derby User list.
For instructions on how to subcribe and post to the Derby User list,
please see:

http://db.apache.org/derby/derby_mail.html

Distributions

Use the links below to download a distribution of Apache Derby from one of our mirrors. You should always verify the integrity of distribution files downloaded from a mirror.

There are four different distributions:

  • bin distribution - contains the documentation, javadoc, and jar files for Derby.
  • lib distribution - contains only the jar files for Derby.
  • lib-debug distribution - contains jar files for Derby with source line numbers.
  • src distribution - contains the Derby source tree at the point which the binaries were built.

db-derby-10.3.3.0-bin.zip [PGP] [MD5]
db-derby-10.3.3.0-bin.tar.gz [PGP] [MD5]

db-derby-10.3.3.0-lib.zip [PGP] [MD5]
db-derby-10.3.3.0-lib.tar.gz [PGP] [MD5]

db-derby-10.3.3.0-lib-debug.zip [PGP] [MD5]
db-derby-10.3.3.0-lib-debug.tar.gz [PGP] [MD5]

db-derby-10.3.3.0-src.zip [PGP] [MD5]
db-derby-10.3.3.0-src.tar.gz [PGP] [MD5] (Note that, due to long filenames, you will need gnu tar to unravel this tarball.)

There are two separate Eclipse plugins for Derby:

  • derby_core_plugin - provides the Derby jar files to other plugins in Eclipse.
  • derby_ui_plugin - provides an Apache Derby Nature in Eclipse for easy database application development.

derby_core_plugin_10.3.3.652961.zip [PGP] [MD5]
derby_ui_plugin_1.1.1.zip [PGP] [MD5]

Please note: both plugins must be installed for full functionality. For information on installing and using the Derby plugins for Eclipse, please see the Using the 10 Core and 1.1 UI Derby plug-ins page.

Release Notes for Derby 10.3.3.0

These notes describe the difference between Derby release 10.3.3.0 and the preceding release 10.3.2.1.

Overview

Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs.

Derby functionality includes:

  • Embedded engine with JDBC drivers
  • Network Server
  • Network client JDBC drivers
  • Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)

Issues

The following issues are addressed by Derby release 10.3.3.0. These issues are not addressed in the preceding 10.3.2.1 release.

Issue IdDescription
DERBY-3658LOBStateTracker should not use SYSIBM.CLOBRELEASELOCATOR when the database is soft-upgraded from 10.2
DERBY-3649can't call a stored function with an aggregate argument without getting the following error: ERROR 42Y29
DERBY-3611ERROR XSDG2: Invalid checksum on Page occurs during mass inserts into two-column bigint PK table
DERBY-3603'IN' clause ignores valid results, incorrect qualifier handling suspected
DERBY-3576Merge EngineBlob and EngineClob into a single interface
DERBY-3571LOB locators are not released if the LOB columns are not accessed by the client
DERBY-3560build failure: Error running ${jdk16}/bin/javac compiler if jdk16 is not set on 10.3
DERBY-3538NullPointerException during execution for query with LEFT OUTER JOIN whose inner table selects all constants.
DERBY-3525Remove unneeded code to get JDBC level in BrokeredConnection and BrokeredStatement classes
DERBY-3496CallableStatement with output parameter leaves cursor open after execution
DERBY-3458dblook fails on TERRITORY_BASED databases
DERBY-3426Remove unused code for autogenerated keys columnNames
DERBY-3422Embedded returns wrong value for DatabaseMetaData.autoCommitFailureClosesAllResultSets()
DERBY-3421Remove unused code for caching of connect bytes
DERBY-3397Derby 10.3.1.4 and 10.3.2.1 break scrollable result sets? Hibernate Query.setFirstResult and/or setMaxResults
DERBY-3379"No Current connection" on PooledConnection.getConnection() if pooled connection is reused during connectionClosed processing
DERBY-3373SQL "distinct" and "order by" needed together
DERBY-3365Network Server stores a duplicate entry in the lob hash map for every lob
DERBY-3354Select from large lob table with embedded gives OutOfMemoryError
DERBY-3347ERROR XSDB3: Container information cannot change once written
DERBY-3343Subsequent calls to PreparedStatement cause SQLIntegrityConstraintViolationException on column that is "Generated always"
DERBY-3321NullPointerException for 'NOT EXISTS' with nested subquery
DERBY-3316Leak in client if ResultSet not closed
DERBY-3315Should UCS_BASIC character types have to look at collation elements when dealing with escape character in the LIKE clause?
DERBY-3309Minor cleanups in ClientPooledConnection40 and ClientPooledConnection
DERBY-3308Broken synchronization for event handling in ClientPooledConnection40
DERBY-3304Explicit commit inside a java procedure makes a dynamic result sets passed out unavailable
DERBY-3303ArrayIndexOutOfBoundsException at MergeSort.compare
DERBY-3302NullPointerException during recovery of database with territory-based collation
DERBY-3301Incorrect result from query with nested EXIST
DERBY-3288wrong query result in presence of a unique index
DERBY-3279Derby 10.3.X ignores ORDER BY DESC when target column has an index and is used in an OR clause or an IN list.
DERBY-3260NullPointerException caused by race condition in GenericActivationHolder
DERBY-3257SELECT with HAVING clause containing OR conditional incorrectly return 1 row - should return 2 rows - works correctly with 10.2 DB
DERBY-3253NullPointer Exception (NPE) from query with IN predicate containing two values and joining a view with a large table. ERROR 38000: The exception 'java.lang.NullPointerException' was thrown while evaluating an expression.
DERBY-3247Activation for a dynamic ResultSet created from an Prepared/CallableStatement will not be closed until garbage collection indicates it is unused to the LCC and the LCC closes it
DERBY-3244NullPointerException in ....B2IRowLocking3.searchLeftAndLockPreviousKey
DERBY-3243(jdbc net client) exception during normal iteration through "ResultSet" of "select * from t"
DERBY-3238When table contains large LOB values (> ~32K) trigger execution fails for that row with ERROR XCL30: An IOException was thrown when reading a 'BLOB'
DERBY-3231Sorting on COUNT with OR and GROUP BY delivers wrong results.
DERBY-3230Selecting data from a Table raises Error XN008: Query processing has been terminated due to an error on the server
DERBY-3229testSysinfoLocale fails if derbyTools.jar is first in the classpath
DERBY-3221"java.sql.SQLException: The conglomerate (-5) requested does not exist." from Derby 10.3.1.4 embedded within Eclipse 3.3 and RAD 7.0
DERBY-3214Optimizer can see negative cost estimates when pulling Optimizables from the join order.
DERBY-3194LOCALIZEDDISPLAY of CURRENT_TIMESTAMP returns only the TIME
DERBY-3094Grouping of expressions causes NullPointerException
DERBY-3044Typos in documentation
DERBY-3037Language ResultSet.finish() is called even when the ResultSet is going to be re-used.
DERBY-3023Different result rows depending on the sequence of INNER JOIN and OUTER JOIN
DERBY-2935DDMReader.readLengthAndCodePoint() decodes long integer incorrectly
DERBY-2892Closing a resultset after retrieving a large > 32665 bytes value with Network Server does not release locks
DERBY-2720remove dead code associated with unsupported National Char implementation
DERBY-2653Expose existing auto-generated key functionality through more JDBC APIs in Derby Client.
DERBY-2559recreating a datasource using javax.naming.Reference from a ClientDataSource40 fails
DERBY-2351ORDER BY with expression with distinct in the select list returns incorrect result
DERBY-2182Documentation for derby.system.bootAll is missing
DERBY-2142NullPointerException while using XAConnection/PooledConnection in a heavily contended multithreaded scenario
DERBY-1585derbylang/procedureInTrigger: not able to create trigger due to an open ResultSet

Compared with the previous release (10.3.2.1), Derby release 10.3.3.0 introduces the following new important fixes. These merit your special attention.

  • Note for DERBY-3347: A bug that could cause unrecoverable database corruption has been fixed.

  • Note for DERBY-3301: Queries with nested EXIST, ANY or IN clauses now return correct results.

  • Note for DERBY-2351: An ORDER BY clause of a DISTINCT query which specifies to order by a column which was not in the DISTINCT list is now rejected, because the intent of the query is ambiguous. Previously, Derby instead produced non-distinct results. Also, an ORDER BY clause which specifies a table-name-qualified column alias is now rejected as invalid, where previously it was accepted.


Note for DERBY-3347

Summary of Change

A bug that could cause unrecoverable database corruption has been fixed.

Symptoms Seen by Applications Affected by Change

A bug that could cause database corruption was introduced in the 10.3 codeline and affects the following releases:

  • Apache Derby 10.3.1.4
  • Apache Derby 10.3.2.1

Users who are hit by this bug may experience exceptions at various times during execution of SQL statements, booting or shutdown of a database, or during checkpointing. It may result in a number of different error messages, including any of these:

ERROR XSDB3: Container information cannot change once written: was 0, now 80
ERROR XSDG1: Page Page(1039,Container(0, 5856)) could not be written to disk, please check if disk is full.
ERROR XSDG2: Invalid checksum on Page Page(0,Container(0, 1313))
ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer4@1afb0c7 could not be accessed
ERROR XSLA1: Log Record has been sent to the stream, but it cannot be applied to the store (Object null). This may cause recovery problems also.
Incompatibilities with Previous Release

None.

Rationale for Change

Database corruption is bad.

Application Changes Required

No changes are required. However, since the database corruption may go unnoticed for a while, users may want to check the consistency of their databases after upgrading Derby. The process is described on the following wiki page: http://wiki.apache.org/db-derby/DatabaseConsistencyCheck . If a corruption is detected, restoring the database from backup is the only reliable way to recover.


Note for DERBY-3301

Summary of Change

Queries with nested EXIST, ANY or IN clauses now return correct results.

Symptoms Seen by Applications Affected by Change

In the previous release, applications that executed SQL statements containing nested EXISTS, ANY or IN clauses could see fewer rows than those satisfying the query. In particular, rows that had the same value for one of the selected columns as another row might not have been returned.

Incompatibilities with Previous Release

None.

Rationale for Change

The previous behavior violated the ANSI SQL standard. The new behavior is correct.

Application Changes Required

Typically none, but applications must handle that the correct results are now returned.


Note for DERBY-2351

Summary of Change

An ORDER BY clause of a DISTINCT query which specifies to order by a column which was not in the DISTINCT list is now rejected, because the intent of the query is ambiguous. Previously, Derby instead produced non-distinct results. Also, an ORDER BY clause which specifies a table-name-qualified column alias is now rejected as invalid, where previously it was accepted.

Symptoms Seen by Applications Affected by Change
New rules for DISTINCT and ORDER BY

Applications which specify certain combinations of SELECT DISTINCT with ORDER BY will now receive an error message, whereas formerly such applications received non-distinct results.

As an example, take the following:

create table person (name varchar(10), age int);
insert into person values ('John', 10);
insert into person values ('John', 30);
insert into person values ('Mary', 20);

SELECT DISTINCT name FROM person ORDER BY age;

The query above is now rejected, with the error message:

If the AGE column is included in the DISTINCT list in the above query, there is no ambiguity

New column alias rules

Applications which specify a column alias for a column in the SELECT statement, and which specify an ORDER BY clause which specifies that column alias qualified by the table name, will now receive an error indicating that the ORDER BY clause is invalid.

As an example, take the following:

create table t1 (i int, j int);
select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by t1.idcolumn1, t1.idcolumn2;

This query is now rejected, as there is no column named 'idcolumn1' in table 't1'. The error message is:

Valid forms of the query above are:

select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by idcolumn1, idcolumn2;

or

select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by t1.id, t1.id;

Rationale for Change

When the query ambiguously specifies both DISTINCT and ORDER BY, Derby was unsure whether to return the rows properly ordered, but non-distinct, or to return a distinct set of rows, but in an unknown order. Since no clear resolution of the ambiguity could be found, we chose instead to reject the query.

The rules for resolving column references in ORDER BY clauses have been enhanced to consider column aliases and column names more fully. Derby now uses different resolution rules depending on whether the ORDER BY column reference is table.column, or just column:

  • if the table name is provided, we match against the underlying table name, and don't consider any aliases
  • if the table name is NOT provided, we first match against the alias name, if present, and if no alias name matches then we match against the underlying source column name.


Application Changes Required

A query which specifies ordering by a non-distinct column should instead include the ORDER BY column in the DISTINCT list, to resolve the ambiguity about which values of that column should be used to distinctly identify the resulting rows.

A query which specifies table-name.alias-name should be rewritten to specify either simply alias-name, or table-name.column-name.

Build Environment

Derby release 10.3.3.0 was built using the following environment:

  • Branch - Source code came from the 10.3 branch.
  • Machine - Cygwin on Microsoft Windows XP Professional Version 2002 Service Pack 2.
  • Ant - Apache Ant version 1.7.0 compiled on December 13 2006.
  • JDK 1.4 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_07-b05)
  • Java 6 - Java(TM) SE Runtime Environment (build 1.6.0_01-b06).
  • OSGi - osgi.jar was used to build org.apache.derby.osgi.EmbeddedActivator.
  • Compiler - The 1.4.2_07-b05 javac was used to compile all classes except for the JDBC4 drivers. The JDBC4 driver classes were compiled using the 1.6.0_01-b06 javac.
  • JSR 169 - Java ME support was built using Java ME CDC/Foundation Specification 1.1 libraries from IBM WebSphere Everyplace Micro Environment 6.1

Verifying releases

It is essential that you verify the integrity of the downloaded files using the PGP and MD5 signatures. MD5 verification ensures the file was not corrupted during the download process. PGP verification ensures that the file came from a certain person.

The PGP signatures can be verified using PGP or GPG. First download the Apache Derby KEYS as well as the asc signature file for the particular distribution. It is important that you get these files from the ultimate trusted source - the main ASF distribution site, rather than from a mirror. Then verify the signatures using ...

% pgpk -a KEYS
% pgpv db-derby-X.Y.tar.gz.asc

or

% pgp -ka KEYS
% pgp db-derby-X.Y.tar.gz.asc

or

% gpg --import KEYS
% gpg --verify db-derby-X.Y.tar.gz.asc

To verify the MD5 signature on the files, you need to use a program called md5 or md5sum, which is included in many unix distributions. It is also available as part of GNU Textutils. Windows users can get binary md5 programs from here, here, or here.

We strongly recommend you verify your downloads with both PGP and MD5.