Derby Developer's Guide
Version 10.4
Derby Document build:
April 14, 2008, 9:48:10 AM (CEST)




Version 10.4   Derby Developer's Guide
   
Contents
Copyright    
License    
About this guide    
Purpose of this guide    
Audience    
How this guide is organized    
Upgrades    
Preparing to upgrade    
Upgrading a database    
Soft upgrade limitations    
JDBC applications and Derby basics    
Application development overview    
Derby embedded basics    
Derby JDBC driver    
Derby JDBC database connection URL    
Derby system    
A Derby database    
Connecting to databases    
Working with the database connection URL attributes    
After installing    
The installation directory    
Batch files and shell scripts    
Derby and JVMs    
Derby libraries and classpath    
UNIX-specific issues    
Configuring file descriptors    
Scripts    
Deploying Derby applications    
Deployment issues    
Embedded deployment application overview    
Deploying Derby in an embedded environment    
Creating Derby databases for read-only use    
Creating and preparing the database for read-only use    
Deploying the database on the read-only media    
Transferring read-only databases to archive (jar or zip) files    
Accessing a read-only database in a zip/jar file    
Accessing databases within a jar file using the classpath    
Databases on read-only media and DatabaseMetaData    
Loading classes from a database    
Class loading overview    
Dynamic changes to jar files or to the database jar classpath    
Derby server-side programming    
Programming database-side JDBC procedures    
Database-side JDBC procedures and nested connections    
Database-side JDBC procedures using non-nested connections    
Database-side JDBC procedures and SQLExceptions    
User-defined SQLExceptions    
Programming trigger actions    
Trigger action overview    
Performing referential actions    
Accessing before and after rows    
Examples of trigger actions    
Triggers and exceptions    
Programming Derby-style table functions    
Overview of Derby-style table functions    
Example Derby-style table function    
Optimizer support for Derby-style table functions    
Controlling Derby application behavior    
The JDBC Connection and Transaction Model    
Connections    
Transactions    
Result set and cursor mechanisms    
Simple non-updatable result sets    
Updatable result sets    
Result sets and auto-commit    
Scrollable result sets    
Holdable result sets    
Locking, concurrency, and isolation    
Isolation levels and concurrency    
Configuring isolation levels    
Lock granularity    
Types and scope of locks in Derby systems    
Deadlocks    
Working with multiple connections to a single database    
Deployment options and threading and connection modes    
Multi-user database access    
Multiple connections from a single application    
Working with multiple threads sharing a single connection    
Pitfalls of sharing a connection among threads    
Multi-thread programming tips    
Example of threads sharing a statement    
Working with database threads in an embedded environment    
Working with Derby SQLExceptions in an application    
Information provided in SQL Exceptions    
Using Derby as a J2EE resource manager    
Classes that pertain to resource managers    
Getting a DataSource    
Shutting down or creating a database    
Derby and Security    
Configuring security for your environment    
Configuring security in a client/server environment    
Configuring security in an embedded environment    
Working with user authentication    
Enabling user authentication    
Defining users    
External directory service    
Built-in Derby users    
List of user authentication properties    
Programming applications for Derby user authentication    
Users and authorization identifiers    
Authorization identifiers, user authentication, and user authorization    
Database owner    
User names and schemas    
Exceptions when using authorization identifiers    
User authorizations    
Setting the SQL standard authorization mode    
Read-only and full access permissions    
Encrypting databases on disk    
Requirements for Derby encryption    
Working with encryption    
Signed jar files    
Notes on the Derby security features    
User authentication and authorization examples    
User authentication example in a client/server environment    
User authentication example in a single-user, embedded environment    
Running Derby under a security manager    
Granting permissions to Derby    
Examples of Java 2 security policy files for embedded Derby    
Developing tools and using Derby with an IDE    
Offering connection choices to the user    
The DriverPropertyInfo Array    
Using Derby with IDEs    
IDEs and multiple JVMs    
SQL tips    
Retrieving the database connection URL    
Supplying a parameter only once    
Defining an identity column    
Using third-party tools    
Tricks of the VALUES clause    
Multiple rows    
Mapping column values to return values    
Creating empty queries    
Localizing Derby    
SQL parser support for Unicode    
Character-based collation in Derby    
Other components with locale support    
Messages libraries    
Derby and standards    
XML data types and operators    
Trademarks    


Derby Developer's Guide
Apache Software FoundationDerby Developer's GuideApache Derby
Copyright
Copyright 2004-2008 The Apache Software Foundation
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.
Related information
License
The Apache License, Version 2.0
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
About this guide
For general information about the Derby documentation, such as a complete list of books, conventions, and further reading, see Getting Started with Derby.
Purpose of this guide
This guide explains how to use the core Derby technology and is for developers building Derby applications.
It describes basic Derby concepts, such as how you create and access Derby databases through JDBC procedures and how you can deploy Derby applications.
Audience
This guide is intended for software developers who already know some SQL and Java.
Derby users who are not familiar with the SQL standard or the Java programming language will benefit from consulting books on those subjects.
How this guide is organized
This document includes the following sections.
 
JDBC applications and Derby basics
Basic details for using Derby, including loading the JDBC driver, specifying a database URL, and starting Derby.
 
After installing
Explains the installation layout.
 
Deploying Derby applications
An overview of different deployment scenarios, and tips for getting the details right when deploying applications.
 
Controlling Derby application behavior
JDBC, cursors, locking and isolation levels, and multiple connections.
 
Using Derby as a J2EE resource manager
Information for programmers developing back-end components in a J2EE system.
 
 
SQL tips
Insiders' tricks of the trade for using SQL.
 
Localizing Derby
An overview of database localization.
Upgrades
To connect to a database created with a previous version of Derby, you must first upgrade that database.
Upgrading involves writing changes to the system tables, so it is not possible for databases on read-only media. The upgrade process:
 
marks the database as upgraded to the current release (Version 10.4).
 
allows use of new features.
See the release notes for more information on upgrading your databases to this version of Derby.
Preparing to upgrade
Upgrading your database occurs the first time the new Derby software connects to the old database.
Before you connect to the database using the new software:
1.
 
Back up your database to a safe location using Derby online/offline backup procedures.
For more information on backup, see the Derby Server and Administration Guide.
2.
 
Update your CLASSPATH with the latest jar files.
3.
 
Make sure that there are no older versions of the Derby jar files in your CLASSPATH. You can determine if you have multiple versions of Derby in your CLASSPATH by using the sysinfo tool.
To use the sysinfo tool, execute the following command:
java org.apache.derby.tools.sysinfo
The sysinfo tool uses information found in the Derby jar files to determine the version of any Derby jar in your CLASSPATH. Be sure that you have only one version of the Derby jar files specified in your CLASSPATH.
Upgrading a database
To upgrade a database, you must explicitly request an upgrade the first time you connect to the database with the new version of Derby.
Ensure that you complete the prerequisite steps before you upgrade:
 
Back up your database before you upgrade.
 
Ensure that only the new Derby jar files are in your CLASSPATH.
When you upgrade the database, you can perform a full upgrade or soft upgrade:
 
A full upgrade is a complete upgrade of the Derby database. When you perform a full upgrade, you cannot connect to the database with an older version of Derby and you cannot revert back to the previous version.
 
A soft upgrade allows you to run a newer version of Derby against an existing database without having to fully upgrade the database. This means that you can continue to run an older version of Derby against the database. However, if you perform a soft upgrade, certain features will not be available to you until you perform a full upgrade.
1.
 
To upgrade the database, select the type of upgrade that you want to perform:
Type of upgrade
Action
Full upgrade
Connect to the database using the upgrade=true database connection URL attribute. For example:
jdbc:derby:sample;upgrade=true
Soft upgrade
Connect to the database. For example:
connect 'jdbc:derby:sample'
In this example, sample is a database from a previous version of Derby.
Soft upgrade limitations
Soft upgrade allows you to run a newer version of Derby against an existing database without having to fully upgrade the database. This means that you can continue to run an older version of Derby against the database.
If you perform a soft upgrade, you will not be able to perform certain functions that are not available in older versions of Derby. For example, the following Derby Version 10.2 features cannot be used in a database that has been soft upgraded from 10.0:
 
SYNONYMS
 
Creating tables using the GENERATED BY DEFAULT option for identity columns
 
Reclaiming unused space using the SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE procedure
The following 10.2 features cannot be accessed from a database which has been soft upgraded from 10.1:
 
GRANT/REVOKE
 
Online backup procedures SYSCS_UTIL.SYSCS_BACKUP_DATABASE_NOWAIT and SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE_NOWAIT.
 
The encryption or re-encryption of a database with a new phrase/key.
Other new features in Derby that do not affect database structure, such as using timestamp arithmetic, are allowed in a soft upgraded database.
To perform a soft upgrade on a database created using an earlier version of Derby:
1.
 
Simply connect to the database, as shown in the following example:
connect 'jdbc:derby:sample'
In this example, the sample database is a Version 10.0 database.
JDBC applications and Derby basics
This section describes the core Derby functionality. In addition, it details the most basic Derby deployment, Derby embedded in a Java application.
Application development overview
Derby application developers use JDBC, the application programming interface that makes it possible to access relational databases from Java programs.
The JDBC API is part of the Java(TM) 2 Platform, Standard Edition and is not specific to Derby. It consists of the java.sql and javax.sql packages, which is a set of classes and interfaces that make it possible to access databases (from a number of different vendors, not just Derby) from a Java application.
To develop Derby applications successfully, you will need to learn JDBC. This section does not teach you how to program with the JDBC API.
This section covers the details of application programming that are specific to Derby applications. For example, all JDBC applications typically start their DBMS's JDBC driver and use a connection URL to connect to a database. This chapter gives you the details of how to start Derby's JDBC driver and how to work with Derby's connection URL to accomplish various tasks. It also covers essential Derby concepts such as the Derby system.
You will find reference information about the particulars of Derby's implementation of JDBC in the Derby Reference Manual.
Derby application developers will need to learn SQL. SQL is the standard query language used with relational databases and is not tied to a particular programming language. No matter how a particular RDBMS has been implemented, the user can design databases and insert, modify, and retrieve data using the standard SQL statements and well-defined data types. SQL-92 is the version of SQL standardized by ANSI and ISO in 1992; Derby supports entry-level SQL-92 as well as some higher-level features. Entry-level SQL-92 is a subset of full SQL-92 specified by ANSI and ISO that is supported by nearly all major DBMSs today. This chapter does not teach you SQL. You will find reference information about the particulars of Derby's implementation of SQL in the Derby Reference Manual.
Derby implements JDBC that allows Derby to serve as a resource manager in a J2EE compliant system.
Derby embedded basics
This section explains how to use and configure Derby in an embedded environment.
Included in the installation is a sample application program, /demo/programs/simple, which illustrates how to run Derby embedded in the calling program.
Derby JDBC driver
Derby consists of both the database engine and an embedded JDBC driver. Applications use JDBC to interact with a database. Applications running on JDK 1.5 or earlier, must load the driver in order to work with the database.
In an embedded environment, loading the driver also starts Derby.
The Derby driver class name for the embedded environment is org.apache.derby.jdbc.EmbeddedDriver.
In a Java application, you typically load the driver with the static Class.forName method or with the jdbc.drivers system property. For example:
Class.forName("org.apache.derby.jdbc.EmbeddedDriver");
java -Djdbc.drivers=org.apache.derby.jdbc.EmbeddedDriver applicationClass
For detailed information about loading the Derby JDBC driver, see "java.sql.Driver interface" in the Derby Reference Manual.
If your application runs on JDK 1.6 or higher, then you do not need to explicitly load the EmbeddedDriver. In that environment, the driver loads automatically.
Derby JDBC database connection URL
A Java application using the JDBC API establishes a connection to a database by obtaining a Connection object.
The standard way to obtain a Connection object is to call the method DriverManager.getConnection, which takes a String containing a connection URL (uniform resource locator). A JDBC connection URL provides a way of identifying a database. It also allows you to perform a number of high-level tasks, such as creating a database or shutting down the system.
An application in an embedded environment uses a different connection URL from that used by applications using the Derby Network Server in a client/server environment. See the Derby Server and Administration Guide for more information on the Network Server.
However, all versions of the connection URL (which you can use for tasks besides connecting to a database) have common features:
 
you can specify the name of the database you want to connect to
 
you can specify a number of attributes and values that allow you to accomplish tasks. For more information about what you can specify with the Derby connection URL, see Database connection examples.
The connection URL syntax is as follows:
jdbc:derby:[subsubprotocol:][databaseName][;attribute=value]*
Subsubprotocol, which is not typically specified, determines how Derby looks for a database: in a directory, in a class path, or in a jar file. Subsubprotocol is one of the following:
 
directory: The default. Specify this explicitly only to distinguish a database that might be ambiguous with one on the class path.
 
classpath: Databases are treated as read-only databases, and all databaseNames must begin with at least a slash, because you specify them "relative" to the classpath directory. See Accessing databases from the classpath for details.
 
jar: Databases are treated as read-only databases. DatabaseNames might require a leading slash, because you specify them "relative" to the jar file. See Accessing databases from a jar or zip file for details.
jar requires an additional element immediately before the database name:
(pathToArchive)
pathToArchive is the path to the jar or zip file that holds the database.
For detailed reference about connection URL attributes and values, see "Setting attributes for the database connection URL" in the Derby Reference Manual.
The following example shows the use of the connection URL:
Connection conn = DriverManager.getConnection("jdbc:derby:sample");
Derby system
A Derby database exists within a system.
A Derby system is a single instance of the Derby database engine and the environment in which it runs. It consists of a system directory, zero or more databases, and a system-wide configuration. The system directory contains any persistent system-wide configuration parameters, or properties, specific to that system in a properties file called derby.properties. This file is not automatically created; you must create it yourself.
The Derby system is not persistent; you must specify the location of the system directory at every startup.
However, the Derby system and the system directory is an essential part of a running database or databases. Understanding the Derby system is essential to successful development and deployment of Derby applications. Derby databases live in a system, which includes system-wide properties, an error log, and one or more databases.
Figure 1. Derby databases live in a system, which includes system-wide properties, an error log, and one or more databases.
 
The system directory can also contain an error log file called derby.log (see The error log).
Each database within that system is contained in a subdirectory, which has the same name as the database (see A Derby database).
In addition, if you connect to a database outside the current system, it automatically becomes part of the current system.
One Derby instance for each Java Virtual Machine
You could potentially have two instances of a Derby system (JVM) running on the same machine at the same time. Each instance must run in a different JVM. Two separate instances of Derby must not access the same database.
For example, in an embedded environment, an application that accesses Derby databases starts up the local JDBC driver, which starts up an instance of Derby. If you start another application, such as ij, and connect to the same database, severe database corruption can result. See Double-booting system behavior.
Booting databases
The default configuration for Derby is to boot (or start) a database when an application first makes a connection to it. When Derby boots a database, it checks to see if recovery needs to be run on the database, so in some unusual cases booting can take some time.
You can also configure your system to automatically boot all databases in the system when it starts up; see derby.system.bootAll in the Tuning Derby manual. Because of the time needed to boot a database, the number of databases in the system directory affects startup performance if you use that configuration.
Once a database has been booted within a Derby system, it remains active until the Derby system has been shut down or until you shut down the database individually.
When Derby boots a database, a message is added to the error log. The message includes the Derby version that the database was booted with, for example:
2006-10-04 03:54:06.196 GMT: Booting Derby version Apache Derby - 10.2.1.5 - (448900): instance c013800d-00fd-0cb0-e736-ffffd1025a25 on database directory sample
The number of databases running in a Derby system is limited only by the amount of memory available in the JVM.
Shutting down the system
In an embedded environment, when an application shuts down, it should first shut down Derby.
If the application that started the embedded Derby quits but leaves the JVM running, Derby continues to run and is available for database connections.
In an embedded system, the application shuts down the Derby system by issuing the following JDBC call:
DriverManager.getConnection("jdbc:derby:;shutdown=true");
Shutdown commands always raise SQLExceptions.
When a Derby system shuts down, a message goes to the error log:
Sat Jan 10 14:31:54 PDT 2005: Shutting down instance 80000001-00d0-8bdf-d115-000a0a0b2d00
Typically, an application using an embedded Derby engine shuts down Derby just before shutting itself down. However, an application can shut down Derby and later restart it in the same JVM session. To restart Derby successfully, the JVM needs to unload org.apache.derby.jdbc.EmbeddedDriver, so that it can reload it when it restarts Derby. (Loading the local driver starts Derby.)
You cannot explicitly request that the JVM unload a class, but you can ensure that the EmbeddedDriver class is unloaded by using a System.gc() to force it to garbage collect classes that are no longer needed. Running with -nogc or -noclassgc definitely prevents the class from being unloaded and makes you unable to restart Derby in the same JVM.
It is also possible to shut down a single database instead of the entire Derby system. See Shutting down Derby or an individual database. You can reboot a database in the same Derby session after shutting it down.
Defining the system directory
You define the system directory when Derby starts up by specifying a Java system property called derby.system.home.
If you do not specify the system directory when starting up Derby, the current directory becomes the system directory.
Derby uses the derby.system.home property to determine which directory is its system directory - and thus what databases are in its system, where to create new databases, and what configuration parameters to use. See Tuning Derby for more information on setting this property.
If you specify a system directory at startup that does not exist, Derby creates this new directory - and thus a new system with no databases-automatically.
The error log
Once you create or connect to a database within a system, Derby begins outputting information and error messages to the error log.
Typically, Derby writes this information to a log called derby.log in the system directory, although you can also have Derby send messages to a stream, using the derby.stream.error.method property. By default, Derby overwrites derby.log when you start the system. You can configure Derby to append to the log with the derby.infolog.append property. For information on setting this and other properties, see Tuning Derby.
derby.properties
The text file derby.properties contains the definition of properties, or configuration parameters that are valid for the entire system.
The derby.properties file is not automatically created. If you want to set Derby properties with this file, you need to create the file yourself. The derby.properties file should be in the format created by the java.util.Properties.save method. For more information about properties and the derby.properties file, see Tuning Derby.
Double-booting system behavior
Derby attempts to prevent two instances of Derby from booting the same database by using a file called db.lck inside the database directory.
On all platforms running with a JDK of 1.4 or higher, Derby can successfully prevent a second instance of Derby from booting the database and thus prevents corruption.
On some platforms running with a JDK lower than 1.4, Derby may prevent a second instance of Derby from booting the database (previous to JDK 1.4 the ability to do this was OS dependent).
If this is the case, you will see an SQLException like the following:
ERROR XJ040: Failed to start database 'sample', see the next exception for details. ERROR XSDB6: Another instance of Derby might have already booted the databaseC:\databases\sample.
The error is also written to the error log.
If you are running a JVM prior to 1.4, Derby issues a warning message on some platforms if an instance of Derby attempts to boot a database that already has a running instance of Derby attached to it. However, it does not prevent the second instance from booting, and thus potentially corrupting, the database. (You can change this behavior with the property derby.database.forceDatabaseLock.)
If a warning message has been issued, corruption might already have occurred. Corruption can occur even if one of the two booting systems has "readonly" access to the database.
The warning message looks like this:
WARNING: Derby (instance 80000000-00d2-3265-de92-000a0a0a0200) is attempting to boot the database /export/home/sky/wombat even though Derby (instance 80000000-00d2-3265-8abf-000a0a0a0200) might still be active. Only one instance of Derby should boot a database at a time. Severe and non-recoverable corruption can result and might have already occurred.
The warning is also written to the error log.
If you see this warning, you should close the connection and exit the JVM, minimizing the risk of a corruption. Close all instances of Derby, then restart one instance of Derby and shut down the database properly so that the db.lck file can be removed. The warning message continues to appear until a proper shutdown of the Derby system can delete the db.lck file.
When developing applications, you might want to configure Derby to append to the log. Doing so will help you detect when you have inadvertently started more than one instance of Derby in the same system. For example, when the derby.infolog.append property is set to true for a system, booting two instances of Derby in the same system produces the following in the log:
Sat Aug 14 09:42:51 PDT 2005: Booting Derby version Apache Derby - 10.0.0.1 - (29612): instance 80000000-00d2-1c87-7586-000a0a0b1300 on database at directory C:\tutorial_system\sample ------------------------------------------------------------ Sat Aug 14 09:42:59 PDT 2005: Booting Derby version Apache Derby - 10.0.0.1 - (29612): instance 80000000-00d2-1c87-9143-000a0a0b1300 on database at directory C:\tutorial_system\HelloWorldDB
Derby allows you to boot databases that are not in the system directory. While this might seem more convenient, check that you do not boot the same database with two JVMs. If you need to access a single database from more than one JVM, you will need to put a server solution in place. You can allow multiple JVMs that need to access that database to connect to the server. The Derby Network Server is provided as a server solution. See the Derby Server and Administration Guide for more information on the Network Server.
Recommended practices
When developing Derby applications, create a single directory to hold your database or databases.
Give this directory a unique name, to help you remember that:
 
All databases exist within a system.
 
System-wide properties affect the entire system, and persistent system-wide properties live in the system directory.
 
You can boot all the databases in the system, and the boot-up times of all databases affect the performance of the system.
 
You can preboot databases only if they are within the system. (Databases do not necessarily have to live inside the system directory, but keeping your databases there is the recommended practice.)
 
Once you connect to a database, it is part of the current system and thus inherits all system-wide properties.
 
Only one instance of Derby can run in a JVM at a single time, and only one instance of Derby should boot a database at one time. Keeping databases in the system directory makes it less likely that you would use more than one instance of Derby.
 
The error log is located inside the system directory.
A Derby database
A Derby database contains dictionary objects such as tables, columns, indexes, and jar files. A Derby database can also store its own configuration information.
The database directory
A Derby database is stored in files that live in a directory of the same name as the database. Database directories typically live in system directories.
A database directory contains the following, as shown in the following figure.
 
log directory
Contains files that make up the database transaction log, used internally for data recovery (not the same thing as the error log).
 
seg0 directory
Contains one file for each user table, system table, and index (known as conglomerates).
 
service.properties file
A text file with internal configuration information.
 
tmp directory
(might not exist.) A temporary directory used by Derby for large sorts and deferred updates and deletes. Sorts are used by a variety of SQL statements. For databases on read-only media, you might need to set a property to change the location of this directory. See "Creating Derby Databases for Read-Only Use".
 
jar directory
(might not exist.) A directory in which jar files are stored when you use database class loading.
Read-only database directories can be archived (and compressed, if desired) into jar or zip files. For more information, see Accessing a read-only database in a zip/jar file.
The following figure shows the files and directories in the Derby database directories that are used by the Derby software.
Figure 2. An example of a Derby database directory and file structure.
 
Derby imposes relatively few limitations on the number and size of databases and database objects. The following table shows some size limitations of Derby databases and database objects:
Table 1. Size limits for Derby database objects
Type of Object
Limit
tables in each database
java.lang.Long.MAX_VALUE
Some operating systems impose a limit to the number of files allowed in a single directory.
indexes in each table
32,767 or storage
columns in each table
1,012
number of columns on an index key
16
rows in each table
No limit.
size of table
No limit. Some operating systems impose a limit on the size of a single file.
size of row
No limit. Rows can span pages. Rows cannot span tables so some operating systems impose a limit on the size of a single file, which results in limiting the size of a table and size of a row in that table.
For a complete list of restrictions on Derby databases and database objects, see the Derby Reference Manual.
Creating, dropping, and backing up databases
You create new databases and access existing ones by specifying attributes to the Derby connection URL.
There is no drop database command. To drop a database, delete the database directory with operating system commands. The database must not be booted when you remove a database. You can get a list of booted databases with getPropertyInfo.
To back up a database, you can use the online backup utility. For information on this utility, see the Derby Server and Administration Guide.
You can also use roll-forward recovery to recover a damaged database. Derby accomplishes roll-forward recovery by using a full backup copy of the database, archived logs, and active logs from the most recent time before a failure. For more information on roll-forward recovery see the Derby Server and Administration Guide.
Single database shutdown
An application can shut down a single database within a Derby system and leave the rest of the system running.
Storage and recovery
A Derby database provides persistent storage and recovery. Derby ensures that all committed transactions are durable, even if the system fails, through the use of a database transaction log.
Whereas inserts, updates, and deletes may be cached before being written to disk, log entries tracking all those changes are never cached but always forced to disk when a transaction commits. If the system or operating system fails unexpectedly, when Derby next starts up it can use the log to perform recovery, recovering the "lost" transactions from the log and rolling back uncommitted transactions. Recovery ensures that all committed transactions at the time the system failed are applied to the database, and all transactions that were active are rolled back. Thus the databases are left in a consistent, valid state.
In normal operation, Derby keeps the log small through periodic checkpoints. Checkpointing marks the portions of the log that are no longer useful, writes changed pages to disk, then truncates the log.
Derby checkpoints the log file as it fills. It also checkpoints the log when a shutdown command is issued. Shutting down the JVM in which Derby is running without issuing the proper shutdown command is equivalent to a system failure from Derby's point of view.
Booting a database means that Derby checks to see if recovery needs to be run on a database. Recovery can be costly, so using the proper shutdown command improves connection or startup performance.
Log on separate device
You can put a database's log on a separate device when you create it.
For more information, see the Derby Server and Administration Guide.
Database pages
Derby tables and indexes, known as conglomerates, consist of two or more pages.
A page is a unit of storage whose size is configurable on a system-wide, database-wide, or conglomerate-specific basis. By default, a conglomerate grows one page at a time until eight pages of user data (or nine pages of total disk use, which includes one page of internal information) have been allocated. (You can configure this behavior; see "derby.storage.initialPages" in Tuning Derby.) After that, it grows eight pages at a time.
The size of a row or column is not limited by the page size. Rows or columns that are longer than the table's page size are automatically wrapped to overflow pages.
Database-wide properties
You can set many Derby properties as database-level properties. When set in this way, they are stored in the database and "travel" with the database unless overridden by a system property.
For more information, see "Database-Wide Properties" in Tuning Derby.
Note: You should work with database-level properties wherever possible.
Derby database limitations
Derby databases have a few limitations.
Indexes
Indexes are not supported for columns defined on CLOB, BLOB, LONG VARCHAR, and XML data types.
If the length of the key columns in an index is larger than half the page size of the index, creating an index on those key columns for the table fails. For existing indexes, an insert of new rows for which the key columns are larger than half of the index page size causes the insert to fail.
Avoid creating indexes on long columns. Create indexes on small columns that provide a quick look-up to larger, unwieldy data in the row. You might not see performance improvements if you index long columns. For information about indexes, see Tuning Derby.
System shutdowns
The system shuts down if the database log cannot allocate more disk space.
A "LogFull" error or some sort of IOException occurs in the derby.log file when the system runs out of space. If the system has no more disk space to append to the derby.log file, you might not see the error messages.
Connecting to databases
You connect to a database using a form of the Derby connection URL as an argument to the DriverManager.getConnection call.
You specify a path to the database within this connection URL.
Connecting to databases within the system
The standard way to access databases is in the file system by specifying the path to the database, either absolute or relative to the system directory. In a client/server environment, this path is always on the server machine.
By default, you can connect to databases within the current system directory (see Defining the system directory). To connect to databases within the current system, just specify the database name on the connection URL. For example, if your system directory contains a database called myDB, you can connect to that database with the following connection URL:
jdbc:derby:myDB
The full call within a Java program would be:
Connection conn =DriverManager.getConnection("jdbc:derby:myDB");
Connecting to databases outside the system directory
You can also connect to databases in other directories (including subdirectories of the system directory) by specifying a relative or absolute path name to identify the database. The way you specify an absolute path is defined by the host operating system.
Using the connection URL as described here, you can connect to databases in more than one directory at a time.
Two examples:
jdbc:derby:../otherDirectory/myDB jdbc:derby:c:/otherDirectory/myDB
Note: Once connected, such a database becomes a part of the Derby system, even though it is not in the system directory. This means that it takes on the system-wide properties of the system and no other instance of Derby should access that database. It is recommended that you connect to databases only in the system directory.
Conventions for specifying the database path
When accessing databases from the file system (instead of from classpath or a jar file), any path that is not absolute is interpreted as relative to the system directory.
The path must do one of the following:
 
refer to a previously created Derby database
 
specify the create=true attribute
The path separator in the connection URL is / (forward slash), as in the standard file:// URL protocol.
You can specify only databases that are local to the machine on which the JVM is running. NFS file systems on UNIX and remote shared files on Windows (//machine/directory) are not guaranteed to work. Using derby.system.home and forward slashes is recommended practice for platform independent applications.
If two different database name values, relative or absolute, refer to the same actual directory, they are considered equivalent. This means that connections to a database through its absolute path and its relative path are connections to the same database. Within Derby, the name of the database is defined by the canonical path of its directory from java.io.File.getCanonicalPath.
Derby automatically creates any intermediate directory that does not already exist when creating a new database. If it cannot create the intermediate directory, the database creation fails.
If the path to the database is ambiguous, i.e., potentially the same as that to a database that is available on the classpath