Apache Derby Web Site

2016: Web site upgraded to Forrest 0.9

The web site is now being built with the latest release of Forrest, Forrest 0.9.

21-Dec-2007: Web site upgraded to Forrest 0.8

The web site was upgraded to build with the latest release of Forrest, Forrest 0.8.

12-Aug-2005: Web site upgraded to Forrest 0.7

The web site was recently upgraded to build with the latest release of Forrest, Forrest 0.7.

If you receive OutOfMemoryErrors while trying to build the site, try increasing the amount of memory available to forrest by setting the ANT_OPTS environment variable to -mx512M.

31-Jul-2005: Web site moved from Incubator to Apache DB

Derby graduated from the Incubator to the Apache DB project, so the web site moved from http://incubator.apache.org/derby to http://db.apache.org/derby. Specific changes, beginning with SVN revision 226719, included:

• src/documentation/skinconf.xml: Changed the group logo, which appears on the masthead, from the Incubator logo to the Apache DB logo, and updated the bread crumb trail.
• Updated references and URLs throughout the site from incubator to db or from incubator.apache.org to db.apache.org.

15-May-2005: Version 10.0 manuals moved

The files that comprise the 10.0 manuals are numerous (about 750), static, and take a while to build (20-40 minutes, depending on the machine). In May they were moved to a separate "mini" forrest site, which is now at http://db.apache.org/derby/docs/10.0/index.html.

27-Jan-2005: Web site upgraded to Forrest 0.6

The Derby web site was initialized in August 2004 with Forrest 0.5.1.

The site was upgraded to Forrest 0.6 on January 27, 2005. To keep the size of commit records down, changes were committed in the blocks listed below:

• Source pages: SVN 134379
• Built pages:
  • SVN 145725 : manuals/admin,getstart
  • SVN 145920 : manuals/develop
  • SVN 146259 : manuals/tools
  • SVN 146516 : manuals/tuning
  • SVN 146733 : manuals/reference
  • SVN 148837 : site/skin
  • SVN 148838 : site

Apache Forrest is an XML documentation framework that is used for generating static/dynamic web sites. For many details, see forrest.apache.org.

This web page walks you through using two Forrest commands: 'forrest site' and 'forrest run'.

The 'forrest site' command consumes the source input files in the src tree of the project hierarchy and spits out a static site composed of HTML pages in the build tree of the project hierarchy.

The 'forrest run' command starts up the Jetty web server that is included in the Forrest distribution on port 8888.

The information in this section is intended for anyone who wants to develop content for the Derby web site (in other words, for both committers and non-committers). It is much easier to integrate new pages by first testing them outside the Derby web site source tree.

Create a Forrest seed site

The Using Forrest: A tutorial on how to use Forrest in your own projects guide provides an excellent overview of Forrest. Below are some simple steps to get started:

• Download Forrest 0.8.
• Install Forrest and set up your environment.
• Create a new directory, cd into it, and enter 'forrest seed' to initialize a new project.
• Enter 'forrest site' to build the site.
• Enter 'forrest run' to start the jetty server that is bundled with Forrest.
• Point your browser at localhost:8888.
• Look at the sample pages that are included in the seed site.

Add your new page to the seed site

If you don't need to add a new page, skip this step. Otherwise, incorporate the new page(s) into the newly-created seed site:

• If you have an HTML file, perhaps named your_file.html,
  • One option is to leave the file as is (previous forrest releases had you rename it to a file named your_file.ihtml, but support for the ihtml extension is deprecated in forrest 0.7). Forrest will convert the file into a new your_file.html file in the build tree and will add the site decoration and navigation to it. The Derby JDBC Implementation Notes paper is an example of an HTML document that is stored in the project source tree as an html file.
  • Another option is to rename the file with a .htm extension. Forrest won't touch it, but will just copy it as is to the build tree; it won't add site decoration and navigation. This is useful for documents that Forrest has trouble converting. The Running DOTS (Database Open Source Test Suite) with Derby is an example of this approach.
• You can also create an XML file that uses the Forrest DTD. The Derby Engine Architecture Overview is an example. Also, the site you just seeded provides examples.
• If you have a file that is ascii text, one option is to create an XML file and include it in a CDATA block, as shown below:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" 
     "http://forrest.apache.org/dtd/document-v20.dtd">
  
  <document>
   <header>
     <title>The Title of your Document</title>
   </header>
  
   <body>
  
      <p>An introductory blurb for your document.</p>
  
      <source><![CDATA[
      Put the contents of your text file here ]]></source>
  
   </body>
  </document>
  

• The seed site includes a "Samples" tab. After you create the file for Forrest to ingest, update the src/documentation/content/xdocs/site.xml file to include your new page in that tab, as shown below:

  <samples label="Samples" href="samples/" tab="samples">
    <your_tag label="Your page" href="your_file.html" description="Your test page" />
  

  Notice that at this point you need to refer to the file as your_file.html even if the source file is your_file.ihtml or your_file.xml.
• Execute 'forrest site' to rebuild your site.
• Restart the jetty server ('forrest run').
• Do you like what Forrest did? If you don't, go back and tweak your source file.

What editor can you use for Forrest XML?

The Forrest web site mentions jEdit, oXygen, xmlspy, and XMLmind XML Editor (XXE) -- and no doubt many more XML products also work.

One Derby community cohort has recently looked at two editors, which are both quite excellent:

• jEdit (GPL) with the jEdit XML plugin. See the Forrest jEdit configuration instructions.
• XMLBuddy eclipse plugin (free). Caches the DTD, but am looking for a way to resolve locally. (If you install this plugin along with the Web Tools Project (WTP) plugin, you may need to do File -> Open With -> XMLBuddy because WTP includes the WST XML editor.)

If you find other tools that also work well, please let others know by posting to derby-dev@db.apache.org.

Resolve Forrest DTDs locally

Many source files in the Derby web site repository are in Forrest XML format. XML files include a DTD declaration at the top; for example, here is an example of what you might see at the top of a Forrest XML file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" 
   "http://forrest.apache.org/dtd/document-v20.dtd">

To reduce requests to the forrest.apache.org site itself, it's important to resolve Forrest DTDs locally. The forrest command resolves DTDs from local copies in the Forrest software distribution, so you don't need to do anything special when you build the Derby web site. However, if you use a tool to edit Forrest XML files that validates the XML, then you should configure that tool to use a catalog entity resolver per the instructions on the Forrest web site.

When you're satisfied with how your new page looks on your seed site, then it is ready to be added to the Derby web site. The mechanics of doing that is described in the next section.

This section explains how to add a new page to the Derby web site or modify an existing page. Steps 1-6 are for both contributors and committers. Step 7 tells contributors how to create a patch. Steps 8-10 tell committers how to commit and publish changes.

1. Install Forrest on your machine

If you haven't installed Forrest yet, install Forrest 0.8 on your local system.

2. Check out the Derby web site pages

Check the Derby web site files out of the subversion repository:

svn co https://svn.apache.org/repos/asf/db/derby/site/trunk

3. Modify files in the src tree

After checking the site out of subversion, the web site source is in src/documentation/content/xdocs and the files processed ("skinned") by Forrest are in build/site. (Forrest also outputs files to build/webapp and build/tmp, but we don't commit those directories to subversion.) Modifying the web site requires changing files in the src tree, then building the site with Forrest.

The Derby web site is composed of many file types and it's helpful to understand what Forrest does with them:

• *.xml and *.html: Forrest reads these files, adds site navigation and decoration, then outputs them as .html files to the build directory.
• *.htm: Forrest copies these files without modifying them to the build/site directory (in other words, they don't get site navigation and decoration).
• binary (*.tar.gz, *.zip) and image (*.gif, *.jpg) files: Forrest copies these files without modifying them to build/site/binaries and build/site/images, respectively.
• *.aart : These files contain ascii stick diagrams that Forrest converts to png, then outputs to the build/site directory.
• *.cgi : Forrest copies these files without modifying them to the build/site directory.

The table below describes the source directory structure and indicates which tabs might link to files in a given directory (the Derby project sometimes changes tab names and linkages):

Two files in xdocs don't get output to the build/site tree; they control the information and links in the resulting navigation bar:

• tabs.xml: sets up the tabs for the web site. Over time the specific tabs have changed.

• site.xml: contains a site map for files on the site. If you add or delete a file you must update site.xml, which can be tricky. If you haven't updated it before, it might be a good idea to ask somebody already familiar with Forrest to help you the first time.

4. Build the site

After modifying files, rebuild the site with the 'forrest site' command. Do this in the trunk (one level above the src and build directories).

Due to FOR-480, the skinned HTML for the release pages will be output to your FORREST_HOME/main/site directory. If you have made changes to the site navigation or to the release pages themselves copy these files to build/site/releases so that they can be checked into subversion with the source changes.

If you run out of memory building the web site, try setting the ANT_OPTS environment variable to -mx512M.

5. Test the changes

There are a couple ways (at least) to review changes.

One way is to open build/site/index.html with your web browser.

Another way is to start the Jetty webserver that is included in the Forrest distribution:

forrest run

After it starts, point your browser at localhost:8888 and review the new or modified pages.

The breadcrumb trail will be different. Instead of this:

apache > db > derby

You'll see this with your local Jetty server:

apache > db 

The difference in bread crumb behavior is alright, though confusing. The skinconf.xml specifies the base links for the trail, then Javascript adds extra breadcrumbs for each directory after the domain. Extra breadcrumbs won't be added when the site is viewed locally.

6. Adjust the site build - odd diffs

Derby developers build the site on a variety of platforms, including Windows, Mac, and Linux. The svn status command might report modifications to files under build/site/skin and to the png files under build/site/papers after you build the site, depending on who last committed those files.

For example, when you run the forrest site command to build the site, Forrest generates inconsistent line endings on Windows machines (see FOR-492 and FOR-805). Also png files generated from aart source may get generated differently on different operating systems.

As a result, you might get odd diffs under build/site/skin and the build/site/papers directories.

If you are using a Windows machine to build the site, you should revert the changes made to the skin directory before you create the patch. For example:

svn revert build\site\skin\*

If you are using TortoiseSVN, you can use Windows Explorer to revert the changes.

7. Create a patch

Create a patch at the trunk level and only include the src directory, as shown below:

svn diff src > changes.diff

Upload the patch to a Jira issue to make it available to a Derby committer.

8. Commit changes to the src and build/site directories

Apply any patch, if one was provided, then build and test the web site (Steps 4-6).

Commit modified pages in the src and build/site trees. The 'forrest site' command also outputs build/webapp and build/tmp directories, but they aren't checked into the subversion repository.

If you add or delete a file, you must 'svn add' or 'svn delete' it in the src and build/site directory hierarchies. Do this after 'forrest site' has been run but before committing changes to the the subversion repository.

One more note about adding new text files. After you add the file to the repository, it's important to set the end-of-line for the file to native. You can do so explicitly with the command shown below:

svn propset svn:eol-style native filename

However, the easiest way to manage svn:eol-style is to automatically set end of line for common file types by adding the Apache dev list of files to your ~/.subversion/config file.

When you're ready to commit, you have to separately commit the changes to the src and build/site directories:

cd src
svn commit --message "here is my commit message" 
cd ../build/site 
svn commit --message "here is my commit message" 

9. Make web site changes visible

A Derby committer can force the source tree to be updated to the latest change as follows:

ssh -l your_apache_login people.apache.org
cd /www/db.apache.org/derby
svn update

However, www.apache.org automatically syncs up with people.apache.org every couple hours (the specific schedule could change) and the build/site tree will get synced right away, so there is no need for this.

10. Considering a quick fix to the build tree?

If you modify a file in the build/site tree, be sure to also change the source file in the src tree. Otherwise, the next 'forrest site' followed by 'svn commit' will likely overwrite your changes to the build tree.

Last updated Dec 21, 2007