Sunitha Kambhampati ( skambha@Yngvi.Org)
I ran DOTS (Database Open Source Test Suite) with
2.2 The test environment that was used in this study:
3. Overview of DOTS and test configurations with Derby
4. Steps to setup and run Dots:
4.2. Database (Derby) related setup
4.3. Start the database server
4.4. Configuring test run parameters for DOTS client
4.6. Checking the output and investigating possible
problems
APPENDIX A: derbydotsscripts.tar.gz contents
APPENDIX B: Sample result file with Derby
Database Opensource Test Suite (DOTS) is an open source test suite that is used to stress test database systems and to measure database reliability (http://ltp.sourceforge.net/dotshowto.php#SEC1 . DOTS is licensed under GNU General Public License (GPL). The test cases are in Java and make JDBC calls to the database. They are aimed to test 100% JDBC API coverage. There are two kinds of test cases, basic and advanced. There are 8 basic tests prefixed with BTCJ and 2 advanced tests prefixed by ATCJ. The advanced test cases simulate real world workloads including user authentication/registration scenario and an online auction scenario.
The aim of this document is to help setup DOTS to run with
This
document basically does a walkthrough of a setup/ run of DOTS with
This section talks about the hardware and software needed to run DOTS. In sub-section 2.1, the requirement as stated by DOTS is given. Sub-section 2.2 lists the actual test environment used.
Database:
OS: Linux 2.4.4
JVM: J2SE1.3.1 or higher
Hardware Requirements:
CPU: 700Mhz or higher
Memory: 512Mb or higher
Disk Capacity: 2 GB or higher
Test machine:
|
OS Name |
Suse Linux |
|
Version |
9.0 |
|
System Manufacturer |
IBM |
|
System Model |
86473FX |
|
System Type |
X86-based PC |
|
Processor |
x86 Family 15 Model 2 Stepping 7 GenuineIntel ~2400 Mhz |
|
Processor |
x86 Family 15 Model 2 Stepping 7 GenuineIntel ~2400 Mhz |
|
Memory |
4Gb |
Software Used:
Instructions to build derby jars and downloads are at
http://db.apache.org/derby/derby_downloads.html
Java -
Java 1.4.2
A high level overview of the main components of DOTS is as follows
DOTS can be run in two modes:
In the example described here: the database, DOTS client and the performance monitor are all on one machine. If you decide to have the DOTS client and database server on different machines, the steps will be some what similar except that you will have to ensure that the necessary jars are present on the client and the database server machines.
The sections 3.1 and 3.2 of the Database Opensource Test Suite User’s Guide provide excellent overview. Section 3.1 (http://ltp.sourceforge.net/dotshowto.php#SEC11) and
Section 3.2 (http://ltp.sourceforge.net/dotshowto.php#SEC12)
1) Download the DOTS 1.1.1 (Dots-1.1.1.gz) from here http://sourceforge.net/project/showfiles.php?group_id=3382
Unzip the gz file
$ tar zxvf Dots-1.1.1.gz
After this, there will be a Dots directory with different sub directories with source of the tests, scripts, make file etc.
In this example, Dots has been installed at /home/perftest/Dots. Let $DOTS_ROOT refer to your dots installation. Here it will be /home/perftest/Dots.
2) Make sure that java is installed, else install it and set the path variable to include jvm.
3) Compiling the DOTS source
Change directory to your Dots installation
cd /home/perftest/Dots/
Before running the make command, make sure that you have javac in your path.
make all
This will compile all the source java files in the src subdirectory of your DOTS installation. If this is successful, you will see no errors during the make, and there will be two jars under $DOTS_ROOT: PerfMon.jar and the Dots.jar
PerfMon.jar – This jar file contains the classes necessary to start the performance monitor client. This jar is required to run performance monitor.
Dots.jar – This jar file has all the test cases that are needed to run tests. Also contains the DOTS client component needed to drive the test runs.
The
database related setup involves setting up the necessary database jar files,
creating the database and loading the schema. For both
There
is a derbydotsscripts.tar.gz file along with this
document that has the scripts and sql files necessary for derby. For more
details on each of the scripts present in the derbydotsscripts.tar.gz file, please
see APPENDIX A. To help reduce the setup
time, these scripts can be easily modified according to your individual setup.
Unzip the derbydotsscripts.tar.gz into a directory called derby and execute the
scripts from the same directory. Modify the myenv.ksh to set the
DERBY_JARS_ROOT as well as the DOTS_JARS_ROOT environment variables
appropriately. DERBY_JARS_ROOT refers to the root directory where the derby
jars are located and DOTS_JARS_ROOT refers to the root directory where the DOTS
related jars are located. Below, for each step the corresponding script that
can be used is given enclosed in brackets. In case you are using the scripts,
make sure that you have first run myenv.ksh or set the DERBY_JARS_ROOT and
DOTS_JARS_ROOT variables in the shell window that you are executing from.
($. ./myenv.ksh)
§ Open a new shell
window and if you are using the scripts, set DERBY_JARS_ROOT and DOTS_JARS_ROOT
appropriately
($. ./myenv.ksh)
§ Obtain derby jars – derby.jar, derbytools.jar ( for embedded mode)
For network server mode, also need derbynet.jar
Set classpath to derby jars.
export DERBY_JARS="/home/perftest/derby"
export CLASSPATH="$DERBY_JARS/derby.jar:$DERBY_JARS/derbytools.jar"
or
($. ./cpderby.ksh
)
§ Ensure JVM is in your path.
export
PATH="/home/perftest/jdk1.4.2_04/bin:${PATH}"
or
($. ./jvm.ksh)
§ Create the database in the current directory and load the schema and procedures
SQL files
o Clean database directory if already present
rm –rf TESTDB
or
($ . ./clean.ksh)
o Create database (TESTDB in location /home/perftest/derby)
java org.apache.derby.tools.ij
ij> connect ‘jdbc:derby:TESTDB;create=true;user=u1;password=p1’;
o Create tables
Check out the createtable_derby.sql in derbydotsscripts.tar.gz
$ java org.apache.derby.tools.ij
createtable_derby.sql
o Create procedures
Check out the createproc_derby.sql in derbydotsscripts.tar.gz
$ java org.apache.derby.tools.ij
createproc_derby.sql
All the steps above or ($. ./load.ksh)
After the derby database is created in the 4.2., things to note:
To
run Dots with
Since derby will run in embedded mode in same JVM as the DOTS client, derby.jar needs to be in the classpath when running the client program. Nothing else needs to be done.
To run Dots with Derby Network Server
Set classpath to derby.jar, derbynet.jar and the location where the DotsJavaProcedures.class is located (obtain it from derbydotsscripts.tar.gz). The DotsJavaProcedures.class contains the java procedures used in one of the test cases (BTCJ8).
export
CLASSPATH=".:$DERBY_JARS_ROOT/derby.jar:$DERBY_JARS_ROOT/derbynet.jar"
Ensure that JVM is in your path
Start the network server ( it will accept connections from the local host)
$ java org.apache.derby.drda.NetworkServerControl
start –h 0.0.0.0
(This example deals
with the server and client on one machine, but if network server needs to
accept connections from outside hosts, network server needs to run under
security manager)
The DOTS client program picks up the test configuration from the config.ini file by default or a file specified with the –config.ini parameter. So the next step is to configure the config.ini file as needed for a test run.
This file contains a list of configurable properties: duration of the test run, summary when results should be logged, the results directory, the number of max connections to be used, amount of max CPU usage etc. For a complete list and explanation of the properties, check out the DOTS User guide. The options of setting the max number of database connections as well as the max CPU load can be changed for different test runs.
For Derby, database connection URL must be edited in the config.ini file.
Using Derby embedded mode:
set DriverClass , dbUrl, UserId and Password appropriately.
An example:
UserID = u1
Password = p1
DriverClass = org.apache.derby.jdbc.EmbeddedDriver
URL = jdbc:derby:TESTDB
If the TESTDB is not in the same directory where the DOTS client is run, then the database name in the URL must point to the exact path.
Using
UserID = u1
Password = p1
DriverClass = com.ibm.db2.jcc.DB2Driver
URL
= jdbc:derby:net://database_server_ip/TESTDB
Here TESTDB is in the same directory where the network
server has been started. If not specify the absolute path for TESTDB on the
server side.
The derbydotsscripts.tar.gz file has derby.ini file which has derby specific connection attributes filled in, in case of the embedded mode else for network server mode, the derbyNS.ini can be used and modified. .
Some common options that are worth noting here are
DURATION
=
Duration you want the test to be run. The above setting sets the time for each run to be 5 minutes
SUMMARY_INTERVAL=
This value determines at the interval at which to gather the summary for the test run.
LOG_DIR=
This value determines where the test reports will be written out to.
The following two parameters are for the performance monitor component. Specify the IP address of the machine where the performance monitor is running (or going to be started for the test).
Default server_port is 8001.
SERVER_IP=9.30.134.81
SERVER_PORT=8001
1) Open a new shell, and start the performance monitor on the machine
on which the database system is.
Ensure classpath is set to PerfMon.jar
$java dots.perfmon.PerfMon
–port 8001
(For convenience purposes, there is a script perfmon.ksh in derbydotsscripts.tar.gz that can be used to start the performance monitor. Make sure to have run myenv.ksh in the shell window before perfmon.ksh)
2) Ensure database is created and loaded and if running against network server, check if the server is running.
If
running against
Set the classpath to
§ the derby jar,
§ to the java procedures implementation class file (DotsJavaProcedures.class in derbydotsscripts.tar.gz) and
§ Dots.jar
export
CLASSPATH=".:$DERBY_JARS_ROOT/derby.jar:$DOTS_JARS_ROOT/Dots.jar"
or
($. ./cpclient.ksh)
To run a test by name BTCJ1,
java dots.framework.Dots -config derby.ini -case BTCJ1
or
($. ./run.ksh)
If
running against
On the dots client side, set the classpath to
§ the network server client jars (db2jcc.jar, db2jcc_license_c.jar)
§ Dots.jar
To run a test by name BTCJ1,
java dots.framework.Dots -config derbyNS.ini -case BTCJ1
On a successful run, go to the path pointed by LOG_DIR in the config.ini (here the test run uses derby.ini) file.
There will be files with three different extensions.
Example of running the BTCJ1 test
BTCJ1-2004-8-31-13-43-40-271.err
BTCJ1-2004-8-31-13-43-40-271.sum