|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.apache.derby.impl.store.raw.log.LogToFile
public final class LogToFile
This is an implementation of the log using a non-circular file system file. No support for incremental log backup or media recovery. Only crash recovery is supported.
The 'log' is a stream of log records. The 'log' is implemented as a series of numbered log files. These numbered log files are logically continuous so a transaction can have log records that span multiple log files. A single log record cannot span more then one log file. The log file number is monotonically increasing.
The log belongs to a log factory of a RawStore. In the current implementation, each RawStore only has one log factory, so each RawStore only has one log (which composed of multiple log files). At any given time, a log factory only writes new log records to one log file, this log file is called the 'current log file'.
A log file is named loglogNumber.dat
Everytime a checkpoint is taken, a new log file is created and all subsequent log records will go to the new log file. After a checkpoint is taken, old and useless log files will be deleted.
RawStore exposes a checkpoint method which clients can call, or a checkpoint is taken automatically by the RawStore when
This LogFactory is responsible for the formats of 2 kinds of file: the log file and the log control file. And it is responsible for the format of the log record wrapper.
Format of log control file
| Format ID | FILE_STREAM_LOG_FILE |
| Purpose | The log control file contains information about which log files are present and where the last checkpoint log record is located. |
| Upgrade | |
| Disk Layout | (pre-v15) int format id int log file version long the log instant (LogCounter) of the last completed checkpoint (v15 onward) int format id int obsolete log file version long the log instant (LogCounter) of the last completed checkpoint int JBMS version int checkpoint interval long spare (value set to 0) long spare (value set to 0) long spare (value set to 0) |
| Field Summary | |
|---|---|
private int |
action
|
private StorageFile |
activeFile
|
private java.lang.String |
activePerms
|
private long |
allowedToReadFileNumber
|
private boolean |
backupInProgress
|
(package private) long |
bootTimeLogFileNumber
|
private static int |
CHECKPOINT_INTERVAL_MAX
|
private static int |
CHECKPOINT_INTERVAL_MIN
|
private DaemonService |
checkpointDaemon
|
private boolean |
checkpointDaemonCalled
|
(package private) long |
checkpointInstant
|
private int |
checkpointInterval
|
private java.util.zip.CRC32 |
checksum
|
protected StandardException |
corrupt
If not null then something is corrupt in the raw store and this represents the original error. |
private CheckpointOperation |
currentCheckpoint
|
private boolean |
databaseEncrypted
|
(package private) java.lang.String |
dataDirectory
|
protected DataFactory |
dataFactory
|
static java.lang.String |
DBG_FLAG
|
private static int |
DEFAULT_CHECKPOINT_INTERVAL
|
private static java.lang.String |
DEFAULT_LOG_ARCHIVE_DIRECTORY
|
private static int |
DEFAULT_LOG_BUFFER_SIZE
|
private static int |
DEFAULT_LOG_SWITCH_INTERVAL
|
static java.lang.String |
DUMP_LOG_FROM_LOG_FILE
|
static java.lang.String |
DUMP_LOG_ONLY
|
protected long |
endPosition
|
private static int |
fid
|
private StorageRandomAccessFile |
firstLog
|
(package private) long |
firstLogFileNumber
|
private boolean |
inCheckpoint
|
private boolean |
inLogSwitch
|
private boolean |
inRedo
|
private boolean |
inReplicationMasterMode
|
private boolean |
inReplicationSlaveMode
|
private boolean |
inReplicationSlavePreMode
True if the database has been booted in replication slave pre mode, effectively turning off writes to the log file. |
private static byte |
IS_BETA_FLAG
|
private static byte |
IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
When the derby.system.durability property is set to 'test', the store system will not force sync calls in the following cases - for the log file at each commit - for the log file before data page is forced to disk - for page allocation when file is grown - for data writes during checkpoint This means it is possible that the recovery system may not work properly, committed transactions may be lost, and/or database may not be in a consistent state. |
private boolean |
isFrozen
If frozen, don't allow anything on disk to change. |
private boolean |
isWriteSynced
Note: Why logging system support file sync and write sync ? |
(package private) ProductVersionHolder |
jbmsVersion
Product Version information. |
private boolean |
jvmSyncErrorChecked
Status for whether the check on the sync error on some JVMs has been done or not. |
private boolean |
keepAllLogs
|
(package private) long |
lastFlush
|
private static int |
LOG_BUFFER_SIZE_MAX
|
private static int |
LOG_BUFFER_SIZE_MIN
|
protected static int |
LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
|
static int |
LOG_FILE_HEADER_SIZE
|
static int |
LOG_RECORD_OVERHEAD
|
private static int |
LOG_SWITCH_INTERVAL_MAX
|
private static int |
LOG_SWITCH_INTERVAL_MIN
|
protected static java.lang.String |
LOG_SYNC_STATISTICS
|
private boolean |
logArchived
|
private boolean |
logBeingFlushed
|
private int |
logBufferSize
|
(package private) java.lang.String |
logDevice
|
(package private) long |
logFileNumber
|
private long |
logFileToBackup
|
private boolean |
logNotSynced
|
protected LogAccessFile |
logOut
|
private WritableStorageFactory |
logStorageFactory
|
private int |
logSwitchInterval
|
private boolean |
logSwitchRequired
|
private long |
logWrittenFromLastCheckPoint
|
private MasterFactory |
masterFactory
|
private long |
maxLogFileNumber
|
private int |
mon_flushCalls
|
private boolean |
mon_LogSyncStatistics
|
private int |
mon_numBytesToLog
|
private int |
mon_numLogFlushWaits
|
private int |
mon_syncCalls
|
private int |
myClientNumber
|
private static int |
OBSOLETE_LOG_VERSION_NUMBER
|
private boolean |
onDiskBeta
|
private int |
onDiskMajorVersion
On disk database version information. |
private int |
onDiskMinorVersion
|
private RawStoreFactory |
rawStoreFactory
|
protected boolean |
ReadOnlyDB
|
private boolean |
recoveryNeeded
|
private StandardException |
replicationSlaveException
If this exception is set while in replication slave mode, the exception will be thrown by the thread doing recovery will. |
private java.lang.Object |
slaveRecoveryMonitor
|
private boolean |
stopped
|
static java.lang.String |
TEST_LOG_FULL
Set to true if we want to simulate a log full condition |
static java.lang.String |
TEST_LOG_INCOMPLETE_LOG_WRITE
Set to true if we want the up comming log record to be only partially written. |
static java.lang.String |
TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES
Set to the number of bytes we want the next log record to actually write out, only used when TEST_LOG_INCOMPLETE_LOG_WRITE is on. |
static java.lang.String |
TEST_LOG_SWITCH_LOG
Set to true if we want the checkpoint to only switch the log but not actually do the checkpoint |
(package private) int |
test_logWritten
DEBUG test only |
static java.lang.String |
TEST_MAX_LOGFILE_NUMBER
Set to true if we want to simulate max possible log file number is being used. |
(package private) int |
test_numRecordToFillLog
|
static java.lang.String |
TEST_RECORD_TO_FILL_LOG
Set to the number of log record we want to write before the log is simulated to be full. |
static java.lang.String |
TEST_SWITCH_LOG_FAIL1
Set to true if we want to simulate a log full condition while switching log |
static java.lang.String |
TEST_SWITCH_LOG_FAIL2
|
private java.io.File |
toFile
|
private static boolean |
wasDBInDurabilityTestModeNoSync
keeps track of if the database was booted previously at any time with derby.system.durability=test |
| Fields inherited from interface org.apache.derby.iapi.store.raw.log.LogFactory |
|---|
LOG_DIRECTORY_NAME, MODULE, RT_READONLY, RUNTIME_ATTRIBUTES |
| Fields inherited from interface org.apache.derby.iapi.services.daemon.Serviceable |
|---|
DONE, REQUEUE |
| Constructor Summary | |
|---|---|
LogToFile()
MT- not needed for constructor |
|
| Method Summary | |
|---|---|
void |
abortLogBackup()
|
long |
appendLogRecord(byte[] data,
int offset,
int length,
byte[] optionalData,
int optionalDataOffset,
int optionalDataLength)
Append length bytes of data to the log prepended by a long log instant and followed by 4 bytes of length information. |
private void |
backupLogFiles(java.io.File toDir,
long lastLogFileToBackup)
|
void |
boot(boolean create,
java.util.Properties startParams)
Boot up the log factory. |
boolean |
canSupport(java.util.Properties startParams)
See if this implementation can support any attributes that are listed in properties. |
private void |
checkCorrupt()
|
protected void |
checkForReplication(LogAccessFile log)
Used by LogAccessFile to check if it should take the replication master role, and thereby send log records to the MasterFactory. |
private boolean |
checkJvmSyncError(StorageFile logFile)
In Java 1.4.2 and newer rws and rwd modes for RandomAccessFile are supported. |
boolean |
checkpoint(RawStoreFactory rsf,
DataFactory df,
TransactionFactory tf,
boolean wait)
Checkpoint the rawStore. |
void |
checkpointInRFR(LogInstant cinstant,
long redoLWM,
long undoLWM,
DataFactory df)
redo a checkpoint during rollforward recovery |
protected boolean |
checkpointWithTran(RawTransaction cptran,
RawStoreFactory rsf,
DataFactory df,
TransactionFactory tf)
checkpoint with pre-start transaction |
(package private) boolean |
checkVersion(int requiredMajorVersion,
int requiredMinorVersion)
Check to see if a database has been upgraded to the required level in order to use a store feature. |
boolean |
checkVersion(int requiredMajorVersion,
int requiredMinorVersion,
java.lang.String feature)
Check to see if a database has been upgraded to the required level in order to use a store feature. |
private void |
createLogDirectory()
Create the directory where transaction log should go. |
protected long |
currentInstant()
Get the current log instant - this is the log instant of the Next log record to be written out MT - This method is synchronized to ensure that it always points to the end of a log record, not the middle of one. |
boolean |
databaseEncrypted()
|
int |
decrypt(byte[] ciphertext,
int offset,
int length,
byte[] cleartext,
int outputOffset)
|
void |
deleteLogFileAfterCheckpointLogFile()
|
private void |
deleteObsoleteLogfiles()
|
void |
deleteOnlineArchivedLogFiles()
|
void |
disableLogArchiveMode()
|
void |
enableLogArchiveMode()
|
int |
encrypt(byte[] cleartext,
int offset,
int length,
byte[] ciphertext,
int outputOffset)
|
void |
endLogBackup(java.io.File toDir)
|
protected long |
endPosition()
|
void |
failoverSlave()
Used to make the slave stop appending log records, complete recovery and boot the database. |
private CheckpointOperation |
findCheckpoint(long checkpointInstant,
FileLogger logger)
|
private long |
firstLogInstant()
Get the first valid log instant - this is the beginning of the first log file MT- synchronized on this |
void |
flush(LogInstant where)
Flush all unwritten log record up to the log instance indicated to disk and sync. |
protected void |
flush(long fileNumber,
long wherePosition)
Flush the log such that the log record written with the instant wherePosition is guaranteed to be on disk. |
void |
flushAll()
Flush all unwritten log record to disk and sync. |
private void |
flushBuffer(long fileNumber,
long wherePosition)
Flush all unwritten log record up to the log instance indicated to disk without syncing. |
void |
freezePersistentStore()
Backup restore - stop sending log record to the log stream |
java.lang.String |
getCanonicalLogPath()
Return the canonical directory of the PARENT of the log directory. |
private StorageFile |
getControlFileName()
Return the control file name MT- read only |
int |
getEncryptedDataLength(int length)
returns the length that will make the data to be multiple of encryption block size based on the given length. |
int |
getEncryptionBlockSize()
return the encryption block size used during encrypted db creation |
private long |
getFirstLogNeeded(CheckpointOperation checkpoint)
Return the "oldest" log file still needed by recovery. |
LogInstant |
getFirstUnflushedInstant()
Get the instant of the first record which was not flushed. |
long |
getFirstUnflushedInstantAsLong()
Get the log instant long value of the first log record that has not been flushed. |
StorageFile |
getLogDirectory()
Return the location of the log directory. |
private java.lang.String |
getLogDirPath(StorageFile logDir)
|
void |
getLogFactoryProperties(PersistentSet set)
Get JBMS properties relavent to the log factory |
protected StorageRandomAccessFile |
getLogFileAtBeginning(long filenumber)
Open a log file and position the file at the beginning. |
protected StorageRandomAccessFile |
getLogFileAtPosition(long logInstant)
Get a read-only handle to the log file positioned at the stated position MT- read only |
private StorageFile |
getLogFileName(long filenumber)
Given a log file number, return its file name MT- read only |
private long |
getLogFileNumber()
Return the current log file number. |
StorageRandomAccessFile |
getLogFileToSimulateCorruption(long filenum)
Get the log file to Simulate a log corruption FOR UNIT TESTING USAGE ONLY |
Logger |
getLogger()
MT- not needed |
private void |
getLogStorageFactory()
|
private StorageFile |
getMirrorControlFileName()
Return the mirror control file name MT- read only |
int |
getTypeFormatId()
Return my format identifier. |
void |
initializeReplicationSlaveRole()
Initializes logOut so that log received from the replication master can be appended to the log file. |
private boolean |
initLogFile(StorageRandomAccessFile newlog,
long number,
long prevLogRecordEndInstant)
Initialize the log to the correct format with the given version and log file number. |
boolean |
inReplicationMasterMode()
Used to determine if the replication master mode has been started, and the logging for unlogged operations needs to be enabled. |
boolean |
inRFR()
|
boolean |
isCheckpointInLastLogFile()
|
boolean |
logArchived()
Backup restore - is the log being archived to some directory? |
protected void |
logErrMsg(java.lang.String msg)
Print error message to user about the log MT - not needed, informational only |
protected void |
logErrMsg(java.lang.Throwable t)
Print error message to user about the log MT - not needed, informational only |
private void |
logErrMsgForDurabilityTestModeNoSync()
In case of boot errors, and if database is either booted with derby.system.durability=test or was previously at any time booted in this mode, mention in the error message that the error is probably because the derby.system.durability was set. |
private long |
logtest_appendPartialLogRecord(byte[] data,
int offset,
int length,
byte[] optionalData,
int optionalDataOffset,
int optionalDataLength)
Writes out a partial log record - takes the appendLogRecord. |
StandardException |
markCorrupt(StandardException originalError)
Once the log factory is makred as corrupt then the raw sto |
protected LogScan |
openBackwardsScan(LogInstant stopAt)
Scan backward from end of log. |
protected LogScan |
openBackwardsScan(long startAt,
LogInstant stopAt)
Scan backward from start position. |
ScanHandle |
openFlushedScan(DatabaseInstant start,
int groupsIWant)
Get a ScanHandle to scan flushed records from the log. |
LogScan |
openForwardsFlushedScan(LogInstant startAt)
Open a forward scan of the transaction log. |
LogScan |
openForwardsScan(LogInstant startAt,
LogInstant stopAt)
Get a forwards scan |
protected LogScan |
openForwardsScan(long startAt,
LogInstant stopAt)
Scan Forward from start position. |
private StorageRandomAccessFile |
openLogFileInWriteMode(StorageFile logFile)
open the given log file name for writes; if file can not be be opened in write sync mode then disable the write sync mode and open the file in "rw" mode. |
int |
performWork(ContextManager context)
Do whatever it is that you want the daemon to do for you. |
private void |
preAllocateNewLogFile(StorageRandomAccessFile log)
|
private void |
printErrorStack(java.lang.Throwable t)
print stack trace from the Throwable including its nested exceptions |
protected boolean |
privCanWrite(StorageFile file)
|
private boolean |
privCopyFile(java.io.File from,
StorageFile to)
|
private boolean |
privCopyFile(StorageFile from,
java.io.File to)
|
protected boolean |
privDelete(StorageFile file)
|
protected boolean |
privExists(StorageFile file)
|
private java.lang.String[] |
privList(java.io.File file)
|
private java.lang.String[] |
privList(StorageFile file)
|
protected boolean |
privMkdirs(StorageFile file)
|
private StorageRandomAccessFile |
privRandomAccessFile(StorageFile file,
java.lang.String perms)
|
private boolean |
privRemoveDirectory(StorageFile file)
|
private long |
readControlFile(StorageFile logControlFileName,
java.util.Properties startParams)
|
void |
recover(RawStoreFactory rsf,
DataFactory df,
TransactionFactory tf)
Recover the rawStore to a consistent state using the log. |
private boolean |
restoreLogs(java.util.Properties properties)
This function restores logs based on the following attributes are specified on connection URL: Attribute.CREATE_FROM (Create database from backup if it does not exist) Attribute.RESTORE_FROM (Delete the whole database if it exists and then restore it from backup) Attribute.ROLL_FORWARD_RECOVERY_FROM:(Perform Rollforward Recovery; except for the log directory everthing else is replced by the copy from backup. log files in the backup are copied to the existing online log directory. |
java.lang.Object |
run()
|
private boolean |
runBooleanAction(int action,
StorageFile file)
|
boolean |
serviceASAP()
If this work should be done as soon as possible, then return true. |
boolean |
serviceImmediately()
If this work should be done immediately on the user thread then return true. |
void |
setDatabaseEncrypted(boolean flushLog)
|
void |
startLogBackup(java.io.File toDir)
|
void |
startNewLogFile()
|
void |
startReplicationMasterRole(MasterFactory masterFactory)
Make this LogFactory pass log records to the MasterFactory every time a log record is appended to the log on disk, and notify the MasterFactory when a log disk flush has taken place. |
void |
stop()
Stop the log factory MT- caller provide synchronization (RESOLVE: this should be called AFTER dataFactory and transFactory are stopped) |
void |
stopReplicationMasterRole()
Stop this LogFactory from passing log records to the MasterFactory and from notifying the MasterFactory when a log disk flush has taken place. |
void |
stopReplicationSlaveRole()
Stop the slave functionality for this LogFactory. |
void |
switchLogFile()
Switch to the next log file if possible. |
private void |
syncFile(StorageRandomAccessFile raf)
Utility routine to call sync() on the input file descriptor. |
protected void |
testLogFull()
Simulate a log full condition if TEST_LOG_FULL is set to true, then the property TEST_RECORD_TO_FILL_LOG indicates the number of times this function is call before an IOException simulating a log full condition is raised. |
private void |
truncateLog(CheckpointOperation checkpoint)
Get rid of old and unnecessary log files MT- only one truncate log is allowed to be taking place at any given time. |
private void |
truncateLog(long firstLogNeeded)
Get rid of old and unnecessary log files |
void |
unfreezePersistentStore()
Backup restore - start sending log record to the log stream |
private boolean |
verifyLogFormat(StorageFile logFileName,
long number)
Verify that we the log file is of the right format and of the right version and log file number. |
private boolean |
verifyLogFormat(StorageRandomAccessFile log,
long number)
Verify that we the log file is of the right format and of the right version and log file number. |
(package private) boolean |
writeControlFile(StorageFile logControlFileName,
long value)
Carefully write out this value to the control file. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
private static int fid
public static final int LOG_FILE_HEADER_SIZE
protected static final int LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
public static final int LOG_RECORD_OVERHEAD
public static final java.lang.String DBG_FLAG
public static final java.lang.String DUMP_LOG_ONLY
public static final java.lang.String DUMP_LOG_FROM_LOG_FILE
protected static final java.lang.String LOG_SYNC_STATISTICS
private static final int OBSOLETE_LOG_VERSION_NUMBER
private static final int DEFAULT_LOG_SWITCH_INTERVAL
private static final int LOG_SWITCH_INTERVAL_MIN
private static final int LOG_SWITCH_INTERVAL_MAX
private static final int CHECKPOINT_INTERVAL_MIN
private static final int CHECKPOINT_INTERVAL_MAX
private static final int DEFAULT_CHECKPOINT_INTERVAL
private static final int DEFAULT_LOG_BUFFER_SIZE
private static final int LOG_BUFFER_SIZE_MIN
private static final int LOG_BUFFER_SIZE_MAX
private int logBufferSize
private static final byte IS_BETA_FLAG
private static final byte IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
This value is written as part of the log control file flags byte.,
Constant Field Valuesprivate static boolean wasDBInDurabilityTestModeNoSync
private static final java.lang.String DEFAULT_LOG_ARCHIVE_DIRECTORY
private int logSwitchInterval
private int checkpointInterval
java.lang.String dataDirectory
private WritableStorageFactory logStorageFactory
private boolean logBeingFlushed
protected LogAccessFile logOut
private StorageRandomAccessFile firstLog
protected long endPosition
long lastFlush
long logFileNumber
long bootTimeLogFileNumber
long firstLogFileNumber
private long maxLogFileNumber
private CheckpointOperation currentCheckpoint
long checkpointInstant
private DaemonService checkpointDaemon
private int myClientNumber
private volatile boolean checkpointDaemonCalled
private long logWrittenFromLastCheckPoint
private RawStoreFactory rawStoreFactory
protected DataFactory dataFactory
protected boolean ReadOnlyDB
private MasterFactory masterFactory
private boolean inReplicationMasterMode
private boolean inReplicationSlaveMode
private volatile StandardException replicationSlaveException
private boolean inReplicationSlavePreMode
SlaveFactoryprivate java.lang.Object slaveRecoveryMonitor
private long allowedToReadFileNumber
private boolean keepAllLogs
private boolean databaseEncrypted
private boolean recoveryNeeded
private boolean inCheckpoint
private boolean inRedo