Class LogFileManager

java.lang.Object
org.objectweb.howl.log.LogObject
org.objectweb.howl.log.LogFileManager

class LogFileManager extends LogObject
Manage a set of log files. This class implements methods that can be called by LogBufferManager to obtain a LogFile for logger IO and to signal the LogFileManager when new buffers are being initialized. LogFileManager manages log files according to implementation specific policies. Some LogFileManagers may use a circular file policy while others may use a set of files. The most simple implementations will use a single file and allow it to grow as needed. QUESTION: do we need multiple implementations, or can we deal with different policies in this one class using configuration?
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    (package private) class 
    helper thread used invoke LogEventListener when log overflow (or other event) is about to occur.
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    (package private) long
    The log key for the oldest active entry in the log.
    (package private) boolean
    indicates whether log files will be marked automatically.
    (package private) final byte[]
    data written to log when autoMark is turned off
    (package private) final byte[]
    data written to log when autoMark is turned on
    (package private) LogBufferManager
    LogBufferManager used by methods that put() records to the log.
    (package private) byte[]
    end of line for log records to make logs readable in text editors.
    (package private) long
    last key returned by put().
    (package private) LogFile
     
    (package private) int
    event to be processed by the eventManagerThread.
    The LogEventListener registered by the application that owns the logger.
    (package private) final Object
    Monitor used by EventManager thread to wait for events that need to be notified.
    (package private) Thread
    Thread used by LogFileManager to send events to the registered LogEventListener.
    (package private) byte[][]
    LogFile header record.
    (package private) ByteBuffer
    ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.
    private final Object
    lock controlling access to LogFile.
    (package private) LogFile[]
    set of LogFile objects associated with the physical log files.
    private boolean
    Indicates that LogFileManager initialization is complete.
    private long
    the first log key generated by this instance of the Logger.
    (package private) int
    workerID to current entry in fileSet[]
    (package private) static final int
    Event types for event manager thread.
    private long
    The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.
    (package private) byte[][]
    MARK control Record.
    (package private) ByteBuffer
    ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.
    (package private) int
    maximum number of blocks to store in each LogFile.
    private int
    Number of times log overflow event was notified.
    (package private) boolean
    The automark value restored during log file initialization.

    Fields inherited from class org.objectweb.howl.log.LogObject

    config
  • Constructor Summary

    Constructors
    Constructor
    Description
    construct LogFileManager with Configuration supplied by caller.
  • Method Summary

    Modifier and Type
    Method
    Description
    (package private) void
    Gracefully close the log files.
    (package private) void
    Write a CLOSE record and shut down the buffer manager if we have one.
    private void
    Detect pending Log Overflow and notify event listener.
    (package private) long
    Returns the highMark from the LogFile that is currently being written.
    (package private) LogFile
    getLogFileForMark(long mark)
    Returns the LogFile that contains the requested mark .
    (package private) LogFile
    Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.
    (package private) String
    Returns an XML node containing statistics for the LogFileManager.
    (package private) void
    validate LogFiles and set member variables.
    (package private) long
    mark(long key)
    calls mark(key, false)
    (package private) long
    mark(long key, boolean force)
    sets the LogFile's mark.
    (package private) void
    open pool of LogFile(s).
    (package private) int
    read(LogBuffer lb, int bsn)
    reads a block of data into LogBuffer lb .
    (package private) long
    setAutoMark(boolean automark)
    Sets the LogFile marking mode.
    (package private) void
    setCurrentKey(long key)
    updates currentKey member variable.
    private boolean
    setLockOnFile(File name, boolean lock)
    Create a JVM wide lock on a File.
    (package private) void
    Registers a LogEventListener for log event notifications.
    (package private) void
    generates MARKKEY data record into supplied data parameter.
    (package private) void
    Compares values in log file header record with current configuration.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Field Details

    • maxBlocksPerFile

      int maxBlocksPerFile
      maximum number of blocks to store in each LogFile.

      controls when logging is switched to a new log file, and/or when a circular log is reset to seek address zero.

      See Also:
    • activeMark

      long activeMark
      The log key for the oldest active entry in the log.

      When automark is enabled (true) the activeMark is updated after every put() operation. When automark is disabled (as should be the case with JOTM) the activeMark is updated manually by a call to Logger.mark(), or during logOverflowNotification processing if a LogEventListener is registered.

      See Also:
    • automark

      boolean automark
      indicates whether log files will be marked automatically.

      When automark is false, the mark() method must be invoked by the log user. When automark is true, the Logger will automatically set the mark at the most recent record.

    • restartAutoMark

      boolean restartAutoMark
      The automark value restored during log file initialization.

      if a set of log files exist from a prior execution, the value of automark is set based on the log file header record processed during the init() processing. The value restored from the log file is saved in restartAutoMark so it can be reported by getStats().

    • currentKey

      long currentKey
      last key returned by put().

      updated by setCurrentKey(long)

    • initialKey

      private long initialKey
      the first log key generated by this instance of the Logger.

      Initialized by init() to the block immediately following the last block written in the prior execution.

    • autoMarkOn

      final byte[] autoMarkOn
      data written to log when autoMark is turned on
    • autoMarkOff

      final byte[] autoMarkOff
      data written to log when autoMark is turned off
    • fileManagerLock

      private final Object fileManagerLock
      lock controlling access to LogFile.
    • fileSet

      LogFile[] fileSet
      set of LogFile objects associated with the physical log files.
      See Also:
    • lfIndex

      int lfIndex
      workerID to current entry in fileSet[]
    • currentLogFile

      LogFile currentLogFile
    • fileHeader

      byte[][] fileHeader
      LogFile header record.

      protected by fileManagerLock

      The first record of every LogFile is a FILE_HEADER record containing information that is used during recovery to reposition the log file and replay records starting from the active mark. byte[1] autoMark byte[1] long activeMark byte[8] global to all log files long lowMark byte[8] low mark for current file == high mark for previous file long prevSwitchTod byte[8] time of previous file switch int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]

    • fileHeaderBB

      ByteBuffer fileHeaderBB
      ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.

      protected by fileManagerLock

    • markRecord

      byte[][] markRecord
      MARK control Record.

      A MARK control record containing the current state of the automark mode and the active mark is stored at the beginning of every block. This strategy allows the logger to read through a log file looking only at the BSN for each block to locate the last written block. Once the last written block is located, the current state of the MARK can be obtained by examining the MARK record at the beginning of the block.

      To avokd garbage collection for a byte[] and ByteBuffer wrapper for each physical block, the class has a single markRecord member that is protected by fileManagerLock.

      protected by fileManagerLock byte[1] autoMark byte[1] long activeMark byte[8] global to all log files int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]

      See Also:
    • markRecordBB

      ByteBuffer markRecordBB
      ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.

      protected by fileManagerLock

    • crlf

      byte[] crlf
      end of line for log records to make logs readable in text editors.
    • bmgr

      LogBufferManager used by methods that put() records to the log.

      For example, the mark() method uses bmgr to put() mark information to the log.

    • eventListener

      private LogEventListener eventListener
      The LogEventListener registered by the application that owns the logger. If eventListener is null then the application is not notified of log events.
    • eventManagerLock

      final Object eventManagerLock
      Monitor used by EventManager thread to wait for events that need to be notified.
    • eventManagerThread

      Thread eventManagerThread
      Thread used by LogFileManager to send events to the registered LogEventListener.
    • event

      int event
      event to be processed by the eventManagerThread.

      When a condition is encountered that requires the LogEventListener to be notified, the value is set to one of the event types.

    • LOG_OVERFLOW_EVENT

      static final int LOG_OVERFLOW_EVENT
      Event types for event manager thread.

      assign this value to the event member when invoking the event manager thread to handle a log overflow notification.

      See Also:
    • lowestSafeLogKey

      private long lowestSafeLogKey
      The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.
    • overflowNotificationCount

      private int overflowNotificationCount
      Number of times log overflow event was notified.
      See Also:
    • initComplete

      private boolean initComplete
      Indicates that LogFileManager initialization is complete.

      Prior to being fully initialized, the LogFileManager should not attempt to put records to the journal because the buffer manager may not be initialized properly.

  • Constructor Details

    • LogFileManager

      LogFileManager(Configuration config)
      construct LogFileManager with Configuration supplied by caller.
      Parameters:
      config - Configuration object.
  • Method Details

    • getLogFileForMark

      LogFile getLogFileForMark(long mark)
      Returns the LogFile that contains the requested mark .

      Called by LogBufferManager to locate a log file needed for a replay() request.

      Parameters:
      mark - A log key previously returned by LogBufferManager.put(). The log key is used to compute the desired file.
      Returns:
      LogFile containing the requested mark .

      Returns null if none of the files in fileSet[] contain the requested mark.

    • getLogFileForWrite

      LogFile getLogFileForWrite(LogBuffer lb) throws LogFileOverflowException
      Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.

      The buffer sequence number of the LogBuffer parameter ( lf.bsn ) represents an implementation specific value that is used to manage log file space. As buffers are written to disk the buffer sequence number is incremented. The LogFileManager is able to compute the seek address for a buffer as a function of lf.bsn and buffer size when using LogBuffer implementations with fixed buffer sizes.

      In all cases, getLogFile records a header record into the buffer containing the current state of the automark mode and the current active mark.

      Parameters:
      lb - LogBuffer that is asking for the LogFile. LogFileManager implementations use lf.bsn to determine when to switch to a new file, or wrap a circular file back to seek address zero.
      Returns:
      a LogFile to use for writing the LogBuffer
      Throws:
      LogFileOverflowException
    • detectLogOverflow

      private void detectLogOverflow(int bsn)
      Detect pending Log Overflow and notify event listener.

      if current file is 50% full then we check to see if the next file contains the active mark. We notify the event listener to move records forward to prevent log overflow.

      Called by: getLogFileForWrite(Logbuffer lb)

      Parameters:
      bsn - The block sequence number of the current LogBuffer.
      See Also:
    • setMarkData

      void setMarkData(ByteBuffer data)
      generates MARKKEY data record into supplied data parameter.
      Parameters:
      data - ByteBuffer wrapping the target byte[]
    • mark

      long mark(long key, boolean force) throws InvalidLogKeyException, IOException, InterruptedException
      sets the LogFile's mark.

      mark() provides a generalized method for callers to inform the Logger that log space can be released for reuse.

      writes a MARKKEY control record to the log.

      Parameters:
      key - is an opaque log key returned by a previous call to put().
      force - when set true causes the caller to be blocked until the new force record is forced to disk.
      Returns:
      log key for the MARK record
      Throws:
      InvalidLogKeyException - if key parameter is out of range. key must be greater than current activeMark and less than the most recent key returned by put().
      IOException
      InterruptedException
    • mark

      calls mark(key, false)
      Parameters:
      key - a log key as described by mark(long,boolean)
      Returns:
      log key of the new mark record.
      Throws:
      InvalidLogKeyException
      IOException
      InterruptedException
      See Also:
    • read

      int read(LogBuffer lb, int bsn) throws IOException, InvalidLogBufferException
      reads a block of data into LogBuffer lb .

      Amount of data read is determined by lb.capacity().

      sets lb.lf with the fileSet[] entry that contains the requested BSN.

      Parameters:
      lb - LogBuffer to read data into.
      bsn - block sequence number of the block to be read. File position is computed as follows:
      1. locate the entry within fileSet[] that contains the requested bsn.
      2. compute position as (requested bsn - first bsn in file) * block size;
      Returns:
      block serial number of block read.

      returns -1 if the requested BSN does not exist in the current fileSet[].

      Throws:
      IOException
      InvalidLogBufferException
    • setAutoMark

      Sets the LogFile marking mode.

      writes an AUTOMARK control record to the log if the log is open.

      Parameters:
      automark - true to indicate automatic marking.
      Returns:
      log key for the generated MARK control record.
      Throws:
      InvalidLogKeyException
      IOException
      InterruptedException
      LogFileOverflowException
    • setCurrentKey

      void setCurrentKey(long key)
      updates currentKey member variable.

      Method must be synchronized to guarantee that only keys with larger values are assigned to currentKey.

      If automark mode is enabled, then the activeMark is updated as well.

      Parameters:
      key - a log key returned by the buffer manager.
    • getHighMark

      long getHighMark()
      Returns the highMark from the LogFile that is currently being written.

      Log keys greater than this value are beyond the logical end of the journal.

      Returns:
      highMark from the current LogFile.
    • setLogEventListener

      void setLogEventListener(LogEventListener eventListener)
      Registers a LogEventListener for log event notifications.
      Parameters:
      eventListener - object to be notified of logger events.
    • setLockOnFile

      private boolean setLockOnFile(File name, boolean lock)
      Create a JVM wide lock on a File.

      Feature 30922

      The java.nio.channel.tryLock method is not guaranteed to respect locks that are set within the JVM by multiple threads. Consequently, it is possible for an application to create multiple instances of Logger resulting in corrupted log files as two separate instances of Logger write to the log.

      To prevent multiple instances of Logger from allocating the same files within a JVM, we create a system property

      Parameters:
      name - File object to be locked
      Returns:
      true if requested lock operation is successful.
    • open

      open pool of LogFile(s).
      Throws:
      FileNotFoundException
      LogConfigurationException
      IOException
      InvalidFileSetException
    • init

      validate LogFiles and set member variables.

      activeMark set based on last block written during previous execution.

      currentKey set to key of the last record written to the log.

      currentLogFile set to the next available LogFile in fileSet[] with file position set to resume writing at the next block following the last block written.

      Throws:
      IOException
      LogConfigurationException
      InvalidLogBufferException
      InterruptedException
    • validateFileHeader

      Compares values in log file header record with current configuration.

      Throws LogConfigurationException if header does not match current configuration.

      Throws:
      LogConfigurationException
      IOException
      InvalidLogBufferException
    • closeBufferManager

      void closeBufferManager() throws InterruptedException, IOException
      Write a CLOSE record and shut down the buffer manager if we have one.

      This routine was originally inline in close(). It was refactored into a separate method to improve readability of close().

      Throws:
      InterruptedException
      IOException
    • close

      void close() throws IOException, InterruptedException
      Gracefully close the log files.
      Throws:
      IOException - If FileChannel.close() encounters an error.
      InterruptedException
      See Also:
    • getStats

      String getStats()
      Returns an XML node containing statistics for the LogFileManager.

      The nested element contain entries for each LogFile object in the set of log files.

      Returns:
      a String containing statistics as an XML node.