Class LogBuffer

java.lang.Object
org.objectweb.howl.log.LogObject
org.objectweb.howl.log.LogBuffer
Direct Known Subclasses:
BlockLogBuffer

abstract class LogBuffer extends LogObject
Classes used as buffers in LogBufferManager must implement this interface.

This abstract class implements methods common to all LogBuffer sub-classes.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    (package private) int
    buffer sequence number.
    (package private) final ByteBuffer
     
    (package private) int
    Number of used data bytes in the buffer.
    (package private) final Adler32
     
    (package private) byte[]
    Local buffer used to compute checksums.
    (package private) boolean
    switch to enable computation of checksum.
    (package private) boolean
    set true by LogFileManager if the buffer must be forced due to file switch.
    (package private) int
    buffer number used by owner (LogBufferManager) to workerID into an array of buffers.
    (package private) IOException
    IOException from last write
    (package private) int
    results of last write.
    (package private) LogFile
    LogFile associated with the current LogBuffer.
    (package private) String
    name of this buffer object.
    (package private) boolean
    set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.
    (package private) long
    currentTimeMillis that buffer was initialized.
    protected int
    number of waiting threads.
    (package private) final Object
    mutex for synchronizing access to waitingThreads

    Fields inherited from class org.objectweb.howl.log.LogObject

    config
  • Constructor Summary

    Constructors
    Constructor
    Description
    default constructor.
  • Method Summary

    Modifier and Type
    Method
    Description
    (package private) int
    Computes a checksum over the the entire byte buffer backing this LogBuffer object.
    (package private) String
    May be used in traces or other purposes for debugging.
    (package private) abstract String
    returns statistics for this LogBuffer object.
    (package private) final int
    returns the number of threads currently waiting for the buffer to be forced to disk.
    (package private) abstract LogBuffer
    init(int bsn, LogFileManager lfm)
    initialize members for LogBuffer implementation class for reuse.
    (package private) abstract long
    put(short type, byte[][] data, boolean sync)
    puts a data record into the buffer and returns a token for record.
    (package private) abstract LogBuffer
    read(LogFile lf, long position)
    read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.
    (package private) final int
    decrements count of waiting threads and returns updated value.
    (package private) abstract boolean
    returns true if the buffer should be forced to disk.
    (package private) final void
    park threads that are waiting for the ByteBuffer to be forced to disk.
    (package private) abstract void
    write ByteBuffer to the LogFile.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Field Details

    • buffer

      final ByteBuffer buffer
    • index

      int index
      buffer number used by owner (LogBufferManager) to workerID into an array of buffers.

      Actual use of workerID is determined by the buffer manager implementation.

    • tod

      long tod
      currentTimeMillis that buffer was initialized.

      Implementations of LogBuffer should provide some means to persist tod to the file to allow recovery operations to determine when a buffer was written.

      Used during replay situations to validate block integrity. The TOD field from the block header is compared with the TOD field of the block footer to verify that an entire block was written.

    • waitingThreads

      protected int waitingThreads
      number of waiting threads.

      Always synchronized on (waitingThreadsLock). Design Note:
      Originally this field was volatile. It was changed at the suggestion of David Jencks to protected.

      We tried using (this) to protect access, but this caused some additional lock contention and performance fell off about 10%.

    • waitingThreadsLock

      final Object waitingThreadsLock
      mutex for synchronizing access to waitingThreads
    • iostatus

      int iostatus
      results of last write.

      Value must be one of the constants defined in LogBuffer interface.

    • bsn

      int bsn
      buffer sequence number.

      LogBufferManager maintains a sequence number of buffers written. The sequence number is stored in the block header of each log block.

      Initialized to zero.

      Set to -1 by read() if bytes read is -1 (end of file)

    • rewind

      boolean rewind
      set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.
    • forceNow

      boolean forceNow
      set true by LogFileManager if the buffer must be forced due to file switch.

      BUG: 300505

    • ioexception

      IOException ioexception
      IOException from last write
    • name

      String name
      name of this buffer object.
    • lf

      LogFile associated with the current LogBuffer.

      The LogBufferManager will have a pool of LogBuffers. If the containing Logger is managing a pool of files, then it is possible that some period of time, some buffers will be written to one file, while other buffers are written to another file. To allow writes and forces to separate files to be performed in parallel, each LogBuffer must keep track of its own LogFile.

      See Also:
    • doChecksum

      boolean doChecksum
      switch to enable computation of checksum.

      Since it takes some CPU to compute checksums over a buffer, it might be useful to disable the checksum, at least for performance measurements.

      When doChecksum is true then an implementation class should compute a checksum and store the value in the buffer during the write() method.

      Use of checksums is optional and depends on the actual implementation class.

    • bytesUsed

      int bytesUsed
      Number of used data bytes in the buffer.

      This is different than buffer capacity(). The bytes used is the number of bytes of usable data in a buffer. Bytes between the bytes used count and the buffer footer are undefined, possibly uninitialized or residue.

      set by operations that read data from files into the buffer such as read().

      checked by operations that retrieve logical records from the buffer get().

    • checksumBuffer

      byte[] checksumBuffer
      Local buffer used to compute checksums. The initial implementation of HOWL used the ByteBuffer.hashCode() method to compute a checksum. Since each vendor's implementation of the JVM is not guaranteed to generate the same value for hashCode, this approach is not portable, and could result in journal files that appear to be invalid if restarted using a JVM by a vendor that is different than the JVM that was used to write the journal initially. The problem is solved by implementing our own checksum method. The checksumBuffer member is used to obtain a local copy of the ByteBuffer contents for purposses of computing the checksum. Instead of obtaining each byte individually, the entire buffer is retrieved into checksumBuffer for computing the checksum.
    • checksum

      final Adler32 checksum
  • Constructor Details

    • LogBuffer

      LogBuffer(Configuration config)
      default constructor.

      after creating a new instance of LogBuffer the caller must invoke config().

  • Method Details

    • release

      final int release()
      decrements count of waiting threads and returns updated value.
      Returns:
      number of threads still waiting after the release.
    • getWaitingThreads

      final int getWaitingThreads()
      returns the number of threads currently waiting for the buffer to be forced to disk.
      Returns:
      current value of waitingThreads
    • sync

      final void sync() throws IOException, InterruptedException
      park threads that are waiting for the ByteBuffer to be forced to disk.

      The count of waiting threads ( waitingThreads ) has been incremented in put() .

      Throws:
      IOException
      InterruptedException
    • checksum

      int checksum()
      Computes a checksum over the the entire byte buffer backing this LogBuffer object.
      Returns:
      the computed checksum.
    • getName

      String getName()
      May be used in traces or other purposes for debugging.
      Returns:
      name of LogBuffer object.
    • init

      abstract LogBuffer init(int bsn, LogFileManager lfm) throws LogFileOverflowException
      initialize members for LogBuffer implementation class for reuse.

      LogBufferManager maintains a pool of LogBuffer objects. Each time a LogBuffer is allocated from the pool for use as the current collection buffer, the init() routine is called. After performing necessary initialization, the LogBuffer invokes the LogFileManager to obtain a LogFile for use when writing and forcing the buffer. If a file switch occurrs, the LogFileManager will store a file header record into this newly initialized buffer.

      Parameters:
      bsn - Logical Block Sequence Number of the buffer. LogBufferManager maintains a block sequence number to ensure correct order of writes to disk. Some implementations of LogBuffer may include the BSN as part of a record or block header.
      lfm - LogFileMaager to call to obtain a LogFile.
      Returns:
      this LogBuffer
      Throws:
      LogFileOverflowException
    • read

      abstract LogBuffer read(LogFile lf, long position) throws IOException, InvalidLogBufferException
      read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.

      Used by LogFileManager implementations to read blocks of data that are formatted by a specific LogBuffer implementation.

      The LogFileManager uses LogBufferManager.getLogBuffer() method to obtain a buffer that is used for reading log files.

      Parameters:
      lf - LogFile to read.
      position - within the LogFile to be read.
      Returns:
      this LogBuffer reference.
      Throws:
      IOException
      InvalidLogBufferException
    • shouldForce

      abstract boolean shouldForce()
      returns true if the buffer should be forced to disk.

      The criteria for determining if a buffer should be forced is implementation dependent. Typically, this method will return true if there are one or more threads waiting for the buffer to be forced and the amount of time the threads has been waiting has been longer than some implementation defined value.

      Returns:
      true if buffer should be forced immediately.
    • put

      abstract long put(short type, byte[][] data, boolean sync) throws LogRecordSizeException
      puts a data record into the buffer and returns a token for record.

      PRECONDITION: caller holds a bufferManager monitor.

      The caller must set the sync parameter true if the thread will ultimately call sync() after a successful put(). This strategy allows the waitingThreads counter to be incremented while the current thread holds the bufferManager monitor.

      Implementations should return a token that can be used later for replay, and for debugging purposes.

      The data record is passed as a byte[][] allowing callers to construct data records from individual bits of information. The arrays are concatenated into a single log record whose size is the sum of the individual array sizes. Each array is preceded by the size of the individual array. The entire record is preceded by the size of all arrays including the individual array size fields.

      The record format is as follows:

       +------+------------+----------------+---------+       +----------------+---------+
       | type | total size | data[0].length | data[0] | . . . | data[n].length | data[n] |
       +------+------------+----------------+---------+       +----------------+---------+
       

      During replay the entire record is returned as a single byte[]. The ReplayListener is responsible for decomposing the record into the original array of byte[].

      Parameters:
      type - short containing implementation defined record type information. The type is stored as the first field of the log record.
      data - byte[][] to be written to log.
      sync - true if thread will call sync following the put. Causes count of waitingThreads to be incremented.
      Returns:
      a long that contains the physical position of the record is returned. The value returned by put() is an encoding of the physical position. The format of the returned value is implementation specific, and should be treated as opaque by the caller. Returns 0 if there is no room for the record in the current buffer.
      Throws:
      LogRecordSizeException - if the sum of all data[] array sizes is larger than the maximum allowed record size for the configured buffer size.
    • write

      abstract void write() throws IOException
      write ByteBuffer to the LogFile.

      updates the buffer header with the number of bytes used. Based on configuration, some implementations may compute a hash code or other integrety value and include the value in some implementation defined header information.

      The buffer is written using the LogFile.write() method to allow the LogFile to manage file position for circular journals.

      forcing and notification of waiting threads is the responsibility of the LogBufferManager that owns this LogBUffer.

      Throws:
      IOException - rethrows any IOExceptions thrown by FileChannel methods.

      QUESTION: should waiters be interupted if IO Error occurs?

      See Also:
    • getStats

      abstract String getStats()
      returns statistics for this LogBuffer object.

      information is returned in the form of an XML node that can be included as a nested element in a larger document containing stats for multiple LogBuffer objects, and any containing objects.

      Returns:
      statistics for this buffer as XML