com.unister.semweb.drums.file
Class HeaderIndexFile<Data extends AbstractKVStorable>

java.lang.Object
  com.unister.semweb.drums.file.AbstractHeaderFile
      com.unister.semweb.drums.file.HeaderIndexFile<Data>

Type Parameters:

Data - an implementation of AbstractKVStorable, e.g. GeneralStorable

public class HeaderIndexFile<Data extends AbstractKVStorable>
extends AbstractHeaderFile

This class represents a file, which contains a bunch of datasets. The file also contains a header with some informations and an IndexForHeaderIndexFile. This class is for managing and handling equal sized storable elements. So be very careful if this precondition is not fulfilled.

Header structure:

+-----------+--------------+---------------+---------------+-------------+---------+
| FILE SIZE | FILLED UP TO | Closed Softly | ReadChunkSize | ElementSize | KeySize |
| 8 bytes . | 8 bytes .... | 1 bytes ..... | 4 bytes ..... | 4 bytes ... | 4 bytes |
+-----------+--------------+---------------+---------------+-------------+---------+
= 1024 bytes (to have enough space for more values)

To use this class correctly, have a look at the following methods:

read(long offset, ByteBuffer destBuffer)

write(long offset, ByteBuffer sourceBuffer)

append(ByteBuffer sourceBuffer)

dbfile.getFilledUpToExclHeader()

The file enlarges automatically. Deletes (writing empty byte[] to a specific position) are leading to fragmentation. The actual version don't have a fragmentation-handling. Example code for reading the whole file:

  ...
  DBFile dbfile = new DBFile(filename, mode, retries);
  ByteBuffer readBuffer = ByteBuffer.allocate(blockSize);
  long offset = 0;
  while(offset < dbfile.getFilledUpFromContentStart()) {
      read(offset, readBuffer);
      // do something with the readBuffer
      ...
      offset += readBuffer.limit();
  }
  ...
 

Author:

Martin Nettling

Nested Class Summary

Nested classes/interfaces inherited from class com.unister.semweb.drums.file.AbstractHeaderFile

AbstractHeaderFile.AccessMode

Field Summary

Fields inherited from class com.unister.semweb.drums.file.AbstractHeaderFile

DEFAULT_SIZE, HEADER_SIZE, osFile

Constructor Summary

HeaderIndexFile(java.lang.String fileName, AbstractHeaderFile.AccessMode mode, int max_retries_connect, DRUMSParameterSet<Data> gp)
This constructor instantiates a new HeaderIndexFile with the given fileName in the given AbstractHeaderFile.AccessMode.

HeaderIndexFile(java.lang.String fileName, int max_retries_connect, DRUMSParameterSet<Data> gp)
This constructor instantiates a new HeaderIndexFile with the given fileName in the given AbstractHeaderFile.AccessMode.

Method Summary

long	append(byte[] sourceBuffer) appends the given sourceBuffer to the file and returns the file position of the appended entry
long	append(java.nio.ByteBuffer sourceBuffer) appends the given sourceBuffer to the file and returns the file position of the appended entry
void	clear() Sets all file-pointers to their initial values.
void	close() closes all open channels and the file
void	enlargeFile(long toEnlarge) Rounds up the given size to the next full chunk, and enlarges the file by these amounts of bytes.
int	getChunkIndex(long offset) calculates the index of the belonging readChunk in the files content
int	getChunkSize()
int	getElementSize()
long	getFreeSpace()
IndexForHeaderIndexFile	getIndex()
boolean	isConsistent() This method checks, if the keys of all inserted elements are incrementing continuously.
boolean	isConsitentWithIndex() This method checks, if the data is consistent with the index.
boolean	isSoftlyClosed()
java.nio.MappedByteBuffer	map(long offset, int length) Create a MappedByteBuffer mapping the given region.
void	openChannel() opens the RandomAccessFile and the corresponding FileChannel.
int	read(long offset, byte[] destBuffer) Reads x bytes from the file to the given ByteBuffer (position was set to zero), where x is the minimum of the capacity of the buffer and the remaining written bytes in the file.
int	read(long offset, java.nio.ByteBuffer destBuffer) Reads x bytes from the file to the given ByteBuffer (position was set to zero), where x is the minimum of the capacity of the buffer and the remaining written bytes in the file.
void	readIndex() reads and instantiates the index from the indexBuffer
void	repairIndex() This method repairs the index.
void	write(long offset, byte[] sourceBuffer) writes the bytes from the given Byte-array to the file beginning at offset.
void	write(long offset, java.nio.ByteBuffer sourceBuffer) writes the bytes from the given ByteBuffer to the file beginning at offset.

Methods inherited from class com.unister.semweb.drums.file.AbstractHeaderFile

delete, getFilledUpFromContentStart, getFilledUpTo, getName, getRemainingBytes, isOpen, lock, unlock

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HeaderIndexFile

public HeaderIndexFile(java.lang.String fileName,
                       AbstractHeaderFile.AccessMode mode,
                       int max_retries_connect,
                       DRUMSParameterSet<Data> gp)
                throws FileLockException,
                       java.io.IOException

This constructor instantiates a new HeaderIndexFile with the given fileName in the given AbstractHeaderFile.AccessMode.

Parameters:

fileName - the filename of the underlying OSfile.

mode - the mode the file should be accessed. READ_ONLY or READ_WRITE

max_retries_connect - the number of retries to open a channel, if the file is locked

gp - the DRUMSParameterSet.

Throws:

FileLockException - if the max_retries_connect is exceeded

java.io.IOException - if another error with the file-access occured

HeaderIndexFile

public HeaderIndexFile(java.lang.String fileName,
                       int max_retries_connect,
                       DRUMSParameterSet<Data> gp)
                throws FileLockException,
                       java.io.IOException

This constructor instantiates a new HeaderIndexFile with the given fileName in the given AbstractHeaderFile.AccessMode. This is a weak constructor and therefore you can only have read-access.

Parameters:

fileName - the filename of the underlying OSfile.

max_retries_connect - the number of retries to open a channel, if the file is locked

gp - the DRUMSParameterSet.

Throws:

FileLockException - if the max_retries_connect is exceeded

java.io.IOException - if another error with the fileaccess occured

Method Detail

map

public java.nio.MappedByteBuffer map(long offset,
                                     int length)
                              throws java.io.IOException

Create a MappedByteBuffer mapping the given region. The size of the HEADER will be respected automatically.

WARNING: This method is for professional use only.

Parameters:

offset -

length -

Returns:

a MappedByteBuffer pointing to the part of the file specified by offset and length.

Throws:

java.io.IOException

write

public void write(long offset,
                  java.nio.ByteBuffer sourceBuffer)
           throws java.io.IOException

writes the bytes from the given ByteBuffer to the file beginning at offset. The size of the HEADER will be respected automatically.

Specified by:

write in class AbstractHeaderFile

Parameters:

offset -

sourceBuffer -

Throws:

java.io.IOException

write

public void write(long offset,
                  byte[] sourceBuffer)
           throws java.io.IOException

Description copied from class: AbstractHeaderFile

writes the bytes from the given Byte-array to the file beginning at offset.

Specified by:

write in class AbstractHeaderFile

Throws:

java.io.IOException

append

public long append(byte[] sourceBuffer)
            throws java.io.IOException

Description copied from class: AbstractHeaderFile

appends the given sourceBuffer to the file and returns the file position of the appended entry

Specified by:

append in class AbstractHeaderFile

Returns:

the position in the file after inserting the given sourceBuffer

Throws:

java.io.IOException

append

public long append(java.nio.ByteBuffer sourceBuffer)
            throws java.io.IOException

Description copied from class: AbstractHeaderFile

appends the given sourceBuffer to the file and returns the file position of the appended entry

Specified by:

append in class AbstractHeaderFile

Returns:

the position in the file after inserting the given sourceBuffer

Throws:

java.io.IOException

read

public int read(long offset,
                java.nio.ByteBuffer destBuffer)
         throws java.io.IOException

Reads x bytes from the file to the given ByteBuffer (position was set to zero), where x is the minimum of the capacity of the buffer and the remaining written bytes in the file. The size of the HEADER will be respected automatically. The position is set to the number of read bytes.

Specified by:

read in class AbstractHeaderFile

Parameters:

offset - the file offset, where to start reading

destBuffer - the buffer to fill

Returns:

the number of bytes read

Throws:

java.io.IOException

read

public int read(long offset,
                byte[] destBuffer)
         throws java.io.IOException

Reads x bytes from the file to the given ByteBuffer (position was set to zero), where x is the minimum of the capacity of the buffer and the remaining written bytes in the file. The size of the HEADER will be respected automatically. The position is set to the number of read bytes.

Specified by:

read in class AbstractHeaderFile

Parameters:

offset - the file offset, where to start reading

destBuffer - the buffer to fill

Returns:

the number of bytes read

Throws:

java.io.IOException

See Also:

read(long, ByteBuffer)

openChannel

public void openChannel()
                 throws FileLockException,
                        java.io.IOException

opens the RandomAccessFile and the corresponding FileChannel. Reads the header and the index. This may overwrite previous set parameters.

Overrides:

openChannel in class AbstractHeaderFile

Throws:

java.io.IOException

FileLockException

getFreeSpace

public long getFreeSpace()

Overrides:

getFreeSpace in class AbstractHeaderFile

Returns:

the number of available bytes

readIndex

public void readIndex()

reads and instantiates the index from the indexBuffer

enlargeFile

public void enlargeFile(long toEnlarge)
                 throws java.io.IOException

Rounds up the given size to the next full chunk, and enlarges the file by these amounts of bytes.

Specified by:

enlargeFile in class AbstractHeaderFile

Parameters:

toEnlarge -

Throws:

java.io.IOException

getIndex

public IndexForHeaderIndexFile getIndex()

Returns:

the instantiated index IndexForHeaderIndexFile

isConsistent

public boolean isConsistent()
                     throws java.io.IOException

This method checks, if the keys of all inserted elements are incrementing continuously.

Returns:

true, if the file fulfills this constraint.

Throws:

java.io.IOException

isConsitentWithIndex

public boolean isConsitentWithIndex()
                             throws java.io.IOException

This method checks, if the data is consistent with the index.

Returns:

true, if the file fulfills this constraint.

Throws:

java.io.IOException

repairIndex

public void repairIndex()
                 throws java.io.IOException

This method repairs the index. It runs over all chunks and writes the largest element into the IndexForHeaderIndexFile. This method should be called, if the file was not closed softly ( isSoftlyClosed()).

Throws:

java.io.IOException

clear

public void clear()

Sets all file-pointers to their initial values. The file itself is not shrinked.

Overrides:

clear in class AbstractHeaderFile

getChunkIndex

public int getChunkIndex(long offset)

calculates the index of the belonging readChunk in the files content

Parameters:

offset - the byteOffset in the content

Returns:

the determined index

getElementSize

public int getElementSize()

Returns:

the size of one element. We assume that all elements are equal sized

getChunkSize

public int getChunkSize()

Returns:

the size in bytes of a chunk

isSoftlyClosed

public boolean isSoftlyClosed()

Returns:

true, if the file was closed softly

close

public void close()

Description copied from class: AbstractHeaderFile

closes all open channels and the file

Overrides:

close in class AbstractHeaderFile