Package org.apache.lucene.util.compress
Class LZ4
java.lang.Object
org.apache.lucene.util.compress.LZ4
LZ4 compression and decompression routines.
https://github.com/lz4/lz4/tree/dev/lib http://fastcompression.blogspot.fr/p/lz4.html
The high-compression option is a simpler version of the one of the original algorithm, and only retains a better hash table that remembers about more occurrences of a previous 4-bytes sequence, and removes all the logic about handling of the case when overlapping matches are found.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final class
Simple lossyLZ4.HashTable
that only stores the last ocurrence for each hash on2^14
bytes of memory.(package private) static class
A record of previous occurrences of sequences of 4 bytes.static final class
A higher-precisionLZ4.HashTable
.private static class
private static class
16 bits per offset.private static class
32 bits per value, only used when inputs exceed 64kB, e.g. -
Field Summary
FieldsModifier and TypeFieldDescription(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate static int
commonBytes
(byte[] b, int o1, int o2, int limit) static void
compress
(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht) Compressbytes[off:off+len]
intoout
using at most 16kB of memory.static void
compressWithDictionary
(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht) Compressbytes[dictOff+dictLen:dictOff+dictLen+len]
intoout
using at most 16kB of memory.static int
decompress
(DataInput compressed, int decompressedLen, byte[] dest, int dOff) Decompress at leastdecompressedLen
bytes intodest[dOff:]
.private static void
encodeLastLiterals
(byte[] bytes, int anchor, int literalLen, DataOutput out) private static void
encodeLen
(int l, DataOutput out) private static void
encodeLiterals
(byte[] bytes, int token, int anchor, int literalLen, DataOutput out) private static void
encodeSequence
(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out) private static int
hash
(int i, int hashBits) private static int
hashHC
(int i) private static int
readInt
(byte[] buf, int i)
-
Field Details
-
MEMORY_USAGE
static final int MEMORY_USAGE- See Also:
-
MIN_MATCH
static final int MIN_MATCH- See Also:
-
MAX_DISTANCE
static final int MAX_DISTANCE- See Also:
-
LAST_LITERALS
static final int LAST_LITERALS- See Also:
-
HASH_LOG_HC
static final int HASH_LOG_HC- See Also:
-
HASH_TABLE_SIZE_HC
static final int HASH_TABLE_SIZE_HC- See Also:
-
-
Constructor Details
-
LZ4
private LZ4()
-
-
Method Details
-
hash
private static int hash(int i, int hashBits) -
hashHC
private static int hashHC(int i) -
readInt
private static int readInt(byte[] buf, int i) -
commonBytes
private static int commonBytes(byte[] b, int o1, int o2, int limit) -
decompress
public static int decompress(DataInput compressed, int decompressedLen, byte[] dest, int dOff) throws IOException Decompress at leastdecompressedLen
bytes intodest[dOff:]
. Please note thatdest
must be large enough to be able to hold all decompressed data (meaning that you need to know the total decompressed length). If the given bytes were compressed using a preset dictionary then the same dictionary must be provided indest[dOff-dictLen:dOff]
.- Throws:
IOException
-
encodeLen
- Throws:
IOException
-
encodeLiterals
private static void encodeLiterals(byte[] bytes, int token, int anchor, int literalLen, DataOutput out) throws IOException - Throws:
IOException
-
encodeLastLiterals
private static void encodeLastLiterals(byte[] bytes, int anchor, int literalLen, DataOutput out) throws IOException - Throws:
IOException
-
encodeSequence
private static void encodeSequence(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out) throws IOException - Throws:
IOException
-
compress
public static void compress(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht) throws IOException Compressbytes[off:off+len]
intoout
using at most 16kB of memory.ht
shouldn't be shared across threads but can safely be reused.- Throws:
IOException
-
compressWithDictionary
public static void compressWithDictionary(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht) throws IOException Compressbytes[dictOff+dictLen:dictOff+dictLen+len]
intoout
using at most 16kB of memory.bytes[dictOff:dictOff+dictLen]
will be used as a dictionary.dictLen
must not be greater than 64kB, the maximum window size.ht
shouldn't be shared across threads but can safely be reused.- Throws:
IOException
-