Class SnappyCompressorOutputStream

  • All Implemented Interfaces:
    java.io.Closeable, java.io.Flushable, java.lang.AutoCloseable

    public class SnappyCompressorOutputStream
    extends CompressorOutputStream
    CompressorOutputStream for the raw Snappy format.

    This implementation uses an internal buffer in order to handle the back-references that are at the heart of the LZ77 algorithm. The size of the buffer must be at least as big as the biggest offset used in the compressed stream. The current version of the Snappy algorithm as defined by Google works on 32k blocks and doesn't contain offsets bigger than 32k which is the default block size used by this class.

    The raw Snappy format requires the uncompressed size to be written at the beginning of the stream using a varint representation, i.e. the number of bytes needed to write the information is not known before the uncompressed size is known. We've chosen to make the uncompressedSize a parameter of the constructor in favor of buffering the whole output until the size is known. When using the FramedSnappyCompressorOutputStream this limitation is taken care of by the warpping framing format.

    Since:
    1.14
    See Also:
    Snappy compressed format description
    • Field Detail

      • os

        private final java.io.OutputStream os
      • oneByte

        private final byte[] oneByte
      • finished

        private boolean finished
      • MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES

        private static final int MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES
        See Also:
        Constant Field Values
      • MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE

        private static final int MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE
        See Also:
        Constant Field Values
      • MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES

        private static final int MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES
        See Also:
        Constant Field Values
      • MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES

        private static final int MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES
        See Also:
        Constant Field Values
      • THREE_SIZE_BYTE_MARKER

        private static final int THREE_SIZE_BYTE_MARKER
        See Also:
        Constant Field Values
      • MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE

        private static final int MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
        See Also:
        Constant Field Values
      • MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE

        private static final int MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
        See Also:
        Constant Field Values
      • MAX_OFFSET_WITH_ONE_OFFSET_BYTE

        private static final int MAX_OFFSET_WITH_ONE_OFFSET_BYTE
        See Also:
        Constant Field Values
      • MAX_OFFSET_WITH_TWO_OFFSET_BYTES

        private static final int MAX_OFFSET_WITH_TWO_OFFSET_BYTES
        See Also:
        Constant Field Values
    • Constructor Detail

      • SnappyCompressorOutputStream

        public SnappyCompressorOutputStream​(java.io.OutputStream os,
                                            long uncompressedSize)
                                     throws java.io.IOException
        Constructor using the default block size of 32k.
        Parameters:
        os - the outputstream to write compressed data to
        uncompressedSize - the uncompressed size of data
        Throws:
        java.io.IOException - if writing of the size fails
      • SnappyCompressorOutputStream

        public SnappyCompressorOutputStream​(java.io.OutputStream os,
                                            long uncompressedSize,
                                            int blockSize)
                                     throws java.io.IOException
        Constructor using a configurable block size.
        Parameters:
        os - the outputstream to write compressed data to
        uncompressedSize - the uncompressed size of data
        blockSize - the block size used - must be a power of two
        Throws:
        java.io.IOException - if writing of the size fails
      • SnappyCompressorOutputStream

        public SnappyCompressorOutputStream​(java.io.OutputStream os,
                                            long uncompressedSize,
                                            Parameters params)
                                     throws java.io.IOException
        Constructor providing full control over the underlying LZ77 compressor.
        Parameters:
        os - the outputstream to write compressed data to
        uncompressedSize - the uncompressed size of data
        params - the parameters to use by the compressor - note that the format itself imposes some limits like a maximum match length of 64 bytes
        Throws:
        java.io.IOException - if writing of the size fails
    • Method Detail

      • write

        public void write​(int b)
                   throws java.io.IOException
        Specified by:
        write in class java.io.OutputStream
        Throws:
        java.io.IOException
      • write

        public void write​(byte[] data,
                          int off,
                          int len)
                   throws java.io.IOException
        Overrides:
        write in class java.io.OutputStream
        Throws:
        java.io.IOException
      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Overrides:
        close in class java.io.OutputStream
        Throws:
        java.io.IOException
      • finish

        public void finish()
                    throws java.io.IOException
        Compresses all remaining data and writes it to the stream, doesn't close the underlying stream.
        Throws:
        java.io.IOException - if an error occurs
      • writeUncompressedSize

        private void writeUncompressedSize​(long uncompressedSize)
                                    throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlock

        private void writeLiteralBlock​(LZ77Compressor.LiteralBlock block)
                                throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockNoSizeBytes

        private void writeLiteralBlockNoSizeBytes​(LZ77Compressor.LiteralBlock block,
                                                  int len)
                                           throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockOneSizeByte

        private void writeLiteralBlockOneSizeByte​(LZ77Compressor.LiteralBlock block,
                                                  int len)
                                           throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockTwoSizeBytes

        private void writeLiteralBlockTwoSizeBytes​(LZ77Compressor.LiteralBlock block,
                                                   int len)
                                            throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockThreeSizeBytes

        private void writeLiteralBlockThreeSizeBytes​(LZ77Compressor.LiteralBlock block,
                                                     int len)
                                              throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockFourSizeBytes

        private void writeLiteralBlockFourSizeBytes​(LZ77Compressor.LiteralBlock block,
                                                    int len)
                                             throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLiteralBlockWithSize

        private void writeLiteralBlockWithSize​(int tagByte,
                                               int sizeBytes,
                                               int len,
                                               LZ77Compressor.LiteralBlock block)
                                        throws java.io.IOException
        Throws:
        java.io.IOException
      • writeLittleEndian

        private void writeLittleEndian​(int numBytes,
                                       int num)
                                throws java.io.IOException
        Throws:
        java.io.IOException
      • writeBackReference

        private void writeBackReference​(LZ77Compressor.BackReference block)
                                 throws java.io.IOException
        Throws:
        java.io.IOException
      • writeBackReferenceWithOneOffsetByte

        private void writeBackReferenceWithOneOffsetByte​(int len,
                                                         int offset)
                                                  throws java.io.IOException
        Throws:
        java.io.IOException
      • writeBackReferenceWithTwoOffsetBytes

        private void writeBackReferenceWithTwoOffsetBytes​(int len,
                                                          int offset)
                                                   throws java.io.IOException
        Throws:
        java.io.IOException
      • writeBackReferenceWithFourOffsetBytes

        private void writeBackReferenceWithFourOffsetBytes​(int len,
                                                           int offset)
                                                    throws java.io.IOException
        Throws:
        java.io.IOException
      • writeBackReferenceWithLittleEndianOffset

        private void writeBackReferenceWithLittleEndianOffset​(int tag,
                                                              int offsetBytes,
                                                              int len,
                                                              int offset)
                                                       throws java.io.IOException
        Throws:
        java.io.IOException
      • createParameterBuilder

        public static Parameters.Builder createParameterBuilder​(int blockSize)
        Returns a builder correctly configured for the Snappy algorithm using the gven block size.
        Parameters:
        blockSize - the block size.
        Returns:
        a builder correctly configured for the Snappy algorithm using the gven block size