- All Implemented Interfaces:
Accountable
DataOutput
storing data in a list of ByteBuffer
s.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final class
An implementation of aByteBuffer
allocation and recycling policy. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final IntFunction
<ByteBuffer> private final IntFunction
<ByteBuffer> ByteBuffer
supplier.private int
Current block size:2^bits
.private final Consumer
<ByteBuffer> ByteBuffer
recycler onreset()
.private final ArrayDeque
<ByteBuffer> Blocks storing data.private ByteBuffer
The current-or-next write block.static final int
DefaultmaxBitsPerBlock
static final int
DefaultminBitsPerBlock
private static final ByteBuffer
private static final byte[]
static final int
LargestmaxBitsPerBlock
allowedstatic final int
SmallestminBitsPerBlock
allowed(package private) static final int
Maximum number of blocks at the currentblockBits
block size before we increase the block size (and thus decrease the number of blocks).private static final int
private final int
Maximum block size:2^bits
.static final Consumer
<ByteBuffer> A singleton instance of "no-reuse" buffer strategy.private long
Cumulative RAM usage across all blocks.Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
Constructor Summary
ConstructorsConstructorDescriptionCreates a new output with all defaults.ByteBuffersDataOutput
(int minBitsPerBlock, int maxBitsPerBlock, IntFunction<ByteBuffer> blockAllocate, Consumer<ByteBuffer> blockReuse) Expert: Creates a new output with custom parameters.ByteBuffersDataOutput
(long expectedSize) Create a new output, suitable for writing a file of aroundexpectedSize
bytes. -
Method Summary
Modifier and TypeMethodDescriptionprivate void
private int
private static int
computeBlockSizeBitsFor
(long bytes) void
Copy numBytes bytes from input to ourself.void
copyTo
(DataOutput output) Copy the current content of this object into anotherDataOutput
.static ByteBuffersDataOutput
long
Return the memory usage of this object in bytes.void
reset()
This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.private void
rewriteToBlockSize
(int targetBlockBits) long
size()
byte[]
Return a contiguous array with the current content written to the output.Return a list of read-only view ofByteBuffer
blocks over the current content written to the output.Return aByteBuffersDataInput
for the set of current buffers (toBufferList()
).toString()
Returns a list of writeable blocks over the (source) content buffers.void
writeByte
(byte b) Writes a single byte.void
writeBytes
(byte[] b) void
writeBytes
(byte[] b, int length) Writes an array of bytes.void
writeBytes
(byte[] src, int offset, int length) Writes an array of bytes.void
writeBytes
(ByteBuffer buffer) void
writeInt
(int v) Writes an int as four bytes (LE byte order).void
writeLong
(long v) Writes a long as eight bytes (LE byte order).private void
writeLongString
(int byteLen, String s) Writes a long string in chunksvoid
writeMapOfStrings
(Map<String, String> map) Writes a String map.void
writeSetOfStrings
(Set<String> set) Writes a String set.void
writeShort
(short v) Writes a short as two bytes (LE byte order).void
Writes a string.Methods inherited from class org.apache.lucene.store.DataOutput
writeGroupVInts, writeGroupVInts, writeVInt, writeVLong, writeZInt, writeZLong
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
Field Details
-
EMPTY
-
EMPTY_BYTE_ARRAY
private static final byte[] EMPTY_BYTE_ARRAY -
ALLOCATE_BB_ON_HEAP
-
NO_REUSE
A singleton instance of "no-reuse" buffer strategy. -
DEFAULT_MIN_BITS_PER_BLOCK
public static final int DEFAULT_MIN_BITS_PER_BLOCKDefaultminBitsPerBlock
- See Also:
-
DEFAULT_MAX_BITS_PER_BLOCK
public static final int DEFAULT_MAX_BITS_PER_BLOCKDefaultmaxBitsPerBlock
- See Also:
-
LIMIT_MIN_BITS_PER_BLOCK
public static final int LIMIT_MIN_BITS_PER_BLOCKSmallestminBitsPerBlock
allowed- See Also:
-
LIMIT_MAX_BITS_PER_BLOCK
public static final int LIMIT_MAX_BITS_PER_BLOCKLargestmaxBitsPerBlock
allowed- See Also:
-
MAX_BLOCKS_BEFORE_BLOCK_EXPANSION
static final int MAX_BLOCKS_BEFORE_BLOCK_EXPANSIONMaximum number of blocks at the currentblockBits
block size before we increase the block size (and thus decrease the number of blocks).- See Also:
-
maxBitsPerBlock
private final int maxBitsPerBlockMaximum block size:2^bits
. -
blockAllocate
ByteBuffer
supplier. -
blockReuse
ByteBuffer
recycler onreset()
. -
blockBits
private int blockBitsCurrent block size:2^bits
. -
blocks
Blocks storing data. -
ramBytesUsed
private long ramBytesUsedCumulative RAM usage across all blocks. -
currentBlock
The current-or-next write block. -
MAX_CHARS_PER_WINDOW
private static final int MAX_CHARS_PER_WINDOW- See Also:
-
-
Constructor Details
-
ByteBuffersDataOutput
public ByteBuffersDataOutput(long expectedSize) Create a new output, suitable for writing a file of aroundexpectedSize
bytes.Memory allocation will be optimized based on the
expectedSize
hint, so that there is less overhead for larger files.- Parameters:
expectedSize
- estimated size of the output file
-
ByteBuffersDataOutput
public ByteBuffersDataOutput()Creates a new output with all defaults. -
ByteBuffersDataOutput
public ByteBuffersDataOutput(int minBitsPerBlock, int maxBitsPerBlock, IntFunction<ByteBuffer> blockAllocate, Consumer<ByteBuffer> blockReuse) Expert: Creates a new output with custom parameters.- Parameters:
minBitsPerBlock
- minimum bits per blockmaxBitsPerBlock
- maximum bits per blockblockAllocate
- block allocatorblockReuse
- block recycler
-
-
Method Details
-
writeByte
public void writeByte(byte b) Description copied from class:DataOutput
Writes a single byte.The most primitive data type is an eight-bit byte. Files are accessed as sequences of bytes. All other data types are defined as sequences of bytes, so file formats are byte-order independent.
- Specified by:
writeByte
in classDataOutput
- See Also:
-
writeBytes
public void writeBytes(byte[] src, int offset, int length) Description copied from class:DataOutput
Writes an array of bytes.- Specified by:
writeBytes
in classDataOutput
- Parameters:
src
- the bytes to writeoffset
- the offset in the byte arraylength
- the number of bytes to write- See Also:
-
writeBytes
public void writeBytes(byte[] b, int length) Description copied from class:DataOutput
Writes an array of bytes.- Overrides:
writeBytes
in classDataOutput
- Parameters:
b
- the bytes to writelength
- the number of bytes to write- See Also:
-
writeBytes
public void writeBytes(byte[] b) -
writeBytes
-
toBufferList
Return a list of read-only view ofByteBuffer
blocks over the current content written to the output. -
toWriteableBufferList
Returns a list of writeable blocks over the (source) content buffers.This method returns the raw content of source buffers that may change over the lifetime of this object (blocks can be recycled or discarded, for example). Most applications should favor calling
toBufferList()
which returns a read-only view over the content of the source buffers.The difference between
toBufferList()
andtoWriteableBufferList()
is that read-only view of source buffers will always returnfalse
fromByteBuffer.hasArray()
(which sometimes may be required to avoid double copying). -
toDataInput
Return aByteBuffersDataInput
for the set of current buffers (toBufferList()
). -
toArrayCopy
public byte[] toArrayCopy()Return a contiguous array with the current content written to the output. The returned array is always a copy (can be mutated).If the
size()
of the underlying buffers exceeds maximum size of Java array, anRuntimeException
will be thrown. -
copyBytes
Description copied from class:DataOutput
Copy numBytes bytes from input to ourself.- Overrides:
copyBytes
in classDataOutput
- Throws:
IOException
-
copyTo
Copy the current content of this object into anotherDataOutput
.- Throws:
IOException
-
size
public long size()- Returns:
- The number of bytes written to this output so far.
-
toString
-
writeShort
public void writeShort(short v) Description copied from class:DataOutput
Writes a short as two bytes (LE byte order).- Overrides:
writeShort
in classDataOutput
- See Also:
-
writeInt
public void writeInt(int v) Description copied from class:DataOutput
Writes an int as four bytes (LE byte order).- Overrides:
writeInt
in classDataOutput
- See Also:
-
writeLong
public void writeLong(long v) Description copied from class:DataOutput
Writes a long as eight bytes (LE byte order).- Overrides:
writeLong
in classDataOutput
- See Also:
-
writeString
Description copied from class:DataOutput
Writes a string.Writes strings as UTF-8 encoded bytes. First the length, in bytes, is written as a
VInt
, followed by the bytes.- Overrides:
writeString
in classDataOutput
- See Also:
-
writeMapOfStrings
Description copied from class:DataOutput
Writes a String map.First the size is written as an
vInt
, followed by each key-value pair written as two consecutiveString
s.- Overrides:
writeMapOfStrings
in classDataOutput
- Parameters:
map
- Input map.
-
writeSetOfStrings
Description copied from class:DataOutput
Writes a String set.First the size is written as an
vInt
, followed by each value written as aString
.- Overrides:
writeSetOfStrings
in classDataOutput
- Parameters:
set
- Input set.
-
ramBytesUsed
public long ramBytesUsed()Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.- Specified by:
ramBytesUsed
in interfaceAccountable
-
reset
public void reset()This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.Sharing byte buffers for reads and writes is dangerous and will very likely lead to hard-to-debug issues, use with great care.
-
newResettableInstance
- Returns:
- Returns a new
ByteBuffersDataOutput
with thereset()
capability.
-
blockSize
private int blockSize() -
appendBlock
private void appendBlock() -
rewriteToBlockSize
private void rewriteToBlockSize(int targetBlockBits) -
computeBlockSizeBitsFor
private static int computeBlockSizeBitsFor(long bytes) -
writeLongString
Writes a long string in chunks- Throws:
IOException
-