Compress-Zstd
view release on metacpan or search on metacpan
ext/zstd/doc/zstd_manual.html view on Meta::CPAN
note : since v1.3.0, ZSTD_CStream and ZSTD_CCtx are the same thing.
Parameters are sticky : when starting a new compression on the same context,
it will re-use the same sticky parameters as previous compression session.
When in doubt, it's recommended to fully initialize the context before usage.
Use ZSTD_CCtx_reset() to reset the context and ZSTD_CCtx_setParameter(),
ZSTD_CCtx_setPledgedSrcSize(), or ZSTD_CCtx_loadDictionary() and friends to
set more specific parameters, the pledged source size, or load a dictionary.
Use ZSTD_compressStream2() with ZSTD_e_continue as many times as necessary to
consume input stream. The function will automatically update both `pos`
fields within `input` and `output`.
Note that the function may not consume the entire input, for example, because
the output buffer is already full, in which case `input.pos < input.size`.
The caller must check if input has been entirely consumed.
If not, the caller must make some room to receive more compressed data,
and then present again remaining input data.
note: ZSTD_e_continue is guaranteed to make some forward progress when called,
but doesn't guarantee maximal forward progress. This is especially relevant
when compressing with multiple threads. The call won't block if it can
consume some input, but if it can't it will wait for some, but not all,
output to be flushed.
@return : provides a minimum amount of data remaining to be flushed from internal buffers
or an error code, which can be tested using ZSTD_isError().
At any moment, it's possible to flush whatever data might remain stuck within internal buffer,
using ZSTD_compressStream2() with ZSTD_e_flush. `output->pos` will be updated.
Note that, if `output->size` is too small, a single invocation with ZSTD_e_flush might not be enough (return code > 0).
In which case, make some room to receive more compressed data, and call again ZSTD_compressStream2() with ZSTD_e_flush.
You must continue calling ZSTD_compressStream2() with ZSTD_e_flush until it returns 0, at which point you can change the
operation.
note: ZSTD_e_flush will flush as much output as possible, meaning when compressing with multiple threads, it will
block until the flush is complete or the output buffer is full.
@return : 0 if internal buffers are entirely flushed,
>0 if some data still present within internal buffer (the value is minimal estimation of remaining size),
or an error code, which can be tested using ZSTD_isError().
Calling ZSTD_compressStream2() with ZSTD_e_end instructs to finish a frame.
It will perform a flush and write frame epilogue.
The epilogue is required for decoders to consider a frame completed.
flush operation is the same, and follows same rules as calling ZSTD_compressStream2() with ZSTD_e_flush.
You must continue calling ZSTD_compressStream2() with ZSTD_e_end until it returns 0, at which point you are free to
start a new frame.
note: ZSTD_e_end will flush as much output as possible, meaning when compressing with multiple threads, it will
block until the flush is complete or the output buffer is full.
@return : 0 if frame fully completed and fully flushed,
>0 if some data still present within internal buffer (the value is minimal estimation of remaining size),
or an error code, which can be tested using ZSTD_isError().
<BR></pre>
<pre><b>typedef ZSTD_CCtx ZSTD_CStream; </b>/**< CCtx and CStream are now effectively same object (>= v1.3.0) */<b>
</b></pre><BR>
<h3>ZSTD_CStream management functions</h3><pre></pre><b><pre>ZSTD_CStream* ZSTD_createCStream(void);
size_t ZSTD_freeCStream(ZSTD_CStream* zcs);
</pre></b><BR>
<h3>Streaming compression functions</h3><pre></pre><b><pre>typedef enum {
ZSTD_e_continue=0, </b>/* collect more data, encoder decides when to output compressed result, for optimal compression ratio */<b>
ZSTD_e_flush=1, </b>/* flush any data provided so far,<b>
* it creates (at least) one new block, that can be decoded immediately on reception;
* frame will continue: any future data can still reference previously compressed data, improving compression.
* note : multithreaded compression will block to flush as much output as possible. */
ZSTD_e_end=2 </b>/* flush any remaining data _and_ close current frame.<b>
* note that frame is only closed after compressed data is fully flushed (return value == 0).
* After that point, any additional data starts a new frame.
* note : each frame is independent (does not reference any content from previous frame).
: note : multithreaded compression will block to flush as much output as possible. */
} ZSTD_EndDirective;
</pre></b><BR>
<pre><b>size_t ZSTD_compressStream2( ZSTD_CCtx* cctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input,
ZSTD_EndDirective endOp);
</b><p> Behaves about the same as ZSTD_compressStream, with additional control on end directive.
- Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
- Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode)
- output->pos must be <= dstCapacity, input->pos must be <= srcSize
- output->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
- When nbWorkers==0 (default), function is blocking : it completes its job before returning to caller.
- When nbWorkers>=1, function is non-blocking : it just acquires a copy of input, and distributes jobs to internal worker threads, flush whatever is available,
and then immediately returns, just indicating that there is some data remaining to be flushed.
The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
- Exception : if the first call requests a ZSTD_e_end directive and provides enough dstCapacity, the function delegates to ZSTD_compress2() which is always blocking.
- @return provides a minimum amount of data remaining to be flushed from internal buffers
or an error code, which can be tested using ZSTD_isError().
if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
This is useful for ZSTD_e_flush, since in this case more flushes are necessary to empty all buffers.
For ZSTD_e_end, @return == 0 when internal buffers are fully flushed and frame is completed.
- after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
only ZSTD_e_end or ZSTD_e_flush operations are allowed.
Before starting a new compression job, or changing compression parameters,
it is required to fully flush internal buffers.
</p></pre><BR>
<pre><b>size_t ZSTD_CStreamInSize(void); </b>/**< recommended size for input buffer */<b>
</b></pre><BR>
<pre><b>size_t ZSTD_CStreamOutSize(void); </b>/**< recommended size for output buffer. Guarantee to successfully flush at least one complete compressed block. */<b>
</b></pre><BR>
<pre><b>size_t ZSTD_initCStream(ZSTD_CStream* zcs, int compressionLevel);
</b>/*!<b>
* Alternative for ZSTD_compressStream2(zcs, output, input, ZSTD_e_continue).
* NOTE: The return value is different. ZSTD_compressStream() returns a hint for
* the next read size (if non-zero and not an error). ZSTD_compressStream2()
* returns the minimum nb of bytes left to flush (if non-zero and not an error).
*/
size_t ZSTD_compressStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output, ZSTD_inBuffer* input);
</b>/*! Equivalent to ZSTD_compressStream2(zcs, output, &emptyInput, ZSTD_e_flush). */<b>
size_t ZSTD_flushStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output);
</b>/*! Equivalent to ZSTD_compressStream2(zcs, output, &emptyInput, ZSTD_e_end). */<b>
size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output);
</b><p>
ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
ZSTD_CCtx_refCDict(zcs, NULL); // clear the dictionary (if any)
ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel);
</p></pre><BR>
<a name="Chapter9"></a><h2>Streaming decompression - HowTo</h2><pre>
A ZSTD_DStream object is required to track streaming operations.
Use ZSTD_createDStream() and ZSTD_freeDStream() to create/release resources.
ZSTD_DStream objects can be re-used multiple times.
Use ZSTD_initDStream() to start a new decompression operation.
@return : recommended first input size
Alternatively, use advanced API to set specific properties.
Use ZSTD_decompressStream() repetitively to consume your input.
The function will update both `pos` fields.
If `input.pos < input.size`, some input has not been consumed.
It's up to the caller to present again remaining data.
The function tries to flush all data decoded immediately, respecting output buffer size.
If `output.pos < output.size`, decoder has flushed everything it could.
But if `output.pos == output.size`, there might be some data left within internal buffers.,
In which case, call ZSTD_decompressStream() again to flush whatever remains in the buffer.
Note : with no additional input provided, amount of data flushed is necessarily <= ZSTD_BLOCKSIZE_MAX.
@return : 0 when a frame is completely decoded and fully flushed,
or an error code, which can be tested using ZSTD_isError(),
or any other value > 0, which means there is still some decoding or flushing to do to complete current frame :
the return value is a suggested next input size (just a hint for better latency)
that will never request more than the remaining frame size.
<BR></pre>
<pre><b>typedef ZSTD_DCtx ZSTD_DStream; </b>/**< DCtx and DStream are now effectively same object (>= v1.3.0) */<b>
</b></pre><BR>
<h3>ZSTD_DStream management functions</h3><pre></pre><b><pre>ZSTD_DStream* ZSTD_createDStream(void);
size_t ZSTD_freeDStream(ZSTD_DStream* zds);
</pre></b><BR>
<h3>Streaming decompression functions</h3><pre></pre><b><pre></pre></b><BR>
<pre><b>size_t ZSTD_DStreamInSize(void); </b>/*!< recommended size for input buffer */<b>
</b></pre><BR>
<pre><b>size_t ZSTD_DStreamOutSize(void); </b>/*!< recommended size for output buffer. Guarantee to successfully flush at least one complete block in all circumstances. */<b>
</b></pre><BR>
<a name="Chapter10"></a><h2>Simple dictionary API</h2><pre></pre>
<pre><b>size_t ZSTD_compress_usingDict(ZSTD_CCtx* ctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
int compressionLevel);
</b><p> Compression at an explicit compression level using a Dictionary.
A dictionary can be any arbitrary data segment (also called a prefix),
or a buffer with specified information (see dictBuilder/zdict.h).
Note : This function loads the dictionary, resulting in significant startup delay.
It's intended for a dictionary used only once.
Note 2 : When `dict == NULL || dictSize < 8` no dictionary is used.
</p></pre><BR>
<pre><b>size_t ZSTD_decompress_usingDict(ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize);
</b><p> Decompression using a known Dictionary.
Dictionary must be identical to the one used during compression.
Note : This function loads the dictionary, resulting in significant startup delay.
It's intended for a dictionary used only once.
Note : When `dict == NULL || dictSize < 8` no dictionary is used.
</p></pre><BR>
<a name="Chapter11"></a><h2>Bulk processing dictionary API</h2><pre></pre>
<pre><b>ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize,
int compressionLevel);
</b><p> When compressing multiple messages / blocks using the same dictionary, it's recommended to load it only once.
ZSTD_createCDict() will create a digested dictionary, ready to start future compression operations without startup cost.
ZSTD_CDict can be created once and shared by multiple threads concurrently, since its usage is read-only.
`dictBuffer` can be released after ZSTD_CDict creation, because its content is copied within CDict.
Consider experimental function `ZSTD_createCDict_byReference()` if you prefer to not duplicate `dictBuffer` content.
Note : A ZSTD_CDict can be created from an empty dictBuffer, but it is inefficient when used to compress small data.
</p></pre><BR>
<pre><b>size_t ZSTD_freeCDict(ZSTD_CDict* CDict);
</b><p> Function frees memory allocated by ZSTD_createCDict().
</p></pre><BR>
<pre><b>size_t ZSTD_compress_usingCDict(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict);
</b><p> Compression using a digested Dictionary.
Recommended when same dictionary is used multiple times.
Note : compression level is _decided at dictionary creation time_,
and frame parameters are hardcoded (dictID=yes, contentSize=yes, checksum=no)
</p></pre><BR>
<pre><b>ZSTD_DDict* ZSTD_createDDict(const void* dictBuffer, size_t dictSize);
</b><p> Create a digested dictionary, ready to start decompression operation without startup delay.
dictBuffer can be released after DDict creation, as its content is copied inside DDict.
</p></pre><BR>
<pre><b>size_t ZSTD_freeDDict(ZSTD_DDict* ddict);
</b><p> Function frees memory allocated with ZSTD_createDDict()
</p></pre><BR>
<pre><b>size_t ZSTD_decompress_usingDDict(ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_DDict* ddict);
</b><p> Decompression using a digested Dictionary.
Recommended when same dictionary is used multiple times.
</p></pre><BR>
<a name="Chapter12"></a><h2>Dictionary helper functions</h2><pre></pre>
<pre><b>unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize);
</b><p> Provides the dictID stored within dictionary.
if @return == 0, the dictionary is not conformant with Zstandard specification.
It can still be loaded, but as a content-only dictionary.
</p></pre><BR>
<pre><b>unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict* ddict);
</b><p> Provides the dictID of the dictionary loaded into `ddict`.
If @return == 0, the dictionary is not conformant to Zstandard specification, or empty.
Non-conformant dictionaries can still be loaded, but as content-only dictionaries.
</p></pre><BR>
<pre><b>unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize);
</b><p> Provides the dictID required to decompressed the frame stored within `src`.
If @return == 0, the dictID could not be decoded.
This could for one of the following reasons :
- The frame does not require a dictionary to be decoded (most common case).
- The frame was built with dictID intentionally removed. Whatever dictionary is necessary is a hidden information.
Note : this use case also happens when using a non-conformant dictionary.
- `srcSize` is too small, and as a result, the frame header could not be decoded (only possible if `srcSize < ZSTD_FRAMEHEADERSIZE_MAX`).
- This is not a Zstandard frame.
When identifying the exact failure cause, it's possible to use ZSTD_getFrameHeader(), which will provide a more precise error code.
</p></pre><BR>
<a name="Chapter13"></a><h2>Advanced dictionary and prefix API</h2><pre>
This API allows dictionaries to be used with ZSTD_compress2(),
ZSTD_compressStream2(), and ZSTD_decompress(). Dictionaries are sticky, and
only reset with the context is reset with ZSTD_reset_parameters or
ZSTD_reset_session_and_parameters. Prefixes are single-use.
<BR></pre>
<pre><b>size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
</b><p> Create an internal CDict from `dict` buffer.
Decompression will have to use same dictionary.
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Special: Loading a NULL (or 0-size) dictionary invalidates previous dictionary,
meaning "return to no-dictionary mode".
Note 1 : Dictionary is sticky, it will be used for all future compressed frames.
To return to "no-dictionary" situation, load a NULL dictionary (or reset parameters).
Note 2 : Loading a dictionary involves building tables.
It's also a CPU consuming operation, with non-negligible impact on latency.
Tables are dependent on compression parameters, and for this reason,
compression parameters can no longer be changed after loading a dictionary.
Note 3 :`dict` content will be copied internally.
Use experimental ZSTD_CCtx_loadDictionary_byReference() to reference content instead.
In such a case, dictionary buffer must outlive its users.
Note 4 : Use ZSTD_CCtx_loadDictionary_advanced()
to precisely select how dictionary content must be interpreted.
</p></pre><BR>
<pre><b>size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict);
</b><p> Reference a prepared dictionary, to be used for all next compressed frames.
Note that compression parameters are enforced from within CDict,
and supersede any compression parameter previously set within CCtx.
The parameters ignored are labled as "superseded-by-cdict" in the ZSTD_cParameter enum docs.
The ignored parameters will be used again if the CCtx is returned to no-dictionary mode.
The dictionary will remain valid for future compressed frames using same CCtx.
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Special : Referencing a NULL CDict means "return to no-dictionary mode".
Note 1 : Currently, only one dictionary can be managed.
Referencing a new dictionary effectively "discards" any previous one.
Note 2 : CDict is just referenced, its lifetime must outlive its usage within CCtx.
</p></pre><BR>
<pre><b>size_t ZSTD_CCtx_refPrefix(ZSTD_CCtx* cctx,
const void* prefix, size_t prefixSize);
</b><p> Reference a prefix (single-usage dictionary) for next compressed frame.
A prefix is **only used once**. Tables are discarded at end of frame (ZSTD_e_end).
Decompression will need same prefix to properly regenerate data.
Compressing with a prefix is similar in outcome as performing a diff and compressing it,
but performs much faster, especially during decompression (compression speed is tunable with compression level).
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Special: Adding any prefix (including NULL) invalidates any previous prefix or dictionary
Note 1 : Prefix buffer is referenced. It **must** outlive compression.
Its content must remain unmodified during compression.
Note 2 : If the intention is to diff some large src data blob with some prior version of itself,
ensure that the window size is large enough to contain the entire source.
See ZSTD_c_windowLog.
Note 3 : Referencing a prefix involves building tables, which are dependent on compression parameters.
It's a CPU consuming operation, with non-negligible impact on latency.
ext/zstd/doc/zstd_manual.html view on Meta::CPAN
Then, consume your input using ZSTD_compressContinue().
There are some important considerations to keep in mind when using this advanced function :
- ZSTD_compressContinue() has no internal buffer. It uses externally provided buffers only.
- Interface is synchronous : input is consumed entirely and produces 1+ compressed blocks.
- Caller must ensure there is enough space in `dst` to store compressed data under worst case scenario.
Worst case evaluation is provided by ZSTD_compressBound().
ZSTD_compressContinue() doesn't guarantee recover after a failed compression.
- ZSTD_compressContinue() presumes prior input ***is still accessible and unmodified*** (up to maximum distance size, see WindowLog).
It remembers all previous contiguous blocks, plus one separated memory segment (which can itself consists of multiple contiguous blocks)
- ZSTD_compressContinue() detects that prior input has been overwritten when `src` buffer overlaps.
In which case, it will "discard" the relevant memory section from its history.
Finish a frame with ZSTD_compressEnd(), which will write the last block(s) and optional checksum.
It's possible to use srcSize==0, in which case, it will write a final empty block to end the frame.
Without last block mark, frames are considered unfinished (hence corrupted) by compliant decoders.
`ZSTD_CCtx` object can be re-used (ZSTD_compressBegin()) to compress again.
<BR></pre>
<h3>Buffer-less streaming compression functions</h3><pre></pre><b><pre>size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); </b>/**< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */<b>
size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); </b>/**< note: fails if cdict==NULL */<b>
size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize); </b>/* compression parameters are already set within cdict. pledgedSrc...
size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); </b>/**< note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */<b>
</pre></b><BR>
<a name="Chapter22"></a><h2>Buffer-less streaming decompression (synchronous mode)</h2><pre>
A ZSTD_DCtx object is required to track streaming operations.
Use ZSTD_createDCtx() / ZSTD_freeDCtx() to manage it.
A ZSTD_DCtx object can be re-used multiple times.
First typical operation is to retrieve frame parameters, using ZSTD_getFrameHeader().
Frame header is extracted from the beginning of compressed frame, so providing only the frame's beginning is enough.
Data fragment must be large enough to ensure successful decoding.
`ZSTD_frameHeaderSize_max` bytes is guaranteed to always be large enough.
@result : 0 : successful decoding, the `ZSTD_frameHeader` structure is correctly filled.
>0 : `srcSize` is too small, please provide at least @result bytes on next attempt.
errorCode, which can be tested using ZSTD_isError().
It fills a ZSTD_frameHeader structure with important information to correctly decode the frame,
such as the dictionary ID, content size, or maximum back-reference distance (`windowSize`).
Note that these values could be wrong, either because of data corruption, or because a 3rd party deliberately spoofs false information.
As a consequence, check that values remain within valid application range.
For example, do not allocate memory blindly, check that `windowSize` is within expectation.
Each application can set its own limits, depending on local restrictions.
For extended interoperability, it is recommended to support `windowSize` of at least 8 MB.
ZSTD_decompressContinue() needs previous data blocks during decompression, up to `windowSize` bytes.
ZSTD_decompressContinue() is very sensitive to contiguity,
if 2 blocks don't follow each other, make sure that either the compressor breaks contiguity at the same place,
or that previous contiguous segment is large enough to properly handle maximum back-reference distance.
There are multiple ways to guarantee this condition.
The most memory efficient way is to use a round buffer of sufficient size.
Sufficient size is determined by invoking ZSTD_decodingBufferSize_min(),
which can @return an error code if required value is too large for current system (in 32-bits mode).
In a round buffer methodology, ZSTD_decompressContinue() decompresses each block next to previous one,
up to the moment there is not enough room left in the buffer to guarantee decoding another full block,
which maximum size is provided in `ZSTD_frameHeader` structure, field `blockSizeMax`.
At which point, decoding can resume from the beginning of the buffer.
Note that already decoded data stored in the buffer should be flushed before being overwritten.
There are alternatives possible, for example using two or more buffers of size `windowSize` each, though they consume more memory.
Finally, if you control the compression process, you can also ignore all buffer size rules,
as long as the encoder and decoder progress in "lock-step",
aka use exactly the same buffer sizes, break contiguity at the same place, etc.
Once buffers are setup, start decompression, with ZSTD_decompressBegin().
If decompression requires a dictionary, use ZSTD_decompressBegin_usingDict() or ZSTD_decompressBegin_usingDDict().
Then use ZSTD_nextSrcSizeToDecompress() and ZSTD_decompressContinue() alternatively.
ZSTD_nextSrcSizeToDecompress() tells how many bytes to provide as 'srcSize' to ZSTD_decompressContinue().
ZSTD_decompressContinue() requires this _exact_ amount of bytes, or it will fail.
@result of ZSTD_decompressContinue() is the number of bytes regenerated within 'dst' (necessarily <= dstCapacity).
It can be zero : it just means ZSTD_decompressContinue() has decoded some metadata item.
It can also be an error code, which can be tested with ZSTD_isError().
A frame is fully decoded when ZSTD_nextSrcSizeToDecompress() returns zero.
Context can then be reset to start a new decompression.
Note : it's possible to know if next input to present is a header or a block, using ZSTD_nextInputType().
This information is not required to properly decode a frame.
== Special case : skippable frames
Skippable frames allow integration of user-defined data into a flow of concatenated frames.
Skippable frames will be ignored (skipped) by decompressor.
The format of skippable frames is as follows :
a) Skippable frame ID - 4 Bytes, Little endian format, any value from 0x184D2A50 to 0x184D2A5F
b) Frame Size - 4 Bytes, Little endian format, unsigned 32-bits
c) Frame Content - any content (User Data) of length equal to Frame Size
For skippable frames ZSTD_getFrameHeader() returns zfhPtr->frameType==ZSTD_skippableFrame.
For skippable frames ZSTD_decompressContinue() always returns 0 : it only skips the content.
<BR></pre>
<h3>Buffer-less streaming decompression functions</h3><pre></pre><b><pre>typedef enum { ZSTD_frame, ZSTD_skippableFrame } ZSTD_frameType_e;
typedef struct {
unsigned long long frameContentSize; </b>/* if == ZSTD_CONTENTSIZE_UNKNOWN, it means this field is not available. 0 means "empty" */<b>
unsigned long long windowSize; </b>/* can be very large, up to <= frameContentSize */<b>
unsigned blockSizeMax;
ZSTD_frameType_e frameType; </b>/* if == ZSTD_skippableFrame, frameContentSize is the size of skippable content */<b>
unsigned headerSize;
unsigned dictID;
unsigned checksumFlag;
} ZSTD_frameHeader;
</pre></b><BR>
<pre><b>size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize); </b>/**< doesn't consume input */<b>
</b>/*! ZSTD_getFrameHeader_advanced() :<b>
* same as ZSTD_getFrameHeader(),
* with added capability to select a format (like ZSTD_f_zstd1_magicless) */
size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format);
size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize); </b>/**< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */<b>
</b><p> decode Frame Header, or requires larger `srcSize`.
@return : 0, `zfhPtr` is correctly filled,
>0, `srcSize` is too small, value is wanted `srcSize` amount,
or an error code, which can be tested using ZSTD_isError()
</p></pre><BR>
<pre><b>typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e;
</b></pre><BR>
<a name="Chapter23"></a><h2>Block level API</h2><pre></pre>
<pre><b></b><p> Frame metadata cost is typically ~18 bytes, which can be non-negligible for very small blocks (< 100 bytes).
But users will have to take in charge needed metadata to regenerate data, such as compressed and content sizes.
A few rules to respect :
- Compressing and decompressing require a context structure
+ Use ZSTD_createCCtx() and ZSTD_createDCtx()
- It is necessary to init context before starting
+ compression : any ZSTD_compressBegin*() variant, including with dictionary
+ decompression : any ZSTD_decompressBegin*() variant, including with dictionary
+ copyCCtx() and copyDCtx() can be used too
- Block size is limited, it must be <= ZSTD_getBlockSize() <= ZSTD_BLOCKSIZE_MAX == 128 KB
+ If input is larger than a block size, it's necessary to split input data into multiple blocks
+ For inputs larger than a single block, consider using regular ZSTD_compress() instead.
Frame metadata is not that costly, and quickly becomes negligible as source size grows larger than a block.
- When a block is considered not compressible enough, ZSTD_compressBlock() result will be 0 (zero) !
===> In which case, nothing is produced into `dst` !
( run in 0.554 second using v1.01-cache-2.11-cpan-39bf76dae61 )