view release on metacpan or  search on metacpan

src/Source/LibJPEG/structure.txt  view on Meta::CPAN

The underlying type is at least a 16-bit signed integer; while "short" is big
enough on all machines of interest, on some machines it is preferable to use
"int" for speed reasons, despite the storage cost.  Coefficients are grouped
into 8x8 blocks (but we always use #defines DCTSIZE and DCTSIZE2 rather than
"8" and "64").

The contents of a coefficient block may be in either "natural" or zigzagged
order, and may be true values or divided by the quantization coefficients,
depending on where the block is in the processing pipeline.  In the current
library, coefficient blocks are kept in natural order everywhere; the entropy
codecs zigzag or dezigzag the data as it is written or read.  The blocks
contain quantized coefficients everywhere outside the DCT/IDCT subsystems.
(This latter decision may need to be revisited to support variable
quantization a la JPEG Part 3.)

Notice that the allocation unit is now a row of 8x8 coefficient blocks,
corresponding to block_size rows of samples.  Otherwise the structure
is much the same as for samples, and for the same reasons.

On machines where malloc() can't handle a request bigger than 64Kb, this data
structure limits us to rows of less than 512 JBLOCKs, or a picture width of
4000+ pixels.  This seems an acceptable restriction.


On 80x86 machines, the bottom-level pointer types (JSAMPROW and JBLOCKROW)
must be declared as "far" pointers, but the upper levels can be "near"
(implying that the pointer lists are allocated in the DS segment).
We use a #define symbol FAR, which expands to the "far" keyword when
compiling on 80x86 machines and to nothing elsewhere.


*** Suspendable processing ***

In some applications it is desirable to use the JPEG library as an
incremental, memory-to-memory filter.  In this situation the data source or
destination may be a limited-size buffer, and we can't rely on being able to
empty or refill the buffer at arbitrary times.  Instead the application would
like to have control return from the library at buffer overflow/underrun, and
then resume compression or decompression at a later time.

This scenario is supported for simple cases.  (For anything more complex, we
recommend that the application "bite the bullet" and develop real multitasking
capability.)  The libjpeg.txt file goes into more detail about the usage and
limitations of this capability; here we address the implications for library
structure.

The essence of the problem is that the entropy codec (coder or decoder) must
be prepared to stop at arbitrary times.  In turn, the controllers that call
the entropy codec must be able to stop before having produced or consumed all
the data that they normally would handle in one call.  That part is reasonably
straightforward: we make the controller call interfaces include "progress
counters" which indicate the number of data chunks successfully processed, and
we require callers to test the counter rather than just assume all of the data
was processed.

Rather than trying to restart at an arbitrary point, the current Huffman
codecs are designed to restart at the beginning of the current MCU after a
suspension due to buffer overflow/underrun.  At the start of each call, the
codec's internal state is loaded from permanent storage (in the JPEG object
structures) into local variables.  On successful completion of the MCU, the
permanent state is updated.  (This copying is not very expensive, and may even
lead to *improved* performance if the local variables can be registerized.)
If a suspension occurs, the codec simply returns without updating the state,
thus effectively reverting to the start of the MCU.  Note that this implies
leaving some data unprocessed in the source/destination buffer (ie, the
compressed partial MCU).  The data source/destination module interfaces are
specified so as to make this possible.  This also implies that the data buffer
must be large enough to hold a worst-case compressed MCU; a couple thousand
bytes should be enough.

In a successive-approximation AC refinement scan, the progressive Huffman
decoder has to be able to undo assignments of newly nonzero coefficients if it
suspends before the MCU is complete, since decoding requires distinguishing
previously-zero and previously-nonzero coefficients.  This is a bit tedious
but probably won't have much effect on performance.  Other variants of Huffman
decoding need not worry about this, since they will just store the same values
again if forced to repeat the MCU.

This approach would probably not work for an arithmetic codec, since its
modifiable state is quite large and couldn't be copied cheaply.  Instead it
would have to suspend and resume exactly at the point of the buffer end.

The JPEG marker reader is designed to cope with suspension at an arbitrary
point.  It does so by backing up to the start of the marker parameter segment,
so the data buffer must be big enough to hold the largest marker of interest.
Again, a couple KB should be adequate.  (A special "skip" convention is used
to bypass COM and APPn markers, so these can be larger than the buffer size
without causing problems; otherwise a 64K buffer would be needed in the worst
case.)

The JPEG marker writer currently does *not* cope with suspension.
We feel that this is not necessary; it is much easier simply to require
the application to ensure there is enough buffer space before starting.  (An
empty 2K buffer is more than sufficient for the header markers; and ensuring
there are a dozen or two bytes available before calling jpeg_finish_compress()
will suffice for the trailer.)  This would not work for writing multi-scan
JPEG files, but we simply do not intend to support that capability with
suspension.


*** Memory manager services ***

The JPEG library's memory manager controls allocation and deallocation of
memory, and it manages large "virtual" data arrays on machines where the
operating system does not provide virtual memory.  Note that the same
memory manager serves both compression and decompression operations.

In all cases, allocated objects are tied to a particular compression or
decompression master record, and they will be released when that master
record is destroyed.

The memory manager does not provide explicit deallocation of objects.
Instead, objects are created in "pools" of free storage, and a whole pool
can be freed at once.  This approach helps prevent storage-leak bugs, and
it speeds up operations whenever malloc/free are slow (as they often are).
The pools can be regarded as lifetime identifiers for objects.  Two
pools/lifetimes are defined:
  * JPOOL_PERMANENT	lasts until master record is destroyed
  * JPOOL_IMAGE		lasts until done with image (JPEG datastream)
Permanent lifetime is used for parameters and tables that should be carried
across from one datastream to another; this includes all application-visible