view release on metacpan or  search on metacpan

src/Source/LibJPEG/structure.txt  view on Meta::CPAN

peak memory usage would be about the same anyway; and having per-pass storage
substantially complicates the virtual memory allocation rules --- see below.)

The memory manager deals with three kinds of object:
1. "Small" objects.  Typically these require no more than 10K-20K total.
2. "Large" objects.  These may require tens to hundreds of K depending on
   image size.  Semantically they behave the same as small objects, but we
   distinguish them for two reasons:
     * On MS-DOS machines, large objects are referenced by FAR pointers,
       small objects by NEAR pointers.
     * Pool allocation heuristics may differ for large and small objects.
   Note that individual "large" objects cannot exceed the size allowed by
   type size_t, which may be 64K or less on some machines.
3. "Virtual" objects.  These are large 2-D arrays of JSAMPLEs or JBLOCKs
   (typically large enough for the entire image being processed).  The
   memory manager provides stripwise access to these arrays.  On machines
   without virtual memory, the rest of the array may be swapped out to a
   temporary file.

(Note: JSAMPARRAY and JBLOCKARRAY data structures are a combination of large
objects for the data proper and small objects for the row pointers.  For
convenience and speed, the memory manager provides single routines to create
these structures.  Similarly, virtual arrays include a small control block
and a JSAMPARRAY or JBLOCKARRAY working buffer, all created with one call.)

In the present implementation, virtual arrays are only permitted to have image
lifespan.  (Permanent lifespan would not be reasonable, and pass lifespan is
not very useful since a virtual array's raison d'etre is to store data for
multiple passes through the image.)  We also expect that only "small" objects
will be given permanent lifespan, though this restriction is not required by
the memory manager.

In a non-virtual-memory machine, some performance benefit can be gained by
making the in-memory buffers for virtual arrays be as large as possible.
(For small images, the buffers might fit entirely in memory, so blind
swapping would be very wasteful.)  The memory manager will adjust the height
of the buffers to fit within a prespecified maximum memory usage.  In order
to do this in a reasonably optimal fashion, the manager needs to allocate all
of the virtual arrays at once.  Therefore, there isn't a one-step allocation
routine for virtual arrays; instead, there is a "request" routine that simply
allocates the control block, and a "realize" routine (called just once) that
determines space allocation and creates all of the actual buffers.  The
realize routine must allow for space occupied by non-virtual large objects.
(We don't bother to factor in the space needed for small objects, on the
grounds that it isn't worth the trouble.)

To support all this, we establish the following protocol for doing business
with the memory manager:
  1. Modules must request virtual arrays (which may have only image lifespan)
     during the initial setup phase, i.e., in their jinit_xxx routines.
  2. All "large" objects (including JSAMPARRAYs and JBLOCKARRAYs) must also be
     allocated during initial setup.
  3. realize_virt_arrays will be called at the completion of initial setup.
     The above conventions ensure that sufficient information is available
     for it to choose a good size for virtual array buffers.
Small objects of any lifespan may be allocated at any time.  We expect that
the total space used for small objects will be small enough to be negligible
in the realize_virt_arrays computation.

In a virtual-memory machine, we simply pretend that the available space is
infinite, thus causing realize_virt_arrays to decide that it can allocate all
the virtual arrays as full-size in-memory buffers.  The overhead of the
virtual-array access protocol is very small when no swapping occurs.

A virtual array can be specified to be "pre-zeroed"; when this flag is set,
never-yet-written sections of the array are set to zero before being made
available to the caller.  If this flag is not set, never-written sections
of the array contain garbage.  (This feature exists primarily because the
equivalent logic would otherwise be needed in jdcoefct.c for progressive
JPEG mode; we may as well make it available for possible other uses.)

The first write pass on a virtual array is required to occur in top-to-bottom
order; read passes, as well as any write passes after the first one, may
access the array in any order.  This restriction exists partly to simplify
the virtual array control logic, and partly because some file systems may not
support seeking beyond the current end-of-file in a temporary file.  The main
implication of this restriction is that rearrangement of rows (such as
converting top-to-bottom data order to bottom-to-top) must be handled while
reading data out of the virtual array, not while putting it in.


*** Memory manager internal structure ***

To isolate system dependencies as much as possible, we have broken the
memory manager into two parts.  There is a reasonably system-independent
"front end" (jmemmgr.c) and a "back end" that contains only the code
likely to change across systems.  All of the memory management methods
outlined above are implemented by the front end.  The back end provides
the following routines for use by the front end (none of these routines
are known to the rest of the JPEG code):

jpeg_mem_init, jpeg_mem_term	system-dependent initialization/shutdown

jpeg_get_small, jpeg_free_small	interface to malloc and free library routines
				(or their equivalents)

jpeg_get_large, jpeg_free_large	interface to FAR malloc/free in MSDOS machines;
				else usually the same as
				jpeg_get_small/jpeg_free_small

jpeg_mem_available		estimate available memory

jpeg_open_backing_store		create a backing-store object

read_backing_store,		manipulate a backing-store object
write_backing_store,
close_backing_store

On some systems there will be more than one type of backing-store object
(specifically, in MS-DOS a backing store file might be an area of extended
memory as well as a disk file).  jpeg_open_backing_store is responsible for
choosing how to implement a given object.  The read/write/close routines
are method pointers in the structure that describes a given object; this
lets them be different for different object types.

It may be necessary to ensure that backing store objects are explicitly
released upon abnormal program termination.  For example, MS-DOS won't free
extended memory by itself.  To support this, we will expect the main program
or surrounding application to arrange to call self_destruct (typically via
jpeg_destroy) upon abnormal termination.  This may require a SIGINT signal
handler or equivalent.  We don't want to have the back end module install its