Figure 1. Typical Content Handler
Our image loader must therefore be designed to accept the image data (e.g.
contents of the file picture.jpg) as a stream, and not as a file. This
means that the image loaders will have to work with sequential data, not
random access data. This is illustrated in Figure 2.
Figure 2. Content Handler invoking ImageDecoder
Architecture
Java side.
The image loader is based around the ImageDecoder Java class (shown in
Figure 2). The content handler is merely the
mechanism that invokes the ImageDecoder. It has no bearing on the way the
image handling works, so it will no longer be discussed.
The ImageDecoder is a class that takes an InputStream containing data from an image file, and returns an Image (a BufferedImage if using Java2D (which is provided with JDK1.2)). Figure 3 shows the internals of this class. From this figure we can see that the decoding of image data into pixel data is done natively. The ImageDecoder passes the image data to the native code, which returns pixel data row at a time. The ImageDecoder converts this pixel data into an Image object and returns it.
The basic algorithm that the ImageDecoder follows is thus:
The first thing to note from this is that the image type is
specifically given to the native code. This is not inferred from the image
data input stream. This simplifies the native code, in that it doesn't have
to read and buffer header information in determining the image type. It
does, however, introduce a problem if the incorrect image type is passed.
For example, a GIF file may have an extension of .bmp. This means that the
image data will not be recognised as a GIF image and will return an error to
that effect. This is desired behaviour as the file extension is a well
defined method of identifying the file contents and should match correctly.
In step 2 of the algorithm, we send all the image data to the native code.
In step 3 we read in the header to determine the image properties. If the
header is at the beginning of the image data, it is not necessary to
transfer the entire image contents before reading the header. These steps
can operate somewhat in parallel. The native code can read in and process
the header, while the rest of the image data is still arriving. In our
implementation this is achieved by running step 2 inside a separate thread.
So Figure 3 can be implemented as shown in
Figure 4. Here step 2 runs in a separate thread in
parallel with steps 3, 4 and 5. There is a case where step 2 is not run in
a separate thread and the algorithm is run sequentially in a single thread.
This is discussed in the next section.
The image that is created from the pixel data is of a DirectColorModel,
using 32 bits per pixel (as in the default Java ColorModel). This uses 8
bits for each of the alpha, red, green and blue colour components. In the
future, the image will be created using 16 bits per pixel (1bit for alpha,
and 5 bits for each of red, green and blue). Due to a Java bug, this can
not be achieved at the moment, but when this is implemented, the memory
usage for each image will halve.
The first (and simplest) method, is to create a temporary file and write all
the image data received from the Java side to this file. When all data has
been received, we then open this file and pass this newly opened file
descriptor to the decoding routines. This requires a sequential approach to
the algorithm described on the Java side. The
reason for this, is that when reading from a file which has another
process/thread writing to it, there is no way to know when all writing has
been finished. So to ensure safety, all writing must be finished and the
file closed, before reading from this temporary file can commence.
There are disadvantages with this method. Firstly, the sequential approach
means that we can't interleave steps of the algorithm which will increase the
time taken to decode an image. But a more pronounced increase in time is
incurred by the use of a temporary file. Having to first write the data to
disk, and then read it back incurs memory to disk, and disk to memory
traffic. This introduces a significant slowdown to the decoding process.
The second method is slightly more complex. It involves setting up a pipe.
The Java side will send data to the writing end of the pipe, and the file
descriptor of the reading end is passed to the image decoding routines. For
this to be achieved, the Java side must send the data in a
separate thread. The reason for this is that when the pipe fills, the
writing thread will block, waiting for the data to be read from the pipe.
If the algorithm proceeds sequentially, the thread will block when the pipe
fills, and will never unblock as there is nothing to empty to pipe.
This method is more efficient in terms of speed, as we skip the overhead of
writing all the data to disk, and then reading it back again. The
programmer must be aware that the file descriptor passed to the image
decoding routines is only capable of sequential access, not random access.
This is essentially not a problem as most image formats can be decoded
sequentially (see the discussion of Tiff for
an exception).
However there is an implementation problem with this method and green
threads under Unix. If the reading end of the pipe tries to read and the
pipe is empty, it will block until data is available (or the writing end is
closed). This is correct behaviour. However with green threads, if one
thread blocks on a call to read(2), then the entire process will block.
This introduces a race condition. Everything will work correctly as long as
the reader does not try to read from an empty pipe. There is unfortunately
no way to guarantee this. This means that under these conditions we must
use the first (and slower) method.
The metrics are for loading in a particular image of dimensions 4096x4096
and of 32 bits colour depth. The memory that such an image should consume
is 64 megabytes. The memory usage figures are approximate as they were
taken using the MEM Usage facility of the NT Task Manager.
Figure 3. ImageDecoder communicating with Native Code.
Figure 4. Threaded ImageDecoder
Native Code.
A commonality with the image libraries that we have, is that they all accept
their image data from an open file descriptor. They do not require a
filename and perform the opening themselves. This has led us to develop two
methods of passing the image data to the decoding routines.
Issues with Tiff
Tiff (Tagged Image Format) is an image format that cannot decode an image by
reading sequentially from the data stream. This format stores a directory
in the image file which contains information about the images contained
within the file. When decoding, we need to seek to the directory, read and
process the directory, then seek to where the offset where the image is
stored in the file. If we are using the second method as described above,
then the decoding will not work because we cannot seek. Our solution to
this was to firstly read all of the image data into memory buffer, and then
write our own read, and seek routines to access the data from this buffer,
rather than using the file descriptor.
Performance
Table 1 shows timings and memory usage comparisons
loading and displaying various image formats using jdk1.1.5, and jdk1.2rc1.
These measurements were taken on a PC running windows NT. Measurements were
all taken under approximately the same cpu and network load. The timings
are not to be taken as absolute, they are given to show the relative
difference between the different methods. Measurements given are the
average of four runs, under near identical conditions.