Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / February 2006

Tip: Looking for answers? Try searching our database.

Loading a jpeg really fast

Thread view: 
E. Naubauer - 06 Feb 2006 06:30 GMT
Hello folks, I have a huge problem and I'm pretty desperate now

For my diploma, I hav to code an app that basically reads jpeg images
from an Axis 2420 cameras mjpeg stream, decodes them, manipulates the
resulting rgb images an draws them back to the screen. I cannot use the
standard jai classes since every image is wrapped in a HTTP message
which I have to remove.

I can read the jfif data to a byte array. I also have access to an input
stream. I tried several options, but they all didn't give me the results
I needed:

- Getting an Image object with

Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j);

and drawing it with g.drawImage paint routine is VERY FAST.
However, since Image is an abstract class, I have no access to the pixel
 data which I need to manipulate. So I tried option number 2:

- Getting a BufferedImage Object with

decoder = JPEGCodec.createJPEGDecoder(datainstream);               
myBufferedImage = decoder.decodeAsBufferedImage();

This gives me direct access to the data, but I had to use a media
tracker so the images are correctly drawn. However, this took very long
(about 200 ms fon one image!) which is inacceptable since I still need
to manipulate the data.

I also tried

- Getting a RenderedImage with

ImageDecoder dec = ImageCodec.createImageDecoder(names[0], myinstream,
null);
image = dec.decodeAsRenderedImage();

and drawing it with

Graphics2D g2 = (Graphics2D)g;
g2.drawRenderedImage(image,AffineTransform.getTranslateInstance(0, 0));

It was even worse.

the fourth option, ImageIO didn't work at all. I fed it with my
InputStream Object via the read method, however it didn't seem to read
the jfif data from the stream right (it seemed to read more bytes than
it was supposed to).

I don't have any more clues how to speed this whole thing up. I have a
JNI dll which does the really hard image processing work. It takes a
byte array representing the image thats what I need that for. I tried a
couple of JIT compilers, also JET etc. and it didn't help.

My machine is a 1.25 Ghz Powerbook Alu with Mac Os X Tiger 10.4.4 and
Java Runtime 1.5 .

I would be really grateful for any ideas. Thank you in advance.
Andrey Kuznetsov - 06 Feb 2006 09:08 GMT
> - Getting a BufferedImage Object with
>
[quoted text clipped - 5 lines]
> ms fon one image!) which is inacceptable since I still need to manipulate
> the data.

forget MediaTracker.
It needed only with old java imaging model (ImageProducer/ImageConsumer)

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

Roedy Green - 06 Feb 2006 09:20 GMT
On Mon, 06 Feb 2006 07:30:43 +0100, "E. Naubauer"
<chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who
said :

>Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j);
>
>and drawing it with g.drawImage paint routine is VERY FAST.
>However, since Image is an abstract class, I have no access to the pixel
>  data which I need to manipulate. So I tried option number 2:

Try this little experiment.  

Object o = Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0,
j);

System.out.println( o.getClass() );

Now you will know what sort of beast createImage ACTUALLY gives you.
You can then cast to that and get access to more methods.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

E. Naubauer - 06 Feb 2006 21:50 GMT
Roedy Green schrieb:
> On Mon, 06 Feb 2006 07:30:43 +0100, "E. Naubauer"
> <chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who
[quoted text clipped - 15 lines]
> Now you will know what sort of beast createImage ACTUALLY gives you.
> You can then cast to that and get access to more methods.

Actually it seems to be an object of the type
OSXImage
which shoudln't surprise since I'm using Mac Os X 10.4   :)
It offers a method named
getBufferedImage
which indeed delivers what it promises. However, it still doesn't run
that fast and I wonder if it isn't a different problem than jpeg decoding.

I might post the critical sourcecode parts, but they are pretty big.
All in all, my program has 3 Threads:

- 2 Threads for decoding the camera images
- 1 Thread for drawing to screen

This is the run Method of the camera threads:

public void run()
{
        //test if still connected
        if(!connected)
            return;
               
           
        try
        {
            //decode stream
            if(connected)
            {
                                                    while(!Thread.currentThread().isInterrupted())
            {       
                //BufferedImage. Can be accessed from
                //outside via getLatestImage() method
                image = null;       
                   
                String header;
                int k = 0;

                   do
                      header = datainstream.readLine();
                   while(++k < 3);
       
                   datainstream.readByte();
                   int j = (new Integer(header.substring("Content-Length:
".length()))).intValue();   
                   datainstream.readByte();   
                   byte abyte1[] = new byte[j + 2];
                   datainstream.readFully(abyte1, 0, j + 2);       
                   datainstream.readLine();
               //header decoding ends here
       
       
                   //abyte1 contains the raw Jfif data now which can be
decoded by the toolkit
                   Image i = Toolkit.getDefaultToolkit().createImage(abyte1,
0,j+2);
                   //Thread.currentThread().sleep(1000);
                   //System.out.println(o.toString());
       

               //I have to use MediaTracker, or the image drawn later is all white
                   MediaTracker tracker = new MediaTracker(new Frame());
                       tracker.addImage(i,0);
                  
                       try {
                       tracker.waitForAll();
                       } catch(InterruptedException iex)
                       {
                           iex.printStackTrace();
                       }
                      
                       tracker.removeImage(i);
       
               //we know that the image is from type apple.awt.OSXImage
                   image = ((apple.awt.OSXImage)i).getBufferedImage();
       
               //invokes stateChanged() methods on all objects that are
interested in the images
                   fireChange();
       
       
                   //decodeNext = false;
                   //((apple.awt.OSXImage)o).preload(this);
       
                   
                }
            }
            catch(Exception ex)
            {
                ex.printStackTrace();
            }
            finally
            {
               
                try {
                datainstream.close();
                } catch(Exception ex)
                {
                    ex.printStackTrace();
                }
            }
}

So the thread calls method stateChanged() via fireChange() on another
object which shows interest in the images. It looks like this:

class MainProgram
extends Frame
implements Runnable,ChangeListener
{

<.....>

BufferedImage currentLeftImage,currentRightImage;
Runnable drawThread;

CameraCanvas leftCameraCanvas = null;
CameraCanvas rightCameraCanvas = null;

<.....>

public void stateChanged(ChangeEvent e)
{
           
            if(e.getSource().equals(leftCamera))
            {
     
            currentLeftImage = leftCamera.getLatestImage();
                       
       
            }
            if(e.getSource().equals(rightCamera))
            {
               
               
            currentRightImage = rightCamera.getLatestImage();
                   
               
            }
           
        //invoke the drawing thread
            EventQueue.invokeLater(drawThread);
           
}

}

The drawing thread resides on the same class and looks like this

drawThread = new Runnable()
{
           
            public void run()
            {
                if(!readyToDraw)
                    return;
               
                repaint();
               
                try {
                Thread.currentThread().sleep(1);
                } catch(InterruptedException iex)
                {
                    iex.printStackTrace();
                }
               
            }
};

This class represents also the main window. It has 2 Canvases,
CameraCanvas leftCameraCanvas = null;
CameraCanvas rightCameraCanvas = null;

This is the paint() Method of MainProgram that is called by the
drawThread thread

 public void paint(Graphics g)
 {
           
        leftCameraCanvas.SetImage(currentLeftImage);
        rightCameraCanvas.SetImage(currentRightImage);
        leftCameraCanvas.repaint();
        rightCameraCanvas.repaint();
 }

And finally, the source of the CameraCanvas class:

public class CameraCanvas extends Canvas
{
    private BufferedImage image = null;
   
    public void paint(Graphics g)
    {
       
        if(image !=null)
        g.drawImage(image,0,0,getWidth(),getHeight(),null);
       
    }
   
    public void SetImage(BufferedImage i)
    {
        image = i;
    }
}

Maybe you have an idea. I can send you the sourcecode files directly if
you need and want to.
Andrey Kuznetsov - 06 Feb 2006 10:32 GMT
> I can read the jfif data to a byte array. I also have access to an input
> stream. I tried several options, but they all didn't give me the results I
[quoted text clipped - 7 lines]
> However, since Image is an abstract class, I have no access to the pixel
> data which I need to manipulate.

you may try following:

step one: only once - create BufferedImage of desired size and get Graphics
from it.
and then for each frame you need to
step two: create Image with
Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j);
step four: draw loaded image to BufferedImage's Graphics.
step fife: manipulate pixels and so on...
step six : goto step two.

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities
.

Chris Uppal - 06 Feb 2006 12:30 GMT
> I don't have any more clues how to speed this whole thing up. I have a
> JNI dll which does the really hard image processing work. It takes a
> byte array representing the image thats what I need that for.

It might be worthwhile looking a free, fast, JPEG decoding library in C and (if
you find one) adding that to your JNI stuff, so that you bypass using Java APIs
to decode the data in the first place.

   -- chris
Andrey Kuznetsov - 06 Feb 2006 13:21 GMT
> It might be worthwhile looking a free, fast, JPEG decoding library in C
> and (if
> you find one) adding that to your JNI stuff, so that you bypass using Java
> APIs
> to decode the data in the first place.

Java already uses free jpeg decoder written in C - libjpeg.

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

Chris Uppal - 06 Feb 2006 14:18 GMT
[me:]
> > It might be worthwhile looking a free, fast, JPEG decoding library in C
[...]
> Java already uses free jpeg decoder written in C - libjpeg.

That sounds very plausible to me.

The point is that if the OP can't find an /API/ which lets him use the built-in
stuff with adequate speed, then -- since he's already using JNI -- he may find
it worthwhile to use JNI for the decoding too.

In case it's not clear, the idea isn't "use JNI becuase it's quicker than Java"
(not true in this case if Java's using JNI anyway ;-), but that /IF/ the
existing implementation is hidden too deeply for him to be able to get at the
features he needs, he might consider a different way of satisfying the
requirement even if that does mean duplicating an existing feature.

   -- chris
Andrey Kuznetsov - 06 Feb 2006 15:39 GMT
> In case it's not clear, the idea isn't "use JNI becuase it's quicker than
> Java"
[quoted text clipped - 3 lines]
> features he needs, he might consider a different way of satisfying the
> requirement even if that does mean duplicating an existing feature.

ImageIO decoder is also extremely quick.
He must doing something wrong.
I tested it with pretty big image (4000x8000) and it was ready in 1-2
seconds.

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

Oliver Wong - 06 Feb 2006 17:03 GMT
>> In case it's not clear, the idea isn't "use JNI becuase it's quicker than
>> Java"
[quoted text clipped - 8 lines]
> I tested it with pretty big image (4000x8000) and it was ready in 1-2
> seconds.

   According to the OP, 200ms is already "unacceptable" for his purposes,
so 1-2 seconds is 5 to 10 times worse than "unacceptable".

   - Oliver
Andrey Kuznetsov - 06 Feb 2006 18:29 GMT
>> ImageIO decoder is also extremely quick.
>> He must doing something wrong.
[quoted text clipped - 3 lines]
>    According to the OP, 200ms is already "unacceptable" for his purposes,
> so 1-2 seconds is 5 to 10 times worse than "unacceptable".

his camera gives surely much smaller images.
4000x4000 shoud be 2 times faster
and 2000x2000 should  be 8 times faster.

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

E. Naubauer - 06 Feb 2006 18:18 GMT
The reason why I didn't use ImageIO was because it doesn't seem to read
from my stream correctly. As I mentioned earlier, the jpeg frames are
encoded as a stream of http frames which each frame wraps one jpeg
image. Usually I remove the http header from the stream, decode the
image from the stream, remove the trailer etc.. All other decoders
exactly removed the jfif data from the stream and decoded it afterwards.
For some reason, however, ImageIO doesn't seem to stop exactly after the
end of the jfif data so I couldn't determine the streams position and
what to do next. It also seems that ImageIO closes the stream after
reading from it. I tried to work around that issue by creating a
JPEGImageDecoder object manually but it didn't work also. Does anyone
know how much data ImageIO gets from the stream and if it blocks etc.?
Roedy Green - 07 Feb 2006 04:13 GMT
On Mon, 06 Feb 2006 19:18:26 +0100, "E. Naubauer"
<chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who
said :

>It also seems that ImageIO closes the stream after
>reading from it
you could suck the bytes out yourself and feed ImageIO the byte[] or
a ByteArrayInputStream wrapper around it.

I solved this particular problem like this in my JPG streaming
protocol:
-----------------------------------
 byte[] rawImage = conx.receiveBytes( imageLength );

     if ( ! isJpgValid ( rawImage ) )
        {
        throw new IOException ( "Invalid JPG image signature: " +
Integer.toHexString( rawImage[0] ) + " " + Integer.toHexString(
rawImage[1] ) );
        }
     //  start process of converting image from jpg to internal
format
     // old AWT style:
     // Image image = toolkit.createImage( rawImage );

     // new ImageIO style, convert raw bytes to BufferedImage
     BufferedImage image = ImageIO.read ( new ByteArrayInputStream (
rawImage ) );
------------------------------
/**
   * Does this image represent a valid JPG file?
   * Does a rough check for the signature.
   *
   * @param image  array of bytes representing the image
   * @return true if this image has a valid JPG signature.
   */
  public static boolean isJpgValid ( byte image[] )
     {
     return( ( image[0] & 0xff )  == 0xff ) && ( (image[1] & 0xff )
== 0xd8 );
     }
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

E. Naubauer - 07 Feb 2006 06:58 GMT
Indeed, it is significantly faster than the toolkit method.
I tried it with 2 cameras on, drawing 2 images to a canvas each at the
same time. With 1 camera there was a delay of 1-2 seconds between
movement and getting drawn to the screen , with both cameras the delay
took about 5 Seconds.

Well, thanks for all your advice so far.

Do you have benchmarks about how your protocol implementation does with,
say ,2 Streams at once?
Roedy Green - 07 Feb 2006 12:35 GMT
On Tue, 07 Feb 2006 07:58:55 +0100, "E. Naubauer"
<chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who
said :

>Do you have benchmarks about how your protocol implementation does with,
>say ,2 Streams at once?

We ran maybe 10 streams at once. These were security cameras, pretty
crappy image quality, some with motion detect.

We had a server running in C that talked to the camera hardware. I
wrote the Java client viewers.  Fun stuff. I had "magnaviewers" as
well where you could zoom the image size up and down smoothly.

We have two modes, live and historic. With historic you could look
back in time like a VCR fast forwarding or reversing with a speed
slider.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

E. Naubauer - 07 Feb 2006 21:49 GMT
Ok, it still doesn't work really good.
I pasted the essential parts of the code into a small demo, added a
frame to it for display and it runs just as bad as the complete thing.
The delay between movement and drawing ia about five seconds, getting
worse with the time. If you have the time to check it out for a sec and
find something problematic, please let me know. Otherwise I will have to
port the whole thing to C. Thanks in any case.

import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.InputStream;
import java.net.*;
import java.awt.*;
import java.awt.image.*;

import javax.imageio.ImageIO;
public class Test
{
   
    private static HttpURLConnection hurlconnection = null;
    private static DataInputStream datainstream = null;
    private static String serialNumber = null;
    private static boolean connected = false;
    private static BufferedImage image = null;
   
    public static void main(String[] args) throws Exception
    {
       
        URL streamurl = new
URL("http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&s
howlength=1&fps=10
");
        hurlconnection = (HttpURLConnection)streamurl.openConnection();
        hurlconnection.setReadTimeout(10000);   
        InputStream instream = hurlconnection.getInputStream();
       
        connected = true;
        BufferedInputStream bis = new BufferedInputStream(instream);
        datainstream = new DataInputStream(bis);
       
        Frame frm = new Frame();
        frm.setSize(200,200);
        frm.setVisible(true);
           
        //decodiere stream
        if(connected)
        {
            Image i;
            while(!Thread.currentThread().isInterrupted())
            {               
                image = null;
                i = null;
               
                String header;
                int k = 0;
               
                while(image == null)
                {
                   
                    k = 0;
                    do
                    {
                        header = datainstream.readLine();
                    }
                    while(++k < 3);
   
                   datainstream.readByte();
                   int j = (new Integer(header.substring("Content-Length:
".length()))).intValue();   
                   datainstream.readByte();   
                   byte abyte1[] = new byte[j + 2];
                   datainstream.readFully(abyte1, 0, j + 2);       
                   datainstream.readLine();
       
                   ByteArrayInputStream bin = new ByteArrayInputStream (abyte1);
                   image = ImageIO.read ( bin );
   
                } //end while image==null
   
                long before = System.currentTimeMillis();
                Graphics2D g2 = (Graphics2D)frm.getGraphics();
                frm.getGraphics().drawImage(image,0,0,null);
                long after = System.currentTimeMillis();
                System.out.println("Duration: " +(after-before));
   
                //Thread.currentThread().sleep(100);
            }
        }
    }
}
bugbear - 07 Feb 2006 10:49 GMT
> The reason why I didn't use ImageIO was because it doesn't seem to read
> from my stream correctly. As I mentioned earlier, the jpeg frames are
[quoted text clipped - 6 lines]
> what to do next. It also seems that ImageIO closes the stream after
> reading from it.

Sounds like simple buffering to me; if it
decorates its InputStream with a BufferredInputStream
(for performance), the BufferredInputStream knows
nothing about the boundaries in the InputStream.

Buffering Systems read ahead - that's the whole point!

   BugBear
bugbear - 06 Feb 2006 16:05 GMT
> Hello folks, I have a huge problem and I'm pretty desperate now
>
[quoted text clipped - 3 lines]
> standard jai classes since every image is wrapped in a HTTP message
> which I have to remove.

Hmm. Any way to create a simple, buffered, stream decorator
(AKA filter) ?

  BugBear
Richard Wheeldon - 06 Feb 2006 22:12 GMT
> Hello folks, I have a huge problem and I'm pretty desperate now
>
> For my diploma, I hav to code an app that basically reads jpeg images
> from an Axis 2420 cameras mjpeg stream, decodes them, manipulates the
> resulting rgb images an draws them back to the screen.

Go to the web server on the Axis camera. Download the jar
file for the display applet. Decompile it. If you want a good
decompiler, try here: http://www.kpdus.com/jad.html

There's also a java program to display images from Axis
cameras on sourceforge,

Richard
E. Naubauer - 06 Feb 2006 23:38 GMT
I know about that implementation. The interesting point here is that
they directly take the abstract image object produced by
Toolkit.getDefaultToolkit().createImage
and pass it to drawimage in the paint method of the displaying frame.
For some reason, this works perfectly even without a media tracker to
ensure that the images are fully loaded before drawing.
Andrey Kuznetsov - 07 Feb 2006 07:20 GMT
>I know about that implementation. The interesting point here is that they
>directly take the abstract image object produced by
> Toolkit.getDefaultToolkit().createImage
> and pass it to drawimage in the paint method of the displaying frame. For
> some reason, this works perfectly even without a media tracker to ensure
> that the images are fully loaded before drawing.

probably because they also passed some Component as ImageObserver to
drawImage method.

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

E. Naubauer - 07 Feb 2006 17:47 GMT
Richard Wheeldon schrieb:

> There's also a java program to display images from Axis
> cameras on sourceforge,
>
> Richard

Would you mind sending me the url (if you have it) ?
Maybe I can learn a few things from it. tx
Richard Wheeldon - 08 Feb 2006 00:02 GMT
> Would you mind sending me the url (if you have it) ?
> Maybe I can learn a few things from it. tx

Sorry. If I had it, I'd have posted it. Try a search
for axis on sf.net,

Richard
slippymississippi@yahoo.com - 07 Feb 2006 22:12 GMT
Let me get this straight.  Unless you can download two images
simultaneously in under 200ms, you will not receive your degree.  WTH
kind of college are you going to?

:)


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.