Java Forum / General / February 2006
Loading a jpeg really fast
E. Naubauer - 06 Feb 2006 06:30 GMT Hello folks, I have a huge problem and I'm pretty desperate now
For my diploma, I hav to code an app that basically reads jpeg images from an Axis 2420 cameras mjpeg stream, decodes them, manipulates the resulting rgb images an draws them back to the screen. I cannot use the standard jai classes since every image is wrapped in a HTTP message which I have to remove.
I can read the jfif data to a byte array. I also have access to an input stream. I tried several options, but they all didn't give me the results I needed:
- Getting an Image object with
Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j);
and drawing it with g.drawImage paint routine is VERY FAST. However, since Image is an abstract class, I have no access to the pixel data which I need to manipulate. So I tried option number 2:
- Getting a BufferedImage Object with
decoder = JPEGCodec.createJPEGDecoder(datainstream); myBufferedImage = decoder.decodeAsBufferedImage();
This gives me direct access to the data, but I had to use a media tracker so the images are correctly drawn. However, this took very long (about 200 ms fon one image!) which is inacceptable since I still need to manipulate the data.
I also tried
- Getting a RenderedImage with
ImageDecoder dec = ImageCodec.createImageDecoder(names[0], myinstream, null); image = dec.decodeAsRenderedImage();
and drawing it with
Graphics2D g2 = (Graphics2D)g; g2.drawRenderedImage(image,AffineTransform.getTranslateInstance(0, 0));
It was even worse.
the fourth option, ImageIO didn't work at all. I fed it with my InputStream Object via the read method, however it didn't seem to read the jfif data from the stream right (it seemed to read more bytes than it was supposed to).
I don't have any more clues how to speed this whole thing up. I have a JNI dll which does the really hard image processing work. It takes a byte array representing the image thats what I need that for. I tried a couple of JIT compilers, also JET etc. and it didn't help.
My machine is a 1.25 Ghz Powerbook Alu with Mac Os X Tiger 10.4.4 and Java Runtime 1.5 .
I would be really grateful for any ideas. Thank you in advance.
Andrey Kuznetsov - 06 Feb 2006 09:08 GMT > - Getting a BufferedImage Object with > [quoted text clipped - 5 lines] > ms fon one image!) which is inacceptable since I still need to manipulate > the data. forget MediaTracker. It needed only with old java imaging model (ImageProducer/ImageConsumer)
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities
Roedy Green - 06 Feb 2006 09:20 GMT On Mon, 06 Feb 2006 07:30:43 +0100, "E. Naubauer" <chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who said :
>Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j); > >and drawing it with g.drawImage paint routine is VERY FAST. >However, since Image is an abstract class, I have no access to the pixel > data which I need to manipulate. So I tried option number 2: Try this little experiment.
Object o = Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j);
System.out.println( o.getClass() );
Now you will know what sort of beast createImage ACTUALLY gives you. You can then cast to that and get access to more methods.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
E. Naubauer - 06 Feb 2006 21:50 GMT Roedy Green schrieb:
> On Mon, 06 Feb 2006 07:30:43 +0100, "E. Naubauer" > <chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who [quoted text clipped - 15 lines] > Now you will know what sort of beast createImage ACTUALLY gives you. > You can then cast to that and get access to more methods. Actually it seems to be an object of the type OSXImage which shoudln't surprise since I'm using Mac Os X 10.4 :) It offers a method named getBufferedImage which indeed delivers what it promises. However, it still doesn't run that fast and I wonder if it isn't a different problem than jpeg decoding.
I might post the critical sourcecode parts, but they are pretty big. All in all, my program has 3 Threads:
- 2 Threads for decoding the camera images - 1 Thread for drawing to screen
This is the run Method of the camera threads:
public void run() { //test if still connected if(!connected) return; try { //decode stream if(connected) { while(!Thread.currentThread().isInterrupted()) { //BufferedImage. Can be accessed from //outside via getLatestImage() method image = null; String header; int k = 0;
do header = datainstream.readLine(); while(++k < 3); datainstream.readByte(); int j = (new Integer(header.substring("Content-Length: ".length()))).intValue(); datainstream.readByte(); byte abyte1[] = new byte[j + 2]; datainstream.readFully(abyte1, 0, j + 2); datainstream.readLine(); //header decoding ends here //abyte1 contains the raw Jfif data now which can be decoded by the toolkit Image i = Toolkit.getDefaultToolkit().createImage(abyte1, 0,j+2); //Thread.currentThread().sleep(1000); //System.out.println(o.toString());
//I have to use MediaTracker, or the image drawn later is all white MediaTracker tracker = new MediaTracker(new Frame()); tracker.addImage(i,0); try { tracker.waitForAll(); } catch(InterruptedException iex) { iex.printStackTrace(); } tracker.removeImage(i); //we know that the image is from type apple.awt.OSXImage image = ((apple.awt.OSXImage)i).getBufferedImage(); //invokes stateChanged() methods on all objects that are interested in the images fireChange(); //decodeNext = false; //((apple.awt.OSXImage)o).preload(this); } } catch(Exception ex) { ex.printStackTrace(); } finally { try { datainstream.close(); } catch(Exception ex) { ex.printStackTrace(); } } }
So the thread calls method stateChanged() via fireChange() on another object which shows interest in the images. It looks like this:
class MainProgram extends Frame implements Runnable,ChangeListener {
<.....>
BufferedImage currentLeftImage,currentRightImage; Runnable drawThread;
CameraCanvas leftCameraCanvas = null; CameraCanvas rightCameraCanvas = null;
<.....>
public void stateChanged(ChangeEvent e) { if(e.getSource().equals(leftCamera)) { currentLeftImage = leftCamera.getLatestImage(); } if(e.getSource().equals(rightCamera)) { currentRightImage = rightCamera.getLatestImage(); } //invoke the drawing thread EventQueue.invokeLater(drawThread); }
}
The drawing thread resides on the same class and looks like this
drawThread = new Runnable() { public void run() { if(!readyToDraw) return; repaint(); try { Thread.currentThread().sleep(1); } catch(InterruptedException iex) { iex.printStackTrace(); } } };
This class represents also the main window. It has 2 Canvases, CameraCanvas leftCameraCanvas = null; CameraCanvas rightCameraCanvas = null;
This is the paint() Method of MainProgram that is called by the drawThread thread
public void paint(Graphics g) { leftCameraCanvas.SetImage(currentLeftImage); rightCameraCanvas.SetImage(currentRightImage); leftCameraCanvas.repaint(); rightCameraCanvas.repaint(); }
And finally, the source of the CameraCanvas class:
public class CameraCanvas extends Canvas { private BufferedImage image = null; public void paint(Graphics g) { if(image !=null) g.drawImage(image,0,0,getWidth(),getHeight(),null); } public void SetImage(BufferedImage i) { image = i; } }
Maybe you have an idea. I can send you the sourcecode files directly if you need and want to.
Andrey Kuznetsov - 06 Feb 2006 10:32 GMT > I can read the jfif data to a byte array. I also have access to an input > stream. I tried several options, but they all didn't give me the results I [quoted text clipped - 7 lines] > However, since Image is an abstract class, I have no access to the pixel > data which I need to manipulate. you may try following:
step one: only once - create BufferedImage of desired size and get Graphics from it. and then for each frame you need to step two: create Image with Toolkit.getDefaultToolkit().createImage(myjfifbytedata, 0, j); step four: draw loaded image to BufferedImage's Graphics. step fife: manipulate pixels and so on... step six : goto step two.
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities .
Chris Uppal - 06 Feb 2006 12:30 GMT > I don't have any more clues how to speed this whole thing up. I have a > JNI dll which does the really hard image processing work. It takes a > byte array representing the image thats what I need that for. It might be worthwhile looking a free, fast, JPEG decoding library in C and (if you find one) adding that to your JNI stuff, so that you bypass using Java APIs to decode the data in the first place.
-- chris
Andrey Kuznetsov - 06 Feb 2006 13:21 GMT > It might be worthwhile looking a free, fast, JPEG decoding library in C > and (if > you find one) adding that to your JNI stuff, so that you bypass using Java > APIs > to decode the data in the first place. Java already uses free jpeg decoder written in C - libjpeg.
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities
Chris Uppal - 06 Feb 2006 14:18 GMT [me:]
> > It might be worthwhile looking a free, fast, JPEG decoding library in C [...]
> Java already uses free jpeg decoder written in C - libjpeg. That sounds very plausible to me.
The point is that if the OP can't find an /API/ which lets him use the built-in stuff with adequate speed, then -- since he's already using JNI -- he may find it worthwhile to use JNI for the decoding too.
In case it's not clear, the idea isn't "use JNI becuase it's quicker than Java" (not true in this case if Java's using JNI anyway ;-), but that /IF/ the existing implementation is hidden too deeply for him to be able to get at the features he needs, he might consider a different way of satisfying the requirement even if that does mean duplicating an existing feature.
-- chris
Andrey Kuznetsov - 06 Feb 2006 15:39 GMT > In case it's not clear, the idea isn't "use JNI becuase it's quicker than > Java" [quoted text clipped - 3 lines] > features he needs, he might consider a different way of satisfying the > requirement even if that does mean duplicating an existing feature. ImageIO decoder is also extremely quick. He must doing something wrong. I tested it with pretty big image (4000x8000) and it was ready in 1-2 seconds.
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities
Oliver Wong - 06 Feb 2006 17:03 GMT >> In case it's not clear, the idea isn't "use JNI becuase it's quicker than >> Java" [quoted text clipped - 8 lines] > I tested it with pretty big image (4000x8000) and it was ready in 1-2 > seconds. According to the OP, 200ms is already "unacceptable" for his purposes, so 1-2 seconds is 5 to 10 times worse than "unacceptable".
- Oliver
Andrey Kuznetsov - 06 Feb 2006 18:29 GMT >> ImageIO decoder is also extremely quick. >> He must doing something wrong. [quoted text clipped - 3 lines] > According to the OP, 200ms is already "unacceptable" for his purposes, > so 1-2 seconds is 5 to 10 times worse than "unacceptable". his camera gives surely much smaller images. 4000x4000 shoud be 2 times faster and 2000x2000 should be 8 times faster.
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities
E. Naubauer - 06 Feb 2006 18:18 GMT The reason why I didn't use ImageIO was because it doesn't seem to read from my stream correctly. As I mentioned earlier, the jpeg frames are encoded as a stream of http frames which each frame wraps one jpeg image. Usually I remove the http header from the stream, decode the image from the stream, remove the trailer etc.. All other decoders exactly removed the jfif data from the stream and decoded it afterwards. For some reason, however, ImageIO doesn't seem to stop exactly after the end of the jfif data so I couldn't determine the streams position and what to do next. It also seems that ImageIO closes the stream after reading from it. I tried to work around that issue by creating a JPEGImageDecoder object manually but it didn't work also. Does anyone know how much data ImageIO gets from the stream and if it blocks etc.?
Roedy Green - 07 Feb 2006 04:13 GMT On Mon, 06 Feb 2006 19:18:26 +0100, "E. Naubauer" <chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who said :
>It also seems that ImageIO closes the stream after >reading from it you could suck the bytes out yourself and feed ImageIO the byte[] or a ByteArrayInputStream wrapper around it.
I solved this particular problem like this in my JPG streaming protocol: ----------------------------------- byte[] rawImage = conx.receiveBytes( imageLength );
if ( ! isJpgValid ( rawImage ) ) { throw new IOException ( "Invalid JPG image signature: " + Integer.toHexString( rawImage[0] ) + " " + Integer.toHexString( rawImage[1] ) ); } // start process of converting image from jpg to internal format // old AWT style: // Image image = toolkit.createImage( rawImage );
// new ImageIO style, convert raw bytes to BufferedImage BufferedImage image = ImageIO.read ( new ByteArrayInputStream ( rawImage ) ); ------------------------------ /** * Does this image represent a valid JPG file? * Does a rough check for the signature. * * @param image array of bytes representing the image * @return true if this image has a valid JPG signature. */ public static boolean isJpgValid ( byte image[] ) { return( ( image[0] & 0xff ) == 0xff ) && ( (image[1] & 0xff ) == 0xd8 ); }
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
E. Naubauer - 07 Feb 2006 06:58 GMT Indeed, it is significantly faster than the toolkit method. I tried it with 2 cameras on, drawing 2 images to a canvas each at the same time. With 1 camera there was a delay of 1-2 seconds between movement and getting drawn to the screen , with both cameras the delay took about 5 Seconds.
Well, thanks for all your advice so far.
Do you have benchmarks about how your protocol implementation does with, say ,2 Streams at once?
Roedy Green - 07 Feb 2006 12:35 GMT On Tue, 07 Feb 2006 07:58:55 +0100, "E. Naubauer" <chrono@mail.isis.de> wrote, quoted or indirectly quoted someone who said :
>Do you have benchmarks about how your protocol implementation does with, >say ,2 Streams at once? We ran maybe 10 streams at once. These were security cameras, pretty crappy image quality, some with motion detect.
We had a server running in C that talked to the camera hardware. I wrote the Java client viewers. Fun stuff. I had "magnaviewers" as well where you could zoom the image size up and down smoothly.
We have two modes, live and historic. With historic you could look back in time like a VCR fast forwarding or reversing with a speed slider.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
E. Naubauer - 07 Feb 2006 21:49 GMT Ok, it still doesn't work really good. I pasted the essential parts of the code into a small demo, added a frame to it for display and it runs just as bad as the complete thing. The delay between movement and drawing ia about five seconds, getting worse with the time. If you have the time to check it out for a sec and find something problematic, please let me know. Otherwise I will have to port the whole thing to C. Thanks in any case.
import java.io.BufferedInputStream; import java.io.ByteArrayInputStream; import java.io.DataInputStream; import java.io.InputStream; import java.net.*; import java.awt.*; import java.awt.image.*;
import javax.imageio.ImageIO; public class Test { private static HttpURLConnection hurlconnection = null; private static DataInputStream datainstream = null; private static String serialNumber = null; private static boolean connected = false; private static BufferedImage image = null; public static void main(String[] args) throws Exception { URL streamurl = new URL("http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&s howlength=1&fps=10"); hurlconnection = (HttpURLConnection)streamurl.openConnection(); hurlconnection.setReadTimeout(10000); InputStream instream = hurlconnection.getInputStream(); connected = true; BufferedInputStream bis = new BufferedInputStream(instream); datainstream = new DataInputStream(bis); Frame frm = new Frame(); frm.setSize(200,200); frm.setVisible(true); //decodiere stream if(connected) { Image i; while(!Thread.currentThread().isInterrupted()) { image = null; i = null; String header; int k = 0; while(image == null) { k = 0; do { header = datainstream.readLine(); } while(++k < 3); datainstream.readByte(); int j = (new Integer(header.substring("Content-Length: ".length()))).intValue(); datainstream.readByte(); byte abyte1[] = new byte[j + 2]; datainstream.readFully(abyte1, 0, j + 2); datainstream.readLine(); ByteArrayInputStream bin = new ByteArrayInputStream (abyte1); image = ImageIO.read ( bin ); } //end while image==null long before = System.currentTimeMillis(); Graphics2D g2 = (Graphics2D)frm.getGraphics(); frm.getGraphics().drawImage(image,0,0,null); long after = System.currentTimeMillis(); System.out.println("Duration: " +(after-before)); //Thread.currentThread().sleep(100); } } } }
bugbear - 07 Feb 2006 10:49 GMT > The reason why I didn't use ImageIO was because it doesn't seem to read > from my stream correctly. As I mentioned earlier, the jpeg frames are [quoted text clipped - 6 lines] > what to do next. It also seems that ImageIO closes the stream after > reading from it. Sounds like simple buffering to me; if it decorates its InputStream with a BufferredInputStream (for performance), the BufferredInputStream knows nothing about the boundaries in the InputStream.
Buffering Systems read ahead - that's the whole point!
BugBear
bugbear - 06 Feb 2006 16:05 GMT > Hello folks, I have a huge problem and I'm pretty desperate now > [quoted text clipped - 3 lines] > standard jai classes since every image is wrapped in a HTTP message > which I have to remove. Hmm. Any way to create a simple, buffered, stream decorator (AKA filter) ?
BugBear
Richard Wheeldon - 06 Feb 2006 22:12 GMT > Hello folks, I have a huge problem and I'm pretty desperate now > > For my diploma, I hav to code an app that basically reads jpeg images > from an Axis 2420 cameras mjpeg stream, decodes them, manipulates the > resulting rgb images an draws them back to the screen. Go to the web server on the Axis camera. Download the jar file for the display applet. Decompile it. If you want a good decompiler, try here: http://www.kpdus.com/jad.html
There's also a java program to display images from Axis cameras on sourceforge,
Richard
E. Naubauer - 06 Feb 2006 23:38 GMT I know about that implementation. The interesting point here is that they directly take the abstract image object produced by Toolkit.getDefaultToolkit().createImage and pass it to drawimage in the paint method of the displaying frame. For some reason, this works perfectly even without a media tracker to ensure that the images are fully loaded before drawing.
Andrey Kuznetsov - 07 Feb 2006 07:20 GMT >I know about that implementation. The interesting point here is that they >directly take the abstract image object produced by > Toolkit.getDefaultToolkit().createImage > and pass it to drawimage in the paint method of the displaying frame. For > some reason, this works perfectly even without a media tracker to ensure > that the images are fully loaded before drawing. probably because they also passed some Component as ImageObserver to drawImage method.
 Signature Andrey Kuznetsov http://uio.imagero.com Unified I/O for Java http://reader.imagero.com Java image reader http://jgui.imagero.com Java GUI components and utilities
E. Naubauer - 07 Feb 2006 17:47 GMT Richard Wheeldon schrieb:
> There's also a java program to display images from Axis > cameras on sourceforge, > > Richard Would you mind sending me the url (if you have it) ? Maybe I can learn a few things from it. tx
Richard Wheeldon - 08 Feb 2006 00:02 GMT > Would you mind sending me the url (if you have it) ? > Maybe I can learn a few things from it. tx Sorry. If I had it, I'd have posted it. Try a search for axis on sf.net,
Richard
slippymississippi@yahoo.com - 07 Feb 2006 22:12 GMT Let me get this straight. Unless you can download two images simultaneously in under 200ms, you will not receive your degree. WTH kind of college are you going to?
:)
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|