Java Forum / General / June 2007
JarFile/ZipFile from byte array without temp file
Karsten Wutzke - 27 Jun 2007 16:29 GMT Hi all!
Subject says it all... how do I create a JarFile/ZipFile instance from a byte array without outputting the byte[] to a temporary file and reading it back via the JarFile/ZipFile constructors??
Currently I do it via temp file (which sucks):
------------------
File flTempJar = new File(RuntimeConfig.getIoTempDir(), "deleteme.jar");
FileOutputStream fos = new FileOutputStream(flTempJar); fos.write(uncompressedBytes); fos.close();
System.out.println("Saving extracted library temporarily as file '" + flTempJar + "' - it sucks......");
JarFile jar = new JarFile(flTempJar);
try { boolean wasSuccessful = flTempJar.delete(); } catch ( Exception e ) { System.err.println("Temporary JAR file '" + flTempJar + "' couldn't be deleted!"); }
//now do something with the JarFile instance....
------------------
I can't and don't want anyone using this code to require disk access. When a SecurityManager prohibits this, this code becomes useless. Furthermore, since this is CLASSLOADER code, all classes to be found and loaded by this class loader will never be available...
Can anyone help what to do here?
Looks like I have to create my own JarFile subclass to provide the byte[] constructor.
If there's a different way, I'm all ears...
I wonder who wrote the ZipFile and JarFile classes... how could they forget byte[] and/or stream constructors? beats me...
TIA Karsten
PS: Using a 3rd library here is out of the question since the code is the 3rd party library class loader's code itself...
Nigel Wade - 27 Jun 2007 17:20 GMT > Hi all! > [quoted text clipped - 46 lines] > I wonder who wrote the ZipFile and JarFile classes... how could they > forget byte[] and/or stream constructors? beats me... Probably because those classes are specific to reading from FileInputStreams? Try looking for other classes related to Jar and Zip streams...
I would try wrapping a ByteArrayInputStream with a JarInputStream. Something along the lines of:
byte[] byteArr; ... ByteArrayInputStream byteIS = new ByteArrayInputStream(byteArr); JarInputStream jarIS = new JarInputStream(byteIS);
 Signature Nigel Wade, System Administrator, Space Plasma Physics Group, University of Leicester, Leicester, LE1 7RH, UK E-mail : nmw@ion.le.ac.uk Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
bencoe@gmail.com - 27 Jun 2007 18:30 GMT > > Hi all! > [quoted text clipped - 63 lines] > E-mail : n...@ion.le.ac.uk > Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555 I happen to have done this before, let me hunt you down the source code...
package com.plink.dolphinnet.assignments;
import com.plink.dolphinnet.*;
import java.util.*;
import java.util.zip.*;
import java.io.*;
/**
This assignment is passed an assignment as a constructor and zips it.
Used for distributing large assignments more efficiently.
*/
public class ZippedAssignment extends Assignment implements Serializable{
///////////////////////////////
// Data.
private byte file[]=null;
/////////////////////////
// Constructor.
public ZippedAssignment(int id,Assignment ZAssignment){
super(id);
loadFile(ZAssignment);
}
/////////////////////////
// Getters.
public Assignment getAssignment(){
try{
ByteArrayOutputStream Buffer=new ByteArrayOutputStream();
ZipInputStream in=new ZipInputStream(new ByteArrayInputStream(file));
ZipEntry data=data=in.getNextEntry();
if(data==null)
return(null);
int i=0;
byte buffer[]=new byte[512];
while((i=in.read(buffer))>0)
Buffer.write(buffer,0,i);
ObjectInputStream oin=new ObjectInputStream(new ByteArrayInputStream(Buffer.toByteArray()));
return((Assignment)oin.readObject());
}catch(Exception e){
e.printStackTrace();
}
return(null);
}
private void loadFile(Assignment ZAssignment){
try{
ByteArrayOutputStream bout=new ByteArrayOutputStream();
ByteArrayOutputStream bout2=new ByteArrayOutputStream();
ZipOutputStream zout=new ZipOutputStream(bout);
zout.setLevel(9);
ObjectOutputStream oout=new ObjectOutputStream(bout2);
oout.writeObject(ZAssignment);
ByteArrayInputStream bin=new ByteArrayInputStream(bout2.toByteArray());
zout.putNextEntry(new ZipEntry("Assignment"));
byte buffer[]=new byte[512];
int index;
while((index=bin.read(buffer))>0)
zout.write(buffer,0,index);
zout.closeEntry();
zout.close();
file=bout.toByteArray();
}catch(Exception e){
e.printStackTrace();
}
}
/** Run the assignments implemented task. */
public Object execute(DataHandler DH){
finish();
return(null);
}
}
That code is from my DolphinNet API, it zips something into a ByteArray on the fly, sends it and has methods for un-zipping it.
----- Ben. http://www.plink-search.com
Karsten Wutzke - 27 Jun 2007 21:21 GMT On 27 Jun., 19:30, ben...@gmail.com wrote:
> > > Hi all! > [quoted text clipped - 205 lines] > That code is from my DolphinNet API, it zips something into a > ByteArray on the fly, sends it and has methods for un-zipping it. Hmm a tried a similar approach...
As far as I understood, the trick here was to write a Serializable object to an output stream being backed by a ByteArrayOutputStream, and then retrieving the raw bytes from the ByteArrayOutputStream... right?
I can't say anything about the size of my JarEntry, its getSize() method always returns -1, whyever it does so. So analogously I create a JarOutputStream backed by a ByteArrayOutputStream, write the JarEntry to the JarOutputStream and retrieve the bytes then.
while ( true ) { JarEntry je = jis.getNextJarEntry();
if ( je == null ) { break; }
//can grow! ByteArrayOutputStream baos = new ByteArrayOutputStream(); JarOutputStream jos = new JarOutputStream(baos);
jos.setLevel(9); //0 - 9 but which one is no compression?
//write JarEntry, now baos should contain the raw bytes only? jos.putNextEntry(je);
int size = baos.size();
byte[] bytes = baos.toByteArray();
jos.close();
... }
Technically it works, however ALL of my entries are only ~60 - 100 bytes long and their sizes correllate with the length of their entry names plus a few bytes. So consequently, it didn't and doesn't work.
Input anyone?
Hmm I'll have to look at my input stream more closely, maybe something is not quite right there...
I create the JarInputStream like:
//just a resource file in the relative path URL url = getClass().getResource("/lib/bcel-5.2.jar"); URLConnection uc = url.openConnection(); FileURLConnection fuc = (FileURLConnection)uc; JarInputStream jis = new JarInputStream(fuc.getInputStream());
When reading from that jis like
byte[] buffer = new byte[8192];
int numTotal = 0;
while ( true ) { int numRead = jis.read(buffer, 0, 8192);
if ( numRead <= 0 ) { break; }
numTotal += numRead; }
numTotal *ALWAYS* is zero! Either now I'm completely off off everything or I must have forgotton to initialize something. No bytes read from the stream at all times.
Please help I'm really starting to get nuts on this. All I want to do is read a JAR file from "lib/bcel-5.2.jar" resource, whether it's currently found in that local dir or another JAR file and read the entries (class files) to decompressed byte arrays.
Karsten
PS: Going home now, 22:22 PM sucks
Karsten Wutzke - 27 Jun 2007 19:29 GMT > > Hi all! > [quoted text clipped - 57 lines] > ByteArrayInputStream byteIS = new ByteArrayInputStream(byteArr); > JarInputStream jarIS = new JarInputStream(byteIS); Hmmm I tried to go via JarInputStream now instead of JarFile. My loop now goes something like this:
JarInputStream jis = ...; //valid JarInputStream
while ( true ) { JarEntry je = jis.getNextJarEntry();
if ( je == null ) { //no more entries break; }
String strName = je.getName(); int size = (int)je.getSize(); //<- RETURNS -1 *always*
System.out.println(strName + " has " + size + " bytes (JarEntry.getSize())");
byte[] uncompressedBytes = new byte[size]; //CAUSES NegativeArraySizeException
//"jar" was an instance to the JarFile, JarInputStream doesn't have equivalent method! //DataInputStream dis = new DataInputStream(jar.getInputStream(je)); DataInputStream dis = new DataInputStream(jis.getInputStream(je)); //UNKNOWN METHOD dis.readFully(uncompressedBytes, 0, uncompressedBytes.length); dis.close();
hmBytes.put(strClassName, uncompressedBytes); }
How do I get the uncompressed bytes of a JarEntry?? Hmmm I looked at the code too many hours now, maybe I'm just blind...
Karsten
bencoe@gmail.com - 27 Jun 2007 20:35 GMT > > > Hi all! > [quoted text clipped - 99 lines] > > Karsten Look at the source I posted, except for dealing with a zip instead of a jar, I think it does exactly what you want, ignore the ObjectInput and ObjectOutput streams.
---- Ben http://www.plink-search.com
Nigel Wade - 28 Jun 2007 16:43 GMT >> > Hi all! >> [quoted text clipped - 98 lines] > > Karsten I think you have hit one of the problems of streamed ZIP, that size isn't set. I think I came across this problem when using streamed ZIPs. I don't really remember, but I have a recollection that getSize() returned -1 when the ZIP wasn't created from an actual file. Presumably when the compression is performed on-the-fly, and the ZipEntry is written to the stream, the size of the uncompressed data is unknown.
I think you just need to modify your code so that it doesn't rely on the getSize(). Read until read() returns -1, appending to, and resizing, the buffer as necessary. As I understand it ZipInputStream.read() reads from a single ZIP entry until the end of that entry, then returns -1. ZipInputStream.getNextEntry() can then be used to position the stream to the next ZIP entry.
 Signature Nigel Wade, System Administrator, Space Plasma Physics Group, University of Leicester, Leicester, LE1 7RH, UK E-mail : nmw@ion.le.ac.uk Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
Roedy Green - 28 Jun 2007 15:12 GMT >Subject says it all... how do I create a JarFile/ZipFile instance from >a byte array without outputting the byte[] to a temporary file and >reading it back via the JarFile/ZipFile constructors?? Here is a piece of code from The Replicator. Full code is posted at http://mindprod.com/products1.html#REPLICATOR
There is a lot of irrelevancy for you, but the key is zip.write.
/** * Add one element to a newly created zip file. * * @param zip ZipOutputStream to tack this element on the end. * @param zd descriptor of the zip file * @param fd descriptor of file to be added. * * @throws IOException */ public static void addOneElement( ZipOutputStream zip, MaxiZD zd, MaxiFD fd ) throws IOException { String elementName = fd.getFilename( MaxiFD.INSIDE_ZIP ); System.out.println( "packing " + elementName ); ZipEntry entry = new ZipEntry( elementName ); entry.setTime( fd.getTimestamp() ); int oldFileLength = (int) fd.getFileLength(); int fileLength; int shrinkage; File elementFile = new File( fd.getFilename( MaxiFD.ON_SOURCE ) ); // strings all interned so == compare is safe, // but use equals for safety. Interning speeds things up. String extension = fd.getExtension(); boolean isHtml = extension.equals( "html" ) || extension.equals( "htm" );
if ( isHtml && ConfigForSender.COMPACT_HTML.equals( "original" ) ) { if ( !elementFile.canWrite() ) { ReplicatorSender .fatal( "Read-only original files cannot be compacted: " + fd.getFilename( MaxiFD.ON_SOURCE ) ); } // compact the original compactor.compactFile( elementFile, true/* quiet */ );
// no need to refresh elementFile even though it is now a different // directory entry. fileLength = (int) elementFile.length(); shrinkage = oldFileLength - fileLength; if ( shrinkage != 0 ) { // set file back to its original date so it will be in the // proper zip. elementFile.setLastModified( fd.getTimestamp() ); if ( ConfigForSender.DEBUGGING ) { System.out .println( "original file compacted from " + oldFileLength + " to " + fileLength + " bytes." ); } // record the new size so we will recognize it. zd.livewood -= shrinkage; fd.setFileLength( fileLength ); } } else { // did not compact the original. It should not have changed. fileLength = (int) elementFile.length(); if ( oldFileLength != fileLength ) { System.err.println( fd );
ReplicatorSender .fatal( "file " + elementName + " has unexpectedly changed length from " + oldFileLength + " to " + fileLength + " bytes." + "\n" + "Please don't update distribution files while the Replicator is running." ); } }
// no need to setCRC, setSize, computed automatically. zip.putNextEntry( entry );
if ( isHtml && ConfigForSender.COMPACT_HTML.equals( "distributed" ) ) { // compact just the distribution, leaving original intact. // read entire file, compact and append as element. String wholeFile = HunkIO.readEntireFile( elementFile ); String compacted = compactor.compactString( wholeFile ); byte[] octets = compacted.getBytes( /* default encoding */ ); fileLength = octets.length; shrinkage = oldFileLength - fileLength; if ( shrinkage != 0 ) { if ( ConfigForSender.DEBUGGING ) { System.out .println( "distributed file compacted from " + oldFileLength + " to " + fileLength + " bytes." ); } // We don't adjust livewood or fd. // We need to recognise the file in future in its fluffy form // with its fluffy length. Its old date is just fine. } zip.write( octets ); } else { // no compacting ft.copy( elementFile, zip, false/* don't close target */ ); } zip.closeEntry(); }
-- Roedy Green Canadian Mind Products The Java Glossary http://mindprod.com
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|