Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / May 2006

Tip: Looking for answers? Try searching our database.

GZIPInputStream (how to uncompress from a succession of byte arrays)

Thread view: 
chattycow - 15 May 2006 19:51 GMT
This message also posted on comp.lang.java.help  --

I have a hopefully simple question regarding gunzipping compressed data
not in a file...

I have a gzipped file/object/whatever residing accross a network (it's
large, about 1GB).
The file is broken up into 32Kb chunks and sent accross the network in
these chunks.

I need to decompress the file using GZIP at the other end of the
network from the bunch of 32Kb chunks.
I obviously can't just load 1GB of bytes into an extremely large
ByteArrayInputStream, then push that into GZIPInputStream as I would
quickly run out of memory as the GZIPInputStream class would have you
do using it's constructor and a ByteArrayInputStream.

I need a way to do this a few 32Kb chunks at a time, and keep adding
data to GZIPInputStream(or whatever) for decompressing, and then
reading the uncompressed data as we go.

Any ideas would be really, really, helpful.  Code examples would be
extremely helpful as I'm pretty new to Java and love the language so
far.
Andrey Kuznetsov - 15 May 2006 20:31 GMT
> I have a hopefully simple question regarding gunzipping compressed data
> not in a file...
[quoted text clipped - 18 lines]
> extremely helpful as I'm pretty new to Java and love the language so
> far.

Unified I/O could be helpful - http://uio.imagero.com
It's free and open source.

Andrey

Signature

http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

chattycow - 15 May 2006 20:38 GMT
I'll take a look and see if it can help me.
Thanks for your reply.
chattycow - 15 May 2006 21:02 GMT
Doesn't apear that unified I/O is going to help.
It doesn't appear to let me create an InputStream, then continually add
to it...unless I'm missing something.   All the while having it
encapsulated within GZIPInputStream.

Thanks anyway....Anybody else have any ideas or have done this before?

Thanks,
  ---Dean.
Chris Uppal - 16 May 2006 07:12 GMT
> I have a gzipped file/object/whatever residing accross a network (it's
> large, about 1GB).
> The file is broken up into 32Kb chunks and sent accross the network in
> these chunks.

I'm not sure what you mean by this. I can think of 3 possible meaning, but each
one has different answers...

1) You have a large lump of gzipped data and you want to read that
incrementally across a network, i.e. without holding the whole thing in memory
at the same time.  That's easy because GZipInputStream is inherently
incremental.

2) You have a server of some sort which provides a sequence of  32K chunks of
data, each of which has been independently gzipped.  Again that is easy -- just
de-gzip each chunk independently and assemble the results.

3) You have an awkwardly designed system which passes out chunks extracted from
a single gzipped file, but doesn't do so as an uninterrupted stream.  Perhaps
some sort of web-service where you keep sending HTTP requests "give me the next
chunk".  If so then your first thought should be "how can I fix the design?".
Assuming that you are unable to do so, and since you clearly have too much data
for a simple hack/workaround to suffice, you will have to create a custom
implementation of ReadStream.  That implementation would satisfy read()
requests by checking an internal buffer to see if it already had enough data
stored and, if not, then it would refill the buffer by issuing a network
request to your server for the next 32K chunk.  You would create an instance of
that class and pass it to the GZipInputStream constructor.  You might find that
quite difficult if other parts of your application need to talk to the server
at the same time, or if the protocol for talking to the server is more
complicated than I'm assuming here.

   -- chris


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.