Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / First Aid / January 2006

Tip: Looking for answers? Try searching our database.

Problems appending data to a file. Using ObjectOutputStream, ObjectInputStream, FileInputStream, FileOutputStream

Thread view: 
JoshRountree@gmail.com - 15 Jan 2006 20:56 GMT
I would like to append data to a file. Ideally I would like to append
any object, but I have decided to use a primitive int for my example
below. I have two problems:

1.) When I append, and then open the file to read the objects back in,
I only get what was just appended. I do not get what previously added.
(I confirm this by running the program multiple times)

2.) What is the preferred way to detect the end of a binary file? I
imagine catching an exception is not the correct way.

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.IOException;
import java.lang.ClassNotFoundException;

class ReadFile {

   ReadFile() {
   }

   public static void main(String[] args) {
    try {
       FileOutputStream out = new FileOutputStream("temp.log", true);
       ObjectOutputStream objStream = new ObjectOutputStream(out);

       objStream.writeInt(1);
       objStream.writeInt(2);
       objStream.writeInt(3);
       objStream.writeInt(4);
       objStream.close();

       FileInputStream in = new FileInputStream("temp.log");
       ObjectInputStream s = new ObjectInputStream(in);

       while (true) {
        System.out.println(s.readInt());
       }
    } catch (IOException exp) {
       System.out.println("Exception");
    } finally {

    }
   }
}
Gordon Beaton - 15 Jan 2006 21:37 GMT
> I would like to append data to a file. Ideally I would like to append
> any object, but I have decided to use a primitive int for my example
> below. I have two problems:

If you append to an existing file using an ObjectOutputStream, you
will not (easily) be able to read anything but the *original* objects.

There must be a one-one correspondence between the ObjectOutputStreams
you use to create the file and the ObjectInputStreams you use to read
from it. This is due to the stateful nature of these streams.

If you want to store primitives, use a different stream type to store
and retrieve your data, e.g. a DataOutputStream and DataInputStream.

To store java Objects, preferably use a *single* ObjectOutputStream
and a *single* ObjectInputStream to store all of the Objects in a
given file.

If you need to store independant Objects in isolation, you can (for
example) use an ObjectOutputStream for each object, write to a byte[]
that you store in the file, then keep track of its position and size.
To recover the object later, read the appropriate bytes from the file,
create an ObjectInputStream from those bytes, and read the Object. Or
use separate files, one per ObjectOutputStream.

To detect the end of any file, read the documentation for the various
methods associated with the particular stream type you've chosen.

/gordon

Signature

[  do not email me copies of your followups  ]
g o r d o n + n e w s @  b a l d e r 1 3 . s e

JoshRountree@gmail.com - 15 Jan 2006 21:50 GMT
Thanks for the reply.

In my case, all I want to do is write out Strings to a file, then at a
later time, I would like to re-open the file, and append more Strings
to the end of it. When I execute my example code multiple times, the
only output I ever get is "1 , 2 , 3,  4". Why is that? Shouldn't I be
getting multiple sequences of "1, 2, 3, 4"? The file gets larger each
time.

I understand having to know exactly what you are reading from the file
(object wise).
Roedy Green - 15 Jan 2006 22:12 GMT
>In my case, all I want to do is write out Strings to a file, then at a
>later time, I would like to re-open the file, and append more Strings
>to the end of it. When I execute my example code multiple times, the
>only output I ever get is "1 , 2 , 3,  4". Why is that? Shouldn't I be
>getting multiple sequences of "1, 2, 3, 4"? The file gets larger each
>time.

then a DataOutputStream with writeUTF is what yo uwant.  It will both
encode the strings compactly in as sort of UTF-8 and put a count on
each one so when you read the string it knows where the end is.

That should work with append. See
http://mindprod.com/jgloss/fileio.html
for sample code.

Tell it you want to write binary to a file.  
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

JoshRountree@gmail.com - 15 Jan 2006 22:59 GMT
Thanks Roedy! That site is very helpful, where was it when I first
started out Java programming, it could have helped/explained things
immensely.

I guess I should have explained my purpose a little more clearly in the
original post. What I am trying to achieve is a log file. It logs
events in a program, but I would like for it to be in a binary format
so it is not easily readable by opening the file in a text editor. The
writeUTF works, but I cannot for the life of me understand why the
ObjectOutputStream would not work. Can anyone explain that?

This does not work, i.e. If I run this program multiple times, the only
output I ever get is "1, 2, 3, 4". Why is that?

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.lang.ClassNotFoundException;

class ReadFile {

   ReadFile() {
   }

   public static void main(String[] args) {
    try {
       FileOutputStream fos = new FileOutputStream("josh.log", true);
       BufferedOutputStream bos = new BufferedOutputStream(fos);
       ObjectOutputStream oos = new ObjectOutputStream(bos);

       oos.writeObject("1");
       oos.writeObject("2");
       oos.writeObject("3");
       oos.writeObject("4");
       oos.close();

       FileInputStream fis = new FileInputStream("josh.log");
       BufferedInputStream bis = new BufferedInputStream(fis);
       ObjectInputStream ois = new ObjectInputStream(bis);

       while (true) {
        System.out.println((String)ois.readObject());
       }
    } catch (IOException exp) {
       System.out.println("Exception");
    } catch (ClassNotFoundException exp) {

    } finally {

    }
   }
}

This example works:

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.lang.ClassNotFoundException;

class ReadFile {

   ReadFile() {
   }

   public static void main(String[] args) {
    try {
       FileOutputStream fos = new FileOutputStream("josh.log", true);
       BufferedOutputStream bos = new BufferedOutputStream(fos);
       DataOutputStream dos = new DataOutputStream(bos);

       dos.writeUTF("1");
       dos.writeUTF("2");
       dos.writeUTF("3");
       dos.writeUTF("4");
       dos.close();

       FileInputStream fis = new FileInputStream("josh.log");
       BufferedInputStream bis = new BufferedInputStream(fis);
       DataInputStream dis = new DataInputStream(bis);

       while (true) {
        System.out.println(dis.readUTF());
       }
    } catch (IOException exp) {
       System.out.println("Exception");
    } finally {

    }
   }
}

Thanks a lot for all your responses.
Roedy Green - 16 Jan 2006 00:15 GMT
>The
>writeUTF works, but I cannot for the life of me understand why the
>ObjectOutputStream would not work

I was similarly puzzled and tried to figure out why.  I figured close
append should be the same as a reset-continue.

IIRC I looked at the hex generated in the file at it was not the same.

The essential problem is that object streams are not simply fields.
They are references to things written much earlier in the stream.

I still think append should work, but what happens it here is some
extra goobledegook at the very head of the file, and getting another
lump of it in the middle of the file confusing the reader.

I suspect the problem is an oversight.  Sun can't fix it now without
changing the protocol, which would make existing files incompatible,
or requiring the reader to understand two protocols.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Knute Johnson - 15 Jan 2006 23:08 GMT
>>In my case, all I want to do is write out Strings to a file, then at a
>>later time, I would like to re-open the file, and append more Strings
[quoted text clipped - 12 lines]
>
> Tell it you want to write binary to a file.  

Any reason why you wouldn't just use a BufferedWriter to write Strings
to a file?

Signature

Knute Johnson
email s/nospam/knute/

JoshRountree@gmail.com - 15 Jan 2006 23:20 GMT
I would like for the log file to *not* be easily readable from a text
viewer. It is a poor attempt at obfuscation.

Ideally, I would only like a person that knows about the format of the
log file to be able to read it, and/or I will embed a log file viewer
in the program.
Knute Johnson - 16 Jan 2006 01:31 GMT
> I would like for the log file to *not* be easily readable from a text
> viewer. It is a poor attempt at obfuscation.
>
> Ideally, I would only like a person that knows about the format of the
> log file to be able to read it, and/or I will embed a log file viewer
> in the program.

you could still use a number or a letter or some other weird ascii code.

Signature

Knute Johnson
email s/nospam/knute/

Roedy Green - 16 Jan 2006 00:17 GMT
On Sun, 15 Jan 2006 15:08:30 -0800, Knute Johnson
<nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone
who said :

>Any reason why you wouldn't just use a BufferedWriter to write Strings
>to a file?

it then becomes your problem to figure out how long they are.  You
need CSV or single field per line. Then you have much more complicated
code to read them back.

See http://mindprod.com/jgloss/csv.html

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Knute Johnson - 16 Jan 2006 01:39 GMT
> On Sun, 15 Jan 2006 15:08:30 -0800, Knute Johnson
> <nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone
[quoted text clipped - 8 lines]
>
> See http://mindprod.com/jgloss/csv.html

separating out the data fields in CSV is only one line of code and you
don't need to know how long they are or even how many fields they have.
 although it does make is simpler if they always have the same number.
 this job I'm working on now has ETX EOT delimited fields but they
don't always include all of the data or fields.  why I have no idea.

Signature

Knute Johnson
email s/nospam/knute/

Roedy Green - 16 Jan 2006 02:31 GMT
On Sun, 15 Jan 2006 17:39:03 -0800, Knute Johnson
<nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone
who said :

>separating out the data fields in CSV is only one line of code and you
>don't need to know how long they are or even how many fields they have.
>  although it does make is simpler if they always have the same number.
>  this job I'm working on now has ETX EOT delimited fields but they
>don't always include all of the data or fields.  why I have no idea.

Problem with text files:
1. you have to reserve a field delimiter
2. you then may need some way of quoting that delimiter if it occurs
as data.
3. you have to process char by char.
4. it is never clear which encoding was used to create the file. It is
not embedded in any way in the file.

With binary with counts, you read field by field. there are no
reserved chars. Your strings may contain embedded " , \n etc. without
any special treatment. When you read ahead, the low level code knows
exactly how many chars/bytes are coming so it knows how big to
allocate the array to hold them.

My thinking is text files are for accepting data from humans or for
preparing it for them. For computer-to-computer, use binary.  This
opinion is considered deviant by the majority, who may secretly have
invested in Verison and want to bloat all communications ten fold.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Knute Johnson - 16 Jan 2006 04:07 GMT
> My thinking is text files are for accepting data from humans or for
> preparing it for them. For computer-to-computer, use binary.  This
> opinion is considered deviant by the majority, who may secretly have
> invested in Verison and want to bloat all communications ten fold.

Now that's funny!

Lately I've been storing form data in text files and transfering it
around in objects.  I've been really happy with the performance.  Things
like log files one would expect to be text, or at least I would but I
guess that's what makes it all go around :-).

Signature

Knute Johnson
email s/nospam/knute/



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.