Java Forum / First Aid / January 2006
Problems appending data to a file. Using ObjectOutputStream, ObjectInputStream, FileInputStream, FileOutputStream
JoshRountree@gmail.com - 15 Jan 2006 20:56 GMT I would like to append data to a file. Ideally I would like to append any object, but I have decided to use a primitive int for my example below. I have two problems:
1.) When I append, and then open the file to read the objects back in, I only get what was just appended. I do not get what previously added. (I confirm this by running the program multiple times)
2.) What is the preferred way to detect the end of a binary file? I imagine catching an exception is not the correct way.
import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.IOException; import java.lang.ClassNotFoundException;
class ReadFile {
ReadFile() { }
public static void main(String[] args) { try { FileOutputStream out = new FileOutputStream("temp.log", true); ObjectOutputStream objStream = new ObjectOutputStream(out);
objStream.writeInt(1); objStream.writeInt(2); objStream.writeInt(3); objStream.writeInt(4); objStream.close();
FileInputStream in = new FileInputStream("temp.log"); ObjectInputStream s = new ObjectInputStream(in);
while (true) { System.out.println(s.readInt()); } } catch (IOException exp) { System.out.println("Exception"); } finally {
} } }
Gordon Beaton - 15 Jan 2006 21:37 GMT > I would like to append data to a file. Ideally I would like to append > any object, but I have decided to use a primitive int for my example > below. I have two problems: If you append to an existing file using an ObjectOutputStream, you will not (easily) be able to read anything but the *original* objects.
There must be a one-one correspondence between the ObjectOutputStreams you use to create the file and the ObjectInputStreams you use to read from it. This is due to the stateful nature of these streams.
If you want to store primitives, use a different stream type to store and retrieve your data, e.g. a DataOutputStream and DataInputStream.
To store java Objects, preferably use a *single* ObjectOutputStream and a *single* ObjectInputStream to store all of the Objects in a given file.
If you need to store independant Objects in isolation, you can (for example) use an ObjectOutputStream for each object, write to a byte[] that you store in the file, then keep track of its position and size. To recover the object later, read the appropriate bytes from the file, create an ObjectInputStream from those bytes, and read the Object. Or use separate files, one per ObjectOutputStream.
To detect the end of any file, read the documentation for the various methods associated with the particular stream type you've chosen.
/gordon
 Signature [ do not email me copies of your followups ] g o r d o n + n e w s @ b a l d e r 1 3 . s e
JoshRountree@gmail.com - 15 Jan 2006 21:50 GMT Thanks for the reply.
In my case, all I want to do is write out Strings to a file, then at a later time, I would like to re-open the file, and append more Strings to the end of it. When I execute my example code multiple times, the only output I ever get is "1 , 2 , 3, 4". Why is that? Shouldn't I be getting multiple sequences of "1, 2, 3, 4"? The file gets larger each time.
I understand having to know exactly what you are reading from the file (object wise).
Roedy Green - 15 Jan 2006 22:12 GMT >In my case, all I want to do is write out Strings to a file, then at a >later time, I would like to re-open the file, and append more Strings >to the end of it. When I execute my example code multiple times, the >only output I ever get is "1 , 2 , 3, 4". Why is that? Shouldn't I be >getting multiple sequences of "1, 2, 3, 4"? The file gets larger each >time. then a DataOutputStream with writeUTF is what yo uwant. It will both encode the strings compactly in as sort of UTF-8 and put a count on each one so when you read the string it knows where the end is.
That should work with append. See http://mindprod.com/jgloss/fileio.html for sample code.
Tell it you want to write binary to a file.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
JoshRountree@gmail.com - 15 Jan 2006 22:59 GMT Thanks Roedy! That site is very helpful, where was it when I first started out Java programming, it could have helped/explained things immensely.
I guess I should have explained my purpose a little more clearly in the original post. What I am trying to achieve is a log file. It logs events in a program, but I would like for it to be in a binary format so it is not easily readable by opening the file in a text editor. The writeUTF works, but I cannot for the life of me understand why the ObjectOutputStream would not work. Can anyone explain that?
This does not work, i.e. If I run this program multiple times, the only output I ever get is "1, 2, 3, 4". Why is that?
import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.BufferedOutputStream; import java.io.BufferedInputStream; import java.io.IOException; import java.lang.ClassNotFoundException;
class ReadFile {
ReadFile() { }
public static void main(String[] args) { try { FileOutputStream fos = new FileOutputStream("josh.log", true); BufferedOutputStream bos = new BufferedOutputStream(fos); ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject("1"); oos.writeObject("2"); oos.writeObject("3"); oos.writeObject("4"); oos.close();
FileInputStream fis = new FileInputStream("josh.log"); BufferedInputStream bis = new BufferedInputStream(fis); ObjectInputStream ois = new ObjectInputStream(bis);
while (true) { System.out.println((String)ois.readObject()); } } catch (IOException exp) { System.out.println("Exception"); } catch (ClassNotFoundException exp) {
} finally {
} } }
This example works:
import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.DataInputStream; import java.io.DataOutputStream; import java.io.BufferedOutputStream; import java.io.BufferedInputStream; import java.io.IOException; import java.lang.ClassNotFoundException;
class ReadFile {
ReadFile() { }
public static void main(String[] args) { try { FileOutputStream fos = new FileOutputStream("josh.log", true); BufferedOutputStream bos = new BufferedOutputStream(fos); DataOutputStream dos = new DataOutputStream(bos);
dos.writeUTF("1"); dos.writeUTF("2"); dos.writeUTF("3"); dos.writeUTF("4"); dos.close();
FileInputStream fis = new FileInputStream("josh.log"); BufferedInputStream bis = new BufferedInputStream(fis); DataInputStream dis = new DataInputStream(bis);
while (true) { System.out.println(dis.readUTF()); } } catch (IOException exp) { System.out.println("Exception"); } finally {
} } }
Thanks a lot for all your responses.
Roedy Green - 16 Jan 2006 00:15 GMT >The >writeUTF works, but I cannot for the life of me understand why the >ObjectOutputStream would not work I was similarly puzzled and tried to figure out why. I figured close append should be the same as a reset-continue. IIRC I looked at the hex generated in the file at it was not the same.
The essential problem is that object streams are not simply fields. They are references to things written much earlier in the stream.
I still think append should work, but what happens it here is some extra goobledegook at the very head of the file, and getting another lump of it in the middle of the file confusing the reader.
I suspect the problem is an oversight. Sun can't fix it now without changing the protocol, which would make existing files incompatible, or requiring the reader to understand two protocols.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
Knute Johnson - 15 Jan 2006 23:08 GMT >>In my case, all I want to do is write out Strings to a file, then at a >>later time, I would like to re-open the file, and append more Strings [quoted text clipped - 12 lines] > > Tell it you want to write binary to a file. Any reason why you wouldn't just use a BufferedWriter to write Strings to a file?
 Signature Knute Johnson email s/nospam/knute/
JoshRountree@gmail.com - 15 Jan 2006 23:20 GMT I would like for the log file to *not* be easily readable from a text viewer. It is a poor attempt at obfuscation.
Ideally, I would only like a person that knows about the format of the log file to be able to read it, and/or I will embed a log file viewer in the program.
Knute Johnson - 16 Jan 2006 01:31 GMT > I would like for the log file to *not* be easily readable from a text > viewer. It is a poor attempt at obfuscation. > > Ideally, I would only like a person that knows about the format of the > log file to be able to read it, and/or I will embed a log file viewer > in the program. you could still use a number or a letter or some other weird ascii code.
 Signature Knute Johnson email s/nospam/knute/
Roedy Green - 16 Jan 2006 00:17 GMT On Sun, 15 Jan 2006 15:08:30 -0800, Knute Johnson <nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone who said :
>Any reason why you wouldn't just use a BufferedWriter to write Strings >to a file? it then becomes your problem to figure out how long they are. You need CSV or single field per line. Then you have much more complicated code to read them back.
See http://mindprod.com/jgloss/csv.html
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
Knute Johnson - 16 Jan 2006 01:39 GMT > On Sun, 15 Jan 2006 15:08:30 -0800, Knute Johnson > <nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone [quoted text clipped - 8 lines] > > See http://mindprod.com/jgloss/csv.html separating out the data fields in CSV is only one line of code and you don't need to know how long they are or even how many fields they have. although it does make is simpler if they always have the same number. this job I'm working on now has ETX EOT delimited fields but they don't always include all of the data or fields. why I have no idea.
 Signature Knute Johnson email s/nospam/knute/
Roedy Green - 16 Jan 2006 02:31 GMT On Sun, 15 Jan 2006 17:39:03 -0800, Knute Johnson <nospam@ljr-2.frazmtn.com> wrote, quoted or indirectly quoted someone who said :
>separating out the data fields in CSV is only one line of code and you >don't need to know how long they are or even how many fields they have. > although it does make is simpler if they always have the same number. > this job I'm working on now has ETX EOT delimited fields but they >don't always include all of the data or fields. why I have no idea. Problem with text files: 1. you have to reserve a field delimiter 2. you then may need some way of quoting that delimiter if it occurs as data. 3. you have to process char by char. 4. it is never clear which encoding was used to create the file. It is not embedded in any way in the file.
With binary with counts, you read field by field. there are no reserved chars. Your strings may contain embedded " , \n etc. without any special treatment. When you read ahead, the low level code knows exactly how many chars/bytes are coming so it knows how big to allocate the array to hold them.
My thinking is text files are for accepting data from humans or for preparing it for them. For computer-to-computer, use binary. This opinion is considered deviant by the majority, who may secretly have invested in Verison and want to bloat all communications ten fold.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Java custom programming, consulting and coaching.
Knute Johnson - 16 Jan 2006 04:07 GMT > My thinking is text files are for accepting data from humans or for > preparing it for them. For computer-to-computer, use binary. This > opinion is considered deviant by the majority, who may secretly have > invested in Verison and want to bloat all communications ten fold. Now that's funny!
Lately I've been storing form data in text files and transfering it around in objects. I've been really happy with the performance. Things like log files one would expect to be text, or at least I would but I guess that's what makes it all go around :-).
 Signature Knute Johnson email s/nospam/knute/
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|