 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / March 2006HELP! OutOfMememyError with arrays
Hi, I am running into an OutOfMemoryError and need some help. My code looks essentially like this: ByteArrayData ref; // has a field: byte[] _data while(i = 0 to 100000 files) { // reads bytes from file (with FileInputStream) into ref._data ref = new ByteArrayData(filename); send_data_over_network(ref);.... // in lieu of free() : ref._data = null; ref = null; // System.gc(); } I am basically creating new objects inside a tight loop. I am evidently running out of memory because the memory referenced by 'data' is not being free'd (to use the C term). Can someone help? > // in lieu of free() : > ref._data = null; > ref = null; > > // System.gc(); That lot isn't going to help... > I am basically creating new objects inside a tight loop. I am evidently > running out of memory because the memory referenced by 'data' is not > being free'd (to use the C term). Can someone help? My best guess is that you are not closing resources, or have a finalisble object that holds onto a large amount of memory. That or one of the files is too large. Tom Hawtin  SignatureUnemployed English Java programmer http://jroller.com/page/tackline/ >> // in lieu of free() : >> ref._data = null; [quoted text clipped - 3 lines] > > That lot isn't going to help... Actually, I say that but: Suppose the constructor assigning to ref allocates a lot of memory. The old ref value is unlikely to be overwritten until after the construction has succeeded. So you could have two large objects allocated at once. Tom Hawtin SignatureUnemployed English Java programmer http://jroller.com/page/tackline/ > > // in lieu of free() : > > ref._data = null; [quoted text clipped - 11 lines] > finalisble object that holds onto a large amount of memory. That or one > of the files is too large. Found the problem. I am using ObjectOutputStream to send objects over sockets. There is evidently a bug that keeps references to sent objects, so ObjectOutputStream's reset must be called. See also: http://java.sun.com/products/jdk/serialization/faq/#OutOfMemoryError > Found the problem. I am using ObjectOutputStream to send objects > over sockets. There is evidently a bug that keeps references to > sent objects, so ObjectOutputStream's reset must be called. Not a bug, but a necessary part of how serialisation works. The output stream has to retain a reference to each object that has been written to it, or else it won't know when/if a second reference to the same object has to be written. If it didn't do that then object networks would not de-serialise correctly. -- chris > Not a bug, but a necessary part of how serialisation works. The output stream > has to retain a reference to each object that has been written to it, or else > it won't know when/if a second reference to the same object has to be written. > If it didn't do that then object networks would not de-serialise correctly. Presumably ObjectOutputStream could keep a weak reference. Tom Hawtin SignatureUnemployed English Java programmer http://jroller.com/page/tackline/ [me:] > > Not a bug, but a necessary part of how serialisation works. The output > > stream has to retain a reference to each object that has been written > > to it, [...] > > Presumably ObjectOutputStream could keep a weak reference. Hmmm, that's an idea. I think that it could introduce subtle inconsistancies when writing object that were themselves only reachable via weak references. E.g. ObjectOutputStream out = //... String oops = "oo" + "ps"; out.writeObject(oops.intern()); // ... time passes ... out.writeObject(oops.intern()); might write two separate objects to the stream, instead of the interned representation of "oops" twice. But it's hard to see that as a serious issue, so I suppose it's more likely that OUS doesn't use a weak collection because Sun either didn't think of it or didn't think it worthwhile. -- chris > ObjectOutputStream out = //... > String oops = "oo" + "ps"; > out.writeObject(oops.intern()); Point of pedantry: oops.intern() there will return the same object as oops. > But it's hard to see that as a serious issue, so I suppose it's more likely > that OUS doesn't use a weak collection because Sun either didn't think of it or > didn't think it worthwhile. Here's the bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4363937 It would slow performance (but I suspect by very little). The main issue is that apparently the serialisation format has no way to reset for a single object. An ObjectInputStream would have to keep the deserialised object strongly referenced, and hence end up with the same memory problems, only not near the source of the problem. As with most cases where WeakHashMap or IdentityHashMap is used, it really wants WeakIdentityHashMap which for some unfathomable reason still hasn't been provided. http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4500542 Tom Hawtin SignatureUnemployed English Java programmer http://jroller.com/page/tackline/ [me:] > > String oops = "oo" + "ps"; > > out.writeObject(oops.intern()); > > Point of pedantry: oops.intern() there will return the same object as > oops. Oops! ;-) > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4363937 Thanks. -- chris Digital Puer schrieb: > Found the problem. I am using ObjectOutputStream to send objects > over sockets. There is evidently a bug that keeps references to > sent objects, so ObjectOutputStream's reset must be called. It's not a bug. This behaviour is intentional: | The ObjectOutputStream class keeps track of each object it serializes | and sends only the handle if the object is written into the stream a | | subsequent time. Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.