 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / September 2007the inefficiency of noncontiguous data - why?
I noticed that breaking up a large array into 2 pieces results in a much higher memory usage for my program. Example: A program with: float[] a = new float[32000000]; uses 138MB of memory on my system. But a program with: float[] b = new float[16000000]; float[] c = new float[16000000]; uses 187MB of memory, even though it's the same amount of data. Why is this? My heap size is 800MB. > I noticed that breaking up a large array into 2 pieces results in a much > higher memory usage for my program. Example: [quoted text clipped - 9 lines] > > Why is this? My heap size is 800MB. How are you measuring memory usage? Dan.  SignatureDaniel Dyer http//www.uncommons.org Using the windows task manager, "Virtual Memory Usage" column. >> I noticed that breaking up a large array into 2 pieces results in a >> much higher memory usage for my program. Example: [quoted text clipped - 16 lines] > --Daniel Dyer > http//www.uncommons.org > >> I noticed that breaking up a large array into 2 pieces results in a > >> much higher memory usage for my program. Example: [quoted text clipped - 16 lines] > > --Daniel Dyer > > http//www.uncommons.org [Top posting fixed] > Using the windows task manager, "Virtual Memory Usage" column. Please don't top post, it confuses the conversation... Windows task manager does not accurately measure Java memory usage. The JVM's allocation of memory, and the Java programs allocation of memory don't always correspond one-to-one. It could be that your test does a garbage collection in one instance, and not in the other... It would be interesting to compare float[] b = new float[16000000]; and compare it to float[] b = new float[32000000]; and see how the memory differs. Also, see if running the program several times has different results, or running it inside an IDE, vs running it outside an IDE. >>>> I noticed that breaking up a large array into 2 pieces results in a >>>> much higher memory usage for my program. Example: [quoted text clipped - 30 lines] > Also, see if running the program several times has different results, > or running it inside an IDE, vs running it outside an IDE. Thanks, I did a little googling to find a more accurate way to measure my program's memory usage, and found out about the Runtime.maxMemory(), Runtime.totalMemory(), and Runtime.freeMemory(). I created a small test program to test the results of allocating memory in one 32000000 length chunk vs 2 16000000 length chunks and these were the results: 1 32000000 length chunk: Max memory: 832438272 Total memory: 130035712 Free memory: 1915488 2 16000000 length chunks Max memory: 832438272 Total memory: 178917376 Free memory: 50663416 So it seems in the latter case the JVM does request and receive more memory from windows for some reason, but that memory is free for my program to use, so it is not wasted, so it's fine. I'm pretty sure there is no big difference in memory consumption. The array itself takes up 16 bytes; that's 8 for the object, 4 for the size, and 4 for alignment. So your overhead is 16 bytes. Add to that what the VM needs for memory allocation, which is also insignificant compared to 64 MB. I should clarify that the 800MB is my maximum heap size, not my actual starting heap size. > I noticed that breaking up a large array into 2 pieces results in a much > higher memory usage for my program. Example: [quoted text clipped - 9 lines] > > Why is this? My heap size is 800MB. >I noticed that breaking up a large array into 2 pieces results in a much >higher memory usage for my program. Normally two smaller pieces would use less ram than one big piece since two small pieces can be fit into the holes between existing objects, where one big piece might require allocating extra VR to get contiguous space. However 1000 small pieces would probably take more space than one big piece since each piece has tracking overhead, perhaps in the order of 8-32 bytes. I would not leap to any great generalisations based on that one example. SignatureRoedy Green Canadian Mind Products The Java Glossary http://mindprod.com Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.