Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / Virtual Machine / October 2005

Tip: Looking for answers? Try searching our database.

IBM JVM and OS heap

Thread view: 
moin - 20 Oct 2005 21:56 GMT
Hi,

we develop a server application, that has the request for very flexible
memory scalabilty: Normaly it uses only 200MiByte and sometimes up to
3GiByte memory. The server (RS6000 / AIX5.1L / JDK1.4.2 (32Bit)) runs
many other applications. Now the problem:

After GC the up to 2,5GiByte free (Java-)heap memory is _not_ returned
to the OS.

We write a small test class to check this issue (see below). It showes
(eg. svmon), that the physical memory is not released. We played around
with the JVM options described in 'IBM JVM Garbage Collection and
Storage Allocation techniques' without any success. We cross checked
our test with the SUN JVM (Linux and Windows 2000) or IBM JVM (Linux)
and it works as expected.

The question:

Does someone knows the correct parameter for the IBM JVM that enables
the shrinkage of the native heap on AIX?
Have we have to modify the AIX environment in a special way (eg
MALLOCTYPE)?

Bye Thomas

-------------- Test class -----------------

public class MemTest {
   public static void main(String argc[]) {
       java.util.ArrayList arrayList = new java.util.ArrayList();
       int allocateStep;
       int character;

       if (argc.length > 0) {
           System.out.println("Allocation Step: " + argc[0] + "MB");
           allocateStep = Integer.parseInt(argc[0]);
       } else {
           System.out.println("Allocation Step: 128MB");
           allocateStep = 128;
       }

       try {
           character = System.in.read();
           while (true) {
               switch (character) {
                   case 'q':
                       System.exit(0);
                       break;

                   case 'a':
                       arrayList.add(allocate(allocateStep));
                       printStorage();
                       break;

                   case 'd':
                       if (arrayList.size() > 0)
                           arrayList.remove(0);
                       else
                           System.out.println("All arrays
deallocated");
                       printStorage();

                       break;

                   case 'g':
                       collectGarbage();
                       printStorage();
                       break;

                   case 'l':
                       printStorage();
                       break;
               }
               character = System.in.read();
           }
       } catch (Throwable aThrowable) {
           aThrowable.printStackTrace();
       }
   }

   static void printStorage() {
       System.out.println("\n\nSpeicher   (Max): " +
Runtime.getRuntime().maxMemory());
       System.out.println("Speicher (Total): " +
Runtime.getRuntime().totalMemory());
       System.out.println("Speicher  (Frei): " +
Runtime.getRuntime().freeMemory());
       System.out.println("\nSpeicher  (Heap): " +
(Runtime.getRuntime().totalMemory() -
Runtime.getRuntime().freeMemory()));
   }

   static void collectGarbage() {
       Runtime.getRuntime().runFinalization() ;
       System.gc();
   }

   static byte[] allocate(int size) {
       return new byte[size  * 1024 * 1024];
   }
}
Ben_ - 21 Oct 2005 17:21 GMT
Hello,

It's implementation dependant.

I better advise you to check in the IBM JVM Diagnostic
Guide(http://www.ibm.com/developerworks/java/jdk/diagnosis), than me telling
something incorrect.
moin - 22 Oct 2005 12:07 GMT
Hi,

we worked also with the advised document. The GC traces shows that it
works as expected: Relesing of objects and compactification of the java
heap is done. Only the last step relesing the allocated memory is not
done (only AIX) and I don't know why ... I think we have here a litle
environment issue that inhibits the jvm releasing the virtual OS
memory.

Bye Thomas
Ben_ - 22 Oct 2005 14:19 GMT
It was feeling that it's by design that memory was not released, because
shrinking the heap and releasing memory to OS are two different things.

I didn't know it for sure, so I didn't tell it. But now, I found the
following in the IBM JVM Diagnostic Guide 1.4.2, p.20, Heap shrinkage:
"
No virtual space from the heap is ever freed
back to the native operating system. When the heap shrinks, it shrinks
inside the
original virtual space.
Whether any physical memory is released depends on the ability of the native
operating system. If it supports paging; that is, the ability of the native
operating
system to commit and decommit physical storage to the virtual storage, the
Garbage Collector uses this function. In this case, physical memory can be
decommitted on a heap shrinkage.
To summarize. You never see the amount of virtual memory that is used by the
JVM decrease. You might see physical memory free size increase after a heap
shrinkage. The native operating system determines what it does with
decommitted
pages.
Also note that, where paging is supported, the Garbage Collector allocates
physical
memory to the initial heap to the amount that is specified by the -Xms
parameter.
Additional memory is committed as the heap grows.
"

Other pointers:
http://www.ibm.com/developerworks/eserver/library/es-javaonaix_memory.html
http://www.jguru.com/faq/view.jsp?EID=478232

I'm not an AIX'er, but it looks from svmon man page that it "captures and
analyzes a snapshot of virtual memory"
(http://publibn.boulder.ibm.com/doc_link/en_US/a_doc_lib/cmds/aixcmds5/svmon
.htm). So, as the Guide indicates, you won't see decreases.

BTW, in a post or another, I read a suggestion that one could spawn a
separate java process for the heap consuming task. Or, if the application is
monitored by a watchdog, commit suicide and let it restart.

HTH.
moin - 24 Oct 2005 22:06 GMT
Ben_ schrieb:

> It was feeling that it's by design that memory was not released, because
> shrinking the heap and releasing memory to OS are two different things.
>
> I didn't know it for sure, so I didn't tell it. But now, I found the
> following in the IBM JVM Diagnostic Guide 1.4.2, p.20, Heap shrinkage:
The following sentice is exactly what we expected.

> "
> No virtual space from the heap is ever freed
> back to the native operating system. When the heap shrinks, it shrinks
> inside the
> original virtual space.
No problem with that. The jvm can allocate 8GiByte virtual memory all
the time, but

> Whether any physical memory is released depends on the ability of the native
> operating system. If it supports paging; that is, the ability of the native
> operating
> system to commit and decommit physical storage to the virtual storage, the
> Garbage Collector uses this function. In this case, physical memory can be
> decommitted on a heap shrinkage.
that does not occur. The unused _physical_ pages to are not released.

> Other pointers:
> http://www.ibm.com/developerworks/eserver/library/es-javaonaix_memory.html
It's unbelievable, how do you find this nice page ... I have to order a
google lead in :-))

> http://www.jguru.com/faq/view.jsp?EID=478232
This discussion is a little bit older. The fist JVMs have not the
functionality to release memory.

> I'm not an AIX'er, but it looks from svmon man page that it "captures and
> analyzes a snapshot of virtual memory"
A snapshot is all what we need.

> (http://publibn.boulder.ibm.com/doc_link/en_US/a_doc_lib/cmds/aixcmds5/svmon
> .htm). So, as the Guide indicates, you won't see decreases.
With a little bit caclulation svmon shows also the used physical memory
pages.

> BTW, in a post or another, I read a suggestion that one could spawn a
> separate java process for the heap consuming task. Or, if the application is
> monitored by a watchdog, commit suicide and let it restart.
Yes, we are thinking about that, but this should be the last step ...

Bye Thomas


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.