Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / June 2007

Tip: Looking for answers? Try searching our database.

-Xmx out of memory errors

Thread view: 
catphive - 28 May 2007 01:21 GMT
That java gets out of memory errors, and that the maximum heap size
needs to be controlled by a flag has always bugged me. Why is this so?
Most systems already have mechanisms in place to set hard limits
(getrlimit, setrlimit) on resource usage, and have mechanisms to
increase the number of pages available for memory management (brk,
sbrk). Why doesn't java just rely on these instead of putting extra
hard limits on memory?

Also, is it at all possible to turn this behavior off? I find that
many java programs eventually run into and crash on their default max
heap size. It would be nice to have a way to either tell individual
java instances, or preferably all java programs to have no maximum
heap size except that defined by rlimit.

Thanks
Arne Vajhøj - 28 May 2007 02:49 GMT
> That java gets out of memory errors, and that the maximum heap size
> needs to be controlled by a flag has always bugged me. Why is this so?
[quoted text clipped - 9 lines]
> java instances, or preferably all java programs to have no maximum
> heap size except that defined by rlimit.

Java is intended to be cross platform. So it makes some sense to
control something that is not guaranteed to be part of all deployment
OS'es.

You can use the maximum supported value as "unlimited". They could
also have defined that -Xmx0 meant maximum, but there are a certain
documentative effect of having the actual number, because it is
not unlimited.

Arne
catphive - 31 May 2007 10:02 GMT
> > That java gets out of memory errors, and that the maximum heap size
> > needs to be controlled by a flag has always bugged me. Why is this so?
[quoted text clipped - 20 lines]
>
> Arne

unlimited eh? Thanks.
catphive - 31 May 2007 10:14 GMT
> > That java gets out of memory errors, and that the maximum heap size
> > needs to be controlled by a flag has always bugged me. Why is this so?
[quoted text clipped - 20 lines]
>
> Arne

actually, upon checking unlimited doesn't seem to be acceptable...
this again raises the problem with -Xmx, that it forces me to put a
hard limit on a dynamically allocated resource...

Now that you mention it, old mac os 9 used to require a max heap size
set for every application (get info, maximum memory). However, that
was a horribly primitive system. I understand that they'd want mac
compatibility, but it seems unfortunate that they did it in such a way
that made everyone else suffer... I'd like my programs not to die
every time they get an unexpectedly large input. However high I set
the -Xmx value it's always possible that more heap space will be
necessary.

Given virtual memory, and the fact the fact that on a reasonable
application only a small subset of memory need to be accessible in
ram, thrashing isn't really a problem, there's really no practical
limit to a maximum heap size.
Chris Dollin - 31 May 2007 10:17 GMT
> Given virtual memory, and the fact the fact that on a reasonable
> application only a small subset of memory need to be accessible in
> ram, thrashing isn't really a problem, there's really no practical
> limit to a maximum heap size.

Those who have experienced systems thrashing may disagree.

Signature

"We did not have time to find out everything               /A Clash of Cymbals/
we wanted to know."                                              - James Blish

Hewlett-Packard Limited                                          registered no:
registered office: Cain Road, Bracknell, Berks RG12 1HN          690597 England

Arne Vajhøj - 01 Jun 2007 03:03 GMT
> actually, upon checking unlimited doesn't seem to be acceptable...
> this again raises the problem with -Xmx, that it forces me to put a
> hard limit on a dynamically allocated resource...

> Given virtual memory, and the fact the fact that on a reasonable
> application only a small subset of memory need to be accessible in
> ram, thrashing isn't really a problem, there's really no practical
> limit to a maximum heap size.

????

The highest limit you can set with Xmx is the limit imposed
by the limits in the virtual address space.

Java can not get more from the operating system.

Arne
Stan - 01 Jun 2007 13:45 GMT
One thing to consider, the larger the heap the harder the java system
gets hit when it needs to do a garbage collection. I know it sounds
crazy but some systems actually behave more efficiently with smaller
heaps. Most people tune their java apps and monitor the gc behavior
with he verbose gc parameter. It will let you monitor how often a gc
event occurs, and how long each gc event takes. Usually while a gc
event occurs all threads may stop working.

Other thing to consider is that java uses more then just heap memory.
It also uses system memory from jni calls. You usually need to balance
both. Having the ability to control the max and min size is not
necessarily a bad thing.

Stan

> That java gets out of memory errors, and that the maximum heap size
> needs to be controlled by a flag has always bugged me. Why is this so?
[quoted text clipped - 11 lines]
>
> Thanks
catphive - 08 Jun 2007 02:02 GMT
The issue is maximum heap... just because the heap is bounded to a
certain volume, or unbounded doesn't mean that java needs to sbrk that
much heap space. Making the default unbounded, which is all I would
like to see change, wouldn't mean that java by default sbrks the whole
address space, and if it does, that's a java implementation problem
that shouldn't be there.

I don't see how -Xmx is useful for balancing "system memory" (by which
I think you mean memory allocated by malloc... which is also on the
heap btw, just not the gc'd section). As near as I can tell
applications pretty much just die when they run into the -Xmx barrier.
I guess it probably throws an exception which could be caught; however
it seems unlikely that such an exception could be recovered from in a
meaningful way given the number of places that could throw such an
exception.

Someone else mentioned that I could make -Xmx the size of the entire
address space; however I would note that isn't portable, as for 32 bit
systems that is merely 4 gigs, whereas 64 bit machines can handle a
lot more depending on the vm's implementation. The problem is that
right now, java users who know nothing about java implementation
details have to go in and mess with the java launching script to make
their java applications work, and they would still have to do that
occasionally if the script writer guessed a common address space size.
It's not the correct solution.

Someone else mentioned that thrashing can happen... but that is the
result of a poorly written program or gc, not a high -Xmx value.
Extremely large heaps can not thrash, and significantly smaller heaps
can depending on memory access patterns.

In conclussion -Xmx seems like a total hack to me, and I'd like to see
it set to an unlimited value by default. It could still be there for
people who need it, and don't know how to do the exact same thing
through the shell with rlimit. The default action is just a nuisance
though.

> One thing to consider, the larger the heap the harder the java system
> gets hit when it needs to do a garbage collection. I know it sounds
[quoted text clipped - 26 lines]
>
> > Thanks


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.