Mapping seems to suck

Thread view:

mtp - 26 Apr 2006 17:05 GMT

Hello,

i can't allocate enough mapped memory on java, and i would like to know
if some of you have a solution or a workaround.

System : Linux localhost 2.6.12-12mdksmp #1 SMP Fri Sep 9 17:43:23 CEST
2005 i686 Intel(R) Pentium(R) D CPU 2.80GHz unknown GNU/Linux

14 calls to FileChannel.map(...)
total allocated space = 1 879 048 192 bytes

JDK 1.5.0_06 or 1.4.2_10 (same behavior)

java.io.IOException: Cannot allocate memory
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)

Thanks for any help

Reply to this Message

Martin Gerner - 26 Apr 2006 18:20 GMT

> 14 calls to FileChannel.map(...)
> total allocated space = 1 879 048 192 bytes

Woha.. 1.8 GB of allocated memory? What you would need to do is take a good
look at the passages where you use that FileChannel.map method. Why can't
you for instance use the normal read/write functions for files?

If you would paste that code, and explain the purpose of your program, we
might be able to help more.

Signature

Martin Gerner

Reply to this Message

Roedy Green - 26 Apr 2006 18:52 GMT

>i can't allocate enough mapped memory on java, and i would like to know
>if some of you have a solution or a workaround.

The general solution is to use a 64 bit CPU and 64 bit JVM and a ton
of RAM.

You can get a bit more by tinkering with the Java.exe command line
options. See http://mindprod.com/jgloss/javaexe.html

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Mark Thornton - 26 Apr 2006 21:15 GMT

> Hello,
>
[quoted text clipped - 14 lines]
>
> Thanks for any help

That is pretty good for a 32 bit OS. Typically on Windows you would get
rather less (using any language not just Java).

Mark Thornton

Reply to this Message

Roedy Green - 27 Apr 2006 00:13 GMT

>i can't allocate enough mapped memory on java, and i would like to know
>if some of you have a solution or a workaround.

Keep in mind that 32 bits can address only 4 gig. The os, dlls, cache
etc etc have to fit in there as well, so your user space could well
be closer to 1 or 2 gigs max. In there must go the JVM, your class
files, your stacks, and finally the heap. It is pretty crowded in
there. You don't get anywhere near entire address space for your
frame.

One other limit is you need enough disk swap space to cover your giant
virtual space AND those of any other jobs running.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Mark Thornton - 27 Apr 2006 19:26 GMT

>>i can't allocate enough mapped memory on java, and i would like to know
>>if some of you have a solution or a workaround.
[quoted text clipped - 8 lines]
> One other limit is you need enough disk swap space to cover your giant
> virtual space AND those of any other jobs running.

Actually mapped regions are swapped to the original file and don't count
against swap space (at least this is what happens on Windows). Otherwise
you're right about the address space shortage.

Mark Thornton

Reply to this Message

Roedy Green - 27 Apr 2006 22:44 GMT

On Thu, 27 Apr 2006 18:26:19 GMT, Mark Thornton
<mark.p.thornton@ntl-spam-world.com> wrote, quoted or indirectly
quoted someone who said :

>Actually mapped regions are swapped to the original file and don't count
>against swap space (at least this is what happens on Windows). Otherwise
>you're right about the address space shortage.

With memory mapped files your user files temporarily become part of
the disk space.

When your hard disk is hundreds of gigs, a 4 gig virtual window on it
does not seem that big.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Roedy Green - 27 Apr 2006 22:45 GMT

On Thu, 27 Apr 2006 21:44:26 GMT, Roedy Green
<my_email_is_posted_on_my_website@munged.invalid> wrote, quoted or
indirectly quoted someone who said :

>With memory mapped files your user files temporarily become part of
>the disk space.

oops that should say:

With memory mapped files, your user files temporarily become part of
the disk swap space.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Mapping seems to suck

Free Magazines