You have to look at some real world possibilities:

Let's for example look at a Niagara server from Sun with Solaris

64 bit virtual ram should hold you for a few years.
+9,223,372,036,854,775,807 [263-1] aka Long.MAX_VALUE    or 9 exabytes,
or 9 billion gig

or double that if they let you use the sign bit.

Let's look at Sun Ultra-20 Opteron
http://www.sun.com/desktop/workstation/ultra20/specifications.jsp
but only 4GIG of real memory.

the ultra 40 goes to 16 gig of real ram.  

If you have a bigger budget, the SunFire 2000 supports 32GIGs or real
RAM.

http://www.sun.com/servers/coolthreads/t2000/features.jsp#Memory

Keep in mind that user processes just as the JVM only know about
virtual memory.  They are not limited to address spaces fitting in
real ram, though of course they run more slowly when they have to page
to disk.

For your other question, is there some VM  limit that JVMs/OSes
typically arbitrarily impose, you will have to ask the man who own
one.

There is the question, is the fully VM address space supported on real
chips or are some of the hardware for the high end bits missing? For
that you could read the chip specs or read up on how the VM->real
address translation works.

I would be highly surprised if a reputable company offered a 64-bit
design where the max USR VM was less than real RAM.

Early in the life an architecture availably of REAL ram is the major
concern. Late in its life, it is virtual ram. We are in the early
stages.