For Windows, look for the module "jvm.dll".

Int isJVMPresent()
{
       HMODULE hModule;
       int             jvmIsPresent;

       hModule = GetModuleHandle(_T("JVM.DLL");
       jvmIsPresent = (hModule != NULL);

       return jvmIsPresent;
}

and/or you could look at the imports for other EXE/DLLs and see if any
of them link to JVM.DLL (and for earlier version of Java, JAVAI.DLL).

Stephen

The trick to solving this problem is knowing the OS and how to observe
what is going on. What files are being accessed? What is the content of
these files?

Someone recommended looking for jvm.dll. What if they renamed the DLL and
it exists in their installation? You might need to dig a little deeper.
You didn't mention what OS. If it is Windows I would use something like
dependency.exe to examine the binaries. I'd look at the functions they are
exporting and see of the names gave any hint. I'd look at the binaries of
Java and see if any of the files in Alias matched.

Bottom line, I know how things work in a computer. I formulate some ideas
as to how they might implement a JVM inside their code. I'd also think
about how they might easily hide it from prying eyes. I'd have to peek and
poke things to see what happens. What I see would determine my next steps.

I dig deeper at these things just so I can learn more about the computer.
It is sort of thing doing a crossword puzzle; I could ask someone else for
answers to the puzzle but I get more satisfaction solving it all on my
own.

So it depends why you want to know. Some people travel to get somewhere.
Others travel for the sake of traveling. What is more important to you?
The experience of traveling or arriving at your destination?