Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / Virtual Machine / January 2006

Tip: Looking for answers? Try searching our database.

Invocation Problems (memory leak)

Thread view: 
Sheol - 15 Jan 2006 01:25 GMT
Why does the GC not collect the local references created in my shared
library? How do I get these objects to collect so that I do not have a
memory leak?

JVM: Sun's 1.5.0_06 for linux.

I have a shared library that I am using behind other larger
(behemoth) programs to start a JVM and execute an existing, complex
system. I use the JNI_CreateJavaVM() to create the JVM using
JNI_VERSION_1_4 arguments. It starts without errors and I am able to use
the existing system for some number of calls. The argument list in the
interface between the shared library and the java system requires that the
shared library create objects and fill them. However, the GC will never
claim the objects created in the shared library through the JNI even
though they show up in a GC root (I have been using a profiler to help
with this). I have used PushLocalFrame()/PopLocalFrame() and
DeleteLocalRef() with no affect. So, why does the GC not collect the local
references created through the JNI? How do I get these objects to
collect so that I do not have a memory leak?

I have an incredibly small test suite that I could post for people if they
want to help find and fix this problem.

I will monitor this posting so please do not send email -- besides, I
think this address is dead anyway.

I have done the obligatory search on google, read Sun's JNI tutorial
(mildly instructive), read a good deal of the JNI specification (more
instructive than the tutorial), and read about as much as I can find on
this topic (including some book whose title I forgot already). That does
not mean I have read everything, just as much as I found.

Lastly, thanks in advance for any and all help.
opalpa@gmail.com opalinski from opalpaweb - 15 Jan 2006 03:32 GMT
Warning: I'm not a JNI expert.

GC does not have to reclaim objects until it feels like it.  If you run
your app and create lots of stuff it still does not reclaim references?
The GC waits until it believes it should clean up because resources
are scarce, is my understanding.  In other words if you have plenty of
resources it will not bother reclaiming even though it could.

Also can you release your resources through some explicit call on the
objects?

Opalinski
opalpa@gmail.com
http://www.geocities.com/opalpaweb/
Sheol - 15 Jan 2006 21:52 GMT
> Warning: I'm not a JNI expert.
>
[quoted text clipped - 10 lines]
> opalpa@gmail.com
> http://www.geocities.com/opalpaweb/

I should have been more explicit. If I let it run long enough, it will
fail with an out-of-memory error. I think that the GC is required to clean
up all that it can before this point.
Chris Uppal - 15 Jan 2006 10:34 GMT
> I have used PushLocalFrame()/PopLocalFrame() and
> DeleteLocalRef() with no affect. So, why does the GC not collect the local
> references created through the JNI? How do I get these objects to
> collect so that I do not have a memory leak?

You are using the right tools.   So the only possibilities I can think of are:

1)  Opalinski's suggestion (another reply in this thread).  The JVM might be
perfectly capable of GCing these objects but just hasn't got around to doing so
yet.  You could try invoking GC explicitly to see if that makes a difference.
OTOH, if these references are showing up as GC roots in your profiler, then
maybe that isn't so likely (assuming you are interpreting what the profiler is
telling you correctly).

2)  There's a bug in this part of the JNI implementation.  That's always a
possibility, especially as you are using the very latest VM, so you don't have
quite the same assurance that the feature is used by lots of other people (who
would surely have noticed the problem before you).  Still, I don't think it's
/very/ likely.

So that leaves:

3)  You are somehow forgetting to DeleteLocalRef() on some of the object
references you create.  To me this sounds quite possible, and is certainly
where I would be looking (and double- and triple-checking) if this were my
code.

> I have an incredibly small test suite that I could post for people if they
> want to help find and fix this problem.

Unless someone else knows of a bug in JNI, then It probably is a good idea to
post some code.  (I can look at it, if it's not too long, but can't try it out
myself it since I don't like running Linux).

   -- chris
Sheol - 15 Jan 2006 22:03 GMT
1) If I let it run long enough, I will always get the out-of-memory
errors. Also, I sent the "snapshots" to the profiler makers and they also
claim that the objects are showing up in a GC root.

2) I have tried JVMs from 1.5.0_01 to 1.5.0_06 with the same results. I
think I have tried them all, but I may have missed a release or two. My
guess is that it is something that I do not understand about invocation
rather than a JNI bug.

3) I have double, triple, and quadruple checked my shared library. That
does not mean it is error free; it simply means this is also my primary
guess at a fault.

I will build a zip file with the code in it. The zip file will be may be
big because of the supporting JAR file, but the actual code in question is
pretty small as things go. Lots of comments, but just a little bit of
work. Also, this code should work on any platform. The C++ code that I
wrote should conform to the ANSI standard (or what tries to be an ANSI
standard) and therefore should compile just about anywhere. I have a
make.sh script that makes the test program, so that can be used to convert
to your platform of preference.

>> I have used PushLocalFrame()/PopLocalFrame() and
>> DeleteLocalRef() with no affect. So, why does the GC not collect the local
[quoted text clipped - 31 lines]
>
>     -- chris
Raymond DeCampo - 15 Jan 2006 22:43 GMT
> 1) If I let it run long enough, I will always get the out-of-memory
> errors. Also, I sent the "snapshots" to the profiler makers and they also
> claim that the objects are showing up in a GC root.

Which GC root?  That is, have you used the output to trace it back to
the canonical object (usually a static reference or other such thing)
that cannot be GC'd?  Tools like JDeveloper have helped me with this in
the past.

HTH,
Ray

Signature

This signature intentionally left blank.

Sheol - 15 Jan 2006 23:35 GMT
I cannot do a uuencoded zip because someone is rejecting it, so here it
is. There are 5 files involved to get the test to build and execute, but
commandInterface.cxx is where the memory leak resides. It takes about an
hour to start seeing the memeory leak without a profiler.

--------- begin make.sh
#! /bin/bash

# JAVA_HOME needs to be set correctly
JAVA_HOME=/usr/local/java

##g++  -I${PWD}  -I${JAVA_HOME}/include  -I${JAVA_HOME}/include/linux -DJAVA_PROFILER -fPIC -c -o commandInterface.cxxo ${HOME}/Source/Testbed/Drivers/commandInterface.cxx
g++  -I${PWD}  -I${JAVA_HOME}/include  -I${JAVA_HOME}/include/linux -fPIC -c -o commandInterface.cxxo ${HOME}/Source/Testbed/Drivers/commandInterface.cxx

g++ -shared -Wl,--start-group commandInterface.cxxo -Wl,--end-group -o libCommandInterface.so \
-L${JAVA_HOME}/jre/lib/amd64 \
-L${JAVA_HOME}/jre/lib/amd64/server \
-ljava -ljvm -lverify \
-Wl,-rpath=/usr/local/java/jre/lib/amd64 \
-Wl,-rpath=/usr/local/java/jre/lib/amd64/server

g++ -Wall -ggdb -O0 -I${PWD} -L${PWD} -lCommandInterface -Wl,-rpath=${PWD}:${JAVA_HOME}/jre/lib/amd64 -o bareTest bareTest.cxx

--------- end make.sh

--------- begin bareTest.cxx

#include <commandInterface.h>
#include <iostream>
#include <stdlib.h>
#include <unistd.h>

int main (int argc, char **argv)
{
 const char *classpath="./jniMemTest.jar";
 const char *commandA = "--exec=localhost --elem=Front Stage.occulter.x --value-units=mm --value-scalar=1.0";
 const char *commandB = "--exec=localhost --elem=Front Stage.occulter.x --value-units=mm --value-scalar=2.0";
 const char *xmlFile="/MY_HOME/Source/Testbed/Coda/localhost.xml";

 bool arrived, result, success, loop = true;
 commandInterface::handle_t *keys = NULL;
 char *response = NULL;
 unsigned long count = 0, pass = 0;

 result = commandInterface::initialize (xmlFile, classpath, 0, false);
 while (loop)
   {
     if ((count % 10000) == 0)
         std::cout << pass++ << std::endl;

     /*
     if ((count % 10000) == 0)
       {
         std::cout << "Disconnecting and then reconnecting..."
                   << std::endl;
         std::cout << "   disconnect() => "
                   << commandInterface::disconnect() << std::endl;
         std::cout << "   initialzie() => "
                   << commandInterface::initialize (xmlFile, classpath, 0, false)
                   << std::endl;
         pass++;
         count = 0;
       }
     */
     usleep (12);
     count++;
     commandInterface::append (commandA, &keys);
     commandInterface::execute (false);
     for (int i = 0 ; keys != NULL && keys[i] != NULL ; i++)
       {
         commandInterface::collect (keys[i],
                                    true,
                                    true,
                                    &arrived,
                                    &success,
                                    &response);
         //std::cout << "Response from command A: " << response << std::endl;
         free (keys[i]);
         free (response);
         keys[i] = response = NULL;
       }

     commandInterface::append (commandB, &keys);
     commandInterface::execute (false);
     for (int i = 0 ; keys != NULL && keys[i] != NULL ; i++)
       {
         commandInterface::collect (keys[i],
                                    true,
                                    true,
                                    &arrived,
                                    &success,
                                    &response);
         //std::cout << "Response from command B: " << response << std::endl;
         free (keys[i]);
         free (response);
         keys[i] = response = NULL;
       }

     //loop = false;
   }

 return 0;
}

--------- end bareTest.cxx

--------- being commandInterface.h

#ifndef CommandInterface
#define CommandInterface

/*
* The interface to sending commands is trivial from this edge. First, one must
* initialize() the library with some layout.xml that conforms to layout.xsd.
* One should then parse the command string and if the return handle is NOT 0,
* then call execute with it. The use must then call either waitFor() or
* ignore or you will have one serious memory leak.
*/
namespace commandInterface
{
 /*
  * Initialize the system by parsing the specified (layoutFileName) XML file
  * and determining what the remote configuration looks like. If the layout
  * is successfully parsed, then all the other functions will work.
  *
  * Notes:
  *  1) This function can be called many times and only the last successful
  *     layout is used. However, it can be used to switch between commanding
  *     several different testbeds.
  *  2) The state of whether this library is initialized is fully dependent
  *     on the last call to initialize(). No history is retained which means
  *     if the library is successfully initialized and used and then
  *     reinitialized but fails, then this library will report failures
  *     with the reason that the library has not properly initialized.
  */
 bool initialize (const char *layoutFileName,
                  const char *classpath,
                  const char *hostname = 0,
                  const bool dump = false);

 /*
  * These functions are to be used in conjunction with one another to obtain
  * command processing equivalent to using the command line. First, it must
  * be noted that all of these functions will fail if the library has not
  * been properly initialized. The user can then subsequently call 'append'
  * with a series of command line parameters that would make up a complete
  * instruction. Once all of the commands are ready the user would then call
  * 'execute'. This will empty the current list of commands and send them all
  * to the testbed in use. The user can then call 'collect' at any time to
  * retrieve the data returned by the testbed from the commands issued.
  * If the user requested data to be retained and then it is never collected,
  * then 'flush' can be used to empty the retention buffer so that excessive
  * memory is not used.
  *
  * Note: All of these methods should be considered thread safe in that
  *       they are internally atomic.
  *
  * Also: Use a call to free to release the handle_t when done with it.
  *       A call to malloc was used to allocate the space originally.
  *       Hence, handle_t should be usable in both C and C++.
  *
  * append()
  *   This function takes a set of command line arguments and generates a sret
  *   of instructions to execute. It then appends a successful translation to
  *   the list of things to do.
  *
  *   INPUTS:
  *     command : the command line arguments in a set of strings. Each string
  *               should reflect what the command line parser would return
  *               given a single long string. The set must be NULL terminated.
  *     handle  : if non-null, it will return a set of handles that allows the
  *               user to 'collect' the results at a later time. Otherwise the
  *               results are simply discarded.
  *               Note, although the pointer should point to a
  *               'handle_t *handles', it should not be preallocated in size.
  *               This call will allocate the space it needs but it does NOT
  *               deallocate whatever is already there.
  *   OUTPUTS:
  *     If the 'command' parses successfully and is appended to the queue of
  *     things to do, then the function will return true. Otherwise it will
  *     return false.
  *
  * collect()
  *   This function allows the user to collect data from a command at some
  *   time after the command was issued. The user must have a valid handle
  *   to retained data that has not yet been erased.
  *
  *   INPUTS:
  *     handle  : a reference to the command sent that will allow the user
  *               to collect data returned from a previous command.
  *     erase   : once read, remove the data from the queue which invalidates
  *               the handle for all subsequent calls.
  *     wait    : if true, waits until the results are returned from the
  *               testbed and returns immediately when false. The user must
  *               test 'arrived' when wait is false.
  *   OUTPUTS:
  *     Returns true if the handle was valid plus all outputs are non-null
  *     and false otherwise.
  *
  *     arrived : if the results are were there and waiting, then true.
  *               Otherwise, it will be false. It should always be true when
  *               'wait' is true as well.
  *     success : if the status of the resultant from the testbed is success
  *               then this will be set to true. Otherwise it be set to false.
  *     result  : if the results have arrived, then result will be set to the
  *               new value. Otherwise it will remain unchanged.
  *
  * execute()
  *   Sends all of the instructions that have been built up using append to
  *   the testbed for execution. It then empties all of the instructions out
  *   of the queue.
  *
  *   INPUTS:
  *     waitForResponse : if true, then the call will not return until all
  *                       the instructions have completed. If false, the call
  *                       returns immediately after sending the commands.
  *   OUTPUTS:
  *     Returns true if all instructions were transmitted and received.
  *     Otherwise, it returns false.
  *
  * flush()
  *   Clears the queue of all retained messages. This is helper function and
  *   should be considered fairly dangerous as all handles still in use will
  *   become instantly invalid. Use with great care in multi-threaded code.
  */
 typedef char* handle_t;
 bool append (const char *command, handle_t **handle = 0);
 bool append (const char **command, handle_t **handle = 0);
 bool collect (const handle_t handle, const bool erase, const bool wait,
               bool *arrived, bool *success, char **resultant);
 bool execute (const bool waitForResponse = false);
 void flush (void);

 /*
  * Start some method for monitoring the remote system. If a GUI is not used,
  * then the information will be dumped to stdout. This will more than likely
  * overwhelm just about anyone, but what else can one do?
  *
  * Note: The monitor can attach to any testbed not just the one used for
  *       for initialization.
  */
 void monitor (const char *xmlfile, const bool useGUI = true);
}

extern "C"
{
 void commandInterfaceExists();
}

#endif // CommandInterface
--------- end commandInterface.h

--------- begin commandInterface.cxx

#include <iostream>
#include <jni.h>

#include "commandInterface.h"

#ifdef _WIN32
#define PATH_SEPARATOR ';'
#else /* UNIX */
#define PATH_SEPARATOR ':'
#endif

static bool isInitialized = false;
static char *gClasspath = NULL, *gDump = NULL, *gHostname = NULL;
static char *gProfiler = NULL;
static JavaVM *jvm = NULL;
static jclass stringClass = 0;
static jmethodID appendMethod = 0;
static jmethodID collectMethod = 0;
static jmethodID executeMethod = 0;
static jmethodID flushMethod = 0;
static jmethodID monitorMethod = 0;
static JNIEnv *env = NULL;
static jobject toolInterface = 0;

static void printNotInitializedMessage()
{
 std::cout << "This library has not been properly initialized. Please call "
           << "initialize to use any of the other desired operations"
           << std::endl;
}

bool commandInterface::initialize (const char *layoutFileName,
                                  const char *classpath,
                                  const char *hostname,
                                  const bool dump)
{
 JavaVMInitArgs vm_args;
#ifdef JAVA_PROFILER
 JavaVMOption options[(hostname == NULL) ? 3 : 4];
#else
 JavaVMOption options[(hostname == NULL) ? 2 : 3];
#endif
 jint resultant;
 jmethodID initID;
 jstring xmlfile;

 // reset the initialization incase this not the first time being called
 isInitialized = false;
 vm_args.version = JNI_VERSION_1_4;
 vm_args.options = options;
#ifdef JAVA_PROFILER
 vm_args.nOptions = (hostname == NULL) ? 3 : 4;
#else
 vm_args.nOptions = (hostname == NULL) ? 2 : 3;
#endif
 vm_args.ignoreUnrecognized = true;

 /*
 std::cout << layoutFileName << std::endl;
 std::cout << classpath << std::endl;
 std::cout << (hostname == NULL ? "null" : hostname) << std::endl;
 std::cout << (dump ? "true" : "false") << std::endl << std::endl;
 */

 // only need to do this if we do not yet have JVM running
 if (jvm == NULL)
   {
     // Define the user class path
     if (gClasspath != NULL) delete[] gClasspath;
     gClasspath = new char[strlen (classpath) + 19];
     sprintf (gClasspath, "-Djava.class.path=%s", classpath);
     options[0].optionString = gClasspath;
     gClasspath[strlen (classpath) + 18] = 0;

     // Define the dump flag for verbosity
     if (gDump != NULL) delete[] gDump;
     gDump = new char[dump ? 41 : 42];
     sprintf (gDump, "-Dgov.nasa.jpl.testbed.control.dump=%s",
              (dump ? "true" : "false"));
     options[1].optionString = gDump;
     gDump[dump ? 40 : 41] = 0;

     // Define hostname if given
     if (hostname != NULL)
       {
         if (gHostname != NULL) delete[] gHostname;
         gHostname = new char[strlen (hostname) + 18];
         sprintf (gHostname, "-Djava.host.name=%s", hostname);
         options[2].optionString = gHostname;
         gHostname[strlen (hostname) + 17] = 0;
       }

#ifdef JAVA_PROFILER
     const char *profiler = "-agentlib:yjpagent";
     gProfiler = new char[strlen (profiler) + 1];
     sprintf (gProfiler, "%s", profiler);
     gProfiler[strlen (profiler)] = 0;
     options[vm_args.nOptions-1].optionString = gProfiler;
#endif

     // Start the JVM
     resultant = JNI_CreateJavaVM (&jvm, (void**)&env, &vm_args);
     if (resultant < 0 || jvm == NULL)
       {
         std::cerr << "Can't create Java VM: " << resultant << std::endl;
         return isInitialized;
       }

     // New JVM, so lets make sure we find the class
     toolInterface = 0;
   }

 // We currently have JVM, but now we need an instance of the Factory.class
 if (toolInterface == 0)
   {
     jclass theClass;
     jclass throwable = env->FindClass ("java/lang/Throwable");
     jmethodID printStackTrace = env->GetMethodID (throwable,
                                                   "printStackTrace",
                                                   "()V");

     // Go get the class
     theClass = env->FindClass
       ("gov/nasa/jpl/testbed/command/impl/ExternalToolInterface");
     if (theClass == 0)
       {
         std::cerr << "Can't find the class "
                   << "gov.nasa.jpl.testbed.command.impl.ExternalToolInterfaces"
                   << std::endl;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     // Get the constructor for the class
     initID = env->GetMethodID (theClass, "<init>", "()V");
     if (initID == 0)
       {
         std::cerr << "Could not find the constructor."
                   << std::endl;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     // Make an object from the class
     toolInterface = env->NewObject (theClass, initID, NULL);
     if (toolInterface == 0)
       {
         std::cerr << "Could not create the tool interface object."
                   << std::endl;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     // Make global so that the JVM does not unload it on me
     toolInterface = env->NewGlobalRef (toolInterface);
     if (toolInterface == 0)
       {
         std::cerr << "Could not globally define the tool interface."
                   << std::endl;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     // Have the class and instance, so lets get the init method
     initID = env->GetMethodID (theClass,
                                "initialize",
                                "(Ljava/lang/String;)Z");
     if (initID == 0)
       {
         std::cerr << "Could not find the method for loading "
                   << "the configuration" << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     // For convience, look up classes and methods being used by everyone else
     stringClass = env->FindClass ("java/lang/String");
     if (stringClass == 0)
       {
         std::cerr << "Could not find the class java.lang.String!!"
                   << std::endl;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     appendMethod =
       env->GetMethodID (theClass,
                         "append",
                         "([Ljava/lang/String;Z)[Ljava/lang/String;");
     if (appendMethod == 0)
       {
         std::cerr << "Could not find the appropriate append method."
                   << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     collectMethod = env->GetMethodID (theClass,
                                       "collect",
                                       "(Ljava/lang/String;ZZLgov/nasa/jpl/testbed/command/impl/ExternalToolResponse;)Z");
     if (collectMethod == 0)
       {
         std::cerr << "Could not find the appropriate collect method."
                   << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     executeMethod = env->GetMethodID (theClass,
                                       "execute",
                                       "(Z)Z");
     if (executeMethod == 0)
       {
         std::cerr << "Could not find the appropriate execute method."
                   << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     flushMethod = env->GetMethodID (theClass,
                                     "flush",
                                     "()V");
     if (flushMethod == 0)
       {
         std::cerr << "Could not find the appropriate flush method."
                   << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }

     monitorMethod = env->GetMethodID (theClass,
                                       "monitor",
                                       "(Ljava/lang/String;Z)V");
     if (monitorMethod == 0)
       {
         std::cerr << "Could not find the appropriate monitor method."
                   << std::endl;
         env->DeleteGlobalRef (toolInterface);
         toolInterface = 0;
         if (env->ExceptionCheck() == JNI_TRUE)
           {
             jthrowable except = env->ExceptionOccurred();
             env->CallVoidMethod (except, printStackTrace);
           }
         return isInitialized;
       }
   }

 if (toolInterface != 0)
   {
     // Finally, initialize the system
     xmlfile = env->NewStringUTF (layoutFileName);
     isInitialized = env->CallBooleanMethod (toolInterface, initID, xmlfile);
   }

 return isInitialized;
}

static commandInterface::handle_t copy (jstring key)
{
 // Create a local frame for cleanup later
 jint pushStatus = env->PushLocalFrame (25);
 if (pushStatus < 0) return NULL;

 jboolean isCopy;
 const char *lkey = env->GetStringUTFChars (key, &isCopy);
 commandInterface::handle_t result =
   (commandInterface::handle_t) malloc (strlen (lkey) + 1);

 result[strlen (lkey)] = 0;
 strcpy (result, lkey);
 env->ReleaseStringUTFChars (key, lkey);

 // Clean up the mess that has been made
 env->PopLocalFrame (NULL);

 return result;
}

static bool isDoubleDash (const char *substring)
{
 return substring[0] == '-' && substring[1] == '-';
}

bool commandInterface::append (const char *command, handle_t **handle)
{
 const int len = strlen (command);
 bool result = false;
 char **parsedCommand;
 int c = 2, s = 0, segments = 1, start = 0;

 // count the number segments in this command
 for (int i = 0 ; i < len ; i++) if (isDoubleDash (&command[i])) segments++;

 // allocate the space for the segments
 parsedCommand = (char**) calloc (segments--, sizeof (char*));
 parsedCommand[segments] = NULL;

 // break up the single line
 while (s < segments && c < len - 1)
   {
     if (isDoubleDash (&command[c]))
       {
         parsedCommand[s] = (char*) malloc (c - start);
         strncpy (parsedCommand[s], &command[start], (c - start));
         parsedCommand[s++][c - start - 1] = 0;
         start = c;
       }

     c++;

     if (c == len - 1)
       {
         parsedCommand[s] = (char*) malloc (len - start + 1);
         strncpy (parsedCommand[s], &command[start], (len - start));
         parsedCommand[s++][len - start] = 0;
       }
   }
 result = append ((const char**)parsedCommand, handle);

 // time to clean up
 for (s = 0 ; parsedCommand[s] != NULL ; s++) free (parsedCommand[s]);
 free (parsedCommand);

 return result;
}

bool commandInterface::append (const char **command, handle_t **handle)
{
 bool result = false;
 int count = 0;

 // quickly get how many commands there are.
 while (command[count] != NULL) count++;

 if (isInitialized)
   {
     // Create a local frame for cleanup later
     jint pushStatus = env->PushLocalFrame (25);
     if (pushStatus < 0) return false;

     // Get the initial array for the commands and fill it
     jobjectArray request = env->NewObjectArray (count,
                                                 stringClass,
                                                 NULL);
     for (int i = 0 ; i < count ; i++)
         env->SetObjectArrayElement (request, i,
                                     env->NewStringUTF (command[i]));

     // We have the method we need as well as the object it belongs to
     // plus something to hold the results, let call it.
     jobjectArray keys = (jobjectArray)env->CallObjectMethod (toolInterface,
                                                              appendMethod,
                                                              request,
                                                              handle != NULL);

     // The work has been done, now we need to determine if it worked
     count = env->GetArrayLength (keys);
     result = count > 0;

     // Now, if the use wanted to hold onto the key for later
     if (handle != NULL)
       {
         *handle = (handle_t*)calloc (count + 1, sizeof(handle_t));
         (*handle)[count] = NULL; // null terminate the sequence
         for (int i = 0 ; i < count ; i++)
           (*handle)[i] =
             copy ((jstring)env->GetObjectArrayElement (keys, i));
       }

     // Clean up the mess that has been made
     env->PopLocalFrame (NULL);
   }
 else printNotInitializedMessage();

 return result;
}

bool commandInterface::collect (const handle_t handle,
                               const bool erase,
                               const bool wait,
                               bool *arrived,
                               bool *success,
                               char **resultant)
{
 const char *RESPONSE_CLASS =
   "gov.nasa.jpl.testbed.command.impl.ExternalToolResponse";
 bool result = false;

 if (isInitialized)
   {
     // Create a local frame for cleanup later
     jint pushStatus = env->PushLocalFrame (25);
     if (pushStatus < 0) return false;

     if (arrived != NULL && success != NULL && resultant != NULL)
       {
         jclass responseClass;
         jmethodID initID;
         jobject response;
         jstring jkey = env->NewStringUTF (handle);

         // get an instance of the response class
         responseClass = env->FindClass
           ("gov/nasa/jpl/testbed/command/impl/ExternalToolResponse");
         if (responseClass == 0)
           {
             // Clean up the mess that has been made
             env->PopLocalFrame (NULL);
             std::cerr << "Could not find the class "
                       << RESPONSE_CLASS
                       << " which means this call failed for what is most "
                       << "likely a configuration problem."
                       << std::endl;
             return false;
           }

         // find the constructor of the response class
         initID = env->GetMethodID (responseClass, "<init>", "()V");
         if (initID == 0)
           {
             // Clean up the mess that has been made
             env->PopLocalFrame (NULL);
             std::cerr << "Could not find the constructor of class "
                       << RESPONSE_CLASS
                       << " which means this call failed for what is most "
                       << "likely a configuration problem."
                       << std::endl;
             return false;
           }

         // have the class and constructor, now make an object out of it
         response = env->NewObject (responseClass, initID, NULL);
         if (response == 0)
           {
             // Clean up the mess that has been made
             env->PopLocalFrame (NULL);
             std::cerr << "Could not create the response object." << std::endl;              return false;
           }

         // Go get the response
         result = env->CallBooleanMethod (toolInterface,
                                          collectMethod,
                                          jkey,
                                          erase,
                                          wait,
                                          response);
         if (result)
           {
             jboolean isCopy;
             jfieldID arrivedID = env->GetFieldID (responseClass,
                                                   "arrived",
                                                   "Z");
             jfieldID successID = env->GetFieldID (responseClass,
                                                   "success",
                                                   "Z");
             jfieldID responseID = env->GetFieldID (responseClass,
                                                    "response",
                                                    "Ljava/lang/String;");

             if (env->GetBooleanField (response, arrivedID))
               {
                 jstring jResponse =
                   (jstring)env->GetObjectField (response, responseID);
                 *success = env->GetBooleanField (response, successID);
                 *arrived = true;

                 // Now, get the string into a more managable form
                 const char *resp = env->GetStringUTFChars (jResponse,
                                                            &isCopy);
                 int length = env->GetStringLength (jResponse);
                 (*resultant) = (char*) malloc (length + 1);
                 (*resultant)[length] = 0;
                 strcpy (*resultant, resp);
                 env->ReleaseStringUTFChars (jResponse, resp);
               }
             else *arrived = false;
           }
       }
     else std::cerr << "Output parameters are NULL. No reason to try and "
                    << "collect any results" << std::endl;

     // Clean up the mess that has been made
     env->PopLocalFrame (NULL);
   }
 else printNotInitializedMessage();

 return result;
}

bool commandInterface::execute (const bool waitForResponse)
{
 bool result = false;

 if (isInitialized)
   {
     // Create a local frame for cleanup later
     jint pushStatus = env->PushLocalFrame (25);
     if (pushStatus < 0) return false;

     result = env->CallBooleanMethod (toolInterface,
                                      executeMethod,
                                      waitForResponse);

     // Clean up the mess that has been made
     env->PopLocalFrame (NULL);
   }
 else printNotInitializedMessage();

 return result;
}

void commandInterface::flush (void)
{
 if (isInitialized)
   {
     // Create a local frame for cleanup later
     jint pushStatus = env->PushLocalFrame (25);
     if (pushStatus < 0) return;

     env->CallVoidMethod (toolInterface, flushMethod);

     // Clean up the mess that has been made
     env->PopLocalFrame (NULL);
   }
 else printNotInitializedMessage();
}

void commandInterface::monitor (const char *xmlfile, const bool useGUI)
{
 if (isInitialized)
   {
     // Create a local frame for cleanup later
     jint pushStatus = env->PushLocalFrame (25);
     if (pushStatus < 0) return;

     env->CallVoidMethod (toolInterface,
                          monitorMethod,
                          env->NewStringUTF (xmlfile),
                          useGUI);

     // Clean up the mess that has been made
     env->PopLocalFrame (NULL);
   }
 else printNotInitializedMessage();
}

void commandInterfaceExists() {}

--------- end commandInterface.cxx

--------- I cannot send binaries, so I cannot give you jniMemTest.jar. If
you want it, give me your email address and I will send it.
Matthias Ernst - 16 Jan 2006 07:11 GMT
Hello Sheol,

> I cannot do a uuencoded zip because someone is rejecting it, so here
> it is. There are 5 files involved to get the test to build and
> execute, but commandInterface.cxx is where the memory leak resides. It
> takes about an hour to start seeing the memeory leak without a
> profiler.

Why don't you tell us:
* what kind of objects are leaked
* which method they were allocated in
* where is the ridiculously small testcase you were talking about?

Matthias
Chris Uppal - 16 Jan 2006 12:15 GMT
> I cannot do a uuencoded zip because someone is rejecting it, so here it
> is.

That's far too long (by factor of about 10, or at least 5) to look through in
detail.  From a quick scan I'm not convinced that you are always deleting
references correctly (although I may be wrong -- there's too much code to see
what's going on clearly).

The basic rules are.

a) PopLocalFrame() will clean up local references created since the matching
call to PushLocalFrame() (whether you want them to be cleaned up or not).  It
will not clean up anything else.

b) Otherwise /all/ local references (jobject, and derived classes like
jthowable and jclass) must be cleaned up with DeleteLocalRef().  Note that if
you create a global ref from a local ref you still have to delete the local
ref.

I happened to notice the function commandInterface::copy().  It seems odd that
it is using the PushLocalFrame()/PopLocalFrame() pair even though it never
creates any local references.  That suggests (to me) that you are using
PushLocalFrame()/PopLocalFrame() more-or-less randomly in an attempt to fix the
problem.  That won't work, you have to think the thing through properly.

There appear to be several places in the long commandInterface::initialise()
function where you create local references and can then return without deleting
them.  E.g. "theClass", "throwable", and "except".   The initial local
reference to "toolInterface" is similarly not cleaned up.   I don't know
whether those apparent errors (and any others I have missed) are directly
responsible for your problem since I assume that code is only executed once --
but if not then it suggests that similar problems may lurk elsewhere.  (And it
/proves/ that your quadruple-checking missed something ;-)

You should be able to create a C++ framework (similar to std::auto_ptr<T>)
which will manage local references for you.  That would take some work to set
up but might easily pay for itself in the long run.  In the short term, check
the code /yet again/ and -- if that doesn't fix it -- then you can use your
profiling information to discover which Java classes have leaked instances.
You can then trace through (in your C++ debugger perhaps) the /exact/ sequence
of events relating to the C++ variable which hold instances of those classes.
You can expect that to take some time, unless you are lucky, but it should find
the problem for you in the end.

BTW, running with the -Xcheck:jni flag may cause the JVM to print some useful
diagnostic messages.  I haven't had a lot of luck with it myself (in fact I
think it's buggy in 1.5.0 and may causes the runtime to abort with a
misdiagnosed error when there is no error at all), but it might help a bit.

   -- chris


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.