jni problem linking into existing s.o. lib with externs

Thread view:

jadrian@wi.rr.com - 20 Apr 2007 22:00 GMT

I'm trying to create a jni library which which will interface with
routines in an existing shared object library. This existing library
has functions available to many C programs and defines 3 variables as
extern; a char * (ERRMSG), a char array(PROGNAME) and a
long(LOGMSG_SW). By declaring them extern any program can include a
header file with the declarations/definitions of the 3 variables and
have them available to the existing library as sort of global
variables across libraries and standalones.

The jni library compiles fine but I get a runtime error:

The java class could not be loaded. java.lang.UnsatisfiedLinkError:
libShare.so (Symbol resolution failed for /Test/libso/libBOB.so
because:
Symbol ERRMSG (number 89) is not exported from dependent
module java.
Symbol LOGMSG_SW (number 90) is not exported from dependent
module java.
Symbol PROGNAME (number 91) is not exported from dependent
module java.
Examine .loader section symbols with the 'dump -Tv' command.)

The jni library is libShare.so and the existing library is libBOB.so.
Running on AIX 5.2, java 1.5.

I've tried many combinations of declaring the variables in the jni
library from including the header that defines them, declaring them in
the jni library and defining them, all different combinations of
including/excluding them from the Share.imp and Share.exp files all to
no avail. It's interesting the that libBOB.so seems to complain about
"dependent module java" instead of libShare.so which is what invokes
libBOB.so. I did add "libBOB.so" to the System.load() call, also to
no avail.

Any thoughts?
Thanks ...

Reply to this Message

Gordon Beaton - 21 Apr 2007 08:52 GMT

> The jni library compiles fine but I get a runtime error:
>
[quoted text clipped - 8 lines]
> module java.
> Examine .loader section symbols with the 'dump -Tv' command.)

In the existing library source, the variables must be defined as usual
(i.e. without extern) and declared separately (with extern, typically
in a header file). Something like this:

BOB.c: char *ERRMSG;

BOB.h: extern char *ERRMSG

In Share.c, you include BOB.h to see the declaration.

If you simply add "extern" to the definition in the BOB.c, it becomes
a declaration and the variable is no longer defined anywhere.

/gordon

--

Reply to this Message

Gordon Beaton - 21 Apr 2007 12:36 GMT

> In the existing library source, the variables must be defined as usual
> (i.e. without extern) and declared separately (with extern, typically
[quoted text clipped - 8 lines]
> If you simply add "extern" to the definition in the BOB.c, it becomes
> a declaration and the variable is no longer defined anywhere.

Sorry, I realize now that I only answered the easy part of the
problem...

If libShared.so has a runtime dependency on libBOB.so, then you need
to encode that information into libShared.so when you build it (two
separate calls to System.load() or System.loadLibrary() will *not*
work).

Exactly how you do that depends on your linker. This is how I would
link (on Linux), including information about libBOB.so in
libShared.so:

gcc -shared -f libBOB.so Shared.o -o libShared.so

At runtime, the dynamic linker (e.g. ld.so or the AIX equivalent)
needs to resolve the dependency, so it isn't sufficient that libBOB.so
can be found in java.library.path, it needs to be in LD_LIBRARY_PATH
or some other place where ld.so can find it. There are also link flags
you can use to encode a path to libBOB.so, but that solution is
fragile.

Run "readelf -d libShared.so" (does AIX use ELF files?) or maybe
"objdump -p libShared.so" to see a NEEDED entry in the library's
dynamic section.

Run "ldd -v libShared.so" to confirm not only that the dependency on
libBOB.so is there, but that ld.so is able to resolve it.

Your Java application should only load libShared.so, not libBOB.so.

/gordon

--

Reply to this Message

Andrew Thompson - 21 Apr 2007 14:08 GMT

>Sorry, I realize now that I only answered the easy part of the
>problem...

I demand a full 'refund' of all moneys paid for the initial answer.

<OT>
As an aside, while I thought my answer to the other 'GIYF'
thread was more informative* than the one you posted -
yours was *way* funnier. ;-)

* Well, OK - it failed to inform the questioner of anything, as
it turns out - but providing information *was* my intention.
There's just no telling what it takes, to get the point across!
</OT>

Signature

Andrew Thompson
http://www.athompson.info/andrew/

Reply to this Message

Gordon Beaton - 21 Apr 2007 19:18 GMT

><OT>
> As an aside, while I thought my answer to the other 'GIYF'
[quoted text clipped - 5 lines]
> There's just no telling what it takes, to get the point across!
></OT>

Actually I wasn't trying to be funny - the page I referred him to does
explain what the acronym means and why he was sent there, in a way
that I thought summed up the thread nicely. He didn't seem to
appreciate my help though.

/gordon

--

Reply to this Message

jadrian@wi.rr.com - 21 Apr 2007 16:30 GMT

> > In the existing library source, the variables must be defined as usual
> > (i.e. without extern) and declared separately (with extern, typically
[quoted text clipped - 42 lines]
>
> --

Hi Gordon,

Thanks for your suggestions. If I understand you, you think that this
is a linker problem, not primarily a storage syntax issue, which makes
sense.

On AIX, the link command for this library is:

xlC_r -brtl -olibShare.so Share.imp -bE:./Share.exp -bM:SRE -bnoentry -
L/Test/libso -lBOB Share.o

libBOB.so is located in /Test/libso and libShare.so is located in the
same directory as Share.class.

I'm not a C/Unix guru but I use and create shared object libraries
frequently and the JVM seemed to be the wildcard here. I'm pretty
sure that LD_LIBRARY_PATH does not include the directory that
libBOB.so is in but for developement purposes libShare.so was in the
same directory as Share.class. It connected with libShare.so nicely at
runtime until I linked libBOB.so into the build of libShare.so. That
was when, at runtime, the error appeared. AIX requires both .imp
and .exp files; the BOB.imp file has the references to ERRMSG, PROGRAM
and LOGMSG_SW.

The most difficult question, as I see it, is whether libBOB.so has a
relationship with Share.class, so to speak, or whether it shares
process space with libShare.so only and is oblivious to Share.class.
If its the latter case, then a solution exists in resolving references
between the two shared object libraries. If, on the other hand,
libBOB.so is trying to find references in the JVM then it has the
potential of being intractable, unless setting LD_LIBRARY_PATH to
include libBOB.so is able to do it.

I don't believe that AIX uses ELF files and the only diagnostic I'm
familiar with to poke around the guts of a library is the one that is
suggested in the error message itself, dump -Tv libShare.so, which
didn't offer much.

I'll try the diagnostics you suggest. If they don't work I'll keep
looking in that direction for an answer. I intended to remove the
load of libBOB.so from the java code--that was just a Hail Mary throw
anyway.

Thanks again for your help.

Jeff Adrian

Reply to this Message

Gordon Beaton - 21 Apr 2007 16:44 GMT

> The most difficult question, as I see it, is whether libBOB.so has a
> relationship with Share.class, so to speak, or whether it shares
[quoted text clipped - 4 lines]
> the potential of being intractable, unless setting LD_LIBRARY_PATH
> to include libBOB.so is able to do it.

One problem here is that when building a shared library is that you
don't get to find out about unresolved symbols until runtime.

Of course all of the libraries share process space with the rest of
the JVM (and any loaded classes), but the visibility of the loaded
symbols is not global. In fact I think you'll find that the JVM uses
dlopen(RTLD_LOCAL) or similar when you do System.loadLibrary(). That's
why your first library needs to "know" about its dependency on the
second one, and why the JVM only loads the first one, letting ld.so
load the second one in order to resolve the dependency.

> I don't believe that AIX uses ELF files and the only diagnostic I'm
> familiar with to poke around the guts of a library is the one that
[quoted text clipped - 3 lines]
> I'll try the diagnostics you suggest. If they don't work I'll keep
> looking in that direction for an answer.

Gnu objdump is available for most systems (part of Gnu objutils).

About ldd for AIX:
http://www.faqs.org/faqs/aix-faq/part4/section-22.html

Also, consult a linking and libraries guide if you've got one.

You might want to ask on an AIX group as well.

/gordon

--

Reply to this Message

jni problem linking into existing s.o. lib with externs

Free Magazines