Big JARs = Big Problems?

Thread view:

kk_oop@yahoo.com - 16 Sep 2005 20:14 GMT

Hi. I'm on a project that has several notional Java components. These
components are each made up of one or more JAR files. Our JAR files
are very big, meaning they have a lot of .class files. All the class
files are somewhat cohesive, since they all are about the functionality
of a particular component. However, my sense is that they can stand to
be broken down into more cohesive pieces--more influenced by the Java
packages than by the notional component.

I've been getting pushback on doing this. Folks are saying that a JAR
is just a zip file, so they're structure and interdependencies are not
that big a deal. When I suggest narrowing the cohesiveness of the big
JARs, they respond "Where do you stop? How small do you go?"

Note that with respect to JAR interdependencies, one way to reduce
those is to have bigger JARs. But this seems like an artificial way to
manage dependencies: Less dependencies but less cohesion at the JAR
level.

So my questions are thus:

1. What are some pros and cons to making more granular JAR files?

2. What are some criteria for knowing how big is too big or how small
is too small?

Thanks!

Ken

Reply to this Message

Roedy Green - 16 Sep 2005 21:04 GMT

> Our JAR files
>are very big, meaning they have a lot of .class files.

That is subjective. What order are you talking about, how many
classes, how fat the final jar?

Problems with big jars.

1. bigger jars take longer to find an element -- a bigger haystack.
However IIRC under some circumstances a ClassLoader will build a
hashmap to help it find the elements quickly. then it would not make
much difference.

2. bigger jars take longer to download. If you are sending updates, if
even one comma in one class changes you have to send the whole jar.

The problems with small jars.

1. You have to manage more jars, track them, make sure they are the
right version. make sure they are on the classpath.

2. there is no obvious way to know you have everything and everything
is designed to fit together. Even if you don't he customer won't and
will restore to create a mixed incompatible bag.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Again taking new Java programming contracts.

Reply to this Message

Hemal Pandya - 17 Sep 2005 07:17 GMT

[snip]

> Problems with big jars.
>
> 1. bigger jars take longer to find an element

jar index might help. Look at the -i option to jar.

[snip]

Reply to this Message

Roedy Green - 17 Sep 2005 08:01 GMT

>jar index might help. Look at the -i option to jar

The jar index is not what you might imagine, some sort of hashmap or
btree. It is a simple list of elements, without pointers to where the
elements are.. Its function is to provide a list of what is in OTHER
jar files, so it really only helps when you have a group of jars.

For very large jars, you can pack them which gives ultra-efficient
compression. See http://mindprod.com/jgloss/jar.html

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Again taking new Java programming contracts.

Reply to this Message

Hemal Pandya - 17 Sep 2005 15:50 GMT

> >jar index might help. Look at the -i option to jar
>
> The jar index is not what you might imagine, some sort of hashmap or
> btree. It is a simple list of elements, without pointers to where the
> elements are.. Its function is to provide a list of what is in OTHER
> jar files, so it really only helps when you have a group of jars.

Hmmm. Thanks for the info.

Reply to this Message

EricF - 18 Sep 2005 05:35 GMT

>>jar index might help. Look at the -i option to jar
>
[quoted text clipped - 5 lines]
>For very large jars, you can pack them which gives ultra-efficient
>compression. See http://mindprod.com/jgloss/jar.html

If they are downloaded when used as an applet, that's a benefit for sure. But
if run on a server, there is overhead to uncompress them.

Reply to this Message

Roedy Green - 18 Sep 2005 07:04 GMT

>If they are downloaded when used as an applet, that's a benefit for sure. But
>if run on a server, there is overhead to uncompress them.

that is true of any jar. I don't know if packed jars take any more
effort to decompress. Has anyone done experiments?

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Again taking new Java programming contracts.

Reply to this Message

Raymond DeCampo - 17 Sep 2005 23:14 GMT

>>Our JAR files
>>are very big, meaning they have a lot of .class files.
[quoted text clipped - 17 lines]
> 1. You have to manage more jars, track them, make sure they are the
> right version. make sure they are on the classpath.

Note that you can use the Class-Path entry in the manifest to help you
here. For example, if foo.jar requires bar.jar and baz.jar, it can list
them in the Class-Path entry in the manifest. Then the classpath
specified to the JVM need only contain foo.jar; that is, bar.jar and
baz.jar are added after the fact.

> 2. there is no obvious way to know you have everything and everything
> is designed to fit together. Even if you don't he customer won't and
> will restore to create a mixed incompatible bag.

HTH,
Ray

Signature

XML is the programmer's duct tape.

Reply to this Message

Andrey Kuznetsov - 17 Sep 2005 16:15 GMT

> Hi. I'm on a project that has several notional Java components. These
> components are each made up of one or more JAR files. Our JAR files
[quoted text clipped - 20 lines]
> 2. What are some criteria for knowing how big is too big or how small
> is too small?

see http://sourceforge.net/projects/jpackit/

Signature

Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

Reply to this Message

Patrick May - 17 Sep 2005 16:31 GMT

> 2. What are some criteria for knowing how big is too big or how
> small is too small?

"The granule of reuse is the granule of release." (See
http://www.objectmentor.com/resources/articles/granularity.pdf.) As I
rule, I create a jar for each package. This minimizes the need to
update jar files and minimizes the size of files that must be
distributed in any given release.

Regards,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc. | The experts in large scale distributed OO
| systems design and implementation.
pjm@spe.com | (C++, Java, Common Lisp, Jini, CORBA, UML)

Reply to this Message

pkriens - 18 Sep 2005 11:27 GMT

This is one of the areas that the OSGi specifications specifically
address. We have developed a modularization of Java by using JARs as
logical modules. The manifest is used to specify imported and exported
packages, which then allows also for private packages. Pacakges are
versioned so that the runtime or deployment time can analyze what will
work together. Different vendors and open source parties have developed
tools to mkae this manageable. Most noticeable is ofcourse Eclipse.
Release 4 of the OSGi specifications were very much driven by the needs
you describe because Websphere and Lotus were key requirement drivers
for the Eclipse case. That is, Eclipse had to run decently with 5000
bundles (=jars). However, OSGi is also available now in Apache (OSCAR)
and with Knopflerfish.

Currently, JSR 277 is working on a further standardization of modules
in Java; several OSGi people are working in this expert group to see if
the OSGi modularization can be adopted in this JSR.

OSGi is perfectly usable as a reliable deployment platform for JARs.
However, there are additional mechanisms that support the needs for
(very) large applications. There is the dynamic service registry to
support service oriented architectures inside a VM and the life cycle
management APIs that allow you update a single JAR without disrupting
the system. Both mechanisms are extremely useful in cocnquering the
problem of large software projects.

Some links:
http://www.knopflerfish.org
http://oscar.objectweb.org
http://www.eclipse.org/osgi
http://www.prosyst.com
http://www-306.ibm.com/software/wireless/smf/
http://en.wikipedia.org/wiki/OSGi

Kind regards,

Peter Kriens

Reply to this Message

Roedy Green - 18 Sep 2005 18:54 GMT

> http://www.knopflerfish.org
> http://oscar.objectweb.org
> http://www.eclipse.org/osgi
> http://www.prosyst.com
> http://www-306.ibm.com/software/wireless/smf/
> http://en.wikipedia.org/wiki/OSGi

for future reference you can find those links in the Java glossary
under OSGi at http://mindprod.com/jgloss/osgi.html

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Again taking new Java programming contracts.

Reply to this Message

Big JARs = Big Problems?

Free Magazines