 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / March 2007Getting a list of Classes in a Package
I've been doing some research online about this subject and I've come across a pretty even balance of "why I need to do it" and "why it isn't possible". I want to maybe get some discussion going, and maybe if there is anyone out there to suggest some alternatives or the best method to accomplish what I want. What I've read so far is that: 1. Packages are not hierarchical, and stand separate to each other, despite similarities. 2. Classes are not fetched until they are requested/needed. Point 1 seems really irrelevant to my purpose as the way I've got my program set up, I know the interface my class will implement (and thus can use it as an instance of that interface), and I know the specific package my classes will be found in - both at compile time. Java offers reflection upon Packages, so they can't be totally redundant... Point 2 is a good point and it is certainly the biggest reason why this is not possible or not done yet: flexibility. But what happens when you import an entire package? For example "javax.swing.*". I'm not having an easy time discerning the behaviour, but wouldn't the classloader reach over and ask for every class in the package? Or am I mistaken, and the classloader only gets classes when they are first interacted with? At that point, even the functionality of getting the names of classes only retrieved by the classloader within the package would suffice. I could write my loaders for each type of runtime-discovered class to request ".*" of the package and relax knowing I'm not requiring everything in a .JAR to be loaded...? IDEs like NetBeans, Eclipse and others do all kinds of wicked reflection when you think about it. To what degree they cheat, I don't know, and would love to! To put my question or intention as simply as possible: During run-time, how do I create one instance of each class in a package when all I know at compile-time is the package's name? > I've been doing some research online about this subject and I've come > across a pretty even balance of "why I need to do it" and "why it > isn't possible". I'm glad you have not only done that research, but thought to mention it. This question often leads to 'short sharp answers'. ;-) > I want to maybe get some discussion going, and maybe if there is > anyone out there to suggest some alternatives or the best method to > accomplish what I want. (shrugs) Sure. > What I've read so far is that: .. > 2. Classes are not fetched until they are requested/needed. I would qualify that to say that classes are not fetched until needed. A JVM would *not* need to fetch a 'package' as such. > Point 1 seems really irrelevant .. > ...Java > offers reflection upon Packages, so they can't > be totally redundant... No, but see *. A quicker way to find the classes in a *specific* jar file is to use the Zip(LookItUp..) class methods to get an enumeration of the zip entries, and simply read through them. > Point 2 is a good point and it is certainly the biggest reason why > this is not possible or not done yet: flexibility. But what happens > when you import an entire package? For example "javax.swing.*". I'm > not having an easy time discerning the behaviour, * 'Entire package' imports are resolved at *compile* time. Java bytecodes only ever have specific imports (fully qualified class names) in them! >..but wouldn't the > classloader reach over and ask for every class in the package? No. > At that point, even the functionality of getting the names of classes > only retrieved by the classloader within the package would suffice. ..hmm. That might be doable. I'll cede to the better knowledge of others. ... > IDEs like NetBeans, Eclipse and others do all kinds of wicked > reflection when you think about it. To what degree they cheat, I > don't know, and would love to! I think it is quite relevant to expect a build, debugging or deployment tool to allow or enable things that might not be appropriate or needed for an "it's in production" application. Reflection can (as I understand) be quite slow. The real questions here, as I see them, are - 1) Why does the code *not* know what classes are available at compile time? 2) If the code is to be told/discover them at run-time, how is it expected to know what to do with the class? Andrew T. > > I've been doing some research online about this subject and I've come > > across a pretty even balance of "why I need to do it" and "why it [quoted text clipped - 72 lines] > > Andrew T. Andrew, First off, thanks for your thorough and helpful response. I'll answer your questions based on the "meat" of my issue. Hopefully I don't miss anything you asked or mentioned, and if I did, let me know... > 1) Why does the code *not* know what classes > are available at compile time? > 2) If the code is to be told/discover them at > run-time, how is it expected to know what to > do with the class? The program ("initial program") is not aware of the classes ("extensions") at compile time because the classes I want it to discover will be created and distributed long after the initial program itself has been compiled and distributed. The initial program will know how to interact with these extension classes through an interface defined during the initial programs compilation. This would offer the flexibility for the initial program to be extended with the new classes (distributed in separate archives too!) it is totally unaware of. These new classes are obviously aware of the initial program and how to work in it, but the initial program itself is only aware of how to use them by their interface...Which would likely consist of a few methods akin to start/ programshuttingdown/suspend etc... For example: The distributed initial program's .JAR file would contain a package "net.my.program.packageofclasses". Inside that package would be the interface "Extension". This is of course alongside all the other stuff that would make up the initial program. The second package would come along as a separate archive that gets included in the classpath, and contain some new classes in "net.my.program.packageofclasses", each of which would implement the interface "Extension". The extension(s) would know about the Extension interface and all the gritty details of the initial program because the extensions would be compiled against the initial program. The classes introduced by this new package would be instantiated by the currently-unavailable/disputed ability of the initial program to reflect on the contents of the package "net.my.program.packageofclasses". It would instantiate them not as instances of themselves, but as instances of the interface (which they do implement). It is also possible to glean the interfaces a class implements through reflection currently - so there is the opportunity for integrity. Simply put, I want to have my program capable of automatically discovering and loading any classes within that package. For convenience and ease of use, I want a minimal (AKA: none) amount of configuration required to have this happen. Scanning the .JAR file - while probably the best (and only?) solution for me, is of course flying in the face of good programming. This process isn't impossible as it is right now, it is simply crippled. If you maintain a list of class "names" for your runtime to instantiate, you can get this. It is this list I want to avoid having to require users/admins to create & maintain. By using recent changes in the 1.6 release of Java, you can include entire directories in the classpath. This is part one of making my desired functionality a reality. It allows the configuration of a specific directory which has all .JAR files located in it to be loaded. I think if this is truly impossible, it will likely be a natural "next step" for Java in the near future, as there has been a substantial amount of discussion about it. > This process isn't impossible as it is right now, it is simply > crippled. If you maintain a list of class "names" for your runtime to [quoted text clipped - 5 lines] > specific directory which has all .JAR files located in it to be > loaded. What I see as a difficulty with what you are asking for is that you don't know how many classes there are on any given system. Sure you *your* system right now there maybe a few thousand, easy enough to iterate though. But on other systems there may be millions or tens of millions of classes. Maybe more than a dead O(n) search should handle. What you really want is a clean way of iterating through only one package, and of detecting when a new .jar file is added to the classpath (or other locations). Then you can detect when only certain classes (ones in the package name space you devise) are added. This could provide needed performance improvement that sysadmins so love. In the mean time, taking a file handle or file name through IPC and iterating through a .jar isn't so bad. Some people even think file systems should be the API for everything. ;-) > > This process isn't impossible as it is right now, it is simply > > crippled. If you maintain a list of class "names" for your runtime to [quoted text clipped - 21 lines] > through a .jar isn't so bad. Some people even think file systems should > be the API for everything. ;-) That is what I asked for. I don't think you have read both my explanations fully. I want to be able to go through a specific package and create one instance of every class I find that implements a specific interface. I don't know how else to put that? .... > Point 2 is a good point and it is certainly the biggest reason why > this is not possible or not done yet: flexibility. But what happens [quoted text clipped - 3 lines] > I mistaken, and the classloader only gets classes when they are first > interacted with? Despite its name, "import" just supplies information to the compile time name resolution system. It does not copy anything in. The class file that the classloader sees only has the resulting qualified names. Patricia > I've been doing some research online about this subject and I've come > across a pretty even balance of "why I need to do it" and "why it [quoted text clipped - 37 lines] > During run-time, how do I create one instance of each class in a > package when all I know at compile-time is the package's name? I think anyone who is interested in having this issue solved and a massively useful set of features made available to them in Java, they should go and vote for the following bug/RFE: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4061452 It isn't far from getting enough votes for recognition, and I think this functionality is well worth having. Omega mentioned: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4061452 People should be required to pass some sort of exam before submitting FREs. Going back to your original question: if you want to know what files (in the broadest sense of the word) are available to be loaded into a JVM (whatever package they may end up being part of) then clearly you can't ask the JVM -- by definition it doesn't know about them. Any more than a a text editor knows what files it may be asked to edit in future. If you want a list of the files in a JAR, or the JARs in a directory, then there are perfectly straightforward ways of getting one without messing around with reflective operations (which are impossible anyway). -- chris On Mar 21, 3:06 pm, "Chris Uppal" <chris.up...@metagnostic.REMOVE- THIS.org> wrote: > Omega mentioned: > [quoted text clipped - 12 lines] > > -- chris I can't possibly agree with you. This is totally what reflection is all about. Not to mention I don't think it is fair that you insult the intelligence of someone who submitted an RFE. People aren't stupid for wanting a feature and I can think of many reasons why using the filesystem to glean the information I desire is very risky (what if there is no filesystem?). I can suck it up and write it this way, no problem. I will for the time being have to do this. I know for a fact that my program will be run on Java SE and not Java ME platforms. Still though, the elegance offered by being able to teach my program how to look inward and discover the classes made available by the inclusion of another .JAR in the classpath is pretty handy and neat too. So, bite your tongue and realize that the JRE is for everyone. Not just you, you have no reason to act so offended. > I can't possibly agree with you. This is totally what reflection is > all about. > > Not to mention I don't think it is fair that you insult the > intelligence of someone who submitted an RFE. As a general facility to be added to ClassLoader, what you are asking for is impossible. The java.net.URLClassLoader can be given a URL which refers to a directory accessed via http. You can't usually enumerate the content of such a directory. Therefore you can't find out what classes are present at that location, yet if you know the name of one you can load it. There are two approaches to this issue which do work. For IDEs, you just insist that any class sources which are to be enumerated are located in places such as jar files or file system directories which can be searched. For general applications, you create files listing all the available implementations of an interface and place these files as resources in the jar files that you create. The standard mechanism for doing this is (as of Java 6) java.util.ServiceLoader. Mark Thornton ... > I think anyone who is interested in having this issue solved and a > massively useful set of features made available to them in Java, they [quoted text clipped - 4 lines] > It isn't far from getting enough votes for recognition, and I think > this functionality is well worth having. I am strongly in favor of some solution to this problem, but doing it through package reflection seems to me to limit its usefulness. As noted in the response, it would only be able to find already loaded classes. The problem is that what you really want is not what has been loaded, but what could be loaded. Some of the solutions other people have suggested duplicated the directory and jar examination capabilities of the system class loader, to see what it could load. I don't like that sort of duplication on general software engineering principles. Here is an alternative suggestion: Add to ClassLoader a method: public String[] getClassNames(Package package) throws UnsupportedOperationException If the method makes sense in a particular ClassLoader, it would search as though it were looking for a class in the specified package, but list all class names it has loaded or could load as classes in package. The method would only find classes that have been loaded or could be loaded by that particular class loader. It would not find classes that exist on some disk drive, nominally in the package, but not loadable by that class loader. It would not find classes that get copied into some directory and become loadable after the call. Some class loaders might not be able to implement this method, and would throw UnsupportedOperationException. For example, an over-the-network class loader might need to specify a complete class name, and not have any interface to ask what could have been loaded. In the simple case, with the system class loader, this would be equivalent to searching the directories and jars on the class path, but without the code duplication, and therefore automatically robust under any changes in how the system class loader finds classes. Patricia > .... >> I think anyone who is interested in having this issue solved and a [quoted text clipped - 44 lines] > without the code duplication, and therefore automatically robust under > any changes in how the system class loader finds classes. 1) Might as well just get all classes, perhaps with a filter interface similar to FilenameFilter. 2) Consider interaction with sealed JARs.  SignatureJohn W. Kennedy Read the remains of Shakespeare's lost play, now annotated! http://pws.prserv.net/jwkennedy/Double%20Falshood/index.html * TagZilla 0.066 * http://tagzilla.mozdev.org ... >> Some of the solutions other people have suggested duplicated the >> directory and jar examination capabilities of the system class loader, [quoted text clipped - 7 lines] >> public String[] getClassNames(Package package) throws >> UnsupportedOperationException ... >> In the simple case, with the system class loader, this would be >> equivalent to searching the directories and jars on the class path, but [quoted text clipped - 3 lines] > 1) Might as well just get all classes, perhaps with a filter interface > similar to FilenameFilter. Could you explain some more? A loadAllInPackage method seems to me to be just as hard to implement, but less flexible. A user who wants to load all the classes in the list could do so with a simple for loop. In the specific case in which I wanted this, an end-user needed to pick one class which would be used during the run. I would have liked to offer a Choice, listing the available class names, rather than a TextField. > 2) Consider interaction with sealed JARs. That seems like a good reason for having the method in the class loader. Presumably, it already has the ability to decide whether a .class file is a legitimate candidate for loading given any sealing issues. A duplicate implementation outside the class loader might easily miss that sort of issue. Patricia > > 2) Consider interaction with sealed JARs. > [quoted text clipped - 3 lines] > duplicate implementation outside the class loader might easily miss that > sort of issue. Apparently, even sealed packages can have dynamically-generated classes added to them. See the JavaDoc for java.lang.reflect.Proxy. More importantly than that quibble, if a package is sealed (using current technology anyway) then it /by definition/ has a readable list of class names somewhere in its META-INF/ directory (I forget exactly where). So this case seems to be the one where a reflection-level solution adds least value. Also, I don't see any particular reason to focus on /packages/ here -- if there is any reasonable way to approximate the OP's vision, then it also ought to be able to cope equally well (or equally badly) with similar requests such as "list all your classes which implement <some interface> regardless of package". -- chris >> 1) Might as well just get all classes, perhaps with a filter interface >> similar to FilenameFilter. > > Could you explain some more? A loadAllInPackage method seems to me to be > just as hard to implement, but less flexible. A user who wants to load > all the classes in the list could do so with a simple for loop. I think you misunderstood my "get". I meant that, rather than public String[] getClassNames(Package package) throws UnsupportedOperationException it would be wiser to define public interface ClassnameFilter { boolean accept (String name); } and public String[] getClassNames(ClassnameFilter filter) throws UnsupportedOperationException thus covering a wider range of possible uses. However, it occurs to me now that there /is/ a real gain to be found in searching by package (because of directories), so I suppose public String[] getClassNames(ClassnameFilter filter, Package package) throws UnsupportedOperationException and the original public String[] getClassNames(Package package) throws UnsupportedOperationException would also be desireable. SignatureJohn W. Kennedy "But now is a new thing which is very old-- that the rich make themselves richer and not poorer, which is the true Gospel, for the poor's sake." -- Charles Williams. "Judgement at Chelmsford" * TagZilla 0.066 * http://tagzilla.mozdev.org > However, it occurs to me now that there /is/ a real gain to be found in > searching by package (because of directories), so I suppose > > public String[] getClassNames(ClassnameFilter filter, Package package) > throws UnsupportedOperationException Might be a bit tricky if some of the classes in the package are loaded from the local filesystem, or a JAR file, while the others are loaded be doing an HTTP GET per class. (Or, of course, any of the other awkward sources for classfiles.) -- chris > Some of the solutions other people have suggested duplicated the > directory and jar examination capabilities of the system class loader, > to see what it could load. I don't like that sort of duplication on > general software engineering principles. I'm against adding anything of the kind to ClassLoader. My reason is that what classloaders do is already widely misunderstood (by people who understandably, but mistakenly, see Java through the static lens enforced by javac); and that adding features which people expect to work in one way, when they actually work in another, would be a mistake. Compare the case of InputStream.read(byte[]); people seem to get that wrong all the time (at least judging by posts here), and I guess there's a /lot/ of production code out there which only works by accident. Misleading APIs cause endless grief... Perhaps a slightly different tack would make more sense. There's a problem with the current classloader architecture in that classloaders are expected to do two unrelated things -- to load classes (and be namespaces, etc); and to find resources. If those two roles were split so that a ClassLoader /might/ have one or more ResourceFinders; and if /some/ ResourceFinders were able to iterate in some way over resources satisfying <some condition>; then the feature the OP (and others) have asked for could be provided in the cases where it makes sense, without having the class architecture suggest that it always made sense (and without the duplication that you deplore). A useful side-effect of that would be to remove the responsibility of fetching stuff from URLs from the classloader hierarchy (where it doesn't belong); and would make it simpler ('cos we're dealing with composition rather than inheritance) to split responsibility for loading resources from URLs and files into two, or more, separate kinds of ResourceFinder. That would mean that we could remove URLClassLoader altogether (backward compatibility aside) -- which, IMO, would be a good thing. Note that some of the standard utilities (javac, javap, maybe others) really need the ability to find and /read/ classfiles without /loading/ them. -- chris On Mar 22, 4:52 am, "Chris Uppal" <chris.up...@metagnostic.REMOVE- THIS.org> wrote: > > Some of the solutions other people have suggested duplicated the > > directory and jar examination capabilities of the system class loader, [quoted text clipped - 34 lines] > > -- chris You're basically denying the obvious use of packages and "Class LOADERS". I mean, this domain & scope trollery is not going to help anything. The best solution available is to allow an instance of Package to have a method: "Class[] getClasses()" or "byte[] getEntries()". This would spare you the need of bellyaching about burdening the classloader (as the Java creators can implement this however they feel needed) and it would put the code in the most obvious place: Reflection. The implementation of everything else around it can be as wild and correct as you envision. Because what I want to do is REFLECT on the contents of a PACKAGE. Whether it is a namespace or a container. There is no need for the concept of Packages to survive through to the .JAR file if they are 100% irrelevant. The only obvious purpose they serve is to logically group and segregate "items" from each other. > On Mar 22, 4:52 am, "Chris Uppal" <chris.up...@metagnostic.REMOVE- > THIS.org> wrote: [quoted text clipped - 7 lines] >> adding features which people expect to work in one way, when they actually work >> in another, would be a mistake. ... > You're basically denying the obvious use of packages and "Class > LOADERS". [quoted text clipped - 7 lines] > Reflection. The implementation of everything else around it can be as > wild and correct as you envision. The problem is that the question may be unanswerable. I can create, at any time, a class that is a member of some package, and that could be loaded by some specialized class loader that I have not, yet, instantiated in a given program run. Asking a class loader "What classes in this package do you currently know how to load?" seems to me to be a much reasonable question. Patricia > Asking a class loader "What classes in this package do you currently > know how to load?" seems to me to be a much reasonable question. Even that isn't useful. For example URL class loaders can't (usually) enumerate the content of directories so all they know about is classes that have already been loaded. There may be other classes that could be loaded, but they have no way of determining their names. Mark Thornton >> Some of the solutions other people have suggested duplicated the >> directory and jar examination capabilities of the system class loader, >> to see what it could load. I don't like that sort of duplication on >> general software engineering principles. Agreed. Writing a classloader that includes this functionality (or just exposes some of the functionality of sun.misc.URLClassPath) would be helpful. Go for it! >I'm against adding anything of the kind to ClassLoader. Agreed. It might be useful as part of a specific ClassLoader, or as a separable component that ClassLoaders can use. -- Mark Rafn dagon@dagon.net <http://www.dagon.net/> > I've been doing some research online about this subject and I've come > across a pretty even balance of "why I need to do it" and "why it [quoted text clipped - 9 lines] > despite similarities. > 2. Classes are not fetched until they are requested/needed. > IDEs like NetBeans, Eclipse and others do all kinds of wicked > reflection when you think about it. To what degree they cheat, I [quoted text clipped - 4 lines] > During run-time, how do I create one instance of each class in a > package when all I know at compile-time is the package's name? It is impossible to get a list of all classes in a package (and therefore also to instantiate them). The absurd but still relevant proof is that I can have a class on my hard drive belong to the package your are looking up. It is possible to get a list of all classes in a dir or jar file. Actually I have some code for processing a jar file if you are interested. Instantiating can become a bit more complex for those classes without zero argument constructor etc., but to some extent it is possible. Arne >> During run-time, how do I create one instance of each class in a >> package when all I know at compile-time is the package's name? As stated, you cannot do this in a general way. You might even have a classloader that generates classes ON THE FLY, so the list of possible classes is infinite. >It is possible to get a list of all classes in a dir or >jar file. Actually I have some code for processing a jar >file if you are interested. Indeed. If you use a custom classloader which limits the source of classfiles it loads, you could easily add this method to it. In the case of running from disk with a fixed local classpath, the classloader probably already has the list available, you just need to expose it. The key is recognizing this as app- or classloader-specific, and writing the code to do it rather than hoping for impossible general support from the core Java libraries. >Instantiating can become a bit more complex for those >classes without zero argument constructor etc., but >to some extent it is possible. Instantiating random classes is probably a huge error anyway. You may want to instantiate all classes which implement a certain interface and have a certain constructor, though. Imagine an extension mechanism for your app, with a special classloader that searches specified directories for jars and classfiles. It has a method to list all the ones it can find that implement MyExtensionInfo, and an easy way to instantiate them all so your app can invoke informational methods on them. -- Mark Rafn dagon@dagon.net <http://www.dagon.net/> > Instantiating random classes is probably a huge error anyway. You may want to > instantiate all classes which implement a certain interface and have a certain > constructor, though. If you read both my explanations, you would see that this is what I'm trying to do - and is more reason why I think Java should have things in place to facilitate this. >> Instantiating random classes is probably a huge error anyway. You may want >> to instantiate all classes which implement a certain interface and have >> a certain constructor, though. >If you read both my explanations, you would see that this is what I'm >trying to do Then go for it. There's nothing stopping you. >and is more reason why I think Java should have things >in place to facilitate this. I repeat: > The key is recognizing this as app- or classloader-specific, and writing the > code to do it rather than hoping for impossible general support from the core > Java libraries. Java _DOES_ have things in place to support it. You can list files, create classloaders that do special things, and use reflection to instantiate classes once you've found them. It doesn't have magic "read my mind and do the right thing" support, but that's probably a good thing, given the confused state of most developer minds :) -- Mark Rafn dagon@dagon.net <http://www.dagon.net/> .. > ...It doesn't have magic "read my mind and do the right > thing" support, .. I am still waiting on the implementation of the static method.. /** Produces the correct result @param theStuff can be any number of any object, or 'null' */ public static void doWhatIMeanNotWhatISay(Object[] theStuff) { ... ;-) Andrew T. > At that point, even the functionality of getting the names of classes > only retrieved by the classloader within the package would suffice. Instrumenting your program with own Java-agent might be another step toward computing that. See (together with its class's package documentation): <http://java.sun.com/javase/6/docs/api/java/lang/instrument/Instrumentation.html# getAllLoadedClasses()> If you know the class-loader (or loaders) classes from your package should be initiated by, possibly faster would be using getInitiatedClasses(). Another possible solution using instrumentation is to implement ClassFileTransformer -- /retransformation incapable/ is enough -- which will cache (probably weekly) the classes from your package, and then, on later request, will return them. > During run-time, how do I create one instance of each class in a > package when all I know at compile-time is the package's name? In general, you can't do that. One (not only) reason, there is no way to refer a class which not exist yet -- classes might be generated even at runtime. BTW -- It's usually preferred by me (hope, I'm not alone with that) to use, and maintain the application configured using clearly defined, not hidden /names/ (like class-names, paths, etc.), than the one with some sophisticated, coded in application, configuration lookup algorithms. It's then often enough for me to use reflection for user configurable/extendable parts of code. The solution is usually as easy as defining an interface (or abstract base class) for extra pluggable code, e.g. public interface Plugin { void hello(); } and using it as follows: Plugin pluginInstance = Class.forName(pluginClassName) .asSubclass(Plugin.class).newInstance(); pluginInstance.hello(); ... Eventually, consider using some more advanced plug-in framework, for example JPF <http://jpf.sourceforge.net/>. piotr Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.