Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / January 2006

Tip: Looking for answers? Try searching our database.

The program that would'nt Quit

Thread view: 
Roedy Green - 26 Jan 2006 02:34 GMT
I have I think discovered a strange bug in both JDK 1.5 and 1.6.

the program ends like this:

      System.out.println( "done" );
      //  System.exit( 0 );
       } // end main

Without that System.exit, the program keeps on running endlessly using
about 94% of my CPU time after printing "done".

The odd thing is this is not a gui and it has but a single thread. I
have no finalizers.

It works fine in Jet.

I don't know where to begin to track this down or how to reproduce it
for Sun. The jar is  271K and the data files it works on are 14 MB.

Any ideas on what could cause such behaviour?

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green - 26 Jan 2006 02:57 GMT
On Thu, 26 Jan 2006 02:34:42 GMT, Roedy Green
<my_email_is_posted_on_my_website@munged.invalid> wrote, quoted or
indirectly quoted someone who said :

>the program ends like this:
>
>       System.out.println( "done" );
>       //  System.exit( 0 );
>        } // end main

there in the class containing main. I don't see anything out of the
ordinary about it.

package com.mindprod.htmlmacros;
import java.io.File;
import java.io.FilenameFilter;
import java.util.ArrayList;

import com.mindprod.filter.ExtensionListFilter;

/**
* Feeds the entire Mindprod website to the Replacer macro mechanism
refreshing
* expansions of all macros. Expands HTML boilerplate refresher macros
e.g. <!--
* macro Include dogs.txt --> <!-- generated --> <img
* src="images/animals/dog.jpg" height="410" width="511" alt="dog">
<!--
* /generated by Include --> Allows arbitrary expander macro code.
Futures:
* Error messages give offset/line number, nearby text check for
generated by
* macro not just generated to remove old text. Can embed a " in a
parm by
* encoding it as &quot;
*/
public class ReplaceMindprod extends Replacer {

   /**
    * True if want debugging output, counts of files gathered from
different
    * dirs.
    */
   private static final boolean DEBUGGING = false;

   /**
    * total count of files gathered
    */
   private static int gatherCount;

   /**
    * arrayList where we build list of files to process.
    */
   static ArrayList<String> filesToProcess;

   /**
    * Estimated files will process.
    */
   public static final int ESTIMATED_TOTAL_FILES = 4500;

   /**
    * Collect all files in the given dirName for processing.
    *
    * @param dirName
    *        Name of fully qualified directory where to look for the
html
    *        files.
    * @param filter
    *        filter to use to decide while files in that dir to
process.
    */
   static void gather ( String dirName, FilenameFilter filter )
       {
       String[] filenames = new File( dirName ).list( filter );
       if ( filenames == null )
           {
           throw new IllegalArgumentException(
               "com.mindprod.htmlmacros.ReplaceMindprod: No such dir:
"
                   + dirName );
           }
       gatherCount += filenames.length;
       if ( DEBUGGING )
           {
           System.out.println( filenames.length
               + " files gathered from "
               + dirName );
           }
       for ( String filename : filenames )
           {
           filesToProcess.add( dirName + "/" + filename );
           }
       }

   /**
    * Get fully qualified name of all the files to process
    */
   public static void gatherFiles ()
       {

       // add *.html files in mindprod

       filesToProcess = new ArrayList<String>( ESTIMATED_TOTAL_FILES
);
       FilenameFilter f = new ExtensionListFilter( new String[] {
           "html"
       } );

       gather( "e:/mindprod", f );
       gather( "e:/mindprod/animalrights", f );
       gather( "e:/mindprod/applets", f );
       gather( "e:/mindprod/applications", f );
       gather( "e:/mindprod/bgloss", f );
       // bookcovers
       gather( "e:/mindprod/contact", f );
       gather( "e:/mindprod/deepthoughts", f );
       // dvdcovers
       gather( "e:/mindprod/environment", f );
       gather( "e:/mindprod/esperanto", f );
       gather( "e:/mindprod/ethics", f );
       gather( "e:/mindprod/feedback", f );
       gather( "e:/mindprod/ggloss", f );
       gather( "e:/mindprod/humanrights", f );
       // icon
       // images
       // include won't work since macros in wrong dir
       gather( "e:/mindprod/jgloss", f );
       // jgloss/include
       // kjv, rarely changes, do manually when it needs it.
       gather( "e:/mindprod/livinglove", f );
       gather( "e:/mindprod/livinglove/methods", f );
       gather( "e:/mindprod/money", f );
       // pads
       gather( "e:/mindprod/phone", f );
       gather( "e:/mindprod/politics", f );
       gather( "e:/mindprod/politics/laser", f );
       // precis
       gather( "e:/mindprod/projects", f );
       gather( "e:/mindprod/quadra", f );

       // redir, only makes sense to expand them in place. Don't
disturb copied
       // versions
       gather( "e:/mindprod/redir", f );
       gather( "e:/mindprod/religion", f );
       // renney
       // replicator
       gather( "e:/mindprod/reunion", f );
       // screenshots
       // snippets
       // sound
       gather( "e:/mindprod/utilities", f );
       gather( "e:/mindprod/webstarts", f );
       // zips

       // all files are ready to go.

       }

   /**
    * Expands macros written in Java in HTML files files.
    *
    * @param args
    *        names of files to process, no wildcards.
    */
   public static void main ( String[] args )
       {
       System.out.println( "\nExpanding macros on entire Mindprod
website" );

       Tools.setWebRoot( "e:\\mindprod" );
       gatherFiles();

       System.out.println( gatherCount
           + " files gathered for mindprod site macro expansion." );

       assert gatherCount <= ESTIMATED_TOTAL_FILES;
       for ( String fileToProcess : filesToProcess )
           {
           assert fileToProcess != null;
           assert new File( fileToProcess ).exists() : "file to
process missing";
           try
               {
               expandMacrosInFile( new File( fileToProcess ), true /*
quiet */);
               }
           catch ( Exception e )
               {
               System.err.println( e.getMessage()
                   + " in file "
                   + fileToProcess );
               System.err.println();
               e.printStackTrace();
               // we don't exit. We keep on processing files.
               }
           } // end for each file loop
       System.out.println( "done" );
       // don't remove the System.exit( 0 ). It bypasses a Sun bug.
       System.exit( 0 );
       } // end main

} // end ReplaceMindProd

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Stefan Ram - 26 Jan 2006 03:12 GMT
 You could use Thread.activeCount() to confirm that there is
 only one thread at the end.

>Tools.setWebRoot( "e:\\mindprod" ); (...)
>gatherFiles(); (...)
>for ( String fileToProcess : filesToProcess ) (...)
>expandMacrosInFile( new File( fileToProcess )

 Comment out (hide) the call of "expandMacrosInFile" and see
 whether the behavior still occurs.

   If yes, remove "gatherFiles" and see whether ...

   If no: The behavior might have been cause by
   something in "expandMacrosInFile": So now, edit
   this method, commenting out its final half or so.

 Recursively repeat this until the cause is found.

 (This approach is not always possible or might not
 always work.)
opalpa@gmail.com opalinski from opalpaweb - 26 Jan 2006 20:02 GMT
Good day Roedy,

I empathize with this situation;  I share an aversion for System.exit .
Some ideas:

There are situations where this occurs according to definition in
non-gui programs.  For example when using a JMF player one needs to
stop(), deallocate(), and close() before graceful exit.  Perhaps you
are inadvartanly and indirectly creating some event loop?
---
I know one program that exits gracefully after about ten seconds.  Have
you waited this long?  The general rule is if there is a thread that
only wakes up every X seconds then you might wait upto X seconds for it
to return from its run().
---
I believe Java spec says that JVM can choose to exit if only demon
threads areleft, but does not have to.  I doubt this is the case
because Sun's jvm always seems to take the exit choice within a couple
of seconds.
---
Perhaps you should view running threads using JConsole.  You will also
see each thread's state.  There is a negative in that you will see
threads related to JConsole listed; this negative is not that large.

Hopefully this email is of some service

All the best,
Opalinski
opalpa@gmail.com
http://www.geocities.com/opalpaweb/

P.S. Jconsole takes a total of maybe five minutes to learn.  Run your
executable with   -Dcom.sun.management.jmxremote and seperately execute
jconsole found within java's bin directory.  That's it

Here is some documenation:
http://java.sun.com/j2se/1.5.0/docs/guide/management/jconsole.html
Roedy Green - 26 Jan 2006 22:11 GMT
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>There are situations where this occurs according to definition in
>non-gui programs.  For example when using a JMF player one needs to
>stop(), deallocate(), and close() before graceful exit.  Perhaps you
>are inadvartanly and indirectly creating some event loop?

This is weird. I am doing pure batch text processing. There are no
events or listeners in the whole program. I don't spawn any threads --
any of the usual suspects in refusal to shut down.

Keep going.  I hope I have overlooked something.  
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

opalpa@gmail.com opalinski from opalpaweb - 27 Jan 2006 03:15 GMT
> This is weird. I am doing pure batch text processing.

Opening files is preceeded by SecurityManager checks.  That may involve
threads.  Stefan Ram suggests a gradual isolation which may be
reasonable.

> 6. There is a quantum mechanical effect. Watching the apps with
> JConsole makes it work!!

Maybe, a long shot, when watching you give extra time which you don't
give when not watching and something time limited gets to complete.

Opalinski
opalpa@gmail.com
http://www.geocities.com/opalpaweb/
Roedy Green - 26 Jan 2006 22:22 GMT
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>I know one program that exits gracefully after about ten seconds.  Have
>you waited this long?  The general rule is if there is a thread that
>only wakes up every X seconds then you might wait upto X seconds for it
>to return from its run().

There are no timers, execs or threads. About the only exotic feature
of this program is using Class.forName to load most of its classes.

Perhaps some sort of incomplete i/o?  I do some auxiliary disk
processing and perhaps a failure is getting lost, maybe an unclosed
file somewhere.  Unfortunately Java has no open method you can grep.
Also I don't know of a way of testing if and what files are still
open.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green - 26 Jan 2006 23:15 GMT
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>Perhaps you should view running threads using JConsole.  You will also
>see each thread's state.  There is a negative in that you will see
>threads related to JConsole listed; this negative is not that large.

I'm running it now  It turns out to be ten times easier to set up than
it first looks.  I have documented this, as is my wont, at
http://mindprod.com/jgloss/jconsole.html

Surprises for me includes:

1. my "single thread' app had 18 threads.

3. garbage collections are extremely frequent.  There over a 1000 of
them in the time it took to connect and start looking. However
JConsole said that only a few seconds were used in GC.  I had assumed
they were much less frequent and far more painful.

4. the different aged collections of objects truly do behave
differently. the older collections are rock stable. The eden has a two
frequency periodicity, like a  sound with harmonics.

5. My app uses about 15 MB of heap space by default. I had never done
much thinking about ram usage.  It is a bit of a shock. In the 1970s I
think I would have pulled off an app like this in  200K including the
run time.

6. There is a quantum mechanical effect. Watching the apps with
JConsole makes it work!!
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Monique Y. Mudama - 27 Jan 2006 00:24 GMT
> 6. There is a quantum mechanical effect. Watching the apps with
> JConsole makes it work!!

That certainly smells like a threading issue.  Every time I've used
tools to inspect the code and the code started to work, it has always
turned out to be a threading problem.

http://catb.org/~esr/jargon/html/H/heisenbug.html

Signature

monique

Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html

Roedy Green - 27 Jan 2006 02:28 GMT
On Thu, 26 Jan 2006 17:24:31 -0700, "Monique Y. Mudama"
<spam@bounceswoosh.org> wrote, quoted or indirectly quoted someone who
said :

>That certainly smells like a threading issue.  Every time I've used
>tools to inspect the code and the code started to work, it has always
>turned out to be a threading problem.
>
>http://catb.org/~esr/jargon/html/H/heisenbug.html

more weirdness. The bug has now gone altogether.  What changed?  I
added some convenience methods to one of the base classes, but I don't
yet use them anywhere.

This is like the old days of FORTRAN when making tiny changes to code
changed which code got clobbered when  subscripts when out of range.

I don't like this. I have been hit twice in a day with programs that
work fine on one compiler but not another. It is  a sign Java has
grown past the manageable bounds of complexity for a group of humans.

There is no JNI in my code.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

opalpa@gmail.com opalinski from opalpaweb - 27 Jan 2006 03:22 GMT
Lack of determinism is a problem.  Tread carefully to be certain it's
the virtual machine.  It won't be first not-exiting bug:

http://developer.java.sun.com/developer/bugParade/bugs/4515058.html
http://developer.java.sun.com/developer/bugParade/bugs/4671025.html
http://developer.java.sun.com/developer/bugParade/bugs/4465537.html

It appears they do get fixed as Sun agrees jvm should exit gracefully.

Opalinski
opalpa@gmail.com
http://www.geocities.com/opalpaweb/
Monique Y. Mudama - 27 Jan 2006 04:16 GMT
> I don't like this. I have been hit twice in a day with programs that
> work fine on one compiler but not another. It is  a sign Java has
> grown past the manageable bounds of complexity for a group of
> humans.

Now, now.  I happen to know for a fact that they've had problems like
this all along.  In my graphics class in college, we were using Java
1.1.7.  I always made sure to test my projects, which I developed on
my windows box, on the lab boxes before submitting them.  My professor
laughed at me and told me that Java works everywhere.

Well.  I was running late for the final project of the semester, and I
was running into some weird environment problem, so I turned in the
project, a wireframe activated by keypresses, which worked flawlessly
on my Windows box.  I got a zero on the project because the keypress
code didn't work the same between Windows and Solaris.  I got a C for
the course instead of an A.

I know @!#! well that Java has *never* worked perfectly across all
systems.

Signature

monique

Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html

abmccullough@gmail.com - 26 Jan 2006 16:02 GMT
Seems like taking a javacore after the "done" message is printed would
a)
tell you what thread is active and what it is doing and b)
provide the information Sun would need to fix it, if it is truly a bug.

-Andrew
Roedy Green - 26 Jan 2006 23:16 GMT
>taking a javacore

What class/method is that?
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

abmccullough@gmail.com - 27 Jan 2006 14:31 GMT
It's not an API, it's a "dump" of the JVM state that you can use to
debug things.  It is automatically created in certain error conditions
(like a bad signal from JNI code), but you can request one to debug a
problem.  On Windows, type Ctrl-BREAK in the command window in which
the Java.exe is active, on *NIX, issue a kill -3 to the java.exe
process.  In windows, it will dump to standard out (the console window
-- usually, you cannot do this in an IDE) a list of all threads, the
currently active thread, held locks and monitors, and a variety of
other highly useful information.  On *NIX it will write the same
information to a file named javacoreTIMESTAMP.txt in the default
directory.  I use it all the time to determine why things don't stop
and other problems.  Some web searching should help you find some
pointers on interpreting the contents, but the thread dump is pretty
straightforward -- it lists all the threads and their current stack.

-Andrew
Roedy Green - 27 Jan 2006 14:45 GMT
>  On Windows, type Ctrl-BREAK in the command window in which
>the Java.exe is active,

All I have even seen is a abort.  does it hide this dump somewhere
without telling you?
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

abmccullough@gmail.com - 27 Jan 2006 17:50 GMT
Hmm.. not sure what you mean, but it works for me, I've been doing it
for years.  You can read a bit more at:
http://java.sun.com/developer/technicalArticles/Programming/Stacktrace/index.html

What does "all I have seen is an abort" mean, exactly?  

-Andrew
Thomas Hawtin - 26 Jan 2006 19:47 GMT
> Without that System.exit, the program keeps on running endlessly using
> about 94% of my CPU time after printing "done".
[quoted text clipped - 8 lines]
>
> Any ideas on what could cause such behaviour?

Have you tried using jstack (or similar) to see what is continuing to run?

Tom Hawtin
Signature

Unemployed English Java programmer
http://jroller.com/page/tackline/

Roedy Green - 26 Jan 2006 23:17 GMT
On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
<usenet@tackline.plus.com> wrote, quoted or indirectly quoted someone
who said :

>Have you tried using jstack
I had never heard of it till now.  Will try that.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green - 26 Jan 2006 23:33 GMT
On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
<usenet@tackline.plus.com> wrote, quoted or indirectly quoted someone
who said :

>Have you tried using jstack (or similar) to see what is continuing to run?
Jstack it turns out does not run on windows.  
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Thomas Hawtin - 27 Jan 2006 00:17 GMT
> On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
> <usenet@tackline.plus.com> wrote, quoted or indirectly quoted someone
> who said :
>
>> Have you tried using jstack (or similar) to see what is continuing to run?
> Jstack it turns out does not run on windows.  

jstack, by default, does much the same as ctrl-\ (which I think is
ctrl-break on Windows) only with less fiddling. There are some options,
like -m which causes it to NPE lots.

Tom Hawtin
Signature

Unemployed English Java programmer
http://jroller.com/page/tackline/



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.