 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / January 2006multiple concurrent processes
I need to create multiple concurrent processes. Is there a way to avoid needing 2 threads per process to handle the output and error streams of the processes? I'm planning to use a kludge at this point, to assume that I will invoke processes using either cmd.exe or /bin/sh depending on the OS (assuming it's either unix or windows), so I can use the 2>&1 operator to merge the 2 streams. Is there a better solution? For example, can I alternate reading the 2 streams of the process somehow? > I need to create multiple concurrent processes. Is there a way to avoid > needing 2 threads per process to handle the output and error streams of the [quoted text clipped - 7 lines] > Is there a better solution? For example, can I alternate reading the 2 > streams of the process somehow? You should be able to alternate the InputStreams by checking the available method, but thats pretty kludgy to (you end up with a busy-wait loop). I think using threads is bettern than that. It would be nice if you could use NIO selectors on the output and error streams, but I haven't found a way to do that. I've always ended up launching at least 1 extra thread to handle the other stream. Your current thread can handle 1 of the streams. >>I need to create multiple concurrent processes. Is there a way to avoid >>needing 2 threads per process to handle the output and error streams of the [quoted text clipped - 17 lines] > I've always ended up launching at least 1 extra thread to handle the > other stream. Your current thread can handle 1 of the streams. Why do you 'need' multiple processes (threads)? Its often a design mistake to do so, most high throughput applications have very few threads. They tend to favour an event driven design rather than a multi-threaded design. (think of Swing, its single threaded and yet has to be very responsive) >>> I need to create multiple concurrent processes. Is there a way to avoid >>> needing 2 threads per process to handle the output and error streams [quoted text clipped - 21 lines] > > Why do you 'need' multiple processes (threads)? In the general case, you need multiple threads because an external Process' stdout and stderr must both be read in order to prevent the process blocking. In some cases, its stdin may need to be fed concurrently as well. There is no way to perform a non-blocking read on a general Java InputStream, nor a non-blocking write on a general OutputStream, so each stream that you need to worry about must be handled in a separate thread. I thought all this was fairly well-known to experienced Java hands. Do you have an idea that would challenge the conventional wisdom? If so then I would be delighted to hear it. > Its often a design mistake to do so, most high throughput applications > have very few threads. They tend to favour an event driven design rather > than a multi-threaded design. > > (think of Swing, its single threaded and yet has to be very responsive) I agree that minimizing the number of threads is a good plan. In the case of a specific external process, you might be able to use knowledge about the process to reduce the required number of threads from three to two, or even to one. For the general case, however, I have yet to hear how you can safely get by with fewer than three.  SignatureJohn Bollinger jobollin@indiana.edu >>>> I need to create multiple concurrent processes. Is there a way to avoid >>>> needing 2 threads per process to handle the output and error streams [quoted text clipped - 30 lines] > OutputStream, so each stream that you need to worry about must be > handled in a separate thread. Sure, the reading of might these need different threads, but the handling of the data that comes across them need not. The data can usually be converted into discrete Events that are used within the application's main thread As far as the OPs application though, we don't know enough about it to say whether an event driven design rather than a multi-threaded design is better or not. > I thought all this was fairly well-known to experienced Java hands. Hopefully it is ;-) > Do you have an idea that would challenge the conventional wisdom? If so > then I would be delighted to hear it. God no - I wasn't trying to claim to have that - merely asking the OP whether they actually need multiple threads or not. >> Its often a design mistake to do so, most high throughput applications >> have very few threads. They tend to favour an event driven design [quoted text clipped - 3 lines] > > I agree that minimizing the number of threads is a good plan. Yes this is the important point, which can easily be missed by developers new to threads (I'm not implying anything about the OP - a general statement is all). I know from personal experience and watching other developers that we tend to start out avoiding threads, then we over use them, then we learn to use them wisely. > In the case of a specific external process, you might be able to use knowledge > about the process to reduce the required number of threads from three to > two, or even to one. For the general case, however, I have yet to hear > how you can safely get by with fewer than three. Agreed, as I say, my question for the op was merely one about whether they had need for multiple threads or whether a simpler threading model could be an alternative. Andrew >> I need to create multiple concurrent processes. Is there a way to avoid >> needing 2 threads per process to handle the output and error streams of the [quoted text clipped - 11 lines] > available method, but thats pretty kludgy to (you end up with a > busy-wait loop). I think using threads is bettern than that. If I use available, how do I tell when the end of the stream has been reached? For the person asking why I need to have concurrent processes, it's a requirement. The processes can last from seconds to an entire day. They need to happen concurrently. There is one event to handle: run all the processes. > >> I need to create multiple concurrent processes. Is there a way to avoid > >> needing 2 threads per process to handle the output and error streams of the [quoted text clipped - 14 lines] > If I use available, how do I tell when the end of the stream has been > reached? I guess you can't tell. I thought it would return -1 or something, but it doesn't. > For the person asking why I need to have concurrent processes, it's a > requirement. The processes can last from seconds to an entire day. They need > to happen concurrently. There is one event to handle: run all the processes. >>>> I need to create multiple concurrent processes. Is there a way to >>>> avoid needing 2 threads per process to handle the output and error [quoted text clipped - 17 lines] > I guess you can't tell. I thought it would return -1 or something, > but it doesn't. InputStream.read(new byte[0],0,0) should return -1 on EOF - but it may block. IMHO it's too complicated to create an algorithm that makes assumptions when the stream might be at EOF. In this case a solution is to check process status by invoking exitStatus() and catching the exception like public static boolean running(Process p) { try { p.exitStatus(); return false; } catch ( IllegalStateException e ) { return true; } } http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Process.html#exitValue() Thread overhead isn't really that high so IMHO it's best to just use threads. At least this is what I'd trie *before* I start optimizing threads away. Kind regards robert Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.