Java Forum / General / August 2007
Parallel processing using Executor?
howa - 14 Aug 2007 20:15 GMT Hello,
I have a method, e.g. foo(int i), which take an integer i and do some heavy processing
now, i want to find the summation of foo() of i = 1..10000, I want to take the advantage of speedup by multithreads, then I use executors to create the threads...,e.g.
public class Foobar implements Runnable {
public Foobar() {}
public void run() { System.out.println("thread is running..."); }
public int foo(int i) { //... }
}
Executor tp = Executors.newFixedThreadPool(10);
for (int i =1; i <=10000; i++) { tp.execute(new Foobar() ); }
but how can i get back the information return from the execute method, i.e. from the foo(int i)?
Thanks...
Daniel Pitts - 14 Aug 2007 20:29 GMT > Hello, > [quoted text clipped - 30 lines] > > Thanks... Try this approach:
public class Foobar extends Callable<Integer> { final int param; public Foobar(int param) { this.param = param; }
public Integer call() { return foo(param); }
public int foo(int i) { return i * 2; } } public class Main { public static void main(String..args) throws Exception { Executor tp = Executors.newFixedThreadPool(10); List<Future<Integer>> futureObjects = new ArrayList<Future<Integer>>(); for (int i =1; i <=10000; i++) { futureObjects.add(tp.execute(new Foobar(i))); } for (Future<Integer> result: futureObjects) { System.out.println(result.get()); } }
Manish Pandit - 14 Aug 2007 20:31 GMT > Hello, > [quoted text clipped - 30 lines] > > Thanks... You'd need a synchronized method that can keep the sum, and then pass that as a callback to the thread.
Also, you'd need to wait till all threads are done executing - which is why I do not think threading is a good solution for this case.
For example:
class Calculator{
private int total;
public synchronized void add(int result){
total+=result;
}
public void spawnThreads(){
for(int i=0;i<10000;i++){ //spawn the threads, passing in this Calculator as a reference //have the thread's run() call add(result) once it has calculated the value //join() on the thread <--- this is why threading will not help here! }
}
In other words, since the sum is dependent on complete execution of all the theads taking part in the computation, evaluating the sum is in a way a blocking call (and hence the join() is needed).
-cheers, Manish
howa - 15 Aug 2007 03:32 GMT Hello,
I am not sure why you said threading can't help much in my case, since the foo() is a cpu intensive method.
I want to have the sum, I don't care the order hwo the sum was calculated.
If we have multi-core machine, we can speed up the process by many times, isn't?
Thanks...
> Also, you'd need to wait till all threads are done executing - which > is why I do not think threading is a good solution for this case. Patricia Shanahan - 15 Aug 2007 05:00 GMT > Hello, > [quoted text clipped - 6 lines] > If we have multi-core machine, we can speed up the process by many > times, isn't? Whether distributing the work to threads helps or not depends on how long foo() takes. It is the sort of thing that would be easier to measure than to estimate.
Patricia
Zig - 15 Aug 2007 05:52 GMT >> Hello, >> I am not sure why you said threading can't help much in my case, since [quoted text clipped - 9 lines] > > Patricia Well, the usual strategy for cpu intensive methods is not to queue up 10000 events, but to partition your events per core. Eg, rather than summing 1..10000, create a couple of java.util.concurrent.Callable(s) to sum 1..5000 and 5001..10000 (or break it up as many ways as you have available cores), and then just sum those results when the threads complete. Of course, if the foo() method were instead IO bound rather than processor bound, then you will want many more threads than cores to minimize CPU downtime.
As another poster commented, the appropriate interface to implement would be java.util.concurrent.Callable (rather than java.lang.Runnable), which will allow you to get a result back from the Executor.
HTH,
-Zig
Patricia Shanahan - 15 Aug 2007 06:13 GMT >>> Hello, >>> I am not sure why you said threading can't help much in my case, since [quoted text clipped - 16 lines] > available cores), and then just sum those results when the threads > complete. ... I would agree with that strategy if, and only if, the time to do the integer adds were the main issue. However, the focus seems to be on foo(), with it described as "cpu intensive".
Of course, depending on how long foo() takes there may be situations in which, after foo() is being done in parallel, the adds do become significant. In that case, it would make sense to parallelize them as well.
Patricia
howa - 15 Aug 2007 07:07 GMT yes, the foo() method is cpu intensive as it contains another big loop to calculate the result but can't be run in parallel, e.g.
foo(int i) { int j = 1; for (int i=0;i<10000000;i++) { // do some calculations on j, must be in the order } return j; }
most of the cpu time will spend on foo() method.
as said by Patricia, partition into sum 1..5000 and 5001..10000 do works... but i believe ths reason of using Executors.newFixedThreadPool(10) is that Java can handle this for me already?
> I would agree with that strategy if, and only if, the time to do the > integer adds were the main issue. However, the focus seems to be on > foo(), with it described as "cpu intensive". blmblm@myrealbox.com - 15 Aug 2007 11:37 GMT > yes, the foo() method is cpu intensive as it contains another big loop > to calculate the result but can't be run in parallel, e.g. [quoted text clipped - 13 lines] > but i believe ths reason of using Executors.newFixedThreadPool(10) is > that Java can handle this for me already? Yes, though there might be some question of whether whatever overhead is added by using a thread pool is acceptable (which might not be much, but which I'd think is nonzero?). As Patricia also says, there might also be some question of whether adding the results sequentially is fast enough, or whether this operation also should be multithreaded (perhaps by having each thread compute a local sum and then combining the results when all work is done).
Some of these questions (how much overhead, should the summing up be parallelized too) might be better addressed by trying something and measuring the results.
> > I would agree with that strategy if, and only if, the time to do the > > integer adds were the main issue. However, the focus seems to be on > > foo(), with it described as "cpu intensive".
 Signature B. L. Massingill ObDisclaimer: I don't speak for my employers; they return the favor.
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|