Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / August 2007

Tip: Looking for answers? Try searching our database.

Parallel processing using Executor?

Thread view: 
howa - 14 Aug 2007 20:15 GMT
Hello,

I have a method, e.g. foo(int i), which take an integer i and do some
heavy processing

now, i want to find the summation of foo() of i = 1..10000, I want to
take the advantage of speedup by multithreads, then I use executors to
create the threads...,e.g.

public  class Foobar implements Runnable {

    public Foobar() {}

    public void run() {
        System.out.println("thread is running...");
    }

      public int foo(int i) {
             //...
     }

}

Executor tp = Executors.newFixedThreadPool(10);

for (int i =1; i <=10000; i++) {
    tp.execute(new Foobar() );
}

but how can i get back the information return from the execute method,
i.e. from the foo(int i)?

Thanks...
Daniel Pitts - 14 Aug 2007 20:29 GMT
> Hello,
>
[quoted text clipped - 30 lines]
>
> Thanks...

Try this approach:

public class Foobar extends Callable<Integer> {
  final int param;
  public Foobar(int param) {
     this.param = param;
  }

  public Integer call() {
     return foo(param);
  }

   public int foo(int i) {
     return i * 2;
   }
}
public class Main {
 public static void main(String..args) throws Exception {
   Executor tp = Executors.newFixedThreadPool(10);
   List<Future<Integer>> futureObjects = new
ArrayList<Future<Integer>>();
   for (int i =1; i <=10000; i++) {
     futureObjects.add(tp.execute(new Foobar(i)));
   }
   for (Future<Integer> result: futureObjects) {
     System.out.println(result.get());
   }
}
Manish Pandit - 14 Aug 2007 20:31 GMT
> Hello,
>
[quoted text clipped - 30 lines]
>
> Thanks...

You'd need a synchronized method that can keep the sum, and then pass
that as a callback to the thread.

Also, you'd need to wait till all threads are done executing - which
is why I do not think threading is a good solution for this case.

For example:

class Calculator{

  private int total;

  public synchronized void add(int result){

   total+=result;

  }

  public void spawnThreads(){

     for(int i=0;i<10000;i++){
        //spawn the threads, passing in this Calculator as a
reference
        //have the thread's run() call add(result) once it has
calculated the value
        //join() on the thread <--- this is why threading will not
help here!
     }

  }

In other words, since the sum is dependent on complete execution of
all the theads taking part in the computation, evaluating the sum is
in a way a blocking call (and hence the join() is needed).

-cheers,
Manish
howa - 15 Aug 2007 03:32 GMT
Hello,

I am not sure why you said threading can't help much in my case, since
the foo() is a cpu intensive method.

I want to have the sum, I don't care the order hwo the sum was
calculated.

If we have multi-core machine, we can speed up the process by many
times, isn't?

Thanks...

> Also, you'd need to wait till all threads are done executing - which
> is why I do not think threading is a good solution for this case.
Patricia Shanahan - 15 Aug 2007 05:00 GMT
> Hello,
>
[quoted text clipped - 6 lines]
> If we have multi-core machine, we can speed up the process by many
> times, isn't?

Whether distributing the work to threads helps or not depends on how
long foo() takes. It is the sort of thing that would be easier to
measure than to estimate.

Patricia
Zig - 15 Aug 2007 05:52 GMT
>> Hello,
>>  I am not sure why you said threading can't help much in my case, since
[quoted text clipped - 9 lines]
>
> Patricia

Well, the usual strategy for cpu intensive methods is not to queue up  
10000 events, but to partition your events per core. Eg, rather than  
summing 1..10000, create a couple of java.util.concurrent.Callable(s) to  
sum 1..5000 and 5001..10000 (or break it up as many ways as you have  
available cores), and then just sum those results when the threads  
complete. Of course, if the foo() method were instead IO bound rather than  
processor bound, then you will want many more threads than cores to  
minimize CPU downtime.

As another poster commented, the appropriate interface to implement would  
be java.util.concurrent.Callable (rather than java.lang.Runnable), which  
will allow you to get a result back from the Executor.

HTH,

-Zig
Patricia Shanahan - 15 Aug 2007 06:13 GMT
>>> Hello,
>>>  I am not sure why you said threading can't help much in my case, since
[quoted text clipped - 16 lines]
> available cores), and then just sum those results when the threads
> complete. ...

I would agree with that strategy if, and only if, the time to do the
integer adds were the main issue. However, the focus seems to be on
foo(), with it described as "cpu intensive".

Of course, depending on how long foo() takes there may be situations in
which, after foo() is being done in parallel, the adds do become
significant. In that case, it would make sense to parallelize them as well.

Patricia
howa - 15 Aug 2007 07:07 GMT
yes, the foo() method is cpu intensive as it contains another big loop
to calculate the result but can't be run in parallel, e.g.

foo(int i) {
  int j = 1;
  for (int i=0;i<10000000;i++) {
     // do some calculations on j, must be in the order
  }
  return j;
}

most of the cpu time will spend on foo() method.

as said by Patricia, partition into sum 1..5000 and 5001..10000 do
works...
but i believe ths reason of using Executors.newFixedThreadPool(10) is
that Java can handle this for me already?

> I would agree with that strategy if, and only if, the time to do the
> integer adds were the main issue. However, the focus seems to be on
> foo(), with it described as "cpu intensive".
blmblm@myrealbox.com - 15 Aug 2007 11:37 GMT
> yes, the foo() method is cpu intensive as it contains another big loop
> to calculate the result but can't be run in parallel, e.g.
[quoted text clipped - 13 lines]
> but i believe ths reason of using Executors.newFixedThreadPool(10) is
> that Java can handle this for me already?

Yes, though there might be some question of whether whatever
overhead is added by using a thread pool is acceptable
(which might not be much, but which I'd think is nonzero?).
As Patricia also says, there might also be some question of
whether adding the results sequentially is fast enough, or
whether this operation also should be multithreaded (perhaps
by having each thread compute a local sum and then combining
the results when all work is done).

Some of these questions (how much overhead, should the summing up
be parallelized too) might be better addressed by trying something
and measuring the results.

> > I would agree with that strategy if, and only if, the time to do the
> > integer adds were the main issue. However, the focus seems to be on
> > foo(), with it described as "cpu intensive".

Signature

B. L. Massingill
ObDisclaimer:  I don't speak for my employers; they return the favor.



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.