As a hypothetical question (not something I am working on, just something I
was thinking about last night), suppose you're implementing a simple
threaded server, for example a chat or game server. The server creates a
new thread for every user/client that connects to it. This thread then
handles all the incoming messages, and sends them to all other clients (for
example using a multicast address).
Now my question is, how many users - and hence how many threads - would you
allow at the same time? Do you select a limit based on the platform (eg
available memory, bandwidth), or are there other things to consider? Or,
would you not impose a limit at all, trusting that the system has enough
resources for your number of users (sounds like a bad plan IMO).
Looking forward to your replies,
zero
iamfractal@hotmail.com - 11 Nov 2005 12:59 GMT
zero skrev:
> The server creates a
> new thread for every user/client that connects to it.
> Looking forward to your replies,
>
> zero
(Presuming you have just one processor, or are at least talking about
threads-per-processor.)
Threads are not fundamentally concerned with muti-user-ness. That
threads can help to serve multiple users is a derivative. Threads are
fundamentally a blocking-management mechanism.
So ignore how many users you have for a start, and think about which
processes will block, for example:
A) Receiving a request (and perhaps putting it into a queue).
B) On-chip processing of the request.
C) Off-chip processing of the request (e.g., database access, perhaps).
D) Sending response.
Straight off the top: you should only have four threads, and I the
limit will probably be the size of the input-queue, and/or a reasonable
respons-time.
.ed
--
www.EdmundKirwan.com - Home of The Fractal Class Composition.
Ben_ - 11 Nov 2005 13:06 GMT
A pattern "1 user = 1 thread" does not scale very well.
One is that each thread has a heap space reserved, and an application can
run out of memory when there are too many threads.
See for example a discussion I had here in the past:
http://groups.google.com/group/comp.lang.java.machine/browse_thread/thread/bd8c8
229ce19ffd7
You can also Google for terms like too many threads, out of threads,
threads scalability, etc.
You'll see that a thousand thread is considered very high already.
That's why J2EE Containers for example implement pool of threads.
Basically, the idea is that a user does not act so rapidly that a thread
needs to be reserved all the time to him. So, you can reuse the thread to
work for another user.
HTH.
Walter Mitty - 11 Nov 2005 15:10 GMT
> A pattern "1 user = 1 thread" does not scale very well.
>
[quoted text clipped - 16 lines]
>
> HTH.
Ditto for handles to a database connection.
Babu Kalakrishnan - 11 Nov 2005 17:56 GMT
> A pattern "1 user = 1 thread" does not scale very well.
>
> One is that each thread has a heap space reserved, and an application can
> run out of memory when there are too many threads.
Agree with your statement on line 1, but a correction to the first part
of the next line.
Each thread DOES NOT have a separate heap space reserved for it - it is
actually stack space that is reserved as private to each thread. The
heap is shared by all threads.
BK
Ben_ - 11 Nov 2005 20:55 GMT
Agreed. Thanks for correcting. :-)
Daniel Dyer - 11 Nov 2005 13:19 GMT
> As a hypothetical question (not something I am working on, just
> something I
[quoted text clipped - 13 lines]
>
> Looking forward to your replies,
This is quite a good article about implementing multi-threaded servers.
The java.util.concurrent package in Java 5 originates from the the author
and concepts of this article.
http://www.awprofessional.com/articles/article.asp?p=167821&rl=1
The NIO packages introduced in Java 1.4 are also worth looking at,
particularly the concepts of channels and selectors. Combined with the
ThreadPoolExecutor in java.util.concurrent, this will allow you to write a
scalable server using a minimal number of threads.
Given enough RAM (for all the stacks), a clever enough JVM and the right
OS, you can have a huge number of Java threads (as many as 10,000), even
on a single-processor server. This does not necessarily correspond to a
similar number of native threads as the JVM may map several Java threads
onto fewer native threads. Whatever approach you take, you are mapping
multiple concurrent tasks onto a few (probably only one) physical
processors. This can be done at the OS level (JVM just maps Java threads
one-to-one with native threads and leaves the OS to do the context
switching), the JVM level (such as the JRockit thin threads
implementation) or the application level (you write Java code to share a
few Java threads between many clients).
You'll probably have to experiment a bit since it depends on the patterns
of behaviour of the clients (how much data they are sending/receiving and
how often, what kind of latency you require etc.).
Dan.

Signature
Daniel Dyer
http://www.dandyer.co.uk
Daniel Dyer - 11 Nov 2005 13:36 GMT
> Given enough RAM (for all the stacks), a clever enough JVM and the right
> OS, you can have a huge number of Java threads (as many as 10,000), even
> on a single-processor server. This does not necessarily correspond to a
> similar number of native threads as the JVM may map several Java threads
> onto fewer native threads.
Of course, I should add that just because you can do it, doesn't mean you
should. This approach (thousands of threads per processor) only really
works when the clients are reasonably idle for most of the time and is
heavily dependent on the cost of context-switching with your OS/JVM
combination.
Dan.

Signature
Daniel Dyer
http://www.dandyer.co.uk