Java networking, 50000 sockets

Thread view:

Chris Miller - 21 Apr 2005 00:07 GMT

Has anyone tried to monitor 50000 socket connections in java?
I have an application that needs to do just that.

I can open 50000 sockets with no problems, but things get unstable when
I try to listen to 5000 to 6000 of them.

My configuration is a combination of 50 threads to listen to 1 socket
each, 500 threads pooled and waiting for the ServerSocket.accept(), and
2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
up to 6000 sockets. I am pretty sure there no deadlocks, or anything
strange.

Has anyone done something similar? Is this idea even sane?

Thanks.

Reply to this Message

George Neuner - 28 Apr 2005 04:46 GMT

>Has anyone tried to monitor 50000 socket connections in java?
>I have an application that needs to do just that.
[quoted text clipped - 11 lines]
>
>Thanks.

Well ...

50000 sockets (real sockets as opposed to local bypasses) would
require something like 1.5 GB of I/O buffer with the default settings.
I'm suprised you can even open that many.

I don't know what you're trying to do, but whatever it is, you should
probably rethink the design to use *many* fewer sockets. Devote some
of the threads exclusively to managing communication and have them
relay data to and from processing threads.

George

Signature

for email reply remove "/" from address

Reply to this Message

millerch - 28 Apr 2005 08:28 GMT

The 50,000 sockets are the primary requirement, so thats not really
negotiable, fortunately I can split this across 2 or 3 boxes.

Opening the sockets themselves are not the problem, w2k (client edition)
can't do it, but linux certainly can. The problem is reading/writing to
all the sockets in a reasonable time.

So far I am up to 13,000 sockets on one box in two threads using nio.

>>Has anyone tried to monitor 50000 socket connections in java?
>>I have an application that needs to do just that.
[quoted text clipped - 24 lines]
>
> George

Reply to this Message

George Neuner - 01 May 2005 23:04 GMT

>The 50,000 sockets are the primary requirement, so thats not really
>negotiable, fortunately I can split this across 2 or 3 boxes.

I would bet the actual spec is 50000 *connections* rather than 50000
sockets. You have to decide whether they are the same.

>The problem is reading/writing to all the sockets in a reasonable time.

Performance is another reason for using fewer sockets. Depending on
the hardware, drivers, TCP and JVM implementation, each socket
operation can result in several context switches and require multiple
copies be made of the user data. It may make a big difference if you
can access the drivers directly using JNI or using an optimized
library rather than going through whatever your JVM supplies..

George

Signature

for email reply remove "/" from address

Reply to this Message

Java networking, 50000 sockets

Free Magazines