Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / April 2006

Tip: Looking for answers? Try searching our database.

how to improve TCP thoughput in Java app

Thread view: 
Shin - 04 Apr 2006 23:47 GMT
Hi there,

 Could anyone share their stories of improving TCP thoughput in Java
apps?  Suppose in my app, small sized message has to be communicated
between different jvms.  Things like, how to reduce the java
input/output stream buffer time, the underlying system buffering time?
In non-Java domain, telnet is a great example showing great response
time for sending/recving small packets.
 Thanks for any tips,

-Shin
noone - 04 Apr 2006 23:55 GMT
> Hi there,
>
[quoted text clipped - 7 lines]
>
> -Shin

When I was learning JAVA a few years ago I wrote a TCP/UDP port sniffer
GUI.  Under windoze the performance was terrible.  Under linux it was
almost as fast as a native C/C++ application.  I really don't have any raw
numbers to show the overhead of IP within the JVM.  About all I can think
of to (maybe) improve your performance (other than good overall program
design) will be to look at JNI but that kind of defeats the purpose of
playing in the sandbox.
Remon van Vliet - 05 Apr 2006 01:44 GMT
>> Hi there,
>>
[quoted text clipped - 15 lines]
> design) will be to look at JNI but that kind of defeats the purpose of
> playing in the sandbox.

I have yet to find a case where bad throughput performance was related to
anything other than errors made by the java developer in question. I have
extensive experience with both old-style blocking IO and non-blocking IO in
java and have not found any OS related throughput issues. It's usually a
case of people forgetting to flush their streams are having a blocking read
that attempts to read more data than the message to be read is long, etc.

Remon van Vliet.
Shin - 05 Apr 2006 20:34 GMT
> When I was learning JAVA a few years ago I wrote a TCP/UDP port sniffer
> GUI.  Under windoze the performance was terrible.  Under linux it was
[quoted text clipped - 3 lines]
> design) will be to look at JNI but that kind of defeats the purpose of
> playing in the sandbox.

Funny I observed the reverse:  for one program I was running, it took
about 10 minutes on a linux box, but 30 seconds on windows.
I agree I might have to turn to NIO at the end.

-Shin
Chris Uppal - 05 Apr 2006 11:48 GMT
>   Could anyone share their stories of improving TCP thoughput in Java
> apps?  Suppose in my app, small sized message has to be communicated
> between different jvms.

Are you sure you mean throughput ?  It sounds to me more as if latency would be
the big issue in this case.

Anyway, for throughput I think the biggest influence is whether you have
carefully considered how and where you use buffering (and other strategies for
minimising the number of small read()s or write()s).

For latency the picture is more complicated.  Some issues:
Do you have to open a new connection for each message ?
If not then how long to you hold connections "idle" before dropping them ?
Is the delay caused by "Nagle's Algorithm" (see Google) a factor ?
How far apart (in network terms) are the servers ?

   -- chris
Remon van Vliet - 05 Apr 2006 18:02 GMT
>>   Could anyone share their stories of improving TCP thoughput in Java
>> apps?  Suppose in my app, small sized message has to be communicated
[quoted text clipped - 16 lines]
>
>    -- chris

These cant be the issues if his problems only occur on Windows and not Linux
etc...sounds like a slight difference in JVM implementations on either OS,
but the actual problem is probably just bad code.
Shin - 05 Apr 2006 20:53 GMT
> "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> wrote in message
>
> These cant be the issues if his problems only occur on Windows and not Linux
> etc...sounds like a slight difference in JVM implementations on either OS,
> but the actual problem is probably just bad code.

First a correction, I really meant reducing latency for each message
exchange.

It could be bad code, indeed.  But is there any experiment on how fast
you can exchange messages between two jvms?  Let me define it more
precisely.  Suppose two dummy java program say "how do you do?" to each
other: 1) through a single Socket connection? 2) a new Socket
connection every time? 3) getting a different Socket from a pool each
time?
How many roundtrips can be performed in say 10 second?  Do whatever you
can to accelerate the proces?

-Shin
Remon van Vliet - 05 Apr 2006 21:50 GMT
>> "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> wrote in message
>>
[quoted text clipped - 17 lines]
>
> -Shin

Well, first of all, we can dismiss the option of establishing a new
connection for each message. Obviously this only adds overhead (and will
probably lower your messages per second figure by about a factor 100). A
pool pattern is typically used when a certain resource is limited and/or
takes a long time to set up (and can be reused). In this case, it doesnt
give you anything a simple persistent connection wouldnt give you.

I use JVM-JVM socket communication for a load balancing solution and i can
typically send the data faster than that i can do something useful with it.
The number of messages you can send should probably be approaching the
practical_bandwidth/message_size number. Latency for such tests is typically
0ms (meaning the granularity of the System.currentTimeMillis() is higher
than the time it takes a roundtrip)....so...fast

Remon van Vliet
Chris Uppal - 06 Apr 2006 13:43 GMT
> [...] is there any experiment on how fast
> you can exchange messages between two jvms?

Easy enough to write a quick test program.  I've appended one which, between
two consumer-grade Windows boxes, connected by 100 MBit Ethernet via two simple
consumer-grade switching hubs, passes about 11K 15-byte messages back and forth
per second.

> Let me define it more
> precisely.  Suppose two dummy java program say "how do you do?" to each
> other: 1) through a single Socket connection? 2) a new Socket
> connection every time? 3) getting a different Socket from a pool each
> time?

Don't even consider opening/closing a TCP connection for each short message
unless you only send a messages at infrequent intervals.

I'm not sure what you mean by pooling -- you can't share a Socket between
difference connections.  If you just mean that you keep a list of recently used
connections, and re-use them instead of opening a new connection, and return
them the pool instead of closing them, then the performance should not differ
greatly from using a dedicated socket.  The code will be more complicated, but
the network can't see that, so it shouldn't care ;-)

   -- chris

=========== code ===========
import java.io.*;
import java.net.*;

public abstract class Test
{
   public static final int PORT_NUMBER = 3344;
   public static final int MESSAGE_SIZE = 15;
   public static final int MESSAGE_PER_UPDATE = 50000;
   public static final boolean USE_BUFFERING = false;
   public static final boolean DISABLE_NAGLE = true;

   private static int s_messages = 0;
   private static long s_lastUpdateTime = System.currentTimeMillis();

   private final byte[] m_buffer;

   // if you don't provide an argument then this runs as
   // the server.  If you do, then this runs as a client
   // and attempts to connect to the named server
   public static void
   main(String[] args)
   {
       System.out.printf("Mesage size: %d%n", MESSAGE_SIZE);
       System.out.printf("Use buffered streams: %b%n", USE_BUFFERING);
       System.out.printf("Disable Nagle's algorithm: %b%n", DISABLE_NAGLE);

       if (args.length > 0)
           new Client(args[0]).start();
       else
           new Server().start();
   }

   private static void
   incrementMessages()
   {
       if (++s_messages < MESSAGE_PER_UPDATE)
           return;

       long now = System.currentTimeMillis();
       double seconds = (now - s_lastUpdateTime) / 1000.0;
       double rate = MESSAGE_PER_UPDATE / seconds;
       System.out.printf("%6.0f msgs / sec%n", rate);

       s_lastUpdateTime = now;
       s_messages = 0;
   }

   Test()
   {
       m_buffer = new byte[MESSAGE_SIZE];
       for (int i = 0; i < m_buffer.length; i++)
           m_buffer[i] = (byte)('A' + i);
   }

   boolean
   readMessage(InputStream in)
   throws IOException
   {
       int todo = m_buffer.length;
       int done = 0;
       while (todo > 0)
       {
           int got = in.read(m_buffer, done, todo);
           if (got < 0)
               return false;
           todo -= got;
           done += got;
       }
       incrementMessages();

       return true;
   }

   void
   writeMessage(OutputStream out)
   throws IOException
   {
       out.write(m_buffer);
       out.flush();
       incrementMessages();
   }

   OutputStream
   outputStreamFor(Socket socket)
   throws IOException
   {
       OutputStream out = socket.getOutputStream();
       if (USE_BUFFERING)
           out = new BufferedOutputStream(out);
       return out;
   }

   InputStream
   inputStreamFor(Socket socket)
   throws IOException
   {
       InputStream in = socket.getInputStream();
       if (USE_BUFFERING)
           in = new BufferedInputStream(in);
       return in;
   }

   void
   maybeDisableNagle(Socket socket)
   throws SocketException
   {
       if (!DISABLE_NAGLE)
           return;
       socket.setTcpNoDelay(true);
       if (!socket.getTcpNoDelay())
           System.err.println("*** FAILED TO DIABLE NAGLE ***");
   }
}

class Server
extends Test
{
   void
   start()
   {
       ServerSocket socket = null;
       try
       {
           socket = new ServerSocket(PORT_NUMBER);
       }
       catch (IOException e)
       {
           System.err.println(e);
           return;
       }

       for (;;)
       {
           try
           {
System.err.println("waiting on " + socket);
               Socket client = socket.accept();
System.err.println("got " + client);
               maybeDisableNagle(client);
               handle(client);
               client.close();
System.err.println("finished with " + client);
           }
           catch (IOException e)
           {
               System.err.println(e);
           }
       }
   }

   private void
   handle(Socket client)
   throws IOException
   {
       InputStream in = inputStreamFor(client);
       OutputStream out = outputStreamFor(client);

       while (readMessage(in))
           writeMessage(out);

       in.close();
       out.close();
   }
}

class Client
extends Test
{
   private final String m_serverName;

   Client(String serverName)
   {
       m_serverName = serverName;
   }

   void
   start()
   {
       try
       {
           Socket server = new Socket(m_serverName, PORT_NUMBER);
           maybeDisableNagle(server);
           pester(server);
           server.close();
       }
       catch (IOException e)
       {
           System.err.println(e);
       }
   }

   private void
   pester(Socket server)
   throws IOException
   {
       InputStream in = inputStreamFor(server);
       OutputStream out = outputStreamFor(server);

       do
           writeMessage(out);
       while (readMessage(in));

       in.close();
       out.close();
   }
}
=========== /code ===========
Shin - 07 Apr 2006 22:39 GMT
> > [...] is there any experiment on how fast
> > you can exchange messages between two jvms?
[quoted text clipped - 24 lines]
> =========== code ===========
> [...]

Thanks for sharing your code; I am obviously too lazy :(  I did run it
on a typical LAN with Linux boxes.  I got about 17K exchanges at msg
size 15, drop to 10K at 256.  Among the parameters,  disable NAGLE
doesn't seem to make a big difference, while disable buffering give you
a little better results.  Using a pool of connections is about the same
performance, a little more cost at the server side where you have to
switch back and forth threads.

So, it seems that for my original goal: I need to disable buffering and
Nagle algo, and call flush at the right place.  There are not many
other options to tweak.

-Shin
Chris Uppal - 09 Apr 2006 08:28 GMT
> Thanks for sharing your code; I am obviously too lazy :(

Well, it helps to have something similar hanging around already ;-)

> So, it seems that for my original goal: I need to disable buffering and
> Nagle algo, and call flush at the right place.

I think you may be reading too much into one simple test.

For this /specific/ test Nagle is probably irrelevant since it always does a
read immediately after each write, so the OS can "realise" that it shouldn't
keep data hanging around in its buffers on the off-chance that the app will
supply a bit more data in a microsecond or two.  If you had a more realistic
simulation, perhaps with several clients, perhaps not sitting in a tight
send/receive loop, or whatever, then the picture might be different (or might
not).

Similarly, for the buffering.  If we change the test very slightly, so that the
code has to read a length (or type) field off the network before it knows how
much big each message is, then the buffering might easily make a big
improvement.

The point is that you have to /think/ about how your app behaves, and then (and
only then) simple tests like the one I posted can help validate your
intuitions, and quantify the effects that you expect to see.

Really all that that one test tells you is a ball-park estimate of the
performance you can expect.  If you need to send 1e6 message/sec from London to
New York, then it tells you that you are out of luck ;-)  If you want to send
1e3 messages/sec over your LAN, then it tells you that network performance
should not be a big problem...

   -- chris
Shin - 05 Apr 2006 20:45 GMT
> Are you sure you mean throughput ?  It sounds to me more as if latency would be
> the big issue in this case.
I meant latency, yes.  You try hard not to mess up these concepts, you
still did.

> For latency the picture is more complicated.  Some issues:
> Do you have to open a new connection for each message ?
> If not then how long to you hold connections "idle" before dropping them ?
> Is the delay caused by "Nagle's Algorithm" (see Google) a factor ?
> How far apart (in network terms) are the servers ?

I did use a pool of connections;  I only dropped them at the end of app
(I limited how many you can create);  At server side, if no data
available, I sleep for 10 microsecons, which I think is reasonable.
Will check out this Nagle factor. Thanks,

-Shin


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.