Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / September 2007

Tip: Looking for answers? Try searching our database.

Socket is still connected after Server-Side socket termination.

Thread view: 
pek - 19 Sep 2007 14:19 GMT
Basic Idea
So I have this client-server project where the client uses Java's
Socket class to communicate with the server using JSON. Let's say that
I have a fake message that the client sends to the server and the
server closes the socket with the client (it doesn't send any response
to the client). In the client side I have a Thread that loops through
the socket's inputstream and prints out any server response. I also
print out the socket's connection status (socket.isConnected()).

Problem
When the client first sends the message the server immediately closes
the socket (I also terminate the server to make sure). The client, on
the other hand, never gets notified that the socket is closed and
everything seems running normal (it still prints out "true" every
socket.isConnect() method call). Only after a few attempts of sending
a message will the client understand that the socket is closed and
tries to reconnect.

Structure
SockeEngine is a class that handles the thread and has a
BlockingQueue<String> for outgoing messages. The thread loops forever.
The loop looks at the queue, the inputstream and outputstream. When
the queue has a message(s) to send it writes it to the outputstream of
the socket. After that it reads the inputstream and finnaly sleeps for
100ms.

I heard that the way we wrote it isn't the problem. We think the
problem relies on the implementation of sockets in Java or Windows. We
also think that it most likely not. Our temporary solution is sending
a termination message from the server to the client and then closing.
Any ideas/suggestions?

Thank you very much for your time.
-pek
Gordon Beaton - 19 Sep 2007 14:18 GMT
> When the client first sends the message the server immediately
> closes the socket (I also terminate the server to make sure). The
> client, on the other hand, never gets notified that the socket is
> closed and everything seems running normal (it still prints out
> "true" every socket.isConnect() method call).

[...]

> I heard that the way we wrote it isn't the problem. We think the
> problem relies on the implementation of sockets in Java or Windows.

No, the problem is that you are relying on isConnected() to tell you
the status of the connection.

The way to determine when the socket is closed is to attempt to read
from it or write to it. Both of these operations will tell you when
you have reached EOF on the correspoding socket stream, indicating
that the remote has closed his end.

/gordon

--
derek - 19 Sep 2007 14:45 GMT
from your description it sounds like you are not using nio. if you only have one thread that handles all of the input and output streams, and it only reads from the inputstream after it does a write, then i would think that of course you wouldnt see it close until after you did a write. i have never found socket.isClosed() to be reliable if the other end of the communication is shut down. i have found that doing a continuous read on the inputstream is reliable. it will let you know when the socket has been shut down on the other end. at least it has always worked for me. i typically use a background thread for each socket input stream. that is if i am not using nio.
pek - 19 Sep 2007 16:09 GMT
> from your description it sounds like you are not using nio. if you only have one thread that handles all of the input and output streams, and it only reads from the inputstream after it does a write, then i would think that of course you wouldnt see it close until after you did a write. i have never found socket.isClosed() to be reliable if the other end of the communication is shut down. i have found that doing a continuous read on the inputstream is reliable. it will let you know when the socket has been shut down on the other end. at least it has always worked for me. i typically use a background thread for each socket input stream. that is if i am not using nio.

First off, I don not use isConnected() to determine if the socket has
terminated or not. I just print it out to see if it works. And it
doens't. So nothing changes in the code.
The Thread writes data only when it is available but it ALWAYS reads
the inputstream. Even when the socket at the server terminates the
read still works! I print out the bytes and I see gibberish. Here  are
my imports in regard to the nio. (No, I don't use it. I just now have
heard of it :S)

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketException;
import java.net.UnknownHostException;
import java.util.List;
import java.util.Queue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

Thank you again for your time.
derek - 19 Sep 2007 16:33 GMT
I think it would behoove you to include a snippet of your code.
From your description its difficult to get a good grasp of what
you are actually doing.
So, the best response your going to get is just some random
guesses.
Maybe just post the portion where you are doing the actual reading
to start with.
pek - 19 Sep 2007 16:41 GMT
I was just about to do that.. ;) Here it is..

            StringBuffer outgoingBuffer;
            List<Message> incomming;
            while (running) {
                try {
                                       // Count the number of
messages the outgoing queue has
                    int messageCount = outgoing.size();
                    outgoingBuffer = new StringBuffer();
                    for (int i=0;i<messageCount;i++) {
                        outgoingBuffer.append(new
String(parser.serialize(outgoing.poll())));
                    }
                    out.write(outgoingBuffer.toString().getBytes());

                    int availableBytes = in.available();
                    byte[] toDeserialize = new byte[availableBytes];
                    in.read(toDeserialize);
                                       // Use parser to serialize/
deserialize message (from/to JSON)
                    incomming = parser.deserialize(toDeserialize);
                    if (incomming.size()>0) {
                        System.out.println("server: "+new String(toDeserialize));
                        for (Message message:incomming) {
                                                       // Send all
registered messageListeners the message that has been received.
                            eventManager.fireEvent(message);
                        }
                    }
                    Thread.sleep(100);
                } catch(SocketException e) {
                    for (int i=0;i<CONNECTION_ATTEMPTS;i++) {
                        try {
                            System.out.println("Attempt to connect...");
                            disconnect();
                            parser.reset();
                            connect(hostname,port);
                            out.write(parser.serialize(createRecoveryMessage()));
                            System.out.println("Connected!");
                            break;
                        } catch(Exception e1) {
                            e1.printStackTrace();
                            try { Thread.sleep(1000); } catch(InterruptedException e2) {}
                        }
                    }
                } catch(IOException e) {
                } catch(InterruptedException e) {

                }
            }
            try {
                socket.close();
            } catch(IOException e) {e.printStackTrace();}
        }
derek - 19 Sep 2007 16:53 GMT
Without running it, and just scanning it, it looks ok.
I would split out the reading though into a separate thread from
the writing.
Then just put that thread into a loop on the in.read(buffer).
I've never had any problems doing it that way.
derek - 19 Sep 2007 16:58 GMT
Also, the inputstream.available() only checks how many bytes
are available to be called and not block.
I dont believe it will tell you when the other side of a
socket is closed.
I believe that is why just putting it into a in.read(buffer) loop
works for me. It will return -1 if the end of the stream has
been reached.

from the javadocs:
read() returns:
the total number of bytes read into the buffer, or -1 is there is no more data because the end of the stream has been reached.
pek - 19 Sep 2007 17:22 GMT
> Also, the inputstream.available() only checks how many bytes
> are available to be called and not block.
[quoted text clipped - 7 lines]
> read() returns:
> the total number of bytes read into the buffer, or -1 is there is no more data because the end of the stream has been reached.

Yes, but read() blocks while read(buffer) doesn't. And I don't want to
use a second thread for concurrency reasons (I am not sure if there
are any, but since there is no harm using one thread, I don't think it
is necessary).

read(buffer) adds bytes to the buffer. Two questions:
what is -1 in bytes..?
the end of stream is reached when the socket is closed or when there
are no data to read? (server isn't sending anything)
derek - 19 Sep 2007 17:59 GMT
> Yes, but read() blocks while read(buffer) doesn't.

false, the javadocs specifically say that both read() and read(byte[]) block.

> read(buffer) adds bytes to the buffer. Two questions:
> what is -1 in bytes..?
> the end of stream is reached when the socket is closed or when > there
> are no data to read? (server isn't sending anything)

-1 means the socket has been closed on the other side.
> 0 means the actual number of bytes that were read into the
byte[]
i dont think a 0 is ever returned.

btw, all of this is in the javadocs.
http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html
Lew - 19 Sep 2007 23:05 GMT
> false, the javadocs [sic] specifically say that both read() and read(byte[]) block.

pek wrote:
>> read(buffer) adds bytes to the buffer. Two questions:
>> what is -1 in bytes..?

What does that question even mean?

It's not relevant to InputStream.read().

You don't put the -1 in the buffer.  You use it to end the loop.  Just like
you don't ever use the value returned by the method in the buffer, even when
greater than or equal to zero.  The return value of read() is not the data
read, but the amount of data read, or -1 if the socket has closed.  As others
have stated.  As the Javadocs state.

> -1 means the socket has been closed on the other side.
> i [sic] dont [sic] think a 0 is ever returned.
>
> btw, all of this is in the javadocs [sic].
> http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html

Make that
<http://java.sun.com/javase/6/docs/api/java/io/InputStream.html>

You might as well be current.

Do read one or the other of those links.  Both links answer all the questions
you've been asking.

Signature

Lew

Gordon Beaton - 19 Sep 2007 17:23 GMT
> I was just about to do that.. ;) Here it is..

You are throwing away the information you need to determine when the
remote has closed the socket.

Look at the return value of in.read(), it will be -1 when the remote
has closed. In other cases, it will tell you how much data it actually
read, which may at times be less than you expect.

Your failure to look at this value is the reason you think you think
you are receiving garbage after the remote has closed the socket.

/gordon

--
pek - 19 Sep 2007 17:52 GMT
> > I was just about to do that.. ;) Here it is..
>
[quoted text clipped - 4 lines]
> has closed. In other cases, it will tell you how much data it actually
> read, which may at times be less than you expect.

Well, read(buffer) returns bytes. And -1 is represented in bytes. So
what is -1 in bytes? Messages that come from the server are strings.
So I use String(theReadBytes) to convert them and read them. How do I
do that with -1?
Also, I use read(buffer) and not read() because the latest blocks.

> Your failure to look at this value is the reason you think you think
> you are receiving garbage after the remote has closed the socket.

I get the same "garbage" even if the server didn't close the socket.
They are probably empty bytes of some kind. The funny is that I get
the same stream of "garbage" even if I close the socket from the
server!

This is an example of the bytes I get when socket opens (I print them
out in each loop)
[B@19bb25a
[B@da6bf4
[B@1e58cb8
[B@179935d
[B@b9e45a
[B@3ef810

And this when the socket on the server has been closed.
[B@17200b4
[B@18c3679
[B@4c47db
[B@1ac3379
[B@6779e6
[B@174219d

Obviously they need to be converted to a human-readable string or
something..
Gordon Beaton - 19 Sep 2007 18:13 GMT
> Well, read(buffer) returns bytes. And -1 is represented in bytes. So
> what is -1 in bytes? Messages that come from the server are strings.
> So I use String(theReadBytes) to convert them and read them. How do
> I do that with -1? Also, I use read(buffer) and not read() because
> the latest blocks.

Both can block.

-1 is the return value of the method at EOF, not the buffer contents.
Do you understand the difference? So:

 byte[] buffer = new byte[someNumber];
 int n = in.read(buffer);

 if (n < 0) {
   System.out.println("Remote has closed");
 }
 else {
   System.out.println("Received " + n " bytes");
 }

> I get the same "garbage" even if the server didn't close the socket.
> They are probably empty bytes of some kind. The funny is that I get
> the same stream of "garbage" even if I close the socket from the
> server!

> This is an example of the bytes I get when socket opens (I print them
> out in each loop)
[quoted text clipped - 4 lines]
> [B@b9e45a
> [B@3ef810

There are no "empty bytes", and the above is not the received data, it
is a representation of the references to your byte arrays. If you want
to display the actual data, you need to iterate over the array
contents (observing n, the return value from in.read()) and display
each byte, perhaps in hex:

 for (int i=0; i<n; i++) {
   System.out.print("  " + Integer.toHexString(ff & buffer[i]));
 }
 System.out.println();

/gordon

--
pek - 19 Sep 2007 16:40 GMT
To make things easier here is the code of the method run of the
looping thread :

            StringBuffer outgoingBuffer;
            List<Message> incomming;
            while (running) {
                try {
                                       // Count the number of
messages the outgoing queue has
                    int messageCount = outgoing.size();
                    outgoingBuffer = new StringBuffer();
                    for (int i=0;i<messageCount;i++) {
                        outgoingBuffer.append(new
String(parser.serialize(outgoing.poll())));
                    }
                    out.write(outgoingBuffer.toString().getBytes());

                    int availableBytes = in.available();
                    byte[] toDeserialize = new byte[availableBytes];
                    in.read(toDeserialize);
                                       // Use parser to serialize/
deserialize message (from/to JSON)
                    incomming = parser.deserialize(toDeserialize);
                    if (incomming.size()>0) {
                        System.out.println("server: "+new String(toDeserialize));
                        for (Message message:incomming) {
                                                       // Send all
registered messageListeners the message that has been received.
                            eventManager.fireEvent(message);
                        }
                    }
                    Thread.sleep(100);
                } catch(SocketException e) {
                    for (int i=0;i<CONNECTION_ATTEMPTS;i++) {
                        try {
                            System.out.println("Attempt to connect...");
                            disconnect();
                            parser.reset();
                            connect(hostname,port);
                            out.write(parser.serialize(createRecoveryMessage()));
                            System.out.println("Connected!");
                            break;
                        } catch(Exception e1) {
                            e1.printStackTrace();
                            try { Thread.sleep(1000); } catch(InterruptedException e2) {}
                        }
                    }
                } catch(IOException e) {
                } catch(InterruptedException e) {

                }
            }
            try {
                socket.close();
            } catch(IOException e) {e.printStackTrace();}
        }
Lew - 19 Sep 2007 23:09 GMT
> To make things easier here is the code of the method run of the
> looping thread :

Please do not embed TABs in Usenet source listings.

Signature

Lew

Lew - 19 Sep 2007 23:14 GMT
> To make things easier here is the code of the method run of the
> looping thread :
>
> StringBuffer outgoingBuffer;

StringBuilder is the preferred class these days.

> List<Message> incomming;

Consider spelling the variable the same as the highly similar natural-language
word to avoid later maintenance errors.

> while (running) {
>   try {
[quoted text clipped - 9 lines]
>     int availableBytes = in.available();
>     byte[] toDeserialize = new byte[availableBytes];

You don't really need to allocate a new buffer each time.  You can re-use a
single buffer.

>     in.read(toDeserialize);

Others have told you of the value of checking the return value of read() to
make sure that you have received any data at all, and if so, all that you expect.

You should do that.  If it returns -1, the socket is closed.

>     incomming = parser.deserialize( toDeserialize );

But you have no certainty that the toDeserialize has everything you want in
it, or anything at all!

You need to check the return value of read().

Signature

Lew

Roedy Green - 19 Sep 2007 16:41 GMT
>When the client first sends the message the server immediately closes
>the socket (I also terminate the server to make sure).

here is how I would tackle this problem.

1. read up on TCP/IP protocol to learn just how it handles close, and
if both ends are even supposed to be immediately notified of the other
end's close.  Keep in mind TCP/IP does not send polling packets when
there is no traffic.

2. use Wireshark or other protocol sniffer
http://mindprod.com/jgloss/sniffer.html
to watch packets during an close on some socket you consider to behave
acceptably.

3. watch packets in your case.

4. read the fine print of what isConnected in supposed to tell you.

5. If it turns out TCP/IP is not designed to tell you of disconnect
sufficiently quickly, introduce some packets into your private higher
level  protocol.

1.SHUT:  other end, please shut down.
2.SHUTTING: other end, I am shutting down, this is the last you will
hear from me.

So the usual sequence in one end issues SHUT the waits for SHUTTING,
then shuts down.  

The other end on hearing a SHUT, issues a shutting, then shuts down.

This then works independently of anything in TCP/IP.  

IIRC this is what I resorted to in one project, then Sun fixed
something so it was no longer necessary, but I left it in as belt and
suspenders.  It fits in with the ISO idea of protocol layering.

The protocol also had heartbeat "are you still alive" packets.  It may
be that Sun added that support to basic TCP/IP, not EOF.  Sorry, it
was a while ago.

Signature

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

pek - 19 Sep 2007 16:56 GMT
On Sep 19, 6:41 pm, Roedy Green <see_webs...@mindprod.com.invalid>
wrote:

> >When the client first sends the message the server immediately closes
> >the socket (I also terminate the server to make sure).
[quoted text clipped - 5 lines]
> end's close.  Keep in mind TCP/IP does not send polling packets when
> there is no traffic.

I'm not quite sure. The server properly closes the socket, so, if this
was true, the client would have listened to that.

> 2. use Wireshark or other protocol snifferhttp://mindprod.com/jgloss/sniffer.html
> to watch packets during an close on some socket you consider to behave
> acceptably.
>
> 3. watch packets in your case.

I am currently developing on localhost. Windows (as wireshock makes it
clear) doesn't support monitoring localhost since there is no physical
interface (modem etc.)

> 4. read the fine print of what isConnected in supposed to tell you.

After reading some other posts, I am now sure that isConnected()
doesn't tell you if the server has closed the socket.

> 5. If it turns out TCP/IP is not designed to tell you of disconnect
> sufficiently quickly, introduce some packets into your private higher
[quoted text clipped - 10 lines]
>
> This then works independently of anything in TCP/IP.

This was our immediate "coding reaction". The thing is, we are on a
tight budget, and using network traffic costs us. We thought that the
simplest way was that server should send a "I closed your socket and
don't care what will you do" message and close the socket immediately
after that.

> IIRC this is what I resorted to in one project, then Sun fixed
> something so it was no longer necessary, but I left it in as belt and
[quoted text clipped - 3 lines]
> be that Sun added that support to basic TCP/IP, not EOF.  Sorry, it
> was a while ago.

This greatens the traffic (and our budget). But of course, if this is
the only solution, then we don't have any other choice.
Roedy Green - 23 Sep 2007 01:14 GMT
>I am currently developing on localhost. Windows (as wireshock makes it
>clear) doesn't support monitoring localhost since there is no physical
>interface (modem etc.)

Hmm. I guess WireShark ties in only at the LAN driver level.  It might
be fairly easy to round up some old beater of a machine to run on a
LAN so you can monitor traffic. Perhaps one of the other sniffers will
intercept local traffic. see http://mindprod.com/jgloss/sniffer.html

I remember years ago working with a Norton virus monitor.  It set up a
proxy mail server on your machine that simply relayed traffic through
to the real mail server.  A protocol sniffer might work like that,
forcing you to talk to a dummy address which then talks to the real
address.
Signature

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

Esmond Pitt - 20 Sep 2007 09:50 GMT
> 1. read up on TCP/IP protocol to learn just how it handles close, and
> if both ends are even supposed to be immediately notified of the other
> end's close.

They aren't. The only way you can find out about a TCP connection close
or abort is to try to read or write. This is well known and readily
available information that's posted here about a thousand times a year.

> 2. use Wireshark or other protocol sniffer
> 3. watch packets in your case.

A totally pointless waste of time & money given the above.

> 4. read the fine print of what isConnected in supposed to tell you.

IOW whether or not you ever connected your Socket in your own JVM.
Nothing about the connection. Nothing else either.

> 5. If it turns out TCP/IP is not designed to tell you of disconnect
> sufficiently quickly,

It isn't. It wasn't.

> The protocol also had heartbeat "are you still alive" packets.  It may
> be that Sun added that support to basic TCP/IP, not EOF.

Socket.setKeepAlive(true). This is an optional part of TCP/IP. Nothing
to do with Sun. Or EOF.
Martin Gregorie - 19 Sep 2007 19:25 GMT
> Basic Idea
> So I have this client-server project where the client uses Java's
> Socket class to communicate with the server using JSON. Let's say that
> I have a fake message that the client sends to the server and the
> server closes the socket with the client (it doesn't send any response
> to the client).

I think this is a poor design. The server should:
- wait for incoming connections from clients
- when a connection is received set up context (if needed)
  for the client
- process requests from the client, generating at least one response
  per request
- when the connection closes discard any client-specific
  context.
- wait for the next connection.

The client should:
- open a connection to the server
- send requests to the server
- read server response(s) to each request
- close the connection when it's done.

IMO the server should NEVER intentionally close a connection: as you've
seen this can cause problems. The method I outlined is much cleaner:
- the client knows when it has finished talking to the server and so
  can close the connection.
- in this scheme any connection closures seen by the client are
  ALWAYS an error.
- the logic of this scheme means that the server will be waiting for
  a new request from the client when the connection is closed and so
  will be ready to handle it or close the connection without needing
  to disentangle incomplete processing.
- designing the protocol so that every client request generates at
  least one server response makes error checking easy (the client
  always gets a response or sees the connection close due to an error.
  If a simple ACK response is short, the overheads are minimal.

If you design the protocol so that the messages contain text then use of
a packet sniffer is a lot easier. If you add debugging code that prints
all messages sent and received then you don't need a packet sniffer and
debugging process:process connections within a single computer is
simple. Specifying the protocol in terms of formatted records and using
record buffers to handle the messages rather than raw
streams also simplifies the protocol logic and helps a lot with
debugging. I normally design messages as length-delimited records
containing comma separated fields, but ymmv.

Lastly, the read/write loop on your client is probably a bad idea.
Unless your client is quite unusual this just adds complexity without
improving throughput. It also chews up CPU with its polling loop. To me
it smacks of inappropriate optimization: queues and scan loops should
only be introduced if monitoring code in the client shows that simple
"write - wait for response - read" logic is positively identified as a
bottleneck.

> When the client first sends the message the server immediately closes
> the socket (I also terminate the server to make sure).

Why? A server should be written to service multiple clients which can
connect and disconnect while the server continues to run. If you want to
stop it, use a dead simple client that connects, sends STOP, waits for
OK and then disconnects. Besides, such a client is often useful for
seeing what the server is doing, getting statistics, etc.

> I heard that the way we wrote it isn't the problem.

Disagree! How many client/server designs had your adviser implemented
successfully?

> We think the
> problem relies on the implementation of sockets in Java or Windows.

Sockets work fine for both C and Java if your message exchange protocol
is a clean design.

Signature

martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

pek - 22 Sep 2007 20:29 GMT
On Sep 19, 9:25 pm, Martin Gregorie <mar...@see.sig.for.address>
wrote:
> > Basic Idea
> > So I have this client-server project where the client uses Java's
[quoted text clipped - 80 lines]
> gregorie. | Essex, UK
> org       |

You probably didn't understand (at all) what is my problem. I know
this isn't the best way. I am not using this. This is an example. I
purposely want to close the connection from server side to see if the
client can catch the closure. The message that I send to the server
(as I mentioned before) is FAKE. That means that I'm just using it for
the example and not in the final version.
Thomas Schodt - 20 Sep 2007 08:04 GMT
> Basic Idea
> In the client side I have a Thread that loops through
> the socket's inputstream and prints out any server response. I also
> print out the socket's connection status (socket.isConnected()).

You misunderstand what isConnected() reports.

It only reports the local status of the socket.

bool Socket.isConnected()
what the javadoc should say:
Indicates if connect() has been called on this socket.
Initially this method returns false. After a connection is established,
this method method returns true. It will never change back to false for
any reason (like the connection failing).


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.