Java Forum / General / September 2007
Socket is still connected after Server-Side socket termination.
pek - 19 Sep 2007 14:19 GMT Basic Idea So I have this client-server project where the client uses Java's Socket class to communicate with the server using JSON. Let's say that I have a fake message that the client sends to the server and the server closes the socket with the client (it doesn't send any response to the client). In the client side I have a Thread that loops through the socket's inputstream and prints out any server response. I also print out the socket's connection status (socket.isConnected()).
Problem When the client first sends the message the server immediately closes the socket (I also terminate the server to make sure). The client, on the other hand, never gets notified that the socket is closed and everything seems running normal (it still prints out "true" every socket.isConnect() method call). Only after a few attempts of sending a message will the client understand that the socket is closed and tries to reconnect.
Structure SockeEngine is a class that handles the thread and has a BlockingQueue<String> for outgoing messages. The thread loops forever. The loop looks at the queue, the inputstream and outputstream. When the queue has a message(s) to send it writes it to the outputstream of the socket. After that it reads the inputstream and finnaly sleeps for 100ms.
I heard that the way we wrote it isn't the problem. We think the problem relies on the implementation of sockets in Java or Windows. We also think that it most likely not. Our temporary solution is sending a termination message from the server to the client and then closing. Any ideas/suggestions?
Thank you very much for your time. -pek
Gordon Beaton - 19 Sep 2007 14:18 GMT > When the client first sends the message the server immediately > closes the socket (I also terminate the server to make sure). The > client, on the other hand, never gets notified that the socket is > closed and everything seems running normal (it still prints out > "true" every socket.isConnect() method call). [...]
> I heard that the way we wrote it isn't the problem. We think the > problem relies on the implementation of sockets in Java or Windows. No, the problem is that you are relying on isConnected() to tell you the status of the connection.
The way to determine when the socket is closed is to attempt to read from it or write to it. Both of these operations will tell you when you have reached EOF on the correspoding socket stream, indicating that the remote has closed his end.
/gordon
--
derek - 19 Sep 2007 14:45 GMT from your description it sounds like you are not using nio. if you only have one thread that handles all of the input and output streams, and it only reads from the inputstream after it does a write, then i would think that of course you wouldnt see it close until after you did a write. i have never found socket.isClosed() to be reliable if the other end of the communication is shut down. i have found that doing a continuous read on the inputstream is reliable. it will let you know when the socket has been shut down on the other end. at least it has always worked for me. i typically use a background thread for each socket input stream. that is if i am not using nio.
pek - 19 Sep 2007 16:09 GMT > from your description it sounds like you are not using nio. if you only have one thread that handles all of the input and output streams, and it only reads from the inputstream after it does a write, then i would think that of course you wouldnt see it close until after you did a write. i have never found socket.isClosed() to be reliable if the other end of the communication is shut down. i have found that doing a continuous read on the inputstream is reliable. it will let you know when the socket has been shut down on the other end. at least it has always worked for me. i typically use a background thread for each socket input stream. that is if i am not using nio. First off, I don not use isConnected() to determine if the socket has terminated or not. I just print it out to see if it works. And it doens't. So nothing changes in the code. The Thread writes data only when it is available but it ALWAYS reads the inputstream. Even when the socket at the server terminates the read still works! I print out the bytes and I see gibberish. Here are my imports in regard to the nio. (No, I don't use it. I just now have heard of it :S)
import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.net.InetAddress; import java.net.InetSocketAddress; import java.net.Socket; import java.net.SocketException; import java.net.UnknownHostException; import java.util.List; import java.util.Queue; import java.util.concurrent.BlockingQueue; import java.util.concurrent.LinkedBlockingQueue;
Thank you again for your time.
derek - 19 Sep 2007 16:33 GMT I think it would behoove you to include a snippet of your code. From your description its difficult to get a good grasp of what you are actually doing. So, the best response your going to get is just some random guesses. Maybe just post the portion where you are doing the actual reading to start with.
pek - 19 Sep 2007 16:41 GMT I was just about to do that.. ;) Here it is..
StringBuffer outgoingBuffer; List<Message> incomming; while (running) { try { // Count the number of messages the outgoing queue has int messageCount = outgoing.size(); outgoingBuffer = new StringBuffer(); for (int i=0;i<messageCount;i++) { outgoingBuffer.append(new String(parser.serialize(outgoing.poll()))); } out.write(outgoingBuffer.toString().getBytes());
int availableBytes = in.available(); byte[] toDeserialize = new byte[availableBytes]; in.read(toDeserialize); // Use parser to serialize/ deserialize message (from/to JSON) incomming = parser.deserialize(toDeserialize); if (incomming.size()>0) { System.out.println("server: "+new String(toDeserialize)); for (Message message:incomming) { // Send all registered messageListeners the message that has been received. eventManager.fireEvent(message); } } Thread.sleep(100); } catch(SocketException e) { for (int i=0;i<CONNECTION_ATTEMPTS;i++) { try { System.out.println("Attempt to connect..."); disconnect(); parser.reset(); connect(hostname,port); out.write(parser.serialize(createRecoveryMessage())); System.out.println("Connected!"); break; } catch(Exception e1) { e1.printStackTrace(); try { Thread.sleep(1000); } catch(InterruptedException e2) {} } } } catch(IOException e) { } catch(InterruptedException e) {
} } try { socket.close(); } catch(IOException e) {e.printStackTrace();} }
derek - 19 Sep 2007 16:53 GMT Without running it, and just scanning it, it looks ok. I would split out the reading though into a separate thread from the writing. Then just put that thread into a loop on the in.read(buffer). I've never had any problems doing it that way.
derek - 19 Sep 2007 16:58 GMT Also, the inputstream.available() only checks how many bytes are available to be called and not block. I dont believe it will tell you when the other side of a socket is closed. I believe that is why just putting it into a in.read(buffer) loop works for me. It will return -1 if the end of the stream has been reached.
from the javadocs: read() returns: the total number of bytes read into the buffer, or -1 is there is no more data because the end of the stream has been reached.
pek - 19 Sep 2007 17:22 GMT > Also, the inputstream.available() only checks how many bytes > are available to be called and not block. [quoted text clipped - 7 lines] > read() returns: > the total number of bytes read into the buffer, or -1 is there is no more data because the end of the stream has been reached. Yes, but read() blocks while read(buffer) doesn't. And I don't want to use a second thread for concurrency reasons (I am not sure if there are any, but since there is no harm using one thread, I don't think it is necessary).
read(buffer) adds bytes to the buffer. Two questions: what is -1 in bytes..? the end of stream is reached when the socket is closed or when there are no data to read? (server isn't sending anything)
derek - 19 Sep 2007 17:59 GMT > Yes, but read() blocks while read(buffer) doesn't. false, the javadocs specifically say that both read() and read(byte[]) block.
> read(buffer) adds bytes to the buffer. Two questions: > what is -1 in bytes..? > the end of stream is reached when the socket is closed or when > there > are no data to read? (server isn't sending anything) -1 means the socket has been closed on the other side.
> 0 means the actual number of bytes that were read into the byte[] i dont think a 0 is ever returned.
btw, all of this is in the javadocs. http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html
Lew - 19 Sep 2007 23:05 GMT > false, the javadocs [sic] specifically say that both read() and read(byte[]) block. pek wrote:
>> read(buffer) adds bytes to the buffer. Two questions: >> what is -1 in bytes..? What does that question even mean?
It's not relevant to InputStream.read().
You don't put the -1 in the buffer. You use it to end the loop. Just like you don't ever use the value returned by the method in the buffer, even when greater than or equal to zero. The return value of read() is not the data read, but the amount of data read, or -1 if the socket has closed. As others have stated. As the Javadocs state.
> -1 means the socket has been closed on the other side. > i [sic] dont [sic] think a 0 is ever returned. > > btw, all of this is in the javadocs [sic]. > http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html Make that <http://java.sun.com/javase/6/docs/api/java/io/InputStream.html>
You might as well be current.
Do read one or the other of those links. Both links answer all the questions you've been asking.
 Signature Lew
Gordon Beaton - 19 Sep 2007 17:23 GMT > I was just about to do that.. ;) Here it is.. You are throwing away the information you need to determine when the remote has closed the socket.
Look at the return value of in.read(), it will be -1 when the remote has closed. In other cases, it will tell you how much data it actually read, which may at times be less than you expect.
Your failure to look at this value is the reason you think you think you are receiving garbage after the remote has closed the socket.
/gordon
--
pek - 19 Sep 2007 17:52 GMT > > I was just about to do that.. ;) Here it is.. > [quoted text clipped - 4 lines] > has closed. In other cases, it will tell you how much data it actually > read, which may at times be less than you expect. Well, read(buffer) returns bytes. And -1 is represented in bytes. So what is -1 in bytes? Messages that come from the server are strings. So I use String(theReadBytes) to convert them and read them. How do I do that with -1? Also, I use read(buffer) and not read() because the latest blocks.
> Your failure to look at this value is the reason you think you think > you are receiving garbage after the remote has closed the socket. I get the same "garbage" even if the server didn't close the socket. They are probably empty bytes of some kind. The funny is that I get the same stream of "garbage" even if I close the socket from the server!
This is an example of the bytes I get when socket opens (I print them out in each loop) [B@19bb25a [B@da6bf4 [B@1e58cb8 [B@179935d [B@b9e45a [B@3ef810
And this when the socket on the server has been closed. [B@17200b4 [B@18c3679 [B@4c47db [B@1ac3379 [B@6779e6 [B@174219d
Obviously they need to be converted to a human-readable string or something..
Gordon Beaton - 19 Sep 2007 18:13 GMT > Well, read(buffer) returns bytes. And -1 is represented in bytes. So > what is -1 in bytes? Messages that come from the server are strings. > So I use String(theReadBytes) to convert them and read them. How do > I do that with -1? Also, I use read(buffer) and not read() because > the latest blocks. Both can block.
-1 is the return value of the method at EOF, not the buffer contents. Do you understand the difference? So:
byte[] buffer = new byte[someNumber]; int n = in.read(buffer);
if (n < 0) { System.out.println("Remote has closed"); } else { System.out.println("Received " + n " bytes"); }
> I get the same "garbage" even if the server didn't close the socket. > They are probably empty bytes of some kind. The funny is that I get > the same stream of "garbage" even if I close the socket from the > server!
> This is an example of the bytes I get when socket opens (I print them > out in each loop) [quoted text clipped - 4 lines] > [B@b9e45a > [B@3ef810 There are no "empty bytes", and the above is not the received data, it is a representation of the references to your byte arrays. If you want to display the actual data, you need to iterate over the array contents (observing n, the return value from in.read()) and display each byte, perhaps in hex:
for (int i=0; i<n; i++) { System.out.print(" " + Integer.toHexString(ff & buffer[i])); } System.out.println();
/gordon
--
pek - 19 Sep 2007 16:40 GMT To make things easier here is the code of the method run of the looping thread :
StringBuffer outgoingBuffer; List<Message> incomming; while (running) { try { // Count the number of messages the outgoing queue has int messageCount = outgoing.size(); outgoingBuffer = new StringBuffer(); for (int i=0;i<messageCount;i++) { outgoingBuffer.append(new String(parser.serialize(outgoing.poll()))); } out.write(outgoingBuffer.toString().getBytes());
int availableBytes = in.available(); byte[] toDeserialize = new byte[availableBytes]; in.read(toDeserialize); // Use parser to serialize/ deserialize message (from/to JSON) incomming = parser.deserialize(toDeserialize); if (incomming.size()>0) { System.out.println("server: "+new String(toDeserialize)); for (Message message:incomming) { // Send all registered messageListeners the message that has been received. eventManager.fireEvent(message); } } Thread.sleep(100); } catch(SocketException e) { for (int i=0;i<CONNECTION_ATTEMPTS;i++) { try { System.out.println("Attempt to connect..."); disconnect(); parser.reset(); connect(hostname,port); out.write(parser.serialize(createRecoveryMessage())); System.out.println("Connected!"); break; } catch(Exception e1) { e1.printStackTrace(); try { Thread.sleep(1000); } catch(InterruptedException e2) {} } } } catch(IOException e) { } catch(InterruptedException e) {
} } try { socket.close(); } catch(IOException e) {e.printStackTrace();} }
Lew - 19 Sep 2007 23:09 GMT > To make things easier here is the code of the method run of the > looping thread : Please do not embed TABs in Usenet source listings.
 Signature Lew
Lew - 19 Sep 2007 23:14 GMT > To make things easier here is the code of the method run of the > looping thread : > > StringBuffer outgoingBuffer; StringBuilder is the preferred class these days.
> List<Message> incomming; Consider spelling the variable the same as the highly similar natural-language word to avoid later maintenance errors.
> while (running) { > try { [quoted text clipped - 9 lines] > int availableBytes = in.available(); > byte[] toDeserialize = new byte[availableBytes]; You don't really need to allocate a new buffer each time. You can re-use a single buffer.
> in.read(toDeserialize); Others have told you of the value of checking the return value of read() to make sure that you have received any data at all, and if so, all that you expect.
You should do that. If it returns -1, the socket is closed.
> incomming = parser.deserialize( toDeserialize ); But you have no certainty that the toDeserialize has everything you want in it, or anything at all!
You need to check the return value of read().
 Signature Lew
Roedy Green - 19 Sep 2007 16:41 GMT >When the client first sends the message the server immediately closes >the socket (I also terminate the server to make sure). here is how I would tackle this problem.
1. read up on TCP/IP protocol to learn just how it handles close, and if both ends are even supposed to be immediately notified of the other end's close. Keep in mind TCP/IP does not send polling packets when there is no traffic.
2. use Wireshark or other protocol sniffer http://mindprod.com/jgloss/sniffer.html to watch packets during an close on some socket you consider to behave acceptably.
3. watch packets in your case.
4. read the fine print of what isConnected in supposed to tell you.
5. If it turns out TCP/IP is not designed to tell you of disconnect sufficiently quickly, introduce some packets into your private higher level protocol.
1.SHUT: other end, please shut down. 2.SHUTTING: other end, I am shutting down, this is the last you will hear from me.
So the usual sequence in one end issues SHUT the waits for SHUTTING, then shuts down.
The other end on hearing a SHUT, issues a shutting, then shuts down.
This then works independently of anything in TCP/IP.
IIRC this is what I resorted to in one project, then Sun fixed something so it was no longer necessary, but I left it in as belt and suspenders. It fits in with the ISO idea of protocol layering.
The protocol also had heartbeat "are you still alive" packets. It may be that Sun added that support to basic TCP/IP, not EOF. Sorry, it was a while ago.
 Signature Roedy Green Canadian Mind Products The Java Glossary http://mindprod.com
pek - 19 Sep 2007 16:56 GMT On Sep 19, 6:41 pm, Roedy Green <see_webs...@mindprod.com.invalid> wrote:
> >When the client first sends the message the server immediately closes > >the socket (I also terminate the server to make sure). [quoted text clipped - 5 lines] > end's close. Keep in mind TCP/IP does not send polling packets when > there is no traffic. I'm not quite sure. The server properly closes the socket, so, if this was true, the client would have listened to that.
> 2. use Wireshark or other protocol snifferhttp://mindprod.com/jgloss/sniffer.html > to watch packets during an close on some socket you consider to behave > acceptably. > > 3. watch packets in your case. I am currently developing on localhost. Windows (as wireshock makes it clear) doesn't support monitoring localhost since there is no physical interface (modem etc.)
> 4. read the fine print of what isConnected in supposed to tell you. After reading some other posts, I am now sure that isConnected() doesn't tell you if the server has closed the socket.
> 5. If it turns out TCP/IP is not designed to tell you of disconnect > sufficiently quickly, introduce some packets into your private higher [quoted text clipped - 10 lines] > > This then works independently of anything in TCP/IP. This was our immediate "coding reaction". The thing is, we are on a tight budget, and using network traffic costs us. We thought that the simplest way was that server should send a "I closed your socket and don't care what will you do" message and close the socket immediately after that.
> IIRC this is what I resorted to in one project, then Sun fixed > something so it was no longer necessary, but I left it in as belt and [quoted text clipped - 3 lines] > be that Sun added that support to basic TCP/IP, not EOF. Sorry, it > was a while ago. This greatens the traffic (and our budget). But of course, if this is the only solution, then we don't have any other choice.
Roedy Green - 23 Sep 2007 01:14 GMT >I am currently developing on localhost. Windows (as wireshock makes it >clear) doesn't support monitoring localhost since there is no physical >interface (modem etc.) Hmm. I guess WireShark ties in only at the LAN driver level. It might be fairly easy to round up some old beater of a machine to run on a LAN so you can monitor traffic. Perhaps one of the other sniffers will intercept local traffic. see http://mindprod.com/jgloss/sniffer.html
I remember years ago working with a Norton virus monitor. It set up a proxy mail server on your machine that simply relayed traffic through to the real mail server. A protocol sniffer might work like that, forcing you to talk to a dummy address which then talks to the real address.
 Signature Roedy Green Canadian Mind Products The Java Glossary http://mindprod.com
Esmond Pitt - 20 Sep 2007 09:50 GMT > 1. read up on TCP/IP protocol to learn just how it handles close, and > if both ends are even supposed to be immediately notified of the other > end's close. They aren't. The only way you can find out about a TCP connection close or abort is to try to read or write. This is well known and readily available information that's posted here about a thousand times a year.
> 2. use Wireshark or other protocol sniffer > 3. watch packets in your case. A totally pointless waste of time & money given the above.
> 4. read the fine print of what isConnected in supposed to tell you. IOW whether or not you ever connected your Socket in your own JVM. Nothing about the connection. Nothing else either.
> 5. If it turns out TCP/IP is not designed to tell you of disconnect > sufficiently quickly, It isn't. It wasn't.
> The protocol also had heartbeat "are you still alive" packets. It may > be that Sun added that support to basic TCP/IP, not EOF. Socket.setKeepAlive(true). This is an optional part of TCP/IP. Nothing to do with Sun. Or EOF.
Martin Gregorie - 19 Sep 2007 19:25 GMT > Basic Idea > So I have this client-server project where the client uses Java's > Socket class to communicate with the server using JSON. Let's say that > I have a fake message that the client sends to the server and the > server closes the socket with the client (it doesn't send any response > to the client). I think this is a poor design. The server should: - wait for incoming connections from clients - when a connection is received set up context (if needed) for the client - process requests from the client, generating at least one response per request - when the connection closes discard any client-specific context. - wait for the next connection.
The client should: - open a connection to the server - send requests to the server - read server response(s) to each request - close the connection when it's done.
IMO the server should NEVER intentionally close a connection: as you've seen this can cause problems. The method I outlined is much cleaner: - the client knows when it has finished talking to the server and so can close the connection. - in this scheme any connection closures seen by the client are ALWAYS an error. - the logic of this scheme means that the server will be waiting for a new request from the client when the connection is closed and so will be ready to handle it or close the connection without needing to disentangle incomplete processing. - designing the protocol so that every client request generates at least one server response makes error checking easy (the client always gets a response or sees the connection close due to an error. If a simple ACK response is short, the overheads are minimal.
If you design the protocol so that the messages contain text then use of a packet sniffer is a lot easier. If you add debugging code that prints all messages sent and received then you don't need a packet sniffer and debugging process:process connections within a single computer is simple. Specifying the protocol in terms of formatted records and using record buffers to handle the messages rather than raw streams also simplifies the protocol logic and helps a lot with debugging. I normally design messages as length-delimited records containing comma separated fields, but ymmv.
Lastly, the read/write loop on your client is probably a bad idea. Unless your client is quite unusual this just adds complexity without improving throughput. It also chews up CPU with its polling loop. To me it smacks of inappropriate optimization: queues and scan loops should only be introduced if monitoring code in the client shows that simple "write - wait for response - read" logic is positively identified as a bottleneck.
> When the client first sends the message the server immediately closes > the socket (I also terminate the server to make sure). Why? A server should be written to service multiple clients which can connect and disconnect while the server continues to run. If you want to stop it, use a dead simple client that connects, sends STOP, waits for OK and then disconnects. Besides, such a client is often useful for seeing what the server is doing, getting statistics, etc.
> I heard that the way we wrote it isn't the problem. Disagree! How many client/server designs had your adviser implemented successfully?
> We think the > problem relies on the implementation of sockets in Java or Windows. Sockets work fine for both C and Java if your message exchange protocol is a clean design.
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
pek - 22 Sep 2007 20:29 GMT On Sep 19, 9:25 pm, Martin Gregorie <mar...@see.sig.for.address> wrote:
> > Basic Idea > > So I have this client-server project where the client uses Java's [quoted text clipped - 80 lines] > gregorie. | Essex, UK > org | You probably didn't understand (at all) what is my problem. I know this isn't the best way. I am not using this. This is an example. I purposely want to close the connection from server side to see if the client can catch the closure. The message that I send to the server (as I mentioned before) is FAKE. That means that I'm just using it for the example and not in the final version.
Thomas Schodt - 20 Sep 2007 08:04 GMT > Basic Idea > In the client side I have a Thread that loops through > the socket's inputstream and prints out any server response. I also > print out the socket's connection status (socket.isConnected()). You misunderstand what isConnected() reports.
It only reports the local status of the socket.
bool Socket.isConnected() what the javadoc should say: Indicates if connect() has been called on this socket. Initially this method returns false. After a connection is established, this method method returns true. It will never change back to false for any reason (like the connection failing).
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|