 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / November 2007NIO, SocketChannel and packet processing
I am using non-blocking NIO with SocketChannel to send data from a server to a client in "packets". Now, I thought that if I send 3 packets of 1024 bytes each and then call read() at the other end 3 times I would be able to read 3 separate packets. However, it seems that the underlying protocol knows nothing about the packet structure so the packets get glued together and just one read() call on the client may read all 3 packets at once. Is this correct? If so, how can I delimit packets and have a separate read for each one? Or is it a case of reading whatever's available and then breaking it up on the client side using some special byte value?  SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) > I am using non-blocking NIO with SocketChannel to send data from a server to > a client in "packets". Now, > I thought that if I send 3 packets of 1024 bytes each and then call > read() at the other end 3 times I would be able to read 3 separate > packets. However, it seems that the underlying protocol knows > nothing about the packet structure so the packets get glued together > and just one read() call on the client may read all 3 packets at > once. That's normal TCP behaviour (unrelated to java.nio). > Is this correct? If so, how can I delimit packets and have a > separate read for each one? Or is it a case of reading whatever's > available and then breaking it up on the client side using some > special byte value? If you know that every packet will be N bytes, then always read N byte packets. Otherwise, define a packet structure for your application that includes the packet length at the start of every packet. Start by reading (say) 4 bytes to get the length, then read again to get the number of bytes that were specified. An alternative mechanism is to separate messages with a delimiter, but that requires you to handle the delimiter specially in those cases where it may occur within a message. It's also more work for the recipient to find the delimiter and separate the messages. /gordon -- Thanks for the helpful reply Gordon. > If you know that every packet will be N bytes, then always read N byte > packets. The packets are of variable length. > Otherwise, define a packet structure for your application that > includes the packet length at the start of every packet. Start by > reading (say) 4 bytes to get the length, then read again to get the > number of bytes that were specified. I like this idea. But how do I read a specific number of bytes? Is it a matter of using a ByteBuffer of the desired size to firstly read the packet size and then allocating another ByteBuffer with the actual size the same as the packet size? > An alternative mechanism is to separate messages with a delimiter, but > that requires you to handle the delimiter specially in those cases > where it may occur within a message. It's also more work for the > recipient to find the delimiter and separate the messages. No, I'll try to get the other system working. Thanks again. SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) > But how do I read a specific number of bytes? Is it a matter of > using a ByteBuffer of the desired size to firstly read the packet > size and then allocating another ByteBuffer with the actual size the > same as the packet size? Either that, or use a single ByteBuffer per client that you know will always be large enough. You can specify both length and offset when you read in either case. There will also be times when you only manage to read part of a message. You need to take note of the return value on each read, and only return a complete message once the actual number of bytes matches the expected number for that client's current message. /gordon -- > Either that, or use a single ByteBuffer per client that you know will > always be large enough. You can specify both length and offset when > you read in either case. I can't see any version of the read() method that allows me to specify how many bytes to read. I think what you are referring to is the form of the read() method that allows you to specify which buffer in a buffer array to use and how many to use. > There will also be times when you only manage to read part of a > message. You need to take note of the return value on each read, and > only return a complete message once the actual number of bytes matches > the expected number for that client's current message. Well I'm pleased to say that it's now working... almost. It works perfectly when the server and client are on the same machine. But when I try the client on a different machine on the LAN, not all the data seems to be getting through. Surely this cannot be because of transmission errors over a tiny LAN? How prevalent would such errors be? What hope do I have of getting it to work over the internet? SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) > I can't see any version of the read() method that allows me to > specify how many bytes to read. I think what you are referring to is > the form of the read() method that allows you to specify which > buffer in a buffer array to use and how many to use. Sorry, my mistake. Strange that there is no method for reading a specified number of bytes into a ByteBuffer. > Well I'm pleased to say that it's now working... almost. It works > perfectly when the server and client are on the same machine. But > when I try the client on a different machine on the LAN, not all the > data seems to be getting through. Surely this cannot be because of > transmission errors over a tiny LAN? How prevalent would such errors > be? What hope do I have of getting it to work over the internet? Since TCP is used for virtually all internet applications, I think your chances are quite good. I can't say from here exactly what problem you're having though. Are you checking the return value from read(), and keeping track of the actual received byte count for each message? Try downloading Wireshark to see the actual network traffic your application is generating. /gordon -- > Since TCP is used for virtually all internet applications, I think > your chances are quite good. I can't say from here exactly what [quoted text clipped - 4 lines] > Try downloading Wireshark to see the actual network traffic your > application is generating. It's sorted now. The problem was that sometimes fewer bytes than the expected packet size were being read so now I just make sure I read until all bytes are processed. This happened more the further the client and server were apart physically. Thanks again Gordon. SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) > The problem was that sometimes fewer bytes than the > expected packet size were being read so now I just make sure I read > until all bytes are processed. Gordon Beaton had just advised: >> Are you checking the return value from >> read(), and keeping track of the actual received byte count for each >> message? SignatureLew >> The problem was that sometimes fewer bytes than the expected packet size >> were being read so now I just make sure I read until all bytes are [quoted text clipped - 4 lines] >>> read(), and keeping track of the actual received byte count for each >>> message? Yes, I was merely pointing out that he was spot on. SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) > It's sorted now. The problem was that sometimes fewer bytes than the > expected packet size were being read so now I just make sure I read > until all bytes are processed. This happened more the further the > client and server were apart physically. This thread was starting to sound familiar to me, and it appears you asked virtually the same questions on Oct 5-6... Did you not solve this problem *then*? /gordon -- >> It's sorted now. The problem was that sometimes fewer bytes than the >> expected packet size were being read so now I just make sure I read [quoted text clipped - 5 lines] > > Did you not solve this problem *then*? That was with plain-old-standard I/O, not NIO. I have moved everything I was doing then over to the improved NIO - successfully thankfully. SignatureAnd loving it, -Q _________________________________________________ Qu0llSixFour@gmail.com (Replace the "SixFour" with numbers to email me) Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2009 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.