Java Forum / General / June 2007
TCP/IP file transfer and a router
aca04pds@shef.ac.uk - 22 Apr 2007 16:48 GMT Hi
I'm trying to create a Java program that can send a file from 1 Windows PC to another over a TCP/IP connection. As many people use [wireless] routers, how can I create a TCP connection to PCs that are connected to the internet via a router? The router has an IP and then each PC connected to that has its own IP for their local network. How would the router know which PC is the right one to send the data to?
All help is appreciated, thanks.
Steven J. Sobol - 22 Apr 2007 17:19 GMT > [wireless] routers, how can I create a TCP connection to PCs that are > connected to the internet via a router? The router has an IP and then > each PC connected to that has its own IP for their local network. How > would the router know which PC is the right one to send the data to? The router manages the connections. You don't have to worry about it, as the programmer. All you need is an IP address that is accessible from the computer running your program, and a properly-configured router.
 Signature Steve Sobol, Professional Geek ** Java/VB/VC/PHP/Perl ** Linux/*BSD/Windows Victorville, California PGP:0xE3AE35ED
It's all fun and games until someone starts a bonfire in the living room.
Martin Gregorie - 22 Apr 2007 17:46 GMT > I'm trying to create a Java program that can send a file from 1 > Windows PC to another over a TCP/IP connection. As many people use > [wireless] routers, how can I create a TCP connection to PCs that are > connected to the internet via a router? The router has an IP and then > each PC connected to that has its own IP for their local network. How > would the router know which PC is the right one to send the data to? The following assumes that by 'router' you mean a router that's acting as a gateway to a private LAN.
If the router is working in transparent mode then all IPs on the inside will be exposed to the world and anybody can connect to them unless prevented by a firewall. However, this is not a typical case.
Usually the router will be working in NAT (Network Address Translation) mode. By default the addresses used by hosts on the LAN are not exposed to the outside world and, in fact, will typically be class C addresses (192.168.x.y) which are intended to be used only on a private LAN that's hidden from the world by the gateway router.
In this case there's no way that you can connect to a host on that LAN. This is by design: a NAT-enabled router is also an effective firewall against inward connections. Hosts on the LAN can still connect outward unless the gateway router has been configured to prevent this. This holds true regardless of whether the router's external IP address is static or dynamic: if anybody knows the router's IP they can contact it but it will refuse the connection.
There is one exception. The owner of the router can configure it to do port forwarding. This means that the router can be configured to pass inbound connection requests for that port to a specific host on the inside LAN, i.e. if an internal host is running a web server the owner might set his router to pass port 80 connections to the web host. However, this can cause problems:
- unless the owner knows exactly what he is doing he can leave his entire LAN open to attack via that port.
- forwarding a port to anything but a well secured, bulletproof server is extremely foolish. IMHO forwarding it to a Windows PC borders on having a death wish.
- many ISPs do not allow users to run servers, so port forwarding may well break the owner's service contract with his ISP.
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
Z. - 22 Apr 2007 18:29 GMT > Usually the router will be working in NAT (Network Address Translation) > mode. By default the addresses used by hosts on the LAN are not exposed [quoted text clipped - 3 lines] > > In this case there's no way that you can connect to a host on that LAN. Not wholly true. A properly constructed unsolicited packet that is sent to a consumer-grade router may be forwarded to a non-routable IP address on the private LAN.
> - forwarding a port to anything but a well secured, bulletproof > server is extremely foolish. IMHO forwarding it to a Windows > PC borders on having a death wish. Ridiculous! Of course, any destination port must be properly protected but that's possible on Windows.
Forwarding to an unsecured port is dangerous on *any* operating system.
Martin Gregorie - 22 Apr 2007 21:37 GMT > Not wholly true. A properly constructed unsolicited packet that is sent > to a consumer-grade router may be forwarded to a non-routable IP address > on the private LAN. Without port forwarding there's no way a NAT-enabled router can know what class C IP to forward the packet or connection to so the only valid option is to drop the packet or connection.
If a router I owned forwarded any such packet or connection I'd return it whence it came accompanied by a complaint.
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
Christian - 22 Apr 2007 23:03 GMT Martin Gregorie schrieb:
>> Not wholly true. A properly constructed unsolicited packet that is >> sent to a consumer-grade router may be forwarded to a non-routable IP [quoted text clipped - 6 lines] > If a router I owned forwarded any such packet or connection I'd > return it whence it came accompanied by a complaint. That is not completely true ... there exists a technique called hole punching... thats what Skype uses for transfering data between peers in a double NAT situation... Basically its trying to trick the NAT to believe the connection was outgoing ..
also some IPPhones use this unconventional behaviour of NATs..
though I think no normal application will implement something like this except networking is really the primary aspect of the program..
Christian
Z. - 23 Apr 2007 01:59 GMT > That is not completely true ... there exists a technique called hole > punching... Exactly.
Good descriptions here: http://en.wikipedia.org/wiki/Hole_punching http://en.wikipedia.org/wiki/UDP_hole_punching
Martin Gregorie - 23 Apr 2007 13:15 GMT > Martin Gregorie schrieb: >>> Not wholly true. A properly constructed unsolicited packet that is [quoted text clipped - 13 lines] > Basically its trying to trick the NAT to believe the connection was > outgoing .. Come on guys, "holepunching" isn't fooling the router, tricking NAT or whatever. It has nothing to do with the subject of this thread, which is whether a NAT-enabled router ever accepts incoming connections without using port forwarding. It doesn't unless it has a bug.
"Holepunching" simply describes a technique where the owner of the router making an outward connection to some sort of message exchange server.
> also some IPPhones use this unconventional behaviour of NATs.. Nope. Skype requires its subscribers to establish a VOIP server on the private LAN. This then connects out to some central Skype server. It may well be subverting company rules but it isn't magically defeating NAT or relying on undocumented features. If you think that then you just don't understand NAT.
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
Christian - 23 Apr 2007 14:36 GMT Martin Gregorie schrieb:
>> Martin Gregorie schrieb: >>>> Not wholly true. A properly constructed unsolicited packet that is [quoted text clipped - 30 lines] > relying on undocumented features. If you think that then you just don't > understand NAT. I rather think you don't understand holepunching.. this outbound server is the rendevouz server... it is needed to establish the connection (sometimes just for initial communication.. in the worstcase helping to establish a connection by spoofing its own ip and pretending to be the other party). But once the connection is established no data runs over this server.
Martin Gregorie - 23 Apr 2007 20:15 GMT > I rather think you don't understand holepunching.. > this outbound server is the rendevouz server... it is needed to > establish the connection (sometimes just for initial communication.. According to the Wikipedia article it doesn't do anything except receive connections from two clients and allow them to send messages to each other.
If you ignore the IM aspect, from a networking POV it really isn't doing anything that an SMTP server doesn't do.
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
Christian - 23 Apr 2007 23:26 GMT Martin Gregorie schrieb:
>> I rather think you don't understand holepunching.. >> this outbound server is the rendevouz server... it is needed to [quoted text clipped - 5 lines] > If you ignore the IM aspect, from a networking POV it really isn't doing > anything that an SMTP server doesn't do. quote from wikipedia - UDP hole punching: "The basic idea is to have each host behind the NAT contact a third well-known server (usually a STUN server) in the public address space and then, once the NAT devices have established UDP state information, to switch to direct communication hoping that the NAT devices will keep the states despite the fact that packets are coming from a different host."
don't mix hole punching with any kind of proxyserver... here the main traffic won't go over the server.. only some traffic for establishing the connection...
Christian
Z. - 23 Apr 2007 01:46 GMT >> Not wholly true. A properly constructed unsolicited packet that is >> sent to a consumer-grade router may be forwarded to a non-routable IP >> address on the private LAN.
> Without port forwarding there's no way a NAT-enabled router can know > what class C IP to forward the packet or connection to so the only > valid option is to drop the packet or connection. It can be done even w/o port forwarding turned on.
Mark Space - 22 Apr 2007 18:53 GMT > Hi > [quoted text clipped - 6 lines] > > All help is appreciated, thanks. What Steven said. But rather than IP addresses, you should assign names to the PCs, and use those instead. Usually, this is called the "host name" and it's the same name that you'd use connecting to load a web page or similar. If you're creating a connection based on a user loading a file to a web server, for example, you should use the URL they already have and extract the host name from that. For www.cnn.com/upload, for example, the host name would be www.cnn.com or cnn.com (because www might not know how to upload random files).
If your Java app is 100% stand alone, allow the user to type in the host name themselves, just like you can with a gui FTP program.
Speaking of gui FTP programs, why not just use one of those? All the hard work is done for you....
aca04pds@shef.ac.uk - 22 Apr 2007 20:06 GMT > If your Java app is 100% stand alone, allow the user to type in the host > name themselves, just like you can with a gui FTP program.
> Speaking of gui FTP programs, why not just use one of those? All the > hard work is done for you.... Well, unless I have misunderstood something, FTP programs require an FTP server which I don't have, nor is it feasable to set up one on every PC that might run this little program I am creating.
Anyone has mention IP address in the form of "192.168.x.y" has got the right idea. It is address like this, ones behind a personal router, that I want to connect to. If this isn't possible then I have a problem. Does anyone have any solutions? The easier the better.
Thanks
Martin Gregorie - 22 Apr 2007 21:48 GMT >> If your Java app is 100% stand alone, allow the user to type in the host >> name themselves, just like you can with a gui FTP program. [quoted text clipped - 10 lines] > that I want to connect to. If this isn't possible then I have a > problem. Does anyone have any solutions? The easier the better. How about using e-mail?
That way firewalls and NAT routers are not an issue because the destination(s) will fetch your message from their ISP's mail servers.
JavaMail is pretty easy to use, especially to send messages, and its easy to send the file as a MIME attachment. Get JavaMail from http://java.sun.com/products/javamail/index.jsp and make sure you grab both the ADI Design Specification (which includes useful example code) and the API documentation. You'll need the JavaBeans Activation Framework as well: http://java.sun.com/products/javabeans/jaf/index.jsp
 Signature martin@ | Martin Gregorie gregorie. | Essex, UK org |
Greg R. Broderick - 23 Apr 2007 18:15 GMT aca04pds@shef.ac.uk wrote in news:1177268794.163530.313070 @e65g2000hsc.googlegroups.com:
> Well, unless I have misunderstood something, FTP programs require an > FTP server which I don't have, nor is it feasable to set up one on > every PC that might run this little program I am creating. Use something like the FTP portion of Apache's commons-net component (http://jakarta.apache.org/commons/net/), which contains the underlying code necessary to construct both the client (sender) and the server (receiver) code.
Cheers!
 Signature --------------------------------------------------------------------- Greg R. Broderick usenet200704@blackholio.dyndns.org
A. Top posters. Q. What is the most annoying thing on Usenet? ---------------------------------------------------------------------
Mark Space - 24 Apr 2007 00:11 GMT >> If your Java app is 100% stand alone, allow the user to type in the host >> name themselves, just like you can with a gui FTP program. [quoted text clipped - 5 lines] > FTP server which I don't have, nor is it feasable to set up one on > every PC that might run this little program I am creating. Java will require that you set up a program, some program, also. There's nothing to receive the transfer request if you don't. Even the other comments about using Apache code will require some kind of server.
There is an FTP server built into most version of Windows. Have you looked into just enabling that?
How about Windows file sharing? Just turn that on and use Explorer. Any already working solution, whether it's FTP, HTTP or Windows, will be much easier to implement than trying to roll your own.
> Anyone has mention IP address in the form of "192.168.x.y" has got the > right idea. It is address like this, ones behind a personal router, Doesn't matter. Everything above should work still, unless your router is borked. And even then everything will still work if your router isn't involved (ie, all PCs are connected via a switch).
> that I want to connect to. If this isn't possible then I have a > problem. Does anyone have any solutions? The easier the better. FTP pretty easy, and supported by most web browsers. Windows file sharing harder on you, but probably easier on the users once it's going.
Michael - 24 Apr 2007 14:19 GMT On Apr 22, 3:06 pm, aca04...@shef.ac.uk wrote:
> > If your Java app is 100% stand alone, allow the user to type in the host > > name themselves, just like you can with a gui FTP program. [quoted text clipped - 11 lines] > > Thanks Set up a VPN or use an SSH tunnel. This requires an initiation from the host behind the NAT to some publicly accessible place. I don't deal with VPNs, but SSH-tunnels are trivial and work on all OS's.
While you may already be aware of how to do this.. You're basically taking a host accessible to the client and defining a host:port combination that is the proxy. All TCP/IP traffic then gets forwarded through the NAT to your target machine where the real server port resides.
The only problem is that your SSH connections need proper keep-alives - or at least detection which will restart the SSH connection (assuming a public/private keypair).
So in the ideal case, you have a publicly accessible host "pubHost" and a NAT host "natHost". natHost needs to SSH to pubHost (possibly via pub/priv keys) and establish a port only accessible to pubHost (e.g. not exposed to the world - which is the default in SSH) say port 5555 which forwards to natHost port 12345 (the server port). Now the client on pubHost connects to localhost:5555 and is really connecting to natHost:12345, but in an encrypted manner and passing through all firewalls.
If pubHost's proxied port was exposed to the world (instead of just localhost), then it had best be behind it's own firewall to restrict accessibility to an extranet. Otherwise it would be easier to just port-forward on the NAT router as others' have suggested.
And FYI, an FTP server is no different than running the java process listening on port 21 and utilizing the FTP protocol (which is rather trivial - I believe there are open-source java libraries that can do this). The only difference is that by default the FTP server likes using a pair of sockets which doesn't work at all with NAT boxes.
Michael - 24 Apr 2007 14:31 GMT On Apr 22, 3:06 pm, aca04...@shef.ac.uk wrote:
> > If your Java app is 100% stand alone, allow the user to type in the host > > name themselves, just like you can with a gui FTP program. [quoted text clipped - 11 lines] > > Thanks Are you talking about arbitrary servers? Meaning some anonymous person downloads your software, runs it, and it's supposed to act as a server? Good lord that would be horrible.. Consider all the personal firewalls which don't even allow arbitrary sockets to open - you'd have tech support nightmares.
Be that as it may, the only way to have a dumb-user get this to work, is to switch the direction of operation for server behind the NAT to client behind the NAT. Meaning the natHost connects to a central server and all connections are thereafter initiated by the server through the existing channel.
If you're trying to do a napster or bittorrent type deal, then what happens is that natHost1 connects to central server - there's nothing to do.. So it sends keep-alive or ping messages down the TCP channel. Later natHost2 connects to the central server, which says there's another host connected - i.e. central-server sends a hostID generated for natHost1 to natHost2.. Now natHost2 wants to send a file to natHost1 (or rather client-id-1), it sends a forward request containing raw file-data. The server forwards the request to natHost1 who then allows the download.
Now either the download can be sequential and the server has to block natHost2 until natHost1 is available (e.g. blocking natHost3 who wants to upload at the same time) or the upload request would require that natHost1 establish a new TCP connection to the server for the express purpose of receiving the file-upload (which presumably will take a while).
The parallel method is more complicated because the new connections have to be associated with the master command-channel. Also more things can go wrong. Plus you can DOS the poor natHost1 by giving it thousands of connections (or even 2).
Obviously the server node needs to be on a publicly available address. But you can even do this through a NAT, so long as you port- forward on your router and use dynDNS or some other such DNS remapper.
Obviously this is a bandwidth hog for the poor server as it has to both upload and download all data for all clients. But there is no other good way to reliably allow arbitrary NAT-hosted machines to talk to one another.
Brandon McCombs - 22 Apr 2007 19:51 GMT > Hi > [quoted text clipped - 6 lines] > > All help is appreciated, thanks. The router will use NAT which keeps track of which packets went out and knows then which packets need to go where when they come back in. But that only works when the PCs behind the router initiate the connection. From the outside you won't be able to address the PCs behind the router directly; that is the whole point of NAT (to hide the PCs behind the router). Those PCs will use non-routable IP addresses (192.168.xxx.xxx for example) so you wouldn't be able to address them because your router will assume you want to address something on your own network and won't let the packets get to the public Internet.
If a small business is your destination and a router is used that is not a personal one (and thus most likely isn't using NAT) then it is possible that the router has an IP and each PC has its own IP and you can address your packets using the PCs' actual addresses.
aca04pds@shef.ac.uk - 22 Apr 2007 20:07 GMT > If your Java app is 100% stand alone, allow the user to type in the host > name themselves, just like you can with a gui FTP program.
> Speaking of gui FTP programs, why not just use one of those? All the > hard work is done for you.... Well, unless I have misunderstood something, FTP programs require an FTP server which I don't have, nor is it feasable to set up one on every PC that might run this little program I am creating.
Anyone has mention IP address in the form of "192.168.x.y" has got the right idea. It is address like this, ones behind a personal router, that I want to connect to. If this isn't possible then I have a problem. Does anyone have any solutions? The easier the better.
Thanks
Karl Uppiano - 22 Apr 2007 21:10 GMT > Hi > [quoted text clipped - 4 lines] > each PC connected to that has its own IP for their local network. How > would the router know which PC is the right one to send the data to? If the LAN PCs are only running java.net.Socket (client sockets), you don't need to do anything, since the NAT router keeps a table of requests, and when the response comes back, the router automatically routes the response to the proper requester using Network Address Translation (NAT).
For any PCs listening on java.net.ServerSocket, the router knows which PC to send the data to using a router feature called "port forwarding". However, you would have to assign a different port to each server behind the router, because the router has a single address on the internet. Clients would have to specify the host name and port in order to select a particular machine behind the router.
Each PC on the LAN can listen on the same port, but you would need to configure the router to publish each PC on a different port on the WAN.
By the way, this is not a Java problem, this is a router configuration problem. Once you understand how NAT routers work, you can write the solution in any language that supports TCP/IP (usually using Berkley Sockets).
Karl Uppiano - 22 Apr 2007 21:26 GMT > Hi > [quoted text clipped - 4 lines] > each PC connected to that has its own IP for their local network. How > would the router know which PC is the right one to send the data to? Is this a peer-to-peer network, or do you have a central server? The latter would be much easier to manage.
In a peer-to-peer configuration, you might need to dynamically configure the router to open and close ports, and so on. Several standards exist for doing things like this, but it is far from foolproof. UPnP (Universal Plug 'n' Play) is one, but everyone would need routers that support the protocol. Furthermore, UPnP is widely regarded as a security risk, and enabling it is not recommended. Finally, you might discover that successfully configuring UPnP from your application is a serious development challenge in itself.
Depending on how loosely coupled your network is, a Virtual Private Network (VPN) would allow you to treat the entire collection of PCs as if they were on the same LAN, considerably simplifying your development problem. But, creating and maintaining a VPN might be problematic if the relationships between peers is casual or ad-hoc.
Christian H - 20 Jun 2007 03:12 GMT Google for "UDP Hole Punching"..
Mark Space - 20 Jun 2007 22:41 GMT > Google for "UDP Hole Punching".. Google for "Thread Necromancy"...
Christian - 20 Jun 2007 23:04 GMT Mark Space schrieb:
>> Google for "UDP Hole Punching".. > > Google for "Thread Necromancy"... something like that:
http://redwing.hutman.net/%7Emreed/warriorshtm/necromancer.htm
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|