Thursday, January 23, 2014

Internal server touching itself from behind another servers/router's external IP

Before I start, I have to say I saw this happening using virtual machines where the network 192.168.1.0/24 was also virtualized, but not when both machines were physical ones. Your situation might be different. YMMV, use with care, this side up, rauchen verbotten. Now that's out, let's say we have one of the following networks:

 {Internet}                     OR       {LAN}
   |                                       |
   | 1.2.3.4 (external IP)                 | 1.2.3.4 (LAN IP)
 [ Router ]                            [ serverA ]
   | 192.168.1.1                           | 192.168.1.1
   |                                       |
   |                                       |
   | 192.168.1.10                          | 192.168.1.10
 [ serverB ]                           [ serverB ]

It is really the same thing for all practical purposes: one machine in an internal network behind another. In any case, we want to reach port 1234 in the internal machine (serverB) from the network outside the router or serverA. So, we can go to the router/serverA do a simple port forwarding iptables rules using DNAT:

iptables -t nat -a PREROUTING -d 1.2.3.4 -p tcp -m tcp --dport 1234 -j DNAT --to-destinaton 192.128.1.10:1234

But, let's say serverB also wants to connect to the same port? Well, the easiest way it to connect to localhost:1224, but what if we want to run the same script that external machines run, which then connect to 1.2.3.4:1234? It might not be what we want:

raub@serverB:~$ nc -v localhost 1234
Connection to localhost 1234 port [tcp/https] succeeded!
^C
raub@serverB:~$ nc -v 1.2.3.4 1234
nc: connect to 1.2.3.4 port 1234 (tcp) failed: Connection timed out
raub@serverB:~$ 

In broad strokes, here is what is happening: ServerB want to connect to 1.2.3.4, a non-local address, so it sends the packet to it's gateway (router or serverA in the diagram). Gateway then assembles its response, which gets put through router/serverA's routing table. Now serverA looks at the IP and realizes that it's a local one, so it sends the response directly to serverB. But serverB is expecting the response from the outside IP, so it does not connect (topplewagon in #centos wrote a better explanation). The solution is mentioned in the session Destination NAT Onto the Same Network of the Linux 2.4 NAT howto: create a SNAT rule:

iptables -t nat -A POSTROUTING -d 192.168.1.10 -s 192.168.1.0/24 -j SNAT --to-source 192.168.1.1