Wednesday, September 28, 2016

DHCP renewal, dynamic DNS, and changing network cards

Another adventure that started with a simple and quick plan and then felt into the rabbit hole! If you did not get the hint, this is one of those examples of why proper documentation is important. Or, what happens when I rely on my memory...

I built a windows VM inside VMWare ESXi a while ago called dr-zaius. When I did the deed, I setup it to use an emulated Intel E1000-series ethernet card. Fast forward and I learned VMWare recommends to use their native NIC, vmxnet3. I am fine with that since that is what I do in KVM, namely use the native interface to the hypervisor and it does result in increased performance. So let's go over the steps for doing the deed.

Note: The following assumes you have the rights to edit a vm client's configuration in ESXi. If not, just contact who does and explain what you want to accomplish.

The Steps

  1. We are going to be lazy and use the vsphere client. Which we will be using while the vm is up and running. When we select the Edit Virtual Machine Settings option, we have the option to add an additional interface, so let's add it and later on delete the old one.

    By default in this version of the vsphere client, we are offered the E1000E emulated interface. And that is not what we want so we need to change the interface as shown on the right (they are calling it an adapter) and select the vmxnet3 one. Don't forget to select the proper VLAN -- Network Label in this dialog box -- while you are there. Also notice by default it will be connected at power on, which is what we want in this article. And then hit the Next button until it finally gets to the screen saying the deed is done.

  2. As soon as we do that, login to the windows machine via console. It will tell us it detected a new device and will start with its driver installation dance. Once that is done, this vm client will end up with two network interfaces. Now we need to set it up.

  3. The best thing to do is disable the old interface -- you are accessing it from console -- inside the vm client, just as you normally do in a physical host. If you use static IP, you might want to delete the IP or change it to a bogus address so the other interface will not get pissed (Windows does like to be very helpful).

  4. Now configure the new interface, be it to use static IP or DHCP. In this example we will be using DHCP.

  5. After testing everything works, we can then shut the vm client down at our convenience and remove the old network interface using the vmsphere client.

And that should be the end of this article. Maybe in another article we can talk about doing the same deed in KVM (and whatever else I get my hands on).

But things did not happen according to the plan

I am not very good with words, so let me just show it:

raub@desktop:~$ ping dr-zaius
PING dr-zaius.in.example.com (192.168.42.105) 56(84) bytes of data.
From desktop.in.example.com (192.168.42.102) icmp_seq=1 Destination Host Unreachable
From desktop.in.example.com (192.168.42.102) icmp_seq=2 Destination Host Unreachable
From desktop.in.example.com (192.168.42.102) icmp_seq=3 Destination Host Unreachable
^C
--- dr-zaius.in.example.com ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3036ms
pipe 3
raub@desktop:~$

But when I login to dr-zaius from a console, I get the IP for its new interface is 192.168.42.126, which seems to work fine when I probulate using netcat:

raub@desktop:~$ nc -v 192.168.42.126 3389
Connection to 192.168.42.126 3389 port [tcp/*] succeeded!
^C

raub@desktop:~$

Er, what is going on? So, when I went to the dhcp/dns server (nameserver) to check. It uses the ISC DHCPD and BIND (I never knew it was an acronym until I wrote this article. Who said you can't learn anything in the web?) to do its thing, and they are configured to do dynamic DNS. So, we cannot just grab the zone files

[root@nameserver ~]# ls /var/named/data/in.example.com.*
/var/named/data/in.example.com.db
/var/named/data/in.example.com.db.jnl
/var/named/data/in.example.com.rev.db
/var/named/data/in.example.com.rev.db.jnl
[root@nameserver ~]#

and edited them. The telltale hint we are doing some dynamic DNS'ing is the journal (.jnl) files. Let's first stop the DNS updates in both forward and reverse zones.

rndc freeze in.example.com
rndc freeze 42.168.192.in-addr.arpa

We now can take a look at the zone files. Here is the entry for dr-zaius in the forward zone file:

dr-zaius                A       192.168.42.105
                        TXT     "316335b72f00f1bf82eb484894d5263cdf"

I do not know about you but it sure seems like it still thinks dr-zaius's IP is ,tt>192.168.42.105. so, we delete those two lines (I like to also update the serial entry on the top of the file just in case). The reverse zone is a bit easier as it has only one line

126                     PTR     dr-zaius.in.example.com.

to be deleted. Once those are done, we can let dynamic DNS take place in that zone again.

rndc thaw 42.168.192.in-addr.arpa
rndc thaw in.example.com

The log seems to show that it is now properly associating the hostname to the IP

Sep 23 13:10:47 nameserver named[1536]: client 192.168.55.10#48383: updating zone 
'in.example.com/IN': adding an RR at 'dr-zaius.in.example.com' A
Sep 23 13:10:47 nameserver named[1536]: client 192.168.55.10#48383: updating zone 
'in.example.com/IN': adding an RR at 'dr-zaius.in.example.com' TXT
Sep 23 13:10:47 nameserver dhcpd: Added new forward map from dr-zaius.in.example.com 
to 192.168.42.126
Sep 23 13:10:47 nameserver named[1536]: client 192.168.55.10#51133: signer 
"dhcpupdate" approved
Sep 23 13:10:47 nameserver named[1536]: client 192.168.55.10#51133: updating zone 
'42.168.192.in-addr.arpa/IN': deleting rrset at '126.42.168.192.in-addr.arpa' PTR
Sep 23 13:10:47 nameserver named[1536]: client 192.168.55.10#51133: updating zone 
'42.168.192.in-addr.arpa/IN': adding an RR at '126.42.168.192.in-addr.arpa' PTR
Sep 23 13:10:47 nameserver dhcpd: added reverse map from 126.42.168.192.in-addr.arpa. 
to dr-zaius.in.example.com.
Sep 23 13:10:47 nameserver dhcpd: DHCPREQUEST for 192.168.42.126 from 
00:0c:29:84:95:8c (dr-zaius) via eth0
Sep 23 13:10:47 nameserver dhcpd: DHCPACK on 192.168.42.126 to 00:0c:29:84:95:8c 
(dr-zai$s) via eth0

So let's try again. nameserver seems to be able to resolve the hostname and IP:

[root@nameserver ~]# nslookup 192.168.42.126
Server:         127.0.0.1
Address:        127.0.0.1#53

126.0.0.10.in-addr.arpa name = dr-zaius.in.example.com.

[root@nameserver ~]# nslookup 192.168.42.126
Server:         127.0.0.1
Address:        127.0.0.1#53

126.0.0.10.in-addr.arpa name = dr-zaius.in.example.com.

[root@nameserver ~]#

Now we go back to the desktop and try again

raub@desktop:~$ nslookup dr-zaius
Server:         192.168.4.1
Address:        192.168.4.1#53

Non-authoritative answer:
Name:   dr-zaius.in.example.com
Address: 192.168.42.105

raub@desktop:~$

What is going on here? I thought I had taken care of this! And who is this 192.168.4.1? nameserver is at 192.168.55.10! We probably need to see which DNS servers desktop is using

raub@desktop:~$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.4.1
nameserver 192.168.55.10
nameserver 192.168.4.1
search in.example.com mgmt.example.com
raub@desktop:~$

A Candle just lit above mt head! Now things make sense! You see, desktop is dual homed in both 192.168.42.0/24 and 192.168.4.0/24 networks, the latter is called mgmt.example.com. desktop also gets its IPs on both networks using dhcp and as a result its nameservers. I do not know why /etc/resolv.conf has two entries for 192.168.4.1, the nameserver as seen in 192.168.4.0/24, but the point is that it is the first nameserver. And, this is not an IP used by the real nameserver. You see, the Juniper router that is doing all this routing is configured to be a forwarding DNS server for a few networks, 192.168.4.0/24 included. And it seems it has not updated its cache. Let's see if I am right:

raub@desktop:~$ nslookup dr-zaius 192.168.4.1
Server:         192.168.4.1
Address:        192.168.4.1#53

Non-authoritative answer:
Name:   dr-zaius.in.example.com
Address: 192.168.42.105

raub@desktop:~$ nslookup dr-zaius 192.168.55.10
Server:         192.168.55.10
Address:        192.168.55.10#53

Name:   dr-zaius.in.example.com
Address: 192.168.42.126

raub@desktop:~$ 

So, all I have to do is wait a bit for the router to catch up.

I feel better now

PS: if you feel that I brushed over the setup of DHCPD and BIND, or how to configure a Juniper router, you are quite right. Those topics, while interesting, were not what we are trying to solve here. And what is that since it seems changing the Windows vm client interface in VMWware is not it? Actually, it is. But things sometimes do not happen according to the plan and we have to figure out what is going on. In this case, the culprit was DNS lease renew times at the DNS primary and forwarding (the router in my example) servers. If you think every time I set it up it is all rainbows and unicorns and things work perfectly at the first time, I have some news for you sunshine.

No comments: