NIC confusion when cloning virtual machines in ESX5i

A few twitter posts have already indicated that I have a new machine in my study-a Supermicro server with the H8DGi6-F motherboard, 2 12-core AMD 6238 processors and 32 GB RAM. And since I have been given a vSphere 5 Enterprise Plus license, what better to test than this combination! I have to say the installation of ESX5i is very straight forward: get the ISO image, burn it, install it. Done-and ESX5i has support for the AMD 6200 processors built in, which suffer from L1 cache invalidations in certain situations.

I have to say that ESX5i is a data center product: I haven’t been able to attach a USB hard disk to it in order to copy my ISO images. Apparently there is a way for ESX 5i (and 4i) to mount vfat formatted USB disks to automatically mount the devices as “NO NAME” but I haven’t been able to get this to work.

Also, the networking is a bit different from xen which I have used extensively before. But it’s all logical and not an issue once you understand it. My first virtual machine is an Oracle Linux 6.2 infrastructure host. Cloning a VM is actually quite simple-in the vSphere client you browse the datastore, create a new folder and copy source into destination folder. Then you right-click the VMX file and “add to inventory”: done!

The first time you start the cloned VM however there is a problem: the MAC addresses for the machine will change after you tell vSphere that you “copied” the virtual machine. This initialises the VMX file with a new UUID and MAC addresses for the VMs. The same is true for a cloning operation in Virtual Box for example. New MAC address will prompt kuduzu to consider the NICs as new devices. In my case, the NICs were eth0 and eth1 on the source, and subsequently ended up being named eth2 and eth3 in the clone-and no eth0 and eth1. Whoops!

But that’s really, really simple to change. In you VM settings, note down the MAC address for each NIC you have, and then open the file /etc/udev/rules.d/70-persistent-net.rules. You may see something like this:

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:5d:d9:10", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:5d:d9:06", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:e2:8f:58", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:e2:8f:4e", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

You can see where this is going… I simply copied and pasted the new MAC addresses into the old udev rule and removed the new ones, as shown here (remember to back-reference to the settings you just wrote down)

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:e2:8f:58", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:e2:8f:4e", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

Now all I needed to do was run system-config-network to update /etc/sysconfig/network-scripts/ifcfg-eth{0,1} with the new MAC addresses and start the network. Result:

[root@ol62infra ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:E2:8F:4E
          inet addr:192.168.0.14  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fee2:8f4e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:54 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7818 (7.6 KiB)  TX bytes:1669 (1.6 KiB)

eth1      Link encap:Ethernet  HWaddr 00:0C:29:E2:8F:58
          inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fee2:8f58/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:257 errors:0 dropped:0 overruns:0 frame:0
          TX packets:193 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:26218 (25.6 KiB)  TX bytes:25039 (24.4 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

See, cloning isn’t that hard after all!