Beacon probing resulting in excessive broadcasts

Not so long ago I posted a link to a VMWare blog in which beacon probing was demystified. This article stated that you only should use beacon probing when there is no link state tracking on the physical switches and you could consider beacon probing as a nice software solution for replacing it.

Well we’ve got our ESX environment set up by a local supplier and they advised us to use beacon probing instead of link state tracking. But for some reason beginning from that moment I got major events from my ProCurve switches stating excessive broadcasts. This wasn’t often, but especially during peak hours I was getting this notification.

When I started sniffing the network packets on the uplink of the switches I noticed what kind of packets it were. It was an almost continuous flow of RARP packets coming from the ESX servers. RARP packets are meant as MAC address table updates for switches. This way when a node is suddenly available on a different MAC address the switch already knows the new path. This is also what happens when a virtual switch detects a link is not functional. It will switch the uplink and notify the switches. When beacon probing isn’t working as expected ESX constantly thinks the uplink isn’t functional so it is constantly switching the uplink and as a result constantly sending out RARP packets.

While you can define both the notify switches parameter and the network failure detection, it isn’t good to just put notify switches to ‘No’. This way the failover is still constantly changing uplinks and this can result in errors (in my case timeouts with TFTP). The real problem was the failure detection. Beacon probing for some reason just didn’t work in our environment. When I changed the failure detection to ‘Link Status only’ all the RARP packets disappeared and my excessive broadcasts were gone.

In my case just link status only is sufficient, but I can imagine there are cases where you would want to use beacon probing. If you enable beacon probing and this results in excessive broadcasts (or just more broadcasts) I do advise to look if you could find those RARP packets. This can indicate that beacon probing is just not working correctly in your environment.

I want to thank Scott Lowe for giving me a push in the right direction.

Standard ESX networking tasks from command line (Part 2)

In part 1 I covered some NIC operations from command line. In part 2 I will cover some standard virtual switch tasks like adding and deleting a virtual switch and making and configuring PortGroups. So here goes for virtual switch operations…

Listing all virtual switches

esxcfg-vswitch -l

This commands gives you a list of all the configured virtual switches with their PortGroups and connected uplinks. Further more a lot of properties are shown about the vSwitches and PortGroups. For vSwitches it shows among other things the name, the uplinks, the number of used ports and the number of configured ports. For PortGroups it shows the PortGroup name, VLAN ID and uplinks.

Add a virtual switch called ‘TestSwitch1′

It’s really simple to add a virtual switch to an ESX server. You simply use the following command:

esxcfg-vswitch -a TestSwitch1

This creates a virtual switch with the name ‘TestSwitch1′. It still has no PortGroups and it has been set with the default amount of configured ports (64). To see all the properties use the command provided earlier to list the virtual switches. If you want to specify the number of configured ports you can use the following command:

esxcfg-vswitch -a TestSwitch1:16

This gives you a virtual switch named ‘TestSwitch1′ with 16 configured ports.

Standard ESX networking tasks from command line (Part 1)

As I was looking around in the command line interface (which is pretty new for me) I came around the esxcfg- command set. In particular the commands to manage the NIC’s (part 1) and the vSwitches (part 2) raised my interest. I decided to explore a bit further and write down how to do some standard actions. So here goes for NIC operations…

Listing all NIC’s

esxcfg-nics -l

This commands gives you a nice list of all the available NIC’s and all their properties. Those properties include name, link, speed, duplex and description.

Setting a specific link speed and duplexity of a NIC

The thing I want to do here is set my ‘vmnic3′ (the name I got from my previous command) to a speed of 100Mbps and I want to set it to full duplex. The command to do this is:

esxcfg-nics -s 100 -d full vmnic3

The ‘-s’ parameter defines the speed. This parameter can hold the values 10, 100, 1000 and 10000 respectively defining the speed to 10Mbps, 100Mbps, 1000Mbps and 10000Mbps.

The ‘-d’ parameter defines the duplexity. This parameter can hold the value ‘full’ for full duplex and ‘half’ for half duplex.

Setting link speed and duplexity of a NIC to automatic detection

To set my ‘vmnic3′ back to automatic detection I use the following command:

esxcfg-nics -a vmnic3

The ‘-a’ parameter simply sets the link speed and duplexity of the NIC back to automatic.

I hope this was useful to someone. At least I got better understanding and a little reminder for myself how to do these things. In part 2 I will cover some standard networking tasks considering virtual switches using the command esxcfg-vswitch.

Beacon probing explained

I was browsing through the VMWare blogs and I ran into the VMWare networking blog. They posted a really nice article about when to use beacon probing. Although it was already posted on the 10th of december I thought it was worth mentioning.

Beaconing is one of those features that often confuses even the most experienced networking admin.

Shudong Zhou, one of our senior engineers, recently posted an entry on the internal blog explaining how it works and how you might use it. He gave me permission to cut and paste his entry. Here it is …

Read the rest of the article here at the VMWare Networking Blog