Problem with VLAN Tagging on Server 2012? Say the Magic Word
The magic word is… VlanFiltering. But more on that later…
I’m working on setting up a new Hyper-V cluster at work, and it has been a slow process. Not because of the technology, but rather because I’m lucky if I get 10 minutes a day to work on it. Everyday I go into work thinking, “Today’s the day! I’m going to shut my door, close Outlook, and get it done!” Fast forward eight (or nine… or more) hours, and there I am leaving the office thinking “Maybe tomorrow.” Projects tend to drag on when you’re getting pulled in fifty different directions. But I digress…
The important part to know is that I’m setting up a couple of Hyper-V hosts (Hyper-V Server 2012 R2 mind you, none of this Windows Server with the Hyper-V role for this guy), and I was setting up my logical switches via System Center Virtual Machine Manager 2012 R2 (VMM). I’ll skip over the issue I ran into with the Management switch and virtual adapter and save that for another day. Once I worked around that problem, I was ready to add my converged switch. The converged switch sits atop a team of two dual-port Broadcom BCM57810 10 GigE NICs. That team connects to a Port Channel on a Cisco Nexus 5000 which is trunking several VLANs including my Live Migration and Cluster CSV networks for the host as well as iSCSI and client VLANs for my guests. I added the logical switches, hit OK, and watched as everything was configured on the hosts.
It was a grand success. Except for the part where I couldn’t pass traffic between the hosts on any of the VLANs. It became clear that the VLAN tags on the traffic were getting dropped and everything was dumped on the default VLAN. I spent the next week poking around, trying to figure out what would cause this problem with little success. I can’t count the number of times I would add the switch, tear it down, add it again, and so on. Online searches weren’t bearing any fruit either. Nothing I found seemed the least bit relevant.
Enter Dinko Fabricni and his blog, IT Solution Braindumps. Back in 2012, Dinko posted an article, “VLAN tagging problems in Windows Server 2012 Hyper-V”, that described my problem and provided the solution for which I had been hunting. It turns out my Broadcom 10 GigE NICs (but not my 1 GigE NICs) were set to filter VLANs by default. To get them to pass the VLAN tags I had to add a registry key to each adapter.
The fix is to add the registry value VlanFiltering (type DWORD) to the following location and set it to 0.
HKEY_LOCAL_MACHINE\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\00xx
where xx is the number for the NIC in question. There are a lot of registry keys under that GUID, so make sure you find all of your NICs because you will need to change it on every NIC in the team. Make that change and restart, and it all starts working. It did for me at least.
This isn’t a new setting. Searching on the term shows references to it going back years, but it was never something I needed to set before. I expect that is probably because now I’m using the native NIC teaming that was introduced in Server 2012. Before I was using the Broadcom utilities to configure teams and VLANs, and I guess it was taking care of that setting itself.
So let me thank Dinko Fabricni and his blog post for saving my sanity and getting me back on track to finish this project.