Update: I found that my core switch was the problem. Replaced, and it’s working.
Switch is a generic 24-Port Gigabit network switch, no routers between hosts, no gateways except to exit my subnet and to the Internet.
I currently have a physical machine running Windows 2003 R2 Standard, running as a small DHCP, DNS, and AD Server.
I have two pysical machines running ESXi Hypervisor. On one of the ESXi servers I am running four virtual machines. They work fine. On the other I just setup a single virtual machine for now, and it is running Windows 2003 R2 and RIS+WDS (WDS Mixed Mode).
I then boot a laptop on the same network, hit F12 for network boot menu in the BIOS choose Network and hit enter, the laptop gets a DHCP address and then tries to Forward me to the ProxyDHCP running on the WDS server. It fails, “WDS Service could not be found.” Then it prompts me to hit F12 to actually start the RIS+WDS Menu, hit that, and it crashes, cannot find \Boot\BDE.
I then setup a VM on the same ESXi host as the WDS server, boot up a new VM and hit F12, instant gets DHCP, no forward to ProxyDHCP bullshit and hit F12 again to start menu. Works great.
I then setup a VM on the opposite ESXi host as the RIS+WDS server. Same results as the laptop.
Seems to only work within the same vSwitch. So I changed out network cards as it was easy, and no huge configuration changes. Same results.
I then did the following on the Windows 2003 R2 DHCP server:
“Running WDS in a mixed mode configuration, and after reading everything needed to add PXEClient to a domain controller running the DHCP service in Windows 2003 R2.”
This post helped out quite a bit: http://felixyon.blogspot.com/2010/12/remote-installation-service.html
Did nothing, and I have since removed the option.
I then decided to take ten minutes and swap out the core switch that I am using, dropped in a Cisco I have lying around and cleared the config, setup as a flat switch same as the other, tried the laptop, instant DHCP, no ProxyDHCP loads menu, done.
Tried the VM that was not working, instant done. Tried the VM that was working, still is.
It seems that this switch is blocking DHCP requests after the first has passed and will not allow clients that just got a DHCP address to hit another DHCP server to get PXE details.