I had the most interesting case yesterday.
What users said: once in a while users could not use some HP printers (4 were affected)
What a local IT expert said: printers do not wake up after a period of sleep
I took over the case and got immediately lucky yesterday: The local IT guy called me and said: one of the printers is not working now.
I went to the web interface of the printer using the printer’s IP and found that although it was in the sleep mode, it was functioning fine. I set a sleep/wake schedule and the printer woke up after a minute, but still the users couldn’t print anything. The printing queue was empty for that day although someone printed something a day before.
I asked the user if he could ping the printer. He could ping the ip address but no longer its FQDN.
My next step was to go to the DHCP server (or rather the IPAM system managing all DHCP servers) to have a look. I discovered the following:
The mac address of the printer was mapped to the correct IP address (the one I was using to log in to the printer) and the correct fqdn,eg.
aaaa.bbbb.cccc 10.60.60.10 printer1234.acme.us (10.60.x.x being a printer vlan)
this was fine.
However, there was an additional entry:
dddd.dddd.eeee 10.10.20.10 printer1234.acme.us (this is a data vlan for normal PCs)
And the log said that this DCHP lease was given two hours before and released 5 minutes later. Someone was pretending to be a printer!
But who was it?
A quick look at our Cisco prime revealed that it was… a wifi user in a different country. dddd.dddd.eeee is the mac address of some guy’s wifi card in Germany. To make the matter even more abstruse, Cisco Prime was showing that the user was moving every 10 minutes between Germany and the Czech Republic (=was associated to APs that are physically in different countries).
Of course, such things are never possible unless he can do bilocation or if he owns a super speedy plane (about 6000km/h). But jokes aside, what was really happening?
I had a look at the two access points:
AP1 was in the flexconnect group Czech Republic connected to controller1. This controller is used for flexconnect APs in the branches of ACME.
AP2 was in a local mode connected to controller2. This controller is usually used for local APs in the HQ.
Next, I tried to locate both APs physically using their mac address. It turned out that the Czech Republic is now located on floor 1 of building 10 in a nice town in central Germany 🙂
So what had happened here?
Someone connected a misconfigured access point to a switch access port, thus broadening the data vlan and enabling any employee in the area to connect and get an IP address from the data range 10.10.x.x. It is important to note that wifi users should get an IP address from range 10.200.x.x instead.
The user was walking around in building 10 in the headquarters in Germany with his 10.200.x.x wifi IP address. However, he suddenly moved into the range of the misconfigured AP which gave him 10.10.20.10 address. This address was mistakenly (mistake no 2) mapped to the printer1234.acme.us FQDN.
The moment the user got his 10.10.20.10 address (an address he should never get unless travelling to the Czech Republic office), he ”took over” the printer’s fqdn because the DHCP server now thought that this user is now printer1234.acme.us