My name is Tomek De Wille. I’m the owner of humanITy, a small IT training shop for all those of you who want to change their lives and believe they can.
I used to be a language teacher but in 2009 I had a sudden change of heart. I knew I could do more than teach. I joined Nokia as a technical editor and started learning about networks. Fast forward to 2017 and i’m a senior network engineer, troubleshooting networks all around the globe. Because I still love teaching, i’ve created humanITy – a home for anyone who wants to become an IT expert. We run IT courses on Saturdays and Sundays in the beautiful city of Wroclaw, the very heart of Europe.
For a very brief (and very true) explanation of why we founded humanity, see this 3minute youtube video
If you speak English and would like to start a successful career in IT, have a look at our site http://www.humanity.pl
Follow our blog for discount codes, information on new and exciting IT courses and much, much more.
Today I came across an interesting case, where two out of three tunnels on one spoke (out of 50 spokes in total) were down. The third (working) tunnel was connected to the same two hubs but via mpls.
- tunnel configuration. It was identical in all other 40+ spokes
- crypto isakmp and ipsec policies. Again, identical.
- state of crypto: both ISAKMP and IPSEC were up, encrypting and decrypting tunnels
So now it was up to GRE and NHRP to do their job of establishing DMVPN.
When I checked the state of dmvpn on the hub, it showed UP, however, the spokes were showing their state as NHRP. What does it mean? After doing NHRP debug I knew that the spoke wasn’t getting the resolution reply for the two bad tunnels.
Finally, I saw that there were 2 NAT entries for GRE traffic. It was strange, because the traffic was sourced by the same host on the LAN and the destinations were (in both cases) the dmvpn hub.
I looked at the nat translation statement: it was a nat overload to the outside (public) interface.
As GRE doesn’t cooperate well with PAT, I decided to clear the entries, especially that the the source host was Incomplete in the ARP table.
After clearing the nats from the ip nat translation table, the tunnels went up.
What does it teach us? We should make a deny statement for GRE traffic in our NAT ACLs and only use TCP and UDP in nat statements rather than IP.
I’m wondering if the same issue will crash my DMVPN lab or not.
We have the following scenario:
IOU 4 has a static route 10.1.9.9 to IOU 9. It redistributes it into OSPF. IOU4 has OSPF adj to IOU5, IOU5 to IOU2.
IOU2 is a BGP neigh to IOU5 and also to IOU4. IOU4 and IOU5 are route reflector clients of IOU2.
The problem starts when we enable BGP synchronization on R5. What do we know about synchronization? the condition is the route in BGP needs to match the same route in IGP to avoid black holes. where R1>R2>R3 but BGP is only enabled on R1 and R3. R2 causes the black hole.
But having the routes match in BGP and IGP is not the only condition, i’m afraid… let’s have a look at this case:
R5 receives the ospf route to IOU9 from IOU4, but it receives the BGP route to IOU9 from IOU2! The route from the internal routing protocol will not get synchronized with the same route in BGP.
The only fix is to change bgp router-id on IOU2 to be the same as OSPF router-id on IOU4.
I’m doing the labs again and so many things are not working out of the box. Today I configured three routers in an NSSA area and one of them insisted on being the translator. The problem was that it wasn’t an ABR. It thought mistakenly that it was connected to the backbone but show ip ospf topology showed status of its backbone area as INACTIVE.
The solution (as always) is to delete the route config and type it in again.
I wonder if the lab images are just as buggy during the exam.
A while ago I considered implementing VTP with pruning on a client network. But it’s a bad idea because of the following (from Cisco documentation)
(In vtp v3) VTP pruning still applies only to VLANs 1 to 1005, and VLANs 1002 to 1005 are still reserved and cannot be modified.
VTP pruning is not designed to function in VTP transparent mode. If one or more switches in the network are in VTP transparent mode, you should do one of these:
Turn off VTP pruning in the entire network
Turn off VTP pruning by making all VLANs on the trunk of the switch upstream to the VTP transparent switch pruning ineligible.
There is a funny cornercase scenario related to this: if you have switch A>switch B>switch C, where switch B is a transparent vtp switch while A and C have vtp enabled with pruning, switch C may request (through B) to prune a vlan. Switch A will prune the vlan from the trunk to switch B (which may have needed it!)
In other words: avoid…
After a short break we will be starting with new courses in September 2018:
- Introduction to IP networks (Tuesday afternoon)
- Introduction to IT / Comptia A+
Any questions? Go to our website humanity.pl or call us on 512056971 to talk to us about the courses.
There will be plenty more to come as we approach the new school year so make sure to visit our website once in a while.
I’ve had a spate of cases recently where not even a single command was needed to solve the problem.
Today a client called me about weak/missing wifi signal in a conference room on floor 2. The problem was that i didn’t have any map so i didn’t know what i was looking for. We called a person onsite who was heading to that building. At the time that person was sure that there was (or had been) an access point there.
On arrival, it turned out that the access point in the hallway near the conference room was nowhere to be seen. What the technician did see was a brand new metal ceiling. Apparently that new metal ceiling was fixed up directly underneath the old ceiling. The access point seemed to be trapped between the two ceilings.
When I asked the technician to connect to wifi, i saw his mac address on another AP on floor 1 instead. It seems that weak signal from floor 1 was leaking to floor 2? Again, we had no way of confirming where both APs were.
Summing up, what do you need to solve cases like this:
- confidence to ask the right question to the right person even if this means that that person needs to walk/drive to the site
- preferably a map with rooms and APs marked
- if you have Cisco Prime then you’re golden because you can see AP channel utilization stats etc. etc.
I had an interesting problem today. The customer has two lines: MPLS and internet with dmvpn running on both.
Suddenly the voice quality dropped, and we saw that the tunnel over MPLS is down.
The route to the DMVPN hub goes through the mpls line with the IP of the CE router being 172.16.0.1
However, the arp entry for 172.16.0.1 was incomplete. Clearly the line was faulty.
A ticket to to the telecom was raised but the answer was unexpected: you haven’t been using the line for the last seven days.
I was like: what do you mean we’re not using the line! we’re not using the line because it’s DOWN, that’s what it is…but i began to think about it: in every answer there’s a grain of truth…
So i pinged the broadcast 172.16.0.255 and VOILA – 172.16.0.2 responded. But what is 172.16.0.2??? Let’s experiment since the line is down anyway
I changed the static route to go through the .2 address and of course the line was working again.
It turned out that the telecom replaced the line a while ago and changed the ip address of the CE router.
Now my point is here: i only used logic and one simple ip route command. No big deal, yet nobody else has come up with the idea to look for other addresses on the subnet. This is great new for beginners in the IT world: keep your mind open, be on the ball and even limited knowledge can bring great results.