One of my blog readers recently experienced an uncommon issue that almost grounded his network. After two days of troubleshooting, the fault was traced to an uncommon component- a poe adapetr. To help reduce the troubleshooting time for those that may find themselves in a similar situation, he has decided to share his story with us.
Hello Timi, many thanks for the good work that you have been doing. I discovered your blog via Google search about four months ago. Since then, I have become a regular visitor. I have followed your posts on Mikrotik, and Cisco. I have also learned from the experiences of other blog readers as shared in the Tech-story section of this blog. To help contribute to the ecosystem and ensure that knowledge is passed around, I have decided to also share my recent experience.
My name is Kelly Collins, I am a field service engineer (FSE) with an internet service provider in Lagos State, South West, Nigeria. We make use of Mikrotik products for the delivering of internet services to cooperate and home customers. This is done in point-to-point or point-to-multipoint mode. We have two points of presence (POP). Let me refer to them as POP-A and POP-B. We installed a backhaul PTP link to feed POP-B from POP-A. From POP-B, we service customers in that areas via the many access points (APs) installed on the mast. Everything was good until recently.
The problem
We observed that pings to a customer premises equipment (CPE) connected to an AP at POP-B returned with high latency. Since this observation was made based on the complaints from a client on a Point-To-Point link, we initially thought it could have been caused by interference and believed that a change in frequency would resolve the issue. After changing the frequency on the AP, the experience remained the same. Further troubleshooting by running pings to all CPEs serviced from same POP revealed the issue was applicable to all customers. Hmmm. Good! It must be an interference on the backhaul Point-to-point link from POP-A.
From POP-A, we ran ping tests to the backhaul station at POP-B and observed the experience was also the same. With this, it was certain that the problem was between the two backhaul radios. So, we decided to change the frequency on the AP. Surprisingly, the experience did not change after alternating between different frequencies on the 5Ghz band. What then could be the problem? At this point, we decided to test from the same LAN switch to the connecting ether1 port on the AP at POP-A. The AP and the test PC were both connected to the same switch via ethernet cables. Unexpectedly, the fault lied between the switch and the AP as ping results recorded high average latency of 78ms instead of 1ms.
Having narrowed the fault down to between the LAN switch and the AP, any of these four components could be the culprit – the switch port on which the AP is plugged, the POE adapter that powers the AP, the cable connecting the AP to the LAN switch, or the access point itself.
The Solution
The fault correction process was carried out from the easiest task to the toughest. Since switching the port was the easiest of the four, it was done first. However, this did not resolve the issue. The next step was the replacement of the 24V POE adapter. After this was done, the issue was resolved. The pings normalized to 1ms and browsing experience improved as a result.
Though the POE could power the AP and allow the passage of packets through its LAN port, for some inexplicable reasons, it impacted negatively on the speed of these packets and at some points, resulted in packet losses.
I hope my story was a good read. Thank you.
Thank you, Collins for sharing. If you are reading this, you owe us your tech story. To send in yours, simply email it to timigateng @gmail.com.
If you enjoyed this tutorial, please subscribe to this blog to receive my posts via email. Also subscribe to my YouTube channel, like my Facebook page and follow me on Twitter.
I confirm your diagnosis because I recently faced the same situation, where I noticed packet losses on the rest of the network, along with a temporary disconnection every night at the same time, specifically at the access point powered by the faulty POE.
The solution was the same as the one mentioned in your tutorial, which was to replace the defective POE with a new one. After doing that, I made sure to prioritize the diagnosis of POEs in the network in case of any future failures.
It would have been easier if these POEs had an interface allowing for advanced diagnostics and early anomaly detection.