Re: C11118p G0/0/1 bounces when G0/0/0 does down.

mcotherman · ‎06-24-2024

I have a C1111-8PLTEEAWB that is connected to a primary ISP cable modem via Gi0/0/0 and has a default gateway static route. I connect a Verizon Inseego M3100 5G hotpot configured in IP Passthrough mode to Gi0/0/1 and set for DHCP and put a floating static default route in the config. (ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0/1 dhcp 200)

When the cable modem loses connectivity because of an outage, in their network, I disconnect the cable, the power, or the ethernet cable, Gi0/0/1 goes down and then back up. This happens with 17.6 and 17.9 and also with a fresh minimal configuration entered after a wr-rel, no special commands or features are used.

The Inseego M3100 on Verizon connects and performs flawlessly at 500Mbps/100Mbps when the cable is moved to a Win10 PC.

Here is a console port output when I shut down Gi0/0/0 manually

C1111-8PLTEEAWB#
C1111-8PLTEEAWB#
C1111-8PLTEEAWB#
C1111-8PLTEEAWB#
Jun 21 09:40:13.329: %LINK-5-CHANGED: Interface GigabitEthernet0/0/0, changed state to adm
inistratively down
Jun 21 09:40:14.329: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/0,
 changed state to down
Jun 21 09:40:32.417: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:40:35.994: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:40:36.003: %IPRT-6-STATICROUTESACROSSTOPO: Cannot install inter-topology static
route 0.0.0.0/0 topology base on outgoing interface GigabitEthernet0/0/1
Jun 21 09:40:43.820: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.65.37.110, mask 255.255.255.252, hostname C1111-8PLTEEAWB

Jun 21 09:40:53.412: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:40:56.990: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:41:04.817: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.120.210.236, mask 255.255.255.248, hostname C1111-8PLTEEAWB

Jun 21 09:41:17.421: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:41:21.003: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:41:21.011: %IPRT-6-STATICROUTESACROSSTOPO: Cannot install inter-topology static
route 0.0.0.0/0 topology base on outgoing interface GigabitEthernet0/0/1
Jun 21 09:41:28.848: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.84.164.172, mask 255.255.255.248, hostname C1111-8PLTEEAWB

Jun 21 09:41:39.418: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:41:42.999: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:41:50.952: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.79.143.119, mask 255.255.255.240, hostname C1111-8PLTEEAWB

Jun 21 09:42:09.412: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:42:12.994: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:42:13.008: %IPRT-6-STATICROUTESACROSSTOPO: Cannot install inter-topology static
route 0.0.0.0/0 topology base on outgoing interface GigabitEthernet0/0/1
Jun 21 09:42:20.948: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.121.49.17, mask 255.255.255.252, hostname C1111-8PLTEEAWB

Jun 21 09:42:31.410: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:42:34.995: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:42:42.944: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/1 assigned DHCP
address 100.100.181.150, mask 255.255.255.252, hostname C1111-8PLTEEAWB

Jun 21 09:43:01.409: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:43:04.986: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to up
Jun 21 09:43:04.996: %IPRT-6-STATICROUTESACROSSTOPO: Cannot install inter-topology static
route 0.0.0.0/0 topology base on outgoing interface GigabitEthernet0/0/1
Jun 21 09:43:08.413: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/1, changed state to down

Jun 21 09:43:09.414: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/1,
 changed state to down
C1111-8PLTEEAWB#
C1111-8PLTEEAWB#

CCIE# 6771

MHM Cisco World · ‎06-24-2024

Try disable ipv6 in port and check.

MHM

mcotherman · ‎06-24-2024

Is there a reason you say that IPV6 enabled could cause this? I did it just in case it revealed a bug or something, but no change in symptoms...

Also, disabling LLDP TxRx on the interface is of no help either...

CCIE# 6771

MHM Cisco World · ‎06-24-2024

It really hard to know what make link down

But let try remove ip add dhcp

Let see if the link stable or not'

If it stable then the IP get from ISP is issue and make ISP shutdown the link

MHM

mcotherman · ‎06-25-2024

The link on Gi0/0/1 is stable until ip connectivity via Gi0/0/0 is interrupted (admin/unplugEth/Unplug coax)

With no address on Gi0/0/1, Gi0/0/1 does not go down

With IP Passthrough on the hotspot, taking down Gi0/0/0 resets Gi0/0/1 (this is the problem, as I want the WAN IP address)

With a static IP Address of 192.168.101.2 on Gi0/0/1 and the Inseego M3100 5G Hotspot set at 192.168.101.1 (it is serving DHCP .100-.199 to wifi) taking down Gi0/0/0 bounces the internet connectivity of the Hotspot, but not the link. How the Hotspot knows the C1111 connection to the internet goes down without a 'routing protocol' running is the question...

CCIE# 6771

MHM Cisco World · ‎06-26-2024

show ip interface brief
are the interface status is down or the protocol is down ?

if the status is down it your router issue
if the protocol is down it SP issue that down link
@Dan Frey point about behave of SP when it see packet with different source, the solution is run NAT so that always SP see source NATing to IP it assign via DHCP

if you can share how you config NAT to talk look if it NAT issue or not

MHM

ivances · ‎06-24-2024

Could you please share the output of "sh int gi0/0/1"? Also, a "show tech" would be great.

mcotherman · ‎06-25-2024

The status of sh int Gi0/0/1 is constantly changing, you do not mention when you would like it?

Show-Tech is not really helpful for diagnosing a problem that is 99% the fault of the hotspot. furthermore is it well over 5000 lines long and would take far too long for me to scrub personal data. As I have stated, I have done a Write Erase/Reload and put in a bare minimum config, which I have provided. I have also used 17.6.6a and 17.9.4a IOS releases.

CCIE# 6771

ivances · ‎06-25-2024

I would like to see interface drops. For example, if you see carrier transitions, it means that it could be a failure with the cabling, or even the power source of the router.

mcotherman · ‎06-25-2024

As I stated in the opening statement, I only have the issue with Gi0/0/1 when IP connectivity to Gi0/0/0 is bounced... cabling issues and carrier transitions would be present at other times. And, as I also stated, when the cable is moved from the router to a Win10 PC, it works fine... this is an issue with communications between the two devices above the hardware layer...

CCIE# 6771

Giuseppe Larosa · ‎06-25-2024

Hello @mcotherman ,

we see an error message:

>>

Jun 21 09:40:36.003: %IPRT-6-STATICROUTESACROSSTOPO: Cannot install inter-topology static
route 0.0.0.0/0 topology base on outgoing interface GigabitEthernet0/0/1

the configuration file does not provide any easy to spot reason on why inter gi0/0/1 is not part of topology base.

Hope to help

Giuseppe

mcotherman · ‎06-25-2024

Doing a "monitor capture CAP interface GigabitEthernet0/0/1" shows that as soon as the Gi0/0/0 interface is shut, a redirect is sent out Gi0/0/1 with the source address of Gi0/0/0, and this affects the Hotspot...

monitor capture CAP interface GigabitEthernet0/0/1 both
monitor capture CAP match any 
monitor capture CAP start
!
conf t
int Gi0/0/0
shut
end
 <wait 2 seconds>

monitor capture CAP stop
conf t
int Gi0/0/0
no shut
end
show monitor capture CAP buffer brief

I saw icmp unreachables: Code: 13 (Communication administratively filtered), so I turned them off on the Gi0/0/1 interface with 'no ip unreachables' in case that was triggering the hotspot to lose its connection to the internet, but it still does.

While it appears that this is leaning towards the hotspot being the problem, the hotspot works fine when connected to a windows PC. Something the C1111 is sending or allowing or responding to is causing the the hotspot to think it has lost its data connection.

...

CCIE# 6771

Dan Frey · ‎06-26-2024

Verizon wireless network requires all packets entering the network to be properly sourced. If a packet enters the wireless network that is not assigned to the device it is classified as a source ip address violation and VZW resets the connection.

mcotherman · ‎06-28-2024

Thanks! That makes perfect sense compared to what I am seeing!

Is there a solution or best practice for failover with such things in place by the carrier? I would think that anyone with a VZW sim in their router for backup would have the same problem, correct? I read this nat-failover-with-dual-isp-on-a-router-configuration-example and the use of event manager to clear the nat translations, is that a recommended solution for this issue?

CCIE# 6771

Dan Frey · ‎06-28-2024

EEM to clear the translations can address the issue. Here is an example.

event manager applet clearNat
 event tag t1 syslog pattern "LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0\/0\/0, changed state to up"
 event tag t2 syslog pattern "LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0\/0\/0, changed state to down"
 trigger
  correlate event t1 or event t2
 action 010 cli command "enable"
 action 020 cli command "clear ip nat tr *"
 action 030 puts "clearing nat translations"