Interface resets with ISP modem

[DSmithQIA]

Hi All

I'm pretty new to the game, slowly learning. I am having a most peculiar issue when connecting my ISR 4321 router to my ISP's DSL modem.

What is happening is the interface is resetting every five minutes, almost to the second, the error sequence looks like this:

*Jan 29 22:31:13.201: %LINK-5-CHANGED: Interface GigabitEthernet0/0/0, changed state to administratively down
*Jan 29 22:31:14.201: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/0, changed state to down
*Jan 29 22:31:16.205: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/0, changed state to down
*Jan 29 22:31:19.209: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/0, changed state to up
*Jan 29 22:31:20.208: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/0, changed state to up
*Jan 29 22:31:28.089: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0/0 assigned DHCP address xx.xx.xx.xx, mask 255.255.255.128, hostname My-RTR

This behaviour repeats itself continually, regardless of the flow of traffic. 

A little info on my ISP - I have a "Business" package, we've purchased a "Fixed" IP address, which is nothing more than a mac reservation  - I am required to set my interface to autonegotiate - it will not function with any speed or duplex settings hard-coded, nor will it work if I assign the IP manually - I have tried all of these.

What really make me wonder is, I have a Wireless modem from a different ISP, and plugging it into the interface on the Router seems to work fine - it picks up an address through DHCP, and there are no interface resets or interruptions of any kind. Below is the sh interface output for the interface in question:

GigabitEthernet0/0/0 is up, line protocol is up
Hardware is ISR4321-2x1GE, address is 80e0.1d0c.2db0 (bia 80e0.1d0c.2db0)
Internet address is 7xx.xx.xx.xx/25
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
Full Duplex, 100Mbps, link type is auto, media type is RJ45
output flow-control is on, input flow-control is on
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:37, output 00:00:37, output hang never
Last clearing of "show interface" counters never
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
317938 packets input, 341232001 bytes, 0 no buffer
Received 130 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 1483 multicast, 126 pause input
225369 packets output, 31675023 bytes, 0 underruns
0 output errors, 0 collisions, 63 interface resets
97 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
68 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

This one is starting to drive me around the bend a little bit - any help would be greatly appreciated!

Cheers

d

one other item - If I turn remove IP NAT OUTSIDE from the interface in question, the errors cease (as does my internet connectivity), could this be NAT-related?

[Philip D'Ath]

A big hint is the error "administratively down".  This means it has been shutdown, on purpose, through your router.  The issue looks like it is on your end.

Do you have any scripts running on the router? The fact that it happens every 5 minutes makes it sound like a scheduled script even more.  Could you post your config?

[DSmithQIA]

Thanks for the insight, I'm not aware of any scheduled scripts - I'm the only one working on the device. Below is my running config, thanks for helping out :)

Building configuration...

Current configuration : 3366 bytes
!
! Last configuration change at 14:16:45 UTC Sat Jan 30 2016
!
version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname MY-RTR
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
enable secret 5 $1$jUVE$woiV..3kMeyjaEH.L1LXj0
!
no aaa new-model
!
!!

ip domain name MY.DOMAIN

!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
username USERNAME secret 5 $1$LIqN$OnQW1Al2TqWd99L/Ovxh1/
!
redundancy
mode none
!
!
!
!
!
no cdp advertise-v2
no cdp run
!
ip tftp source-interface GigabitEthernet0
ip ssh source-interface GigabitEthernet0
ip ssh version 2
!
!
interface GigabitEthernet0/0/0
ip address dhcp
ip nat outside
negotiation auto
!
interface GigabitEthernet0/0/1
ip address 11.1.11.1 255.255.255.252
ip nat inside
negotiation auto
!
interface GigabitEthernet0
vrf forwarding Mgmt-intf
ip address 192.168.1.1 255.255.255.0
negotiation auto
!
ip nat inside source static 11.1.11.2 interface GigabitEthernet0/0/0
ip forward-protocol nd
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0/0 dhcp
!
!
!
!
!
control-plane
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
exec-timeout 40 0
login local
transport input ssh
!
!
end

[Philip D'Ath]

Nothing wrong with that config.  Nice and simple.  However that doesn't negate the "administratively down" log entry, meaning someone or something on the router shut down the interface.

The current gold star software release for your platform is  3.16.1aS.  If you are not using it I would try changing to that.

[DSmithQIA]

Thanks again for having a look. I'm at sea on this one - I've switched out the DSL modem with my Wireless modem that I normally use for travel and the issue does not happen even once. This is not an acceptable (permanent) workaround as the Wireless Modem is much slower than the DSL, and has a pretty low bandwidth cap.

I have not changed a single thing in the router config, just physically plugging the Wireless Modem into the interface instead of the DSL modem.

My limited troubleshooting expertise suggests that my NAT configuration is somehow causing this - i.e. there is something with the way my ISP handles NAT on their side of the of the network that is disagreeing with my router...does that sound reasonable at all to you?

We currently do not have a service contract with Cisco which makes getting the IOS update file a problem, I am in the process of getting one and am going to try the updated IOS to see of that sorts me out.

Cheers,

d

[Philip D'Ath]

It you say you can plug the 4321 into a different router and it works fine, and then when you plug it back into the ISP's DSL router it does not - that makes it sounds like an issue with the ISPs DSL router.

[DSmithQIA]

That pretty much sums up what I keep coming back to.

I did place a call with them, and they weren't too interested in anything I had to say - they just kept telling me they could communicate with their end device and that the problem was definitely on my side.

I have two of these DSL lines, both of them behave the same way when plugged into my router...but if you plug them directly into a PC they seem to work fine.

They also seemed to work fine when I had an older TP Link load balancing router in place.

So troubleshooting this has taken me in circles a bit...but I have to figure it out somehow.

I appreciate your input - sooner or later I will stumble across a solution.

cheers,

d

[Philip D'Ath]

Bit of a long shot - try a cross over Ethernet cable.

[DSmithQIA]

ah yes...I don't have time just now, but I will come back to the office later on and give that a shot.

I'm kind of excited to try something I haven't tried yet!

Cheers,

d

[DSmithQIA]

Well, I tried the crossover cable and no dice. The connection came up and traffic was flowing nicely, for five minutes, then the same old administratively down sequence I am seeing using a normal cable.

Some condition must be present to be causing these errors...soon might be time to engage Cisco Support on it.

Thanks again for your suggestions!

cheers

d

[Philip D'Ath]

How about trying a different port on the 4321, in case there is a fault with that specific port.

[DSmithQIA]

I had tried that early on in troubleshooting - switched the DSL modem from Gig0/0/0 to 0/0/1, adjusted NAT and the default route, and got the exact same behaviour, albeit it took closer to 10 minutes for the first interface shutdown to occur.

Something is out of place here - the challenge is identifying if it's my router (hardware failure) or my configuration.

My thought now is to enable every kind of logging I can think of, perhaps the answer will reveal itself in an error log...

cheers,

d

[DSmithQIA]

By George, I think I figured it out!

Focusing on NAT as the culprit, I removed my existing NAT statement, removed the NAT inside and outside statements from their respective interfaces, entered a NAT overload statement and corresponding ACL, then re-entered the NAT inside/outside statements on their respective interfaces and Voila!

I noticed my NAT translation table increased from approximately 50 translations to over 600, and now after about a half hour I have had no interface shutdowns, and the show interface output shows no resets and no lost carrier errors.

Thanks Philip for taking the time to respond to my posts, although we didn't hit the exact cause you certainly helped me through the troubleshooting process!

Cheers,

d

[Philip D'Ath]

When you plug a computer directly in and it works - are you using the same network cable as you use on the 4321?