WAN Ports on Opposite Sides of a Circuit Go Down Simultaneously

[ts7575]

Hello,

I don't have a lot of experience with WAN routing, so this issue I am facing is perplexing. I have two WAN routers in two different geographical locations that are virtually connected via a tunnel. Recently we have been experiencing occasional outages maybe twice a week. Most of the time they only last a few minutes, but there have been outages that have lasted for hours. These outages begin with our outside interfaces going down simultaneously on both routers. At first we were convinced there is an issue with our circuit, but our ISP has stated that there are no issues on their end. Additionally, we have noticed during a reload of one router that the outside interface of the remote router goes down. Any ideas on what could be causing this to happen? I am perplexed because I don't understand how the outside interface on one router going down could possibly affect the outside interface of another router in a different geographical location. 

Thankyou

[balaji.bandi]

Frist start with some basic inputs :

1. what Router models both the sides

2. what IOS Code running on it.

3. what kind of circuits - MPLS , VPLS, Internet ?

4. what kind of Tunnels P2P ?

5. can you post the configuration removing confidential information.

6. when the Tunnel go down ( what Logs you see both the routers)

7. Are you able to ping each other Underlay IP address (if that is allowed) to confirm Circuit working as expected when the tunnel down 

Depends on the circuits and connection, its normal when you reboot the one side router, other side WAN interface go down.

IPSEC troubleshooting guide for reference :

https://www.cisco.com/c/en/us/support/docs/security-vpn/ipsec-negotiation-ike-protocols/5409-ipsec-debug-00.html

[ts7575]

1.) ISR 4000 series on both sides.

2.) 17.9.5, but this issue has occurred on past software versions.

3.) I believe MPLS, but I could be wrong. I will have to look into that further.

4.) IPSEC

5.) I am not permitted to.

6.) %LINK-3-UPDOWN: Interface GigabitEthernetx/x/x, changed state to down

     %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernetx/x/x, changed  state to         down

     %LINEPROTO-5-UPDOWN: Line protocol on interface Tunnel x, changed state to down

     %LINK-3-UPDOWN: Interface GigabitEthernetx/x/x, changed state to up

     %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernetx/x/x, changed  state to         up

     %LINEPROTO-5-UPDOWN: Line protocol on interface Tunnelx, changed state to up

[balaji.bandi]

unfortunately there is little information to suggest here.

6. when the Tunnel go down ( what Logs you see both the routers)

this clearly shows your outside interface x/x/x going down and coming up ?

If this x/x/x connected to ISP circuits - then contact ISP - what other information you need to prove that interface Physically going down, that is the reason IPSEC  tear down. and when that come up - tunnel come up

this is based on the information you provided here, how your network looks like you have better idea.

[MHM Cisco World]

Can I see the IPsec config 

MHM

[Georg Pauwen]

Hello,

can you post the running configs of both routers ? The problem to describe makes it very likely that your ISP actually IS the culprit. In my experience, it usually takes a few calls for them to acknowledge the fact that the problem is on their side. I would call them at the very moment you are experiencing problems.