Urgent help required---Regarding BGP flap

[jatinder sharma]

Hello ,

We have 2 MPLS links at one of our remote site ,one link is connected at Cisco 7206 VXR router and 2nd link is connected on Cisco 3750 Switch .

Presently traffic is flowing via router link and it is active link and link on switch is backup link.we have single subnet at that site and also we have announced that site subnet x.x.x.x/24 in  BGP at router but at switch we have bifurcated same subnet into x.x.x.x/26,x.x.x.x/27,.x.x.x.x/28 and announced these subnets into BGP at switch. IBGP is configured between router and switch. Today backup link at switch got flapped due to this some routes got refreshed in active link and user reported issue. please suggest what was the reason behind this issue. either these  bifurcated subnets at switch caused issue or some thing else.

Regards

Jatinder Sharma

[Richard Burts]

Jatinder

We do not have enough information to be able to identify what caused the issue. In general BGP would prefer the more specific route advertisements and I would have thought that would cause BGP to prefer the link through the switch. But you post says that traffic is flowing through the router. So I can not offer much advice about what caused the issue.

HTH

Rick

[jatinder sharma]

Hello Richard,

Yes ,IBGP is configured between both router and switch ,both router and switch has same no. of prefixes ,but in switch we have broken same LAN subnet into /26,/27,/28 and same published in BGP at switch end. in switch BGP has more specific routes than router ,is that the case or some thing else.

Below is BGP config of both devices. i have masked IP addresses..

OSCC-RTR#sh running-config | begin bgp
router bgp 64xxx
 no bgp log-neighbor-changes
 network 10.x.x.0 mask 255.255.255.0  --LAN Subnet
 timers bgp 5 15
 neighbor 10.x.x.250 remote-as 64xxx  -IBGP Config
 neighbor 10.x.x.250 next-hop-self
 neighbor 172.36.x.x remote-as 9xxx --EBGP Peer IP
 neighbor 172.36.x.x description primary-Link
 neighbor 172.36.x.x fall-over bfd
 neighbor 172.36.x.x allowas-in 10
 neighbor 172.36.x.x route-map SETLOCALIN in
 neighbor 172.36.x.x filter-list 10 out

Switch Config-
router bgp 64xxx
 no synchronization
 no bgp log-neighbor-changes
 network 10.x.x.0 mask 255.255.255.0
 network 10.x.x.0 mask 255.255.255.128
 network 10.x.x.128 mask 255.255.255.192
 network 10.x.x.192 mask 255.255.255.224
 timers bgp 5 15
 neighbor 10.x.x.249 remote-as 64xxx--IBGP Config
 neighbor 10.x.x.249 next-hop-self
 neighbor 172.x.x.17 remote-as 9xxx  --EBGP Peer IP
 neighbor 172.x.x.17 description Secondary-Link
 neighbor 172.x.x.17 fall-over bfd
 neighbor 172.x.x.17 allowas-in 10
 neighbor 172.x.x.17 route-map SETASPATH out
 neighbor 172.x.x.17 filter-list 10 out
 

please suggest on this.

Regards

Jatinder Sharma

[Richard Burts]

Jatinder

Thank you for the additional information. There are still some things that might impact it that we do not know. For example the router config has a route map inbound but we do not know what this route map is doing. And the switch has a route map outbound but we do not know what this route map is doing. And both router and switch apply filter-list 10 but we do not know what is in this filter list.

It is not clear to me why one device has a route map inbound and the other device has a route map outbound. I am not sure if this inconsistency is significant or not.

But even if we had answers to all of these items I do not believe that it would help us understand what caused the flap or to understand what was really going on when the users experienced problems.

HTH

Rick

[jatinder sharma]

Hello Richard,

these links are working in active standby mode, presently link on router is active and link on switch is in standby mode. please find below route -map config .

route-map SETLOCALIN permit 10
 set local-preference 200
!
route-map SETASPATH permit 20
 set as-path prepend 64xxx 64xxx 64xxx

filter-list 10 out is used to filter routes which was received from PE router,because we have 2 links at site from 2 PE's.

Regards

Jatinder Sharma

[jatinder sharma]

Hello Richard,

please suggest on this.

[Richard Burts]

Jatinder

In your original post you gave us this description " Today backup link at switch got flapped due to this some routes got refreshed in active link and user reported issue"

We have no way to know what caused the flap on the switch, no way to know which routes got refreshed, and no way to know what issue impacted the users. What can I comment on?

HTH

Rick 

[jatinder sharma]

hello Richard,

yes ,backup link was flapped on switch due to flap some routes got refreshed in active link and user reported issue. as I  earlier told in switch we have broken same LAN subnet /24  into parts /26,/27,/28 and same published in BGP. I think BGP prefers more specific routes ,so it caused issue or  something else. 

IN router we have published same LAN subnet as /24 .

Regards

Jatinder Sharma

[Richard Burts]

Jatinder

If the link at the switch flapped and went down it could be that the smaller parts of the subnet were removed from the BGP routing table. When the link came back up the smaller parts of the subnet would be added back to the BGP routing table. If that happened several times, and if the BGP peer has configured dampening then the advertisement of the smaller parts of the subnet could have been suppressed for a period of time. This is normal BGP behavior. I do not believe that at this point we can determine whether this caused the issue that users experienced.

HTH

Rick

[John Blakley]

Jatinder,

I agree with Rick. There's not really much information here to work off of. From your description, it sounds like you have an ebgp peering to the provider, but you have an ibgp peering between your switch and your router. Is that the case? The only reason that I could see is what Rick has already suggested in that the switch probably received a better route through the switch somehow. Without knowing why it would be difficult considering the different levels that bgp uses to select the best path.

HTH,

John

[jatinder sharma]

Hello John,

Yes ,IBGP is configured between both router and switch ,both router and switch has same no. of prefixes ,but in switch we have broken same LAN subnet into /26,/27,/28 and same published in BGP at switch end. in switch BGP has more specific routes than router ,is that the case or some thing else.

Below is BGP config of both devices. i have masked IP addresses..

OSCC-RTR#sh running-config | begin bgp
router bgp 64xxx
 no bgp log-neighbor-changes
 network 10.x.x.0 mask 255.255.255.0  --LAN Subnet
 timers bgp 5 15
 neighbor 10.x.x.250 remote-as 64xxx  -IBGP Config
 neighbor 10.x.x.250 next-hop-self
 neighbor 172.36.x.x remote-as 9xxx --EBGP Peer IP
 neighbor 172.36.x.x description primary-Link
 neighbor 172.36.x.x fall-over bfd
 neighbor 172.36.x.x allowas-in 10
 neighbor 172.36.x.x route-map SETLOCALIN in
 neighbor 172.36.x.x filter-list 10 out

Switch Config-
router bgp 64xxx
 no synchronization
 no bgp log-neighbor-changes
 network 10.x.x.0 mask 255.255.255.0
 network 10.x.x.0 mask 255.255.255.128
 network 10.x.x.128 mask 255.255.255.192
 network 10.x.x.192 mask 255.255.255.224
 timers bgp 5 15
 neighbor 10.x.x.249 remote-as 64xxx--IBGP Config
 neighbor 10.x.x.249 next-hop-self
 neighbor 172.x.x.17 remote-as 9xxx  --EBGP Peer IP
 neighbor 172.x.x.17 description Secondary-Link
 neighbor 172.x.x.17 fall-over bfd
 neighbor 172.x.x.17 allowas-in 10
 neighbor 172.x.x.17 route-map SETASPATH out
 neighbor 172.x.x.17 filter-list 10 out
 

please suggest on this.

Regards

Jatinder Sharma