Routing Assistance

[gboyce]

Hi Folks,
I need some routing assistance if you will. I have no routing education but I have been tasked with it anyway.
I have included a network diagram that shows how things are currently.
The routing statements in our router(ASR1002x xxx.xxx.120.1) are
0.0.0.0 0.0.0.0 ourBGPaddress
xxx.xxx.121.0 255.255.255.0 xxx.xxx.120.2
xxx.xxx.122.0 255.255.255.0 xxx.xxx.120.2

We have most of our fiber customers on SW2 (xxx.xxx.120.2). Until recently each fiber customer has come to us on a fiber pair. Our fiber carrier said that this is too expensive and delivered
a new fiber pair for an NNI for all new fiber customers to go through. We want to have that NNI go into SW1 (xxx.xxx.120.4).
I tried changing the static routes in our router from the above two statements to
xxx.xxx.121.32 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.64 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.96 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.128 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.160 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.192 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.200 255.255.255.252 xxx.xxx.120.2
xxx.xxx.121.208 255.255.255.240 xxx.xxx.120.2
xxx.xxx.121.248 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.232 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.240 255.255.255.248 xxx.xxx.120.2
xxx.xxx.122.0 255.255.255.224 xxx.xxx.120.2
xxx.xxx.122.32 255.255.255.248 xxx.xxx.120.2
xxx.xxx.122.64 255.255.255.240 xxx.xxx.120.2
xxx.xxx.122.80 255.255.255.252 xxx.xxx.120.2
xxx.xxx.122.40 255.255.255.248 xxx.xxx.120.4
xxx.xxx.122.47 255.255.255.252 xxx.xxx.120.4

Both switches have 0.0.0.0 0.0.0.0 xxx.xxx.120.1
When I remove the 2 existing routing statements and put in the new ones and move the NNI to SW1 none of the fiber customers can get to the internet. I am on the xxx.xxx.120.0 network and can get to each of their devices.
I hope that someone here can point me in the right direction.

[Georg Pauwen]

Hello,

post the full configs of all three devices (1 router and 2 switches). I assume the switches are Layer 3 switches ? What routing is configured on SW1 ?

[gboyce]

Yes the switches are Layer 3. Both switches routing statement is 0.0.0.0 0.0.0.0 x.x.120.1

Router:

version 15.3
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
no platform punt-keepalive disable-kernel-core
!
hostname ASR1002-X
!
boot-start-marker
boot system bootflash:asr1002x-universalk9.03.10.04.S.153-3.S4-ext.SPA.bin
boot-end-marker
!
aqm-register-fnf
!
vrf definition Mgmt-intf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
enable secret 
!
no aaa new-model
clock timezone EST -5 0
!
!
!
!
!

ip flow-cache timeout inactive 10
ip flow-cache timeout active 5
ip domain name telpage.net
!
!
!
ipv6 multicast rpf use-bgp
ipv6 multicast vrf Mgmt-intf rpf use-bgp
!
!
!
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
license boot level adventerprise
spanning-tree extend system-id
!

!
redundancy
 mode none
!
!
!
!
!
cdp run
!
!

!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0/0
 no ip address
 negotiation auto
!
interface GigabitEthernet0/0/1
 description Uplink to Cogent
 bandwidth 1000000
 ip address xxx.xxx.xxx.xxx 255.255.255.248
 ip access-group 102 in
 no negotiation auto
!
interface GigabitEthernet0/0/2
 description Core Network
 ip address xxx.xxx.120.1 255.255.255.0
 ip flow egress
 no negotiation auto
!
interface GigabitEthernet0/0/3
 description Wireless Customers
 ip address xxx.xxx.126.1 255.255.255.0 secondary
 ip address xxx.xxx.127.1 255.255.255.0 secondary
 ip address xxx.xxx.125.1 255.255.255.0
 ip flow egress
 negotiation auto
!
interface GigabitEthernet0/0/4
 description test port
 no ip address
 negotiation auto
!
interface GigabitEthernet0/0/5
 ip address xxx.xxx.123.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0
 vrf forwarding Mgmt-intf
 no ip address
 shutdown
 negotiation auto
!
router bgp 46445
 bgp log-neighbor-changes
 network xxx.xxx.120.0
 network xxx.xxx.121.0
 network xxx.xxx.122.0
 network xxx.xxx.123.0
 network xxx.xxx.124.0
 network xxx.xxx.125.0
 network xxx.xxx.126.0
 network xxx.xxx.127.0
 neighbor x.x.x.x remote-as 174
 neighbor x.x.x.x version 4
!
ip forward-protocol nd
!
ip flow-export version 9
ip flow-export destination xxx.xxx.126.7 2501
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 x.x.x.x
ip route xxx.xxx.121.0 255.255.255.0 xxx.xxx.120.2
ip route xxx.xxx.122.0 255.255.255.0 xxx.xxx.120.2
ip route xxx.xxx.123.0 255.255.255.0 xxx.xxx.120.5
!
access-list 102 deny   tcp any host 38.122.67.106 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.1 eq 22
access-list 102 deny   tcp any host xxx.xxx.125.1 eq 22
access-list 102 deny   tcp any host xxx.xxx.126.1 eq 22
access-list 102 deny   tcp any host xxx.xxx.127.1 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.2 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.4 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.34 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.35 eq 22
access-list 102 deny   tcp any host xxx.xxx.120.38 eq 22
access-list 102 permit ip any any
!
snmp-server community xxxxxx RO
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!
line con 0
 logging synchronous
 login local
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
ntp logging
ntp peer xxx.xxx.125.131 prefer
!
end

SW1:

Current configuration : 17223 bytes
!
! Last configuration change at 12:37:10 UTC Tue Jan 15 2019
!
version 16.6
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname NOC
!
!
vrf definition Mgmt-vrf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
enable secret 5 $1$USYX$imGdI84R/YDofIT1pvS2E/
enable password set^a$43T
!
no aaa new-model
switch 1 provision c9300-48t
!
!
!
!
ip routing
!
ip domain name telpage.net
!
!
!
!
!
!
!
!
cpp system-default
!
crypto pki trustpoint TP-self-signed-2551947109
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-2551947109
 revocation-check none
 rsakeypair TP-self-signed-2551947109
!
!
crypto pki certificate chain TP-self-signed-2551947109
 certificate self-signed 01
  30820330 30820218 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
  31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
  69666963 6174652D 32353531 39343731 3039301E 170D3138 31313136 31333035
  30375A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
  4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D32 35353139
  34373130 39308201 22300D06 092A8648 86F70D01 01010500 0382010F 00308201
  0A028201 0100DA06 D9772694 D898A1B9 E3DEBF23 BAED0C5C BDB2D18B 4AEF1D45
  96B8A99E 2310EE9A 703DED0A 677AF7A3 EDB30E27 30440271 58768DDA B3560712
  AC09DDD4 AFDB563E 6FEF0E98 104D9442 F23DCD7F 6488FBD5 838CE715 C66B2A27
  21AD0BF6 6F5ED9A2 E5859068 5DC6B48A 0A47C93C 199CEABF 58D32889 7631332E
  9124D7B4 44FB2A97 EE068107 DCD20628 D987CED1 302DF9FA 6C8AE524 A5303766
  B99CD853 CD2C95F8 8FC9E2C3 0237EF8E 5892B2BD 9BA94AB1 3A830C81 C0776811
  8416BA42 1352D64F FADA6A97 532AB02E 64B4741B 594A1BD6 A77B0DC7 3F915023
  D396F6E9 9C46969C 2F6D1D1A CFB9CAC1 7DB04FE0 4F67AB68 50FE1B0A D51EB55E
  61D3A6AF AB350203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF
  301F0603 551D2304 18301680 1441BFB1 EE0BD5A9 47B23E39 5F818D72 D84AE89F
  DE301D06 03551D0E 04160414 41BFB1EE 0BD5A947 B23E395F 818D72D8 4AE89FDE
  300D0609 2A864886 F70D0101 05050003 82010100 D4CDF76D E5A9B86C 0FD663FC
  7A40D39A 007132AE C0C085DB 3974C26A 36117886 A10D93A9 48B26879 A2926139
  E063E5DB 2D527995 E85EC526 138FC958 9E8DD247 16B6A523 E5E5B4F4 3BB226F1
  70E7AF89 7C2B7A26 26D7CC37 86EF4CB3 5FB8040F 10DD6B24 7271A1FE F5421CD9
  D71E3313 F1D0EA27 0EE5949E B221D638 6A93D112 C94CE7A5 0CCACE71 2FD46A4F
  110385D0 3F1B7D15 97241CB6 26063F0C 1704B2E8 E0331378 2F3D6F6C 84F1E99F
  8DF3D30B FE4315CA 8C5A3E6F 5A34368B A43033A8 936F8A05 2D89D155 11CECA5A
  74A2F00E F0B0DC50 2227D073 3CD6728D C615AA07 7C0E2A13 0044FC62 C4343429
  167F0E20 4C2A278D 4E43C0CB 6A15CD8E 1639A096
   quit
!
!
!
diagnostic bootup level minimal
spanning-tree mode rapid-pvst
spanning-tree extend system-id
!

!
redundancy
 mode sso
!
!
vlan 2247-2248,2251
!
!
class-map match-any system-cpp-police-topology-control
  description Topology control
class-map match-any system-cpp-police-sw-forward
  description Sw forwarding, L2 LVX data, LOGGING
class-map match-any system-cpp-default
  description DHCP Snooping, EWLC control, EWCL data
class-map match-any system-cpp-police-sys-data
  description Learning cache ovfl, Crypto Control, Exception, EGR Exception, NFL SAMPLED DATA, Gold Pkt, RPF Failed
class-map match-any system-cpp-police-punt-webauth
  description Punt Webauth
class-map match-any system-cpp-police-l2lvx-control
  description L2 LVX control packets
class-map match-any system-cpp-police-forus
  description Forus Address resolution and Forus traffic
class-map match-any system-cpp-police-multicast-end-station
  description MCAST END STATION
class-map match-any system-cpp-police-multicast
  description Transit Traffic and MCAST Data
class-map match-any system-cpp-police-l2-control
  description L2 control
class-map match-any system-cpp-police-dot1x-auth
  description DOT1X Auth
class-map match-any system-cpp-police-data
  description ICMP redirect, ICMP_GEN and BROADCAST
class-map match-any system-cpp-police-stackwise-virt-control
  description Stackwise Virtual
class-map match-any system-cpp-police-control-low-priority
  description General punt
class-map match-any non-client-nrt-class
class-map match-any system-cpp-police-routing-control
  description Routing control
class-map match-any system-cpp-police-protocol-snooping
  description Protocol snooping
!
policy-map system-cpp-policy
 class system-cpp-police-data
  police rate 600 pps
 class system-cpp-police-sys-data
  police rate 100 pps
 class system-cpp-police-sw-forward
  police rate 1000 pps
 class system-cpp-police-multicast
  police rate 500 pps
 class system-cpp-police-multicast-end-station
  police rate 2000 pps
 class system-cpp-police-punt-webauth
  police rate 1000 pps
 class system-cpp-police-l2-control
  police rate 2000 pps
 class system-cpp-police-stackwise-virt-control
  police rate 8000 pps
 class system-cpp-police-routing-control
  police rate 5400 pps
 class system-cpp-police-control-low-priority
  police rate 200 pps
 class system-cpp-police-l2lvx-control
  police rate 1000 pps
 class system-cpp-police-topology-control
  police rate 13000 pps
 class system-cpp-police-dot1x-auth
  police rate 1000 pps
 class system-cpp-police-protocol-snooping
  police rate 2000 pps
 class system-cpp-police-forus
  police rate 1000 pps
 class system-cpp-default
  police rate 1000 pps
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Port-channel1
 description Port Channel 1
 switchport mode trunk
 spanning-tree portfast disable
 lacp max-bundle 2
!
interface GigabitEthernet0/0
 vrf forwarding Mgmt-vrf
 no ip address
 speed 1000
 negotiation auto
!
interface GigabitEthernet1/0/1
 description Uplink to 2960-0 GB1/0/21
 switchport mode trunk
 channel-group 1 mode active
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/2
 description Uplink to 2960-1 GB2/0/21
 switchport mode trunk
 channel-protocol lacp
 channel-group 1 mode active
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/3
 description VS0 GB2 SAN Management
 switchport access vlan 10
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/4
 description VS1 GB1 iSCSI
 switchport access vlan 10
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/5
 description cloud.modsbyus.com
 switchport access vlan 123
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/6
 description speedtest.telpage.net
 switchport access vlan 123
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/7
 description VS1 GB2 VMs
 switchport mode trunk
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/8
 description Uplink to Corp-Temp
 switchport mode trunk
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/9
 description NOC Uplink
 switchport trunk native vlan 120
 switchport mode trunk
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/10
 description Charlie_NT Access
 switchport access vlan 150
 switchport trunk native vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/11
 switchport trunk native vlan 30
 switchport mode trunk
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
 description Communigate1
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/14
 description Communigate 2
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/15
 description NOC Sensor Probe
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/16
 description Ziggy
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/17
 description cameras.telpage.net
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/18
 description FlightAware Pi
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/19
 description TelpageBench
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/20
 description Uplink to ASA-0 GiG0/0
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/21
 description Uplink to ASA-1 GiG0/0
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/22
!
interface GigabitEthernet1/0/23
!
interface GigabitEthernet1/0/24
!
interface GigabitEthernet1/0/25
 description Main to FG PTP670
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/26
 description Main to Lodge PTP250
 switchport access vlan 30
 switchport mode access
 speed 100
 duplex full
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/27
 description Main 5.2 AP
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/28
 description Main 2.4 AP
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/29
 description RemoteLink PC
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/30
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/31
 description Garys Bench
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/32
 description Tylers Bench
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/33
 description PTP670 to Phoenix
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/0/34
!
interface GigabitEthernet1/0/35
!
interface GigabitEthernet1/0/36
 description Laptop
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast trunk
!
interface GigabitEthernet1/0/37
!
interface GigabitEthernet1/0/38
!
interface GigabitEthernet1/0/39
!
interface GigabitEthernet1/0/40
!
interface GigabitEthernet1/0/41
!
interface GigabitEthernet1/0/42
!
interface GigabitEthernet1/0/43
!
interface GigabitEthernet1/0/44
!
interface GigabitEthernet1/0/45
!
interface GigabitEthernet1/0/46
!
interface GigabitEthernet1/0/47
!
interface GigabitEthernet1/0/48
 switchport access vlan 2247
 switchport mode access
 spanning-tree portfast disable
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
 description ASR 120 Uplink
 switchport access vlan 120
 switchport mode access
 speed nonegotiate
 spanning-tree portfast trunk
!
interface TenGigabitEthernet1/1/2
 description ASR 30 Uplink
 switchport access vlan 30
 switchport mode access
 spanning-tree portfast trunk
!
interface TenGigabitEthernet1/1/3
 description MBC Switch Uplink
 switchport access vlan 120
 switchport trunk native vlan 120
 switchport mode access
 spanning-tree portfast disable
!
interface TenGigabitEthernet1/1/4
 description Telpage to Davis Fiber Uplink
 switchport trunk native vlan 30
 switchport mode trunk
 spanning-tree portfast disable
!
interface TenGigabitEthernet1/1/5
 description MBC NNI
 switchport trunk native vlan 30
 switchport trunk allowed vlan 2247,2251
 switchport mode trunk
 speed nonegotiate
 spanning-tree portfast disable
!
interface TenGigabitEthernet1/1/6
!
interface TenGigabitEthernet1/1/7
!
interface TenGigabitEthernet1/1/8
!
interface FortyGigabitEthernet1/1/1
!
interface FortyGigabitEthernet1/1/2
!
interface Vlan1
 no ip address
!
interface Vlan10
 no ip address
!
interface Vlan20
 no ip address
!
interface Vlan30
 description VLAN30
 ip address 10.6.6.1 255.255.255.0
!
interface Vlan100
 no ip address
!
interface Vlan120
 description Vlan120
 ip address xxx.xxx.120.4 255.255.255.0
!
interface Vlan123
 no ip address
!
interface Vlan150
 no ip address
!
interface Vlan155
 no ip address
!
ip forward-protocol nd
ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 xxx.xxx.120.1
ip route xxx.xxx.125.0 255.255.255.0 xxx.xxx.125.1
ip route xxx.xxx.126.0 255.255.255.0 xxx.xxx.126.1
ip route xxx.xxx.127.0 255.255.255.0 xxx.xxx.127.1
ip ssh authentication-retries 2
!
!
ip access-list standard Corp-Access
 permit 0.0.0.5 255.255.255.0
ip access-list standard GaryHomeAccess
 permit 0.0.0.129 255.255.255.0
ip access-list standard GaryLaptop
 permit 0.0.0.199 255.255.255.0
ip access-list standard PRTG
 permit 0.0.0.254 255.255.255.0
!
!
snmp-server community tpstat*7 RO
snmp-server location Telpage-NOC
snmp-server contact Telpage
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps flowmon
snmp-server enable traps entity-perf throughput-notif
snmp-server enable traps call-home message-send-fail server-fail
snmp-server enable traps tty
snmp-server enable traps eigrp
snmp-server enable traps ospf state-change
snmp-server enable traps ospf errors
snmp-server enable traps ospf retransmit
snmp-server enable traps ospf lsa
snmp-server enable traps ospf cisco-specific state-change nssa-trans-change
snmp-server enable traps ospf cisco-specific state-change shamlink interface
snmp-server enable traps ospf cisco-specific state-change shamlink neighbor
snmp-server enable traps ospf cisco-specific errors
snmp-server enable traps ospf cisco-specific retransmit
snmp-server enable traps ospf cisco-specific lsa
snmp-server enable traps rep
snmp-server enable traps vtp
snmp-server enable traps vlancreate
snmp-server enable traps vlandelete
snmp-server enable traps port-security
snmp-server enable traps license
snmp-server enable traps cpu threshold
snmp-server enable traps memory bufferpeak
snmp-server enable traps stackwise
snmp-server enable traps fru-ctrl
snmp-server enable traps flash insertion removal lowspace
snmp-server enable traps energywise
snmp-server enable traps power-ethernet police
snmp-server enable traps entity
snmp-server enable traps envmon
snmp-server enable traps cef resource-failure peer-state-change peer-fib-state-change inconsistency
snmp-server enable traps ike policy add
snmp-server enable traps ike policy delete
snmp-server enable traps ike tunnel start
snmp-server enable traps ike tunnel stop
snmp-server enable traps ipsec cryptomap add
snmp-server enable traps ipsec cryptomap delete
snmp-server enable traps ipsec cryptomap attach
snmp-server enable traps ipsec cryptomap detach
snmp-server enable traps ipsec tunnel start
snmp-server enable traps ipsec tunnel stop
snmp-server enable traps ipsec too-many-sas
snmp-server enable traps ipsla
snmp-server enable traps entity-diag boot-up-fail hm-test-recover hm-thresh-reached scheduled-test-fail
snmp-server enable traps bfd
snmp-server enable traps config-copy
snmp-server enable traps config
snmp-server enable traps config-ctid
snmp-server enable traps event-manager
snmp-server enable traps hsrp
snmp-server enable traps bridge newroot topologychange
snmp-server enable traps stpx inconsistency root-inconsistency loop-inconsistency
snmp-server enable traps syslog
snmp-server enable traps trustsec-sxp conn-srcaddr-err msg-parse-err conn-config-err binding-err conn-up conn-down binding-expn-fail oper-nodeid-change binding-conflict
snmp-server enable traps trustsec authz-file-error cache-file-error keystore-file-error keystore-sync-fail random-number-fail src-entropy-fail
snmp-server enable traps trustsec-interface unauthorized sap-fail authc-fail supplicant-fail authz-fail
snmp-server enable traps trustsec-server radius-server provision-secret
snmp-server enable traps lisp
snmp-server enable traps dhcp
snmp-server enable traps auth-framework sec-violation
snmp-server enable traps rsvp
snmp-server enable traps ipmulticast
snmp-server enable traps msdp
snmp-server enable traps pim neighbor-change rp-mapping-change invalid-pim-message
snmp-server enable traps isis
snmp-server enable traps ospfv3 state-change
snmp-server enable traps ospfv3 errors
snmp-server enable traps vlan-membership
snmp-server enable traps errdisable
snmp-server enable traps rf
snmp-server enable traps transceiver all
snmp-server enable traps bulkstat collection transfer
snmp-server enable traps mac-notification change move threshold
snmp-server enable traps vrfmib vrf-up vrf-down vnet-trunk-up vnet-trunk-down
snmp-server host xxx.xxx.126.7 version 2c tpstat*7
!
!
control-plane
 service-policy input system-cpp-policy
!
!
line con 0
 stopbits 1
line vty 0
 login
 length 0
line vty 1 4
 login
 length 0
 transport input ssh
line vty 5 15
 login
 transport input ssh
!
!
wsma agent exec
!
wsma agent config
!
wsma agent filesys
!
wsma agent notify
!
!
end

SW2:

v1.4.5.02 / R800_NIK_1_4_194_194
CLI v1.0
set system mode router

file SSD indicator plaintext
@
spanning-tree priority 28672
vlan database
vlan 2,22,30,32,64,68,80,96,120-121,128,155,160,192,208,224,232,240
vlan 248,284,320,2247,2251
exit
voice vlan oui-table add 0001e3 Siemens_AG_phone________
voice vlan oui-table add 00036b Cisco_phone_____________
voice vlan oui-table add 00096e Avaya___________________
voice vlan oui-table add 000fe2 H3C_Aolynk______________
voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone
voice vlan oui-table add 00d01e Pingtel_phone___________
voice vlan oui-table add 00e075 Polycom/Veritel_phone___
voice vlan oui-table add 00e0bb 3Com_phone______________
no eee enable
bonjour interface range vlan 1
hostname MBC
line ssh
exec-timeout 5
exit
line telnet
exec-timeout 1
exit
logging host xxx.xxx.125.178
logging host xxx.xxx.125.131 severity debugging
logging origin-id hostname

snmp-server server
snmp-server location Datacenter
snmp-server contact Telpage
snmp-server community ******* ro xxx.xxx.126.7 view Default
ip http timeout-policy 300
clock timezone EST -5
clock summer-time web recurring usa
clock source sntp
sntp unicast client poll
sntp server 10.6.6.9 poll
ip domain name telpage.net
ip name-server  xxx.xxx.120.9 xxx.xxx.120.10
!
interface vlan 1
 ip address xxx.xxx.121.249 255.255.255.248
 no ip address dhcp
!
interface vlan 2
 name Mrrj
 ip address xxx.xxx.121.1 255.255.255.224
!
interface vlan 22
 name MarkPope
 ip address xxx.xxx.122.1 255.255.255.224
!
interface vlan 30
 name Wireless
!
interface vlan 32
 name Hampton
 ip address xxx.xxx.121.33 255.255.255.224
!
interface vlan 64
 name PandS
 ip address xxx.xxx.121.65 255.255.255.224
!
interface vlan 68
 name test68
!
interface vlan 80
 name VaPallets
 ip address xxx.xxx.122.81 255.255.255.252
!
interface vlan 96
 name Fairfield
 ip address xxx.xxx.121.97 255.255.255.224
!
interface vlan 120
 name Core/Fiber
 ip address xxx.xxx.120.2 255.255.255.0
!
interface vlan 121
 name Fiber
!
interface vlan 128
 name OwenFord
 ip address xxx.xxx.121.129 255.255.255.224
!
interface vlan 155
 name CIMC
 ip address xxx.xxx.121.201 255.255.255.252
!
interface vlan 160
 name GCC
 ip address xxx.xxx.121.161 255.255.255.224
!
interface vlan 192
 name Boyd
 ip address xxx.xxx.121.193 255.255.255.248
!
interface vlan 208
 name BeachMold
 ip address xxx.xxx.121.209 255.255.255.240
!
interface vlan 232
 name Dominion
 ip address xxx.xxx.121.233 255.255.255.248
!
interface vlan 240
 name Fluor
 ip address xxx.xxx.121.241 255.255.255.248
!
interface vlan 248
 name BeachMoldMngmnt
!
interface vlan 284
 name test
!
interface vlan 320
 name OSG
 ip address xxx.xxx.122.33 255.255.255.248
!
interface vlan 2247
 name Remac
 ip address xxx.xxx.122.41 255.255.255.248
!
interface vlan 2251
 name "Wright Lville"
 ip address xxx.xxx.122.49 255.255.255.252
!
interface gigabitethernet1
 description "Telpage Bench GE1"
 switchport mode access
 switchport access vlan 30
 no macro auto persistent
!
interface gigabitethernet2
 description "Uplink to Ziggy GE2"
 switchport mode access
 switchport access vlan 120
 no macro auto persistent
!
interface gigabitethernet3
 description "Uplink to ASA-0 GiG0/0 GE3"
 switchport mode access
 switchport access vlan 120
 no macro auto persistent
!
interface gigabitethernet4
 description "Uplink to ASA-1 GiG0/0 GE4"
 switchport mode access
 switchport access vlan 120
 no macro auto persistent
!
interface gigabitethernet5
 speed 100
 no negotiation
 description "GE5"
 switchport trunk native vlan 2
 no macro auto persistent
!
interface gigabitethernet6
 speed 100
 no negotiation
 description "Hampton Inn GE6"
 switchport trunk native vlan 32
 no macro auto persistent
!
interface gigabitethernet7
 no negotiation
 description "Fairfield Inn GE7"
 switchport trunk native vlan 96
 no macro auto persistent
!
interface gigabitethernet8
 speed 100
 no negotiation
 description "Owen Ford GE8"
 rate-limit 31744 31744
 traffic-shape 21504 21504
 switchport trunk native vlan 128
 no macro auto persistent
!
interface gigabitethernet9
 speed 100
 description "Greensville County Campus GE9"
 switchport trunk native vlan 160
 no macro auto persistent
!
interface gigabitethernet10
 speed 100
 no negotiation
 description "Boyd Chevrolet GE10"
 switchport trunk native vlan 192
 no macro auto persistent
!
interface gigabitethernet11
 speed 100
 description "CIMC GE11"
 switchport trunk allowed vlan add 30
 switchport trunk native vlan 155
 no macro auto persistent
!
interface gigabitethernet12
 speed 100
 no negotiation
 description "Mark Pope GE12"
 switchport trunk native vlan 22
 no macro auto persistent
!
interface gigabitethernet13
 speed 100
 no negotiation
 description "P&S Transportation GE13"
 switchport trunk allowed vlan add 30,64,120,208,248
 no macro auto persistent
!
interface gigabitethernet14
 speed 100
 description "GE14"
 no macro auto persistent
!
interface gigabitethernet15
 description "OSG GE15"
 rate-limit 11264 128000
 traffic-shape 11264
 switchport trunk native vlan 320
!
interface gigabitethernet16
 no negotiation
 description "Pleasant Shade GE16"
 switchport trunk allowed vlan add 30,120,232,240
 no macro auto persistent
!
interface gigabitethernet17
 speed 100
 description "Remac GE17"
 switchport mode access
 switchport access vlan 30
 no macro auto persistent
!
interface gigabitethernet18
 description "Virginia Pallets GE18"
 switchport mode access
 switchport access vlan 80
 no macro auto persistent
!
interface gigabitethernet19
 description "Wrights Auto Lville GE19"
 switchport trunk allowed vlan add 155
 switchport trunk native vlan 30
 no macro auto persistent
!
interface gigabitethernet20
 description "GE20"
 port monitor GigabitEthernet 16
 no macro auto persistent
!
interface gigabitethernet21
 description "GE21"
!
interface gigabitethernet22
 description "GE22"
 switchport mode access
 switchport access vlan 30
 no macro auto persistent
!
interface gigabitethernet23
 description "GE23"
 switchport mode access
 switchport access vlan 30
!
interface gigabitethernet24
 description "GE24"
 switchport mode access
 no macro auto persistent
!
interface gigabitethernet25
 description "Uplink to NOC Core GiG/048 GE25"
 switchport trunk allowed vlan add 2,22,30,32,64,68,80,96,121,128,155
 switchport trunk allowed vlan add 160,192,208,224,232,240,284,320
 switchport trunk native vlan 120
 switchport default-vlan tagged
!
interface gigabitethernet26
 description "Davis Tower GE26"
 switchport mode access
 switchport access vlan 30
 no macro auto persistent
!
interface gigabitethernet27
 description "Gig Fiber Circuit GE27"
 switchport trunk allowed vlan add 2247,2251
 switchport trunk native vlan 30
!
interface gigabitethernet28
 no negotiation
 description "Uplink to ASR GiG0/0/2 GE28"
 switchport mode access
 switchport access vlan 120
 no macro auto persistent
!
exit
macro auto disabled
macro auto processing type ip_phone disabled
macro auto processing type ip_phone_desktop disabled
macro auto processing type switch disabled
macro auto processing type ap disabled
ip default-gateway xxx.xxx.120.1
ip ssh-client key rsa key-pair

[Jon Marshall]

It is not entirely clear how the addressing works but when you pointed the routes to 120.4 did you also move the customer L3 vlan interfaces to that switch as well because that switch will now be routing between vlans. 

Jon

[gboyce]

Im sorry, Yes, I moved the VLANs that are on the NNI to the other switch when I made the routing changes and removed them from the original switch.

[Jon Marshall]

Did you create the L3 interfaces for the vlans ? 

I am trying to match up the configurations you posted with your diagram ie. your diagram shows vlans 1216 and 1224 but I cannot see any L3 interfaces for those vlans on either switch ? 

Jon

[gboyce]

---

@Jon Marshall wrote:

Did you create the L3 interfaces for the vlans ? 

I am trying to match up the configurations you posted with your diagram ie. your diagram shows vlans 1216 and 1224 but I cannot see any L3 interfaces for those vlans on either switch ? 

Jon

---

I'm sorry for the confusion. The network diagram was just a mockup. The VLANs and IPs on that are arbitrary. The structure is accurate. I didn't realize I would be putting up my configs when I started this. 1216 and 1224 are actually 2247 and 2251. When I created the VLANs, I put the gateway address for the subnet assigned to that customer on the VLAN.

[Jon Marshall]

That makes more sense :)

So when you moved the vlan across you also had for each of those vlans a L3 interface eg. 

int vlan <x>
ip address x.x.x.x <subnet mask>

and you checked the vlan interfaces were up "sh ip int br | inc Vlan" . 

Finally bear in mind the clients will have an arp cache mapping the default gateway IP to the mac address of the old switch vlan interfaces so they may need to clear their arp cache. 

Jon

[gboyce]

---

@Jon Marshall wrote:

That makes more sense :)

So when you moved the vlan across you also had for each of those vlans a L3 interface eg. 

int vlan <x>
ip address x.x.x.x <subnet mask>

and you checked the vlan interfaces were up "sh ip int br | inc Vlan" . 

Finally bear in mind the clients will have an arp cache mapping the default gateway IP to the mac address of the old switch vlan interfaces so they may need to clear their arp cache. 

Jon

---

That's exactly right. However, most of my fiber customers remained in SW2 and their routes back to the router didn't change. None of them or the moved NNI customers could get online after the updated router routes.

The routes in the router only changed from sending the entire /24 subnet down to SW2 to sending each subnetted network to SW2.

so...

From: 

x.x.121.0 255.255.255.0 x.x.120.2

x.x.122.0 255.255.255.0 x.x120.2

To:

xxx.xxx.121.32 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.64 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.96 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.128 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.160 255.255.255.224 xxx.xxx.120.2
xxx.xxx.121.192 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.200 255.255.255.252 xxx.xxx.120.2
xxx.xxx.121.208 255.255.255.240 xxx.xxx.120.2
xxx.xxx.121.248 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.232 255.255.255.248 xxx.xxx.120.2
xxx.xxx.121.240 255.255.255.248 xxx.xxx.120.2
xxx.xxx.122.0 255.255.255.224 xxx.xxx.120.2
xxx.xxx.122.32 255.255.255.248 xxx.xxx.120.2
xxx.xxx.122.64 255.255.255.240 xxx.xxx.120.2
xxx.xxx.122.80 255.255.255.252 xxx.xxx.120.2

The only routes that changed to a different switch were the two NNI customers

xxx.xxx.122.40 255.255.255.248 xxx.xxx.120.4
xxx.xxx.122.47 255.255.255.252 xxx.xxx.120.4

The problem is that all of these routes failed to get all the customer out to the internet. I could still get to the WAN interface of their router/firewall from any part of my network, including the x.x.125.0, x.x.126.0, x.x.127.0, and x.x.120.0 networks.

[Jon Marshall]

 You haven't included the first two octets of the subnets so this is a bit of guess work but I think the issue is with your BGP configuration. 

So for BGP to advertise a network using the network statement it must find an exact match for that route in the IP routing table which when you had just the two static routes it did match.  But you now have a lot of /27 subnets and this will not match with BGP network statements. 

Assuming your provider will accept /24's at a minimum you will need, in addition to your /27 static routes to also have a /24 static route as before to match the BGP network statements and the next hop could be either switch as the more specific route should always be used (or you could use null0 as the next hop for the same reason).

Like I say a bit of guesswork but I suspect that is the issue. 

Hope that makes sense. 

Jon

[gboyce]

How can I help take the guess work out of the equation without giving my public IP information to everyone? I don't mind communicating that privately. 

[Jon Marshall]

Are your public IP's class C networks ? 

Jon

[gboyce]

The BGP address is 38.122.x.x 255.255.255.248   

The customer Publics are x.x.120.0/24, 121.0/24, 122.0/24, 123.0/24, 124.0/24, 125.0/24, 126.0/24, 127.0/24

[Jon Marshall]

Stupid me, just realised your subnets have 255.255.255.0 masks so yes class C. 

Can you just confirm when you changed all the static routes to the more specific ones did you remove the original two static routes ? 

Jon

[gboyce]

Yes, I removed the original 2 statements from the router, then added the more specific ones.