Solved: New MOE Connection Problems

Ed Willson · ‎03-02-2012

Hi All,

I've just turned up a new MPLS MOE connection between a new site and and the HQ. Currently all the other branches are running another providers MPLS network, and we're working on transitioning to this new carrier. I was expecting to route to the IP I was given by the carrier, but it's not working and I'm at a loss as to why.

Here's the setup:

Headquarters has a 2800 router and the remote site has a 3750x switch.

The carrier gave me a /30 for inside addresses at each end. I can ping the inside address from each location, but I cannot ping any other interfaces that are on one side from the other.

Any help would be great. I've been banging my head for a dozen hours so far.

Thanks,

Skymeat

Peter Paluch · ‎03-03-2012

Hello Ed,

From what you wrote I assume that from one CE router, you can ping the PE/CE link on the other CE router, however, you can not ping behind that CE router - am I correct? What kind of MPLS interconnection is this - L2 pseudowire or L3 VPN? I assume the latter.

If that is true then in my opinion, the problem is in the exchange of the routing information between your sites and the MPLS L3VPN. You are using static routing but in MPLS L3 VPN, it is necessary for provider's PE routers to know what networks are located on your individual sites. I do not assume that this knowledge has been put into the PE routers statically by your service provider; instead, I believe, you should run a routing protocol between your CE and PE routers. The exact details should have been told you by your service provider.

Another problem I see here is the definition of the default route on your 3750 switch: it is always wrong to define a static default route out via a multiaccess interface, such as Vlan300. That makes the route appear as directly connected, and the switch assumes that all stations within that network are directly connected. Surely, the entire internet is not directly sitting on your Vlan300 This makes your switch to totally depend on ProxyARP of the neighboring PE router, causes exorbitantly large ARP tables and traffic, and if the ProxyARP is deactivated, it does not work at all. I strongly recommend removing the line

ip route 0.0.0.0 0.0.0.0 Vlan300

and replace it with

ip route 0.0.0.0 0.0.0.0 216.206.123.177

Beware that doing this remotely may cause lock you out from the device - it is better performed from a console, or by first entering the second command, and only then removing the first command (the existing one).

Best regards,

Peter

View solution in original post

Ed Willson · ‎03-02-2012

Here's the HQ config. The interface in question is g 0/1.832:

HQ#show run

Building configuration...

Current configuration : 5470 bytes

!

version 12.4

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

!

hostname FWG-Hen-RTR-MPLS

!

boot-start-marker

boot-end-marker

!

card type t1 0 0

card type t1 0 1

logging message-counter syslog

logging buffered 10000000

enable secret 5

!

no aaa new-model

no network-clock-participate wic 0

no network-clock-participate wic 1

!

dot11 syslog

ip source-route

!

ip cef

!

multilink bundle-name authenticated

!

archive

log config

hidekeys

!

controller T1 0/0/0

fdl ansi

cablelength long 0db

channel-group 0 timeslots 1-24

!

controller T1 0/0/1

fdl ansi

cablelength long 0db

channel-group 0 timeslots 1-24

!

controller T1 0/1/0

fdl ansi

cablelength long 0db

channel-group 0 timeslots 1-24

!

controller T1 0/1/1

fdl ansi

cablelength long 0db

channel-group 0 timeslots 1-24

!

class-map match-all Oracle

match access-group 160

class-map match-all RDP

match access-group 102

class-map match-all voice-priority

match access-group 101

!

policy-map POLICY1

class voice-priority

set ip precedence 5

priority percent 40

class RDP

bandwidth percent 20

set ip precedence 4

class Oracle

set ip precedence 4

bandwidth percent 15

class class-default

fair-queue

set ip precedence 0

!

buffers tune automatic

!

interface Multilink1

ip address 192.168.253.2 255.255.255.252

ip flow ingress

ip flow egress

ppp multilink

ppp multilink interleave

ppp multilink group 1

ppp multilink fragment disable

service-policy output POLICY1

!

interface GigabitEthernet0/0

ip address 10.10.30.1 255.255.0.0 secondary

ip address 10.10.30.2 255.255.0.0

ip flow ingress

ip flow egress

duplex full

speed 100

!

interface GigabitEthernet0/1

description SWI_2

no ip address

ip flow ingress

ip flow egress

duplex auto

speed auto

!

interface GigabitEthernet0/1.832

encapsulation dot1Q 832

ip address 63.156.54.138 255.255.255.252

ip flow ingress

ip flow egress

!

interface Serial0/0/0:0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

interface Serial0/0/1:0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

interface Serial0/1/0:0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

interface Serial0/1/1:0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

interface Serial0/2/0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

interface Serial0/3/0

bandwidth 1536

no ip address

ip flow ingress

ip flow egress

encapsulation ppp

ppp multilink

ppp multilink group 1

!

ip forward-protocol nd

ip route 0.0.0.0 0.0.0.0 10.10.10.3

ip route 10.5.0.0 255.255.0.0 216.206.123.177

ip route 10.16.0.0 255.255.0.0 192.168.253.1

ip route 10.20.0.0 255.255.0.0 192.168.253.1

ip route 10.21.0.0 255.255.0.0 192.168.253.1

ip route 10.22.0.0 255.255.0.0 192.168.253.1

ip route 10.23.0.0 255.255.0.0 192.168.253.1

ip route 10.30.0.0 255.255.0.0 192.168.253.1

ip route 10.40.0.0 255.255.0.0 192.168.253.1

ip route 10.50.0.0 255.255.0.0 192.168.253.1

ip route 10.60.0.0 255.255.0.0 192.168.253.1

ip route 10.70.0.0 255.255.0.0 192.168.253.1

ip route 10.80.0.0 255.255.0.0 192.168.253.1

ip route 10.85.0.0 255.255.0.0 192.168.253.1

ip route 10.90.0.0 255.255.0.0 192.168.253.1

ip route 10.100.0.0 255.255.0.0 192.168.253.1

ip route 10.110.0.0 255.255.0.0 10.10.10.3

ip route 10.120.0.0 255.255.0.0 192.168.253.1

ip route 10.130.0.0 255.255.0.0 192.168.253.1

ip route 74.123.58.128 255.255.255.240 192.168.253.1

ip route 74.123.60.32 255.255.255.240 192.168.253.1

ip route 140.85.0.0 255.255.0.0 192.168.253.1

ip route 141.146.128.0 255.255.128.0 192.168.253.1

ip route 172.25.106.0 255.255.255.0 192.168.253.1

ip route 172.25.234.192 255.255.255.192 192.168.253.1

ip route 172.26.16.0 255.255.255.0 192.168.253.1

ip route 172.26.74.0 255.255.255.0 192.168.253.1

ip route 172.26.75.0 255.255.255.0 192.168.253.1

ip route 192.168.0.0 255.255.255.0 192.168.253.1

ip route 192.168.253.0 255.255.255.0 192.168.253.1

ip route 216.206.123.176 255.255.255.252 GigabitEthernet0/1.832

no ip http server

no ip http secure-server

!

ip flow-export source GigabitEthernet0/0

ip flow-export version 5

ip flow-export destination 10.10.80.9 2055

!

logging trap notifications

logging source-interface GigabitEthernet0/0

logging 10.10.80.9

access-list 101 permit ip any any dscp ef

access-list 102 permit tcp any eq 3389 any

access-list 102 permit tcp any any eq 3389

access-list 160 permit ip any 172.25.106.0 0.0.0.255

access-list 160 permit ip any 172.25.234.192 0.0.0.63

access-list 160 permit ip any 172.24.106.0 0.0.0.255

access-list 160 permit ip any 172.24.234.192 0.0.0.63

!

snmp-server community luigi RO

snmp-server community locutus RW

snmp-server ifindex persist

snmp-server trap-source GigabitEthernet0/0

snmp-server enable traps config

snmp mib persist circuit

!

control-plane

!

line con 0

password 7

login

line aux 0

password 7

login

modem InOut

transport input all

stopbits 1

flowcontrol hardware

line vty 0 4

password 7

login

!

scheduler allocate 20000 1000

end

HQ#show ip route

Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route

Gateway of last resort is 10.10.10.3 to network 0.0.0.0

S 149.85.0.0/16 [1/0] via 192.168.253.1

149.146.0.0/17 is subnetted, 1 subnets

S 141.146.128.0 [1/0] via 192.168.253.1

172.25.0.0/16 is variably subnetted, 2 subnets, 2 masks

S 172.25.234.192/26 [1/0] via 192.168.253.1

S 172.25.106.0/24 [1/0] via 192.168.253.1

172.26.0.0/24 is subnetted, 3 subnets

S 172.26.16.0 [1/0] via 192.168.253.1

S 172.26.74.0 [1/0] via 192.168.253.1

S 172.26.75.0 [1/0] via 192.168.253.1

216.206.123.0/30 is subnetted, 1 subnets

S 216.206.123.176 is directly connected, GigabitEthernet0/1.832

10.0.0.0/16 is subnetted, 19 subnets

C 10.10.0.0 is directly connected, GigabitEthernet0/0

S 10.5.0.0 [1/0] via 216.206.123.177

S 10.30.0.0 [1/0] via 192.168.253.1

S 10.16.0.0 [1/0] via 192.168.253.1

S 10.22.0.0 [1/0] via 192.168.253.1

S 10.23.0.0 [1/0] via 192.168.253.1

S 10.20.0.0 [1/0] via 192.168.253.1

S 10.21.0.0 [1/0] via 192.168.253.1

S 10.40.0.0 [1/0] via 192.168.253.1

S 10.60.0.0 [1/0] via 192.168.253.1

S 10.50.0.0 [1/0] via 192.168.253.1

S 10.70.0.0 [1/0] via 192.168.253.1

S 10.90.0.0 [1/0] via 192.168.253.1

S 10.80.0.0 [1/0] via 192.168.253.1

S 10.85.0.0 [1/0] via 192.168.253.1

S 10.110.0.0 [1/0] via 10.10.10.3

S 10.100.0.0 [1/0] via 192.168.253.1

S 10.120.0.0 [1/0] via 192.168.253.1

S 10.130.0.0 [1/0] via 192.168.253.1

S 192.168.0.0/24 [1/0] via 192.168.253.1

63.0.0.0/30 is subnetted, 1 subnets

C 63.156.54.136 is directly connected, GigabitEthernet0/1.832

192.168.253.0/24 is variably subnetted, 3 subnets, 3 masks

C 192.168.253.1/32 is directly connected, Multilink1

C 192.168.253.0/30 is directly connected, Multilink1

S 192.168.253.0/24 [1/0] via 192.168.253.1

74.0.0.0/28 is subnetted, 2 subnets

S 74.123.60.32 [1/0] via 192.168.253.1

S 74.123.58.128 [1/0] via 192.168.253.1

S* 0.0.0.0/0 [1/0] via 10.10.10.3

Ed Willson · ‎03-02-2012

Here's the remote site. The interface in question is vlan 300:

Remote#show run

Building configuration...

Current configuration : 2158 bytes

!

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

!

hostname FRS-Van-Swi-ACore

!

boot-start-marker

boot-end-marker

!

enable password 7

!

no aaa new-model

switch 1 provision ws-c3750x-24

system mtu routing 1500

ip routing

!

vtp domain Stuff

vtp mode transparent

!

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

vlan 300,1000

!

interface FastEthernet0

no ip address

no ip route-cache cef

no ip route-cache

no ip mroute-cache

!

interface GigabitEthernet1/0/1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface GigabitEthernet1/1/1

!

interface GigabitEthernet1/1/2

!

interface GigabitEthernet1/1/3

!

interface GigabitEthernet1/1/4

!

interface TenGigabitEthernet1/1/1

!

interface TenGigabitEthernet1/1/2

!

interface Vlan1

ip address 10.5.41.1 255.255.0.0

!

interface Vlan300

ip address 216.206.123.178 255.255.255.252

!

ip classless

ip route 0.0.0.0 0.0.0.0 Vlan300

no ip http server

no ip http secure-server

!

ip sla enable reaction-alerts

logging trap notifications

logging 10.10.80.9

!

line con 0

line vty 0 4

password 7

login

line vty 5 15

login

!

end

Remote#show ip route

Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

216.206.123.0/30 is subnetted, 1 subnets

C 216.206.123.176 is directly connected, Vlan300

10.0.0.0/16 is subnetted, 1 subnets

C 10.5.0.0 is directly connected, Vlan1

S* 0.0.0.0/0 is directly connected, Vlan300

FRS-Van-Swi-ACore#show int tru

Port Mode Encapsulation Status Native vlan

Gi1/0/1 on 802.1q trunking 1

Port Vlans allowed on trunk

Gi1/0/1 1-4094

Port Vlans allowed and active in management domain

Gi1/0/1 1,300,1000

Port Vlans in spanning tree forwarding state and not pruned

Gi1/0/1 1,300,1000

Peter Paluch · ‎03-03-2012

Hello Ed,

From what you wrote I assume that from one CE router, you can ping the PE/CE link on the other CE router, however, you can not ping behind that CE router - am I correct? What kind of MPLS interconnection is this - L2 pseudowire or L3 VPN? I assume the latter.

If that is true then in my opinion, the problem is in the exchange of the routing information between your sites and the MPLS L3VPN. You are using static routing but in MPLS L3 VPN, it is necessary for provider's PE routers to know what networks are located on your individual sites. I do not assume that this knowledge has been put into the PE routers statically by your service provider; instead, I believe, you should run a routing protocol between your CE and PE routers. The exact details should have been told you by your service provider.

Another problem I see here is the definition of the default route on your 3750 switch: it is always wrong to define a static default route out via a multiaccess interface, such as Vlan300. That makes the route appear as directly connected, and the switch assumes that all stations within that network are directly connected. Surely, the entire internet is not directly sitting on your Vlan300 This makes your switch to totally depend on ProxyARP of the neighboring PE router, causes exorbitantly large ARP tables and traffic, and if the ProxyARP is deactivated, it does not work at all. I strongly recommend removing the line

ip route 0.0.0.0 0.0.0.0 Vlan300

and replace it with

ip route 0.0.0.0 0.0.0.0 216.206.123.177

Beware that doing this remotely may cause lock you out from the device - it is better performed from a console, or by first entering the second command, and only then removing the first command (the existing one).

Best regards,

Peter

Ed Willson · ‎03-03-2012

Peter,

Thanks for the heads up on default route. I was at the point of just trying things and banging away senselessly at the keys when I posted those configs.

I Just got done talking to the provider, and they forgot to add our networks to thier routers. So nothing can be done till Monday, but I'm hoping that it can be resolved then. I'm pretty sure you called the problem correctly.

Thanks,

Ed

Peter Paluch · ‎03-03-2012

Hi Ed,

    I Just got done talking to the provider, and they forgot to add our networks to thier routers.

Man... they're deploying a MPLS L3 VPN infrastructure and forget to deal with routing which is the very essence of this VPN type? I can't believe my eyes

Nevertheless, please let me know if they got it running on Monday.

Best regards,

Peter

Ed Willson · ‎03-05-2012

The proider added our internal networks and everything's looking peachy. Thanks for the help!

Thanks,

Ed

Peter Paluch · ‎03-05-2012

Hi Ed,

Don't mention it - I actually did not help at all here. Glad to see you've got it working!

Best regards,

Peter