cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
907
Views
0
Helpful
1
Replies

Same VSI but cannot PING from two of them

Biotek
Level 1
Level 1

Hello everyone, 

 

I am new to here and also to VPLS service which we just ordered from XO Comminication.

 

We have 3 sites and try to connect all of them together, so in the beginning we decided to go with a full-mesh network and furthermore we asked our DS-1 provider XO and they said VPLS is the best solution for us, which is a layer-2 service.

 

Long story short. In the end we find out it is not a "full mesh" but a "hub and spoke".

 

Say Site 1, 2, 3.

 

Site 1 and Site 2 cannot PING each other at all, but can see the MAC address, as for Site 1, the arp table "occasionally" could find site 2 but most of the time the site 2 CE router MAC shows "Incomplete".

 

The topology we finally could make it work, is to set Site 3 as the Hub, and let other 2 as spoke to communicate. We are not sure whether the structure was determined by XO or not, but it is definitely not we want.

 

As of the configuration, I can paste them as following (All using OSPF), and it works for us:

 

Site 1 Cisco ISR 4321

 

version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname Site1_VPLS
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
enable secret 5 XXXXX
!
no aaa new-model
!
transport-map type persistent telnet TELNET_XX
connection wait none
!
transport-map type persistent telnet XXXXX_vpls_telnet
transport interface GigabitEthernet0
subscriber templating
multilink bundle-name authenticated
!
!
!
!
license udi pid ISR4321/K9 sn XXXXXXXX
license accept end user agreement
license boot level appxk9
!
spanning-tree extend system-id
!
!
redundancy
mode none
!
!
vlan internal allocation policy ascending

interface Loopback0
ip address 10.1.1.1 255.255.255.0
ip ospf network point-to-point
!
interface GigabitEthernet0/0/0
ip address 192.168.3.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet0/0/1
no ip address
speed 100
no negotiation auto
!
interface GigabitEthernet0/0/1.100
encapsulation dot1Q 347
ip address 10.10.10.1 255.255.255.0
ip mtu 1492
ip ospf network point-to-multipoint non-broadcast
!
interface GigabitEthernet0/1/0
shutdown
!
interface GigabitEthernet0/1/1
shutdown
!
interface GigabitEthernet0/1/2
shutdown
!
interface GigabitEthernet0/1/3
shutdown
!
interface GigabitEthernet0
vrf forwarding Mgmt-intf
no ip address
shutdown
negotiation auto
!
interface Vlan1
no ip address
shutdown
!
router ospf 347
passive-interface GigabitEthernet0/0/0
network 10.10.10.1 0.0.0.0 area 0
network 10.10.10.3 0.0.0.0 area 0
network 10.10.10.0 0.0.0.255 area 0
neighbor 10.10.10.3 cost 500
!
ip forward-protocol nd
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet0
!
!
ip access-list extended ALLOWPING
permit ip any any
permit icmp any 10.10.10.0 0.0.0.255 echo-reply
!
!
!
!
control-plane
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
password XXXXX
login
transport type persistent telnet input TELNET_XX
end

 

==================================================

 

Site 2 Cisco ISR 4321  (Also our data center)

 

version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
hostname Site2_VPLS
boot-start-marker
boot-end-marker
vrf definition Mgmt-intf
address-family ipv4
exit-address-family
address-family ipv6
exit-address-family
enable secret 5 XXXXX
no aaa new-model
transport-map type persistent telnet TELNET_XX
connection wait none
transport-map type persistent telnet xxxxx_vpls_telnet
transport interface GigabitEthernet0
subscriber templating
multilink bundle-name authenticated

license udi pid ISR4321/K9 sn XXXXXXXX
license accept end user agreement
license boot level appxk9
spanning-tree extend system-id
redundancy
mode none
vlan internal allocation policy ascending

interface Loopback0
ip address 10.1.1.2 255.255.255.0
ip ospf network point-to-point
!
interface GigabitEthernet0/0/0
ip address 192.168.7.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet0/0/1
no ip address
speed 100
no negotiation auto
!
interface GigabitEthernet0/0/1.100
encapsulation dot1Q 347
ip address 10.10.10.2 255.255.255.0
ip mtu 1492
ip ospf network point-to-multipoint non-broadcast
!
interface GigabitEthernet0/1/0
shutdown
!
interface GigabitEthernet0/1/1
shutdown
!
interface GigabitEthernet0/1/2
shutdown
!
interface GigabitEthernet0/1/3
shutdown
!
interface GigabitEthernet0
vrf forwarding Mgmt-intf
no ip address
shutdown
negotiation auto
!
interface Vlan1
no ip address
shutdown
!
router ospf 347
passive-interface GigabitEthernet0/0/0
network 10.10.10.2 0.0.0.0 area 0
network 10.10.10.3 0.0.0.0 area 0
network 10.10.10.0 0.0.0.255 area 0
neighbor 10.10.10.3 cost 500
!
ip forward-protocol nd
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet0
!
ip access-list extended ALLOWPING
permit ip any any
permit icmp any 10.10.10.0 0.0.0.255 echo-reply
!
control-plane
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
password XXXXX
login
!
transport type persistent telnet input TELNET_XX
!
end

 

=====================================================

 

Site 3 Cisco ISR 4321  (Currently seems to be our hub   :-(  )

 

version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname Site3_VPLS
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
enable secret 5 XXXXX
!
no aaa new-model
!
transport-map type persistent telnet TELNET_XX
connection wait none
!
transport-map type persistent telnet xxxxx_vpls_telnet
transport interface GigabitEthernet0
subscriber templating
multilink bundle-name authenticated

license udi pid ISR4321/K9 sn XXXXXXXX
license accept end user agreement
license boot level appxk9
!
spanning-tree extend system-id
!
!
redundancy
mode none
!
!
vlan internal allocation policy ascending

interface Loopback0
ip address 10.1.1.3 255.255.255.0
ip ospf network point-to-point
!
interface GigabitEthernet0/0/0
ip address 192.168.4.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet0/0/1
no ip address
speed 100
no negotiation auto
!
interface GigabitEthernet0/0/1.100
encapsulation dot1Q 347
ip address 10.10.10.3 255.255.255.0
ip mtu 1492
ip ospf network point-to-multipoint non-broadcast
!
interface GigabitEthernet0/1/0
shutdown
!
interface GigabitEthernet0/1/1
shutdown
!
interface GigabitEthernet0/1/2
shutdown
!
interface GigabitEthernet0/1/3
shutdown
!
interface GigabitEthernet0
vrf forwarding Mgmt-intf
no ip address
shutdown
negotiation auto
!
interface Vlan1
no ip address
shutdown
!
router ospf 347
passive-interface GigabitEthernet0/0/0

network 10.10.10.1 0.0.0.0 area 0
network 10.10.10.2 0.0.0.0 area 0
network 10.10.10.3 0.0.0.0 area 0
network 10.10.10.0 0.0.0.255 area 0
neighbor 10.10.10.1 cost 500

neighbor 10.10.10.3 cost 500
!
ip forward-protocol nd
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet0
!
!
ip access-list extended ALLOWPING
permit ip any any
permit icmp any 10.10.10.0 0.0.0.255 echo-reply
!
!
!
!
control-plane
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
password XXXXX
login
!
transport type persistent telnet input TELNET_XX
!
!
end

 

 

 

 

 

We feel very strange that why CE 1 and CE 2 cannot talk to each other, after endless contacting & opening case with them, here is the configuration on their end which they insist that all MAC addresses have found are all in the same VSI. I guess they are using Juniper devices however I am not a Juniper guy. Here is what they showed me.

 

============================================================

Site 1

jxxxxx@MCR3.Chicago-IL> show vpls mac-table instance VSI-XXXXX-XXXXXXXXXX

MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC)

Routing instance : VSI-XXXXX-XXXXXXXXXX
Bridging domain : __VSI-XXXXX-XXXXXXXXXX__, VLAN : NA
MAC MAC Logical NH RTR
addresssss flags interface Index ID
50:f7:22:1d:89:31 D ae10.3521
ec:1d:8b:91:7b:21 D lsi.1048687
ec:1d:8b:d0:9f:31 D ae10.1233 <=== This circuit

============================================================
Site 2

jxxxxx@MCR3.Chicago-IL> show vpls mac-table instance VSI-XXXXX-XXXXXXXXXX

MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC)

Routing instance : VSI-XXXXX-XXXXXXXXXX
Bridging domain : __VSI-XXXXX-XXXXXXXXXX__, VLAN : NA
MAC MAC Logical NH RTR
addresssss flags interface Index ID
50:f7:22:1d:89:31 D ae10.3521 <=== This circuit
ec:1d:8b:91:7b:21 D lsi.1048687
ec:1d:8b:d0:9f:31 D ae10.1233

================================================================
Site 3

jxxxx@MCR4.Chicago-IL> show vpls mac-table instance VSI-XXXXX-XXXXXXXXXX

MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC)

Routing instance : VSI-XXXXX-XXXXXXXXXX
Bridging domain : __VSI-XXXXX-XXXXXXXXXX__, VLAN : NA
MAC MAC Logical NH RTR
addresssss flags interface Index ID
50:f7:22:1d:89:31 D lsi.1048671
ec:1d:8b:91:7b:21 D ae10.2182 <=== This circuit
ec:1d:8b:d0:9f:31 D lsi.1048671

 

 

 

We saw that Site 1 and Site 2 seems like connects to the same PE router on XO side, and Site 3 connects to another PE, however if i only set both of them with "neighbor 10.10.10.2" and "neighor 10.10.10.1" in the OSPF, and same "ip ospf network point-to-multipoint non-broadcast" applied to the subinterface, seems like they still can see, but they cannot PING each other thus OSPF cannot load FULL for all adjacency, the states always show as "Down".

 

Since we don't want to have this "Hub and Spoke" structure and XO claimed that in their VSI they can see all 3 connected routers MAC addresses and they only handle the layer 2, does anyone see where the problem is? and how to make Site 1 and Site 2 to PING each other without going through Site 3? We do not familiar with SP end how they set up the configuration for us, could that because our 2 sites logical Aggregated Ethernet interface all connects to the same MCR3 router port that blocks the BROADCAST??

 

Thanks a lot!

 

1 Reply 1

Biotek
Level 1
Level 1

Updates:

We turned off the Site 3 since we knew it has been working all good. Only dealing with 1 and 2.

 

For Site 1 and Site 2 they are all connect to the same PE router (XO-MCR3) even we don't have the direct access.

 

We found if we configure both subinterface as NON-BROADCAST, as soon as we clear the arp on both Site 1 and Site 2 CE router, which ever "PING" first, can send the ARP request to another end show the other site MAC address as 000.000.000, so does the other site can receive and response, see the full MAC address, however the other end can never receive the echo back.

 

For example, after site 2 clear arp PING first.

 

SITE_2#ping 10.10.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:

*May 30 17:41:08.002: IP ARP: sent req src 10.10.10.2 50f7.221d.8931,
                 dst 10.10.10.1 0000.0000.0000 GigabitEthernet0/0/1.100.
*May 30 17:41:10.005: IP ARP: sent req src 10.10.10.2 50f7.221d.8931,
                 dst 10.10.10.1 0000.0000.0000 GigabitEthernet0/0/1.100.
*May 30 17:41:12.010: IP ARP: sent req src 10.10.10.2 50f7.221d.8931,
                 dst 10.10.10.1 0000.0000.0000 GigabitEthernet0/0/1.100.
*May 30 17:41:14.014: IP ARP: sent req src 10.10.10.2 50f7.221d.8931,
                 dst 10.10.10.1 0000.0000.0000 GigabitEthernet0/0/1.100.
*May 30 17:41:16.014: IP ARP throttled out the ARP Request for 10.10.10.1.
Success rate is 0 percent (0/5)

and from Site 1 shows:

 

*May 30 17:43:07.707: ARP: flushing ARP entries for all interfaces
*May 30 17:43:07.708: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.1 ffff.ffff.ffff GigabitEthernet0/0/1.100
*May 30 17:43:10.679: IP ARP: rcvd rep src 10.10.10.2 50f7.221d.8931, dst 10.10.10.2 GigabitEthernet0/0/1.100
*May 30 17:43:34.156: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:34.157: IP ARP: creating entry for IP address: 10.10.10.2, hw: 50f7.221d.8931
*May 30 17:43:34.157: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:36.162: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:36.162: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:38.165: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:38.165: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:40.171: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:40.171: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:42.174: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:42.174: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:45.921: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:45.921: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:47.924: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:47.924: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:49.929: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:49.929: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100
*May 30 17:43:51.933: IP ARP: rcvd req src 10.10.10.2 50f7.221d.8931, dst 10.10.10.1 GigabitEthernet0/0/1.100
*May 30 17:43:51.933: IP ARP: sent rep src 10.10.10.1 ec1d.8bd0.9f31,
                 dst 10.10.10.2 50f7.221d.8931 GigabitEthernet0/0/1.100