EoMPLS VC is up, but with very small MTU (42 bytes)

Eric Xu · ‎05-15-2016

Good afternoon experts,

I have two directly connected Cisco 7609 routers running MPLS with OSPF being IGP, and a EoMPLS VC has been created.

On router 1:

interface Vlan100
ip address 192.168.168.2 255.255.255.0
xconnect 10.200.0.158 100 encapsulation mpls

On router 2:

interface Vlan100
ip address 192.168.168.3 255.255.255.0
xconnect 10.200.0.157 100 encapsulation mpls

VC is up on both routers with MTU 1500:

router1#sh mpls l2transport vc 100 detail
Local interface: Vl100 up, line protocol up, Eth VLAN 100 up
Interworking type is Ethernet
Destination address: 10.200.0.158, VC ID: 100, VC status: up
Output interface: Te1/0/0.18000064, imposed label stack {16}
Preferred path: not configured
Default path: active
Next hop: 10.200.0.38
Create time: 00:12:14, last status change time: 00:12:14
Last label FSM state change time: 00:12:14
Signaling protocol: LDP, peer 10.200.0.158:0 up
Targeted Hello: 10.200.0.157(LDP Id) -> 10.200.0.158, LDP is UP
Status TLV support (local/remote) : enabled/supported
LDP route watch : enabled
Label/status state machine : established, LruRru
Last local dataplane status rcvd: No fault
Last BFD dataplane status rcvd: Not sent
Last BFD peer monitor status rcvd: No fault
Last local AC circuit status rcvd: No fault
Last local AC circuit status sent: No fault
Last local PW i/f circ status rcvd: No fault
Last local LDP TLV status sent: No fault
Last remote LDP TLV status rcvd: No fault
Last remote LDP ADJ status rcvd: No fault
MPLS VC labels: local 214, remote 16
Group ID: local 0, remote 0
MTU: local 1500, remote 1500
Remote interface description:
Sequencing: receive disabled, send disabled
Control Word: On (configured: autosense)

router2#sh mpls l2transport vc 100 detail
Local interface: Vl100 up, line protocol up, Eth VLAN 100 up
Interworking type is Ethernet
Destination address: 10.200.0.157, VC ID: 100, VC status: up
Output interface: Te2/11.18000064, imposed label stack {214}
Preferred path: not configured
Default path: active
Next hop: 10.200.0.37
Create time: 00:13:06, last status change time: 00:12:53
Last label FSM state change time: 00:12:53
Signaling protocol: LDP, peer 10.200.0.157:0 up
Targeted Hello: 10.200.0.158(LDP Id) -> 10.200.0.157, LDP is UP
Status TLV support (local/remote) : enabled/supported
LDP route watch : enabled
Label/status state machine : established, LruRru
Last local dataplane status rcvd: No fault
Last BFD dataplane status rcvd: Not sent
Last BFD peer monitor status rcvd: No fault
Last local AC circuit status rcvd: No fault
Last local AC circuit status sent: No fault
Last local PW i/f circ status rcvd: No fault
Last local LDP TLV status sent: No fault
Last remote LDP TLV status rcvd: No fault
Last remote LDP ADJ status rcvd: No fault
MPLS VC labels: local 16, remote 214
Group ID: local 0, remote 0
MTU: local 1500, remote 1500
Remote interface description:
Sequencing: receive disabled, send disabled
Control Word: On (configured: autosense)
SSO Descriptor: 10.200.0.157/100, local label: 16
Dataplane:
SSM segment/switch IDs: 4144/4142 (used), PWID: 9
VC statistics:
transit packet totals: receive 32, send 15
transit byte totals: receive 2318, send 1020
transit packet drops: receive 0, seq error 0, send 0

MTU between the routers are more than 9000:

router1#ping 10.200.0.158 size 9000 df-bit source l0
Type escape sequence to abort.
Sending 5, 9000-byte ICMP Echos to 10.200.0.158, timeout is 2 seconds:
Packet sent with a source address of 10.200.0.157
Packet sent with the DF bit set
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/13/16 ms

router2#ping 10.200.0.157 size 9000 df-bit source l0
Type escape sequence to abort.
Sending 5, 9000-byte ICMP Echos to 10.200.0.157, timeout is 2 seconds:
Packet sent with a source address of 10.200.0.158
Packet sent with the DF bit set
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/20/44 ms

However, MTU on the VC is only 42 bytes, which makes it unusable:

router1#ping 192.168.168.3 size 42 df-bit source vlan 100
Type escape sequence to abort.
Sending 5, 42-byte ICMP Echos to 192.168.168.3, timeout is 2 seconds:
Packet sent with a source address of 192.168.168.2
Packet sent with the DF bit set
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/12 ms

router1#ping 192.168.168.3 size 43 df-bit source vlan 100
Type escape sequence to abort.
Sending 5, 43-byte ICMP Echos to 192.168.168.3, timeout is 2 seconds:
Packet sent with a source address of 192.168.168.2
Packet sent with the DF bit set
....!
Success rate is 20 percent (1/5), round-trip min/avg/max = 8/8/8 ms

router1#ping 192.168.168.3 size 44 df-bit source vlan 100
Type escape sequence to abort.
Sending 5, 44-byte ICMP Echos to 192.168.168.3, timeout is 2 seconds:
Packet sent with a source address of 192.168.168.2
Packet sent with the DF bit set
.....
Success rate is 0 percent (0/5)

Any ideas why?

Thank you.

Victor Acevedo · ‎05-16-2016

Hey Eric,

It looks like you are trying to implement a Routed Pseudowire and are generating the pings on the local PE's. Do you also see this problem if you connect another device to each PE and send the pings across the PE's?

Please share a "show ver" and "show mod" from both PE's

Thanks,

Vic

Eric Xu · ‎05-16-2016

Hi Vic,

Thanks for your reply.

I have the same problem if I connected another device to each PE and send the pings across the PEs.

"show ver" and "show mod" on both PE. The cards in use are in bold, and serial numbers have been removed.

1. router1
router1#sh ver
Cisco IOS Software, c7600s72033_rp Software (c7600s72033_rp-ADVIPSERVICESK9-M), Version 15.2(4)S6, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Fri 08-Aug-14 09:47 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)S4, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, c7600s72033_rp Software (c7600s72033_rp-ADVIPSERVICESK9-M), Version 15.2(4)S6, RELEASE SOFTWARE (fc1)

SydEquinix7609 uptime is 37 weeks, 3 days, 16 hours, 1 minute
Uptime for this control processor is 37 weeks, 3 days, 15 hours, 55 minutes
System returned to ROM by reload (SP by reload)
System restarted at 08:11:56 EST Fri Aug 28 2015
System image file is "disk0:/c7600s72033-advipservicesk9-mz.152-4.S6.bin"
Last reload type: Normal Reload
Last reload reason: Reload Command

cisco CISCO7609 (R7000) processor (revision 1.1) with 458720K/65536K bytes of memory.
Processor board ID
SR71000 CPU at 600MHz, Implementation 1284, Rev 1.2, 512KB L2 Cache
Last reset from s/w reset
2 controllers (4 TenGigabitEthernet).
9 Virtual Ethernet interfaces
76 Gigabit Ethernet interfaces
12 Ten Gigabit Ethernet interfaces
1917K bytes of non-volatile configuration memory.
8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102

router1#sh module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 2 ESM20G 7600-ES20-10G3C
3 20 7600 ES+ 7600-ES+20G3C
4 20 7600 ES+ 7600-ES+20G3C
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B
6 2 Supervisor Engine 720 (Hot) WS-SUP720-3B
7 16 SFM-capable 16 port 10/100/1000mb RJ45 WS-X6516-GE-TX
8 16 SFM-capable 16 port 10/100/1000mb RJ45 WS-X6516-GE-TX
9 2 ESM20G 7600-ES20-10G3C

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ----- ------------- ------------ -------
1 0022.9046.2700 to 0022.9046.277f 1.4 12.2(33r)SRB 15.2(4)S Ok
3 8843.e12d.6040 to 8843.e12d.607f 1.3 12.2(33r)SRD 15.2(4)S Ok
4 5475.d0d1.f940 to 5475.d0d1.f97f 1.3 12.2(33r)SRD 15.2(4)S Ok
5 0015.c62f.d010 to 0015.c62f.d013 4.6 8.4(2 15.2(4)S Ok
6 0019.e7d4.4100 to 0019.e7d4.4103 5.6 8.5(2 15.2(4)S Ok
7 0005.7486.25e8 to 0005.7486.25f7 2.2 6.3(1) (sierra_main Ok
8 0009.11e2.c434 to 0009.11e2.c443 2.2 6.3(1) (sierra_main Ok
9 001c.58b0.e880 to 001c.58b0.e8ff 1.0 12.2(33r)SRB 15.2(4)S Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
1 ESM20G/PFC3C Distributed Fo 7600-ES20-D3C Ok
1/0 1x10GE XFP Port 7600-ES20-2X10G 1.0 Ok
1/1 1x10GE XFP Port 7600-ES20-2X10G 1.0 Ok
3 7600 ES+ DFC LITE 7600-ES+3C 1.1 Ok
3 7600 ES+ 20xGE SFP 7600-ES+20G 1.1 Ok
4 7600 ES+ DFC LITE 7600-ES+3C 1.1 Ok
4 7600 ES+ 20xGE SFP 7600-ES+20G 1.1 Ok
5 Policy Feature Card 3 WS-F6K-PFC3B 2.3 Ok
5 MSFC3 Daughterboard WS-SUP720 2.5 Ok
6 Policy Feature Card 3 WS-F6K-PFC3B 2.3 Ok
6 MSFC3 Daughterboard WS-SUP720 3.1 Ok
9 ESM20G/PFC3C Distributed Fo 7600-ES20-D3C 1.0 Ok
9/0 1x10GE XFP Port 7600-ES20-2X10G 1.0 Ok
9/1 1x10GE XFP Port 7600-ES20-2X10G 1.0 Ok

Mod Online Diag Status
---- -------------------
1 Pass
1/0 Pass
1/1 Pass
3 Pass
4 Pass
5 Pass
6 Pass
7 Minor Error
8 Pass
9 Pass
9/0 Pass
9/1 Pass

router2#sh ver
Cisco IOS Software, c7600s72033_rp Software (c7600s72033_rp-ADVIPSERVICESK9-M), Version 15.2(4)S6, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Fri 08-Aug-14 09:47 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)SX7, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, c7600s72033_rp Software (c7600s72033_rp-ADVIPSERVICESK9-M), Version 15.2(4)S6, RELEASE SOFTWARE (fc1)

MelNextDC7609 uptime is 28 weeks, 5 days, 14 hours, 2 minutes
Uptime for this control processor is 28 weeks, 5 days, 13 hours, 40 minutes
System returned to ROM by SSO Switchover at 00:27:45 UTC Mon Oct 20 2014 (SP by power on)
System restarted at 11:39:37 AEST Wed Oct 28 2015
System image file is "disk0:c7600s72033-advipservicesk9-mz.152-4.S6.bin"
Last reload type: Normal Reload
Last reload reason: reload

cisco CISCO7609 (R7000) processor (revision 1.2) with 458720K/65536K bytes of memory.
Processor board ID
SR71000 CPU at 600MHz, Implementation 1284, Rev 1.2, 512KB L2 Cache
Last reset from s/w reset
6 Virtual Ethernet interfaces
34 Gigabit Ethernet interfaces
3 Ten Gigabit Ethernet interfaces
2045K bytes of non-volatile configuration memory.
8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102

router2#sh module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 11 7600 ES+ 76-ES+XC-20G3C
2 11 7600 ES+ 76-ES+XC-20G3C
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B
6 2 Supervisor Engine 720 (Hot) WS-SUP720-3B
9 11 7600 ES+ 76-ES+XC-20G3C

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ----- ------------- ------------ -------
1 7081.05ac.6ed0 to 7081.05ac.6eef 1.5 12.2(33r)SRD 15.2(4)S Warning
2 a493.4c50.f880 to a493.4c50.f89f 1.5 12.2(33r)SRE 15.2(4)S Ok
5 0016.c85e.c7c8 to 0016.c85e.c7cb 5.7 8.5(4 15.2(4)S Ok
6 0017.9444.4d70 to 0017.9444.4d73 5.3 8.5(4 15.2(4)S Ok
9 c84c.7541.ad30 to c84c.7541.ad4f 1.3 12.2(33r)SRE 15.2(4)S Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
1 7600 ES+ DFC LITE 7600-ES+3C 1.2 Warning
1 ES+ 1xTGE XFP 10xGE SFP 7600-ES+20C 1.1 Warning
2 7600 ES+ DFC LITE 7600-ES+3C 1.2 Ok
2 ES+ 1xTGE XFP 10xGE SFP 7600-ES+20C 1.1 Ok
5 Policy Feature Card 3 WS-F6K-PFC3B 2.5 Ok
5 MSFC3 Daughterboard WS-SUP720 3.5 Ok
6 Policy Feature Card 3 WS-F6K-PFC3B 2.5 Ok
6 MSFC3 Daughterboard WS-SUP720 2.11 Ok
9 7600 ES+ DFC LITE 7600-ES+3C 1.1 Ok
9 ES+ 1xTGE XFP 10xGE SFP 7600-ES+20C 1.0 Ok

Mod Online Diag Status
---- -------------------
1 Not Applicable
2 Pass
5 Pass
6 Pass
9 Pass

I also have other similar set up between r1 and r3 (another 7609) and r2 and r4 (yet another 7609). They all work fine.

Thanks.

Eric Xu · ‎05-17-2016

Any help, please?

Thanks.

Victor Acevedo · ‎05-29-2016

Hey Eric,

Can you move the core facing interface in Router1 from module 1 to module 2? Do we see this same problem when we use the ES+ card instead of the ES20 card the core facing interface card? I see on Router 2 you are already using the ES+ card as the core facing.

Thx,

Vic

Eric Xu · ‎05-30-2016

Hi Victor,

As this is a production router, will need to organise outage for that.

Will keep you posted when I do it.

Thanks.

Eric

Vinit Jain · ‎05-24-2016

Hello Eric

I do not see the line cards highlighted in bold which are core facing and customer facing.

Also, could you please share the loopback to loopback ping between the PE routers with size interface MTU and df-bit set. Also, please perform MPLS ping as well.

Would like to know if this is a day 1 problem or was this working before?

Do you have any other working setup. I would like to get the show module from that setup. The reason i am asking for this is, you are running SUP720-3B where as the linecards are 3C.

Regards

Vinit

Thanks
--Vinit

Vinit Jain · ‎05-16-2016

On which module is the core and the customer facing interface on. Are you using any CoPP policy on the edge or core devices?

Please share the sh module n show version output from both the PE routers.

Also, as Victor mentioned, try attaching a CE device to it and initiate the ping from there.

Regards

Vinit

Thanks
--Vinit