cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4559
Views
19
Helpful
41
Replies

MPLS LDP Session Establishment failed

Steve
Level 1
Level 1

Hello,

i have tried to connect an

ASR1004(RP2,ESP20,SIP10) and a C6504E(VS-S720-10G)

over 10G direct links.

10G port on C6504E

is located on a

WS-X6708-10GE ( with WS-F6700-DFC3C)

linecard. Both interfaces are

MPLS enabled via mpls ip

IP connectivity between the

mpls loopbacks

of the routers is working fine, can ping with max

MTU 9216 and df-bit set

The tcpdump (via monitor session) from the interface on the

C6504E

shows "LDP Hello Messages" from both routers.

ASR1004

which has the higher

LSR-ID

initiates the LDP TCP Connection to

C6504E

but connection is refused on

C6504E

by sending  TCP (RST,ACK).

What could be the reason for dropping the connection ? i have attached a zip with the pcap trace from interface traffic

 

ASR1004 loopback1 172.16.217.254/32
C6504E loopback1 172.16.211.254/32
ASR1004 te1/0/0 172.16.220.14/30
C6504E te2/1 172.16.220.13/30

Global setting on both routers:

mpls ldp router-id Loopback1 force
C6504E#ping 172.16.220.14 size 9216 df-bit source lo1 repeat 10
Type escape sequence to abort.
Sending 10, 9216-byte ICMP Echos to 172.16.220.14, timeout is 2 seconds:
Packet sent with a source address of 172.16.211.254
Packet sent with the DF bit set
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 1/2/4 ms

ASR1004#ping 172.16.220.13 source lo1 size 9216 df-bit repeat 10
Type escape sequence to abort.
Sending 10, 9216-byte ICMP Echos to 172.16.220.13, timeout is 2 seconds:
Packet sent with a source address of 172.16.217.254
Packet sent with the DF bit set
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 2/2/3 ms

 

41 Replies 41

I run lab and test some thing here 
the mLDP is not effect the LDP session 
the LDP use two protocol UPD for hello and TCP for LDP binding 
so I need some time to more check 

that first thing and I mention before, 
C6504E have CoPP can drop any packet toward the CPU (LO for MPLS ID)
here in lab I add ACL deny udp 646 and result as you have both side have Xmit ONLY 
share below for C6500


show class-map
show policy-map
show policy-map control-plane

Screenshot (772).pngScreenshot (773).png

Hi - no CoPP active

C6504E# show class-map
Class Map match-any class-default (id 0)
Match any

C6504E#show policy-map
Policy Map PE-SET-MPLS-EXP-IMP-VIDEO
Description: ### Mark MPLS experimental Bits to reflect QoS / VIDEO / CoS-ToS: 4 ###
Class class-default
set mpls experimental imposition 4

Policy Map PE-SET-MPLS-EXP-IMP-RESERVED
Description: ### Mark MPLS experimental Bits to reflect QoS / RESERVED / CoS-ToS: 1 ###
Class class-default
set mpls experimental imposition 1

Policy Map PE-SET-MPLS-EXP-IMP-BUSINESS
Description: ### Mark MPLS experimental Bits to reflect QoS / BUSINESS / CoS-ToS: 3 ###
Class class-default
set mpls experimental imposition 3

Policy Map PE-SET-MPLS-EXP-IMP-BESTEFFORT
Description: ### Mark MPLS experimental Bits to reflect QoS / BESTEFFORT / CoS-ToS: 0 ###
Class class-default
set mpls experimental imposition 0

Policy Map PE-SET-MPLS-EXP-IMP-REALTIME
Description: ### Mark MPLS experimental Bits to reflect QoS / REALTIME / CoS-ToS: 5 ###
Class class-default
set mpls experimental imposition 5

Policy Map PE-SET-MPLS-EXP-IMP-STANDARD
Description: ### Mark MPLS experimental Bits to reflect QoS / STANDARD / CoS-ToS: 2 ###
Class class-default
set mpls experimental imposition 2


C6504E#show policy-map control-plane
C6504E#

 

 

SW(config)#mls qos protocol mpls ldp  precedence change ip-precedence(used to map the dscp to cos value) 

there is no CoPP run I check some bug and find one that can cause this issue, the QoS of LDP is low-priority and this make CPU drop it, check the workaround  
CSCsd31503 : Bug Search Tool (cisco.com)

Nice catch - i have to check this later and will update you asap.

Steve
Level 1
Level 1

I have checked BUG Report and tried suggested workaround but it doesn't help. i think i have to check ASR1004 on site again and eliminate the packetloss before we do any further investigation. There is another strange thing with the ASR1004 - why does the LDP interface state stucks in

xmit

even we can see receiving echos in LDP and the resulting tcp session init ?

*Jun 9 13:26:43.485: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 9 13:26:47.323: ldp: Rcvd ldp hello; TenGigabitEthernet1/0/0, from 172.16.220.13 (172.16.211.254:0), intf_id 0, opt 0xC
*Jun 9 13:26:47.323: ldp: ldp Hello from 172.16.220.13 (172.16.211.254:0) to 224.0.0.2, opt 0xC
*Jun 9 13:26:47.323: ldp: New adj 0x7F19D75FA358 for 172.16.211.254:0, TenGigabitEthernet1/0/0
*Jun 9 13:26:47.323: ldp: adj_addr/xport_addr 172.16.220.13/172.16.211.254
*Jun 9 13:26:47.323: ldp: Request adj send hello back on TenGigabitEthernet1/0/0 to (xport addr 172.16.211.254) in 50 msec
*Jun 9 13:26:47.323: ldp: local interface = TenGigabitEthernet1/0/0, holdtime = 15000, peer 172.16.220.13 holdtime = 15000
*Jun 9 13:26:47.323: ldp: Link intvl min cnt 2, intvl 5000, interface TenGigabitEthernet1/0/0
*Jun 9 13:26:47.323: ldp: Opening ldp conn; adj 0x7F19D75FA358, 172.16.217.254 <-> 172.16.211.254; with normal priority
*Jun 9 13:26:47.323: ldp: :{ldp conn 172.16.217.254:47272=>172.16.211.254:646}: Registered tcb 0x7F19CA59D528 [key 189511] with LDP TCB database, total 1
*Jun 9 13:26:47.324: ldp: Conn failed (TCP connect notify)!; adj 0x7F19D75FA358, 172.16.220.13
*Jun 9 13:26:47.324: ldp: : rx_notify tcp_rc 5, adj_state 3
*Jun 9 13:26:47.324: ldp: {ldp conn 172.16.217.254:47272=>172.16.211.254:646} (Te1/0/0) (adj 0x7F19D75FA358): processing transport close request
*Jun 9 13:26:47.324: ldp: Unregistered from LDP TCB database tcb 0x7F19CA59D528 [key 189511], total 0
*Jun 9 13:26:47.325: ldp: Close tcp connection to 172.16.211.254
*Jun 9 13:26:47.325: ldpx_tcp: {ldp conn 172.16.217.254:47272=>172.16.211.254:646}: client requesting close of tcb
*Jun 9 13:26:47.325: ldp: Adj 0x7F19D75FA358; state set to closed

ASR1004#sh mpls ldp discovery
Local LDP Identifier:
172.16.217.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet1/0/0 (ldp): xmit

I will update this thread next week after i was on site

can you share 

show mpls ldp

neighbor detail 

Sorry for the late response -haven't seen your last reply

ASR1004#show mpls ldp neighbor detail
ASR1004#

C6504E#sh mpls ldp neighbor detail | begin 172.16.217.254
C6504#

No entries on both sites - anyway, i have to check hardware on site during this week

 

Denis_from_BE
Level 1
Level 1

Hey Steve, here are some of my current thoughts that can help: 

  • Yes, xmit means it's transmitting its hello messages but does not read any hello's from the facing LSR.
  • Ensure mpls is globaly enabled (no shutdown)
  • LDP uses the IGP for reverse path forwarding to verify the LDP router-ID, as you're using static routing at this moment, the LDP hellos are supposed to be entering the same interface that is used to route to the other LSR. Is it the case at this moment ?
  • Maybe you can set up your new nodes within your final IGP domain as planned. Therefore the router-id must be advertised within the IGP but you know that already. Otherwise, you can also temporary try to establish the LDP neighborship without using Lo1 as source of the LDP messages, but between the physical interface IP, in order to not involve IP routing between the LSR Loopback IP's.
  • Also, I read about some software bugs of using a non /32 IP addr as router-id (Lo1). Which is not your current case anyway.
  • I would not wonder too much about a potential TCP connection problem that is used later on to exchange bindings between peers and create transport sessions. As far as I know, as long as both or LSR can't properly exchange Hello link message, it won't go further in the process.

Hey Denis,

thanks for your input 

>Ensure mpls is globaly enabled (no shutdown)
double checked - nothing disabled

>LDP uses the IGP for reverse path forwarding to verify the LDP router-ID .....
>Is it the case at this moment ?
Yes, the involved interfaces on

ASR1004 and C6504E

are connecetd with a
fiber patch cable and are configured with a

simple /30 subnet


LSR Loopbacks are routed through this interface IPs

>....in order to not involve IP routing between the LSR Loopback IP's
i will give it a try tomorrow

>I would not wonder too much about a potential TCP connection .....
i suppose you are right - will try to simulate this in lab tomorrow
(blocking UDP hellos in one direction to make one LSR

xmit

only
and see what LDP TCP is doing)

Hey Denis

>.....in order to not involve IP routing between the LSR Loopback IP's

Have tried it but nothing has changed..

#ASR1004
interface TenGigabitEthernet1/0/0
mtu 9216
ip address 172.16.220.14 255.255.255.252
no ip redirects
no ip proxy-arp
load-interval 30
mpls ip
mpls ldp discovery transport-address interface
no mpls mldp
!

#C6504E
interface TenGigabitEthernet2/1
mtu 9216
ip address 172.16.220.13 255.255.255.252
no ip redirects
no ip proxy-arp
mpls ldp discovery transport-address interface
no mpls mldp
mpls ip

DEBUG-LOG ASR1004

*Jun 13 06:42:16.611: ldp: Opening ldp conn; adj 0x7F19D75FA358, 172.16.220.14 <-> 172.16.220.13; with normal priority
*Jun 13 06:42:16.611: ldp: :{ldp conn 172.16.220.14:26996=>172.16.220.13:646}: Registered tcb 0x7F19DBD104E0 [key 510840] with LDP TCB database, total 1
*Jun 13 06:42:16.611: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 13 06:42:16.612: ldp: Conn failed (TCP connect notify)!; adj 0x7F19D75FA358, 172.16.220.13
*Jun 13 06:42:16.612: ldp: : rx_notify tcp_rc 5, adj_state 3
*Jun 13 06:42:16.612: ldp: {ldp conn 172.16.220.14:26996=>172.16.220.13:646} (Te1/0/0) (adj 0x7F19D75FA358): processing transport close request
*Jun 13 06:42:16.612: ldp: Unregistered from LDP TCB database tcb 0x7F19DBD104E0 [key 510840], total 0
*Jun 13 06:42:16.613: ldp: Close tcp connection to 172.16.220.13
*Jun 13 06:42:16.613: ldpx_tcp: {ldp conn 172.16.220.14:26996=>172.16.220.13:646}: client requesting close of tcb
*Jun 13 06:42:16.613: ldp: Adj 0x7F19D75FA358; state set to closed
*Jun 13 06:42:21.324: ldp: Rcvd ldp hello; TenGigabitEthernet1/0/0, from 172.16.220.13 (172.16.211.254:0), intf_id 0, opt 0xC
*Jun 13 06:42:21.324: ldp: ldp Hello from 172.16.220.13 (172.16.211.254:0) to 224.0.0.2, opt 0xC
*Jun 13 06:42:21.324: ldp: New adj 0x7F19D75FA358 for 172.16.211.254:0, TenGigabitEthernet1/0/0
*Jun 13 06:42:21.324: ldp: adj_addr/xport_addr 172.16.220.13/172.16.220.13
*Jun 13 06:42:21.324: ldp: Request adj send hello back on TenGigabitEthernet1/0/0 to (xport addr 172.16.220.13) in 50 msec
*Jun 13 06:42:21.324: ldp: local interface = TenGigabitEthernet1/0/0, holdtime = 15000, peer 172.16.220.13 holdtime = 15000
*Jun 13 06:42:21.324: ldp: Link intvl min cnt 2, intvl 5000, interface TenGigabitEthernet1/0/0
*Jun 13 06:42:21.324: ldp: Opening ldp conn; adj 0x7F19D75FA358, 172.16.220.14 <-> 172.16.220.13; with normal priority
*Jun 13 06:42:21.324: ldp: :{ldp conn 172.16.220.14:46043=>172.16.220.13:646}: Registered tcb 0x7F19CA59D528 [key 510845] with LDP TCB database, total 1
*Jun 13 06:42:21.325: ldp: Conn failed (TCP connect notify)!; adj 0x7F19D75FA358, 172.16.220.13
*Jun 13 06:42:21.325: ldp: : rx_notify tcp_rc 5, adj_state 3
*Jun 13 06:42:21.325: ldp: {ldp conn 172.16.220.14:46043=>172.16.220.13:646} (Te1/0/0) (adj 0x7F19D75FA358): processing transport close request
*Jun 13 06:42:21.325: ldp: Unregistered from LDP TCB database tcb 0x7F19CA59D528 [key 510845], total 0
*Jun 13 06:42:21.325: ldp: Close tcp connection to 172.16.220.13
*Jun 13 06:42:21.325: ldpx_tcp: {ldp conn 172.16.220.14:46043=>172.16.220.13:646}: client requesting close of tcb
*Jun 13 06:42:21.325: ldp: Adj 0x7F19D75FA358; state set to closed
*Jun 13 06:42:21.373: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 13 06:42:25.310: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 13 06:42:26.160: ldp: Rcvd ldp hello; TenGigabitEthernet1/0/0, from 172.16.220.13 (172.16.211.254:0), intf_id 0, opt 0xC
*Jun 13 06:42:26.160: ldp: ldp Hello from 172.16.220.13 (172.16.211.254:0) to 224.0.0.2, opt 0xC
*Jun 13 06:42:26.160: ldp: New adj 0x7F19D75FA358 for 172.16.211.254:0, TenGigabitEthernet1/0/0
*Jun 13 06:42:26.160: ldp: adj_addr/xport_addr 172.16.220.13/172.16.220.13
*Jun 13 06:42:26.160: ldp: Request adj send hello back on TenGigabitEthernet1/0/0 to (xport addr 172.16.220.13) in 1 msec
*Jun 13 06:42:26.160: ldp: local interface = TenGigabitEthernet1/0/0, holdtime = 15000, peer 172.16.220.13 holdtime = 15000
*Jun 13 06:42:26.160: ldp: Link intvl min cnt 2, intvl 5000, interface TenGigabitEthernet1/0/0
*Jun 13 06:42:26.160: ldp: Opening ldp conn; adj 0x7F19D75FA358, 172.16.220.14 <-> 172.16.220.13; with normal priority
*Jun 13 06:42:26.160: ldp: :{ldp conn 172.16.220.14:41024=>172.16.220.13:646}: Registered tcb 0x7F19DBD104E0 [key 510850] with LDP TCB database, total 1
*Jun 13 06:42:26.161: ldp: Conn failed (TCP connect notify)!; adj 0x7F19D75FA358, 172.16.220.13
*Jun 13 06:42:26.161: ldp: : rx_notify tcp_rc 5, adj_state 3

DEBUG-LOG C6504E

033313: Jun 13 09:46:15.912 CEST: ldp: bytes_written = 34, at offset = 160
033314: Jun 13 09:46:15.912 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033315: Jun 13 09:46:20.704 CEST: ldp: bytes_written = 34, at offset = 160
033316: Jun 13 09:46:20.704 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033317: Jun 13 09:46:25.661 CEST: ldp: bytes_written = 34, at offset = 160
033318: Jun 13 09:46:25.661 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033319: Jun 13 09:46:29.817 CEST: ldp: bytes_written = 34, at offset = 160
033320: Jun 13 09:46:29.817 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033321: Jun 13 09:46:34.457 CEST: ldp: bytes_written = 34, at offset = 160
033322: Jun 13 09:46:34.457 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033323: Jun 13 09:46:39.001 CEST: ldp: bytes_written = 34, at offset = 160
033324: Jun 13 09:46:39.001 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033325: Jun 13 09:46:43.885 CEST: ldp: bytes_written = 34, at offset = 160
033326: Jun 13 09:46:43.885 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033327: Jun 13 09:46:47.657 CEST: ldp: bytes_written = 34, at offset = 160
033328: Jun 13 09:46:47.657 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033329: Jun 13 09:46:52.173 CEST: ldp: bytes_written = 34, at offset = 160
033330: Jun 13 09:46:52.173 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033331: Jun 13 09:46:56.065 CEST: ldp: bytes_written = 34, at offset = 160
033332: Jun 13 09:46:56.065 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033333: Jun 13 09:47:00.117 CEST: ldp: bytes_written = 34, at offset = 160
033334: Jun 13 09:47:00.117 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033335: Jun 13 09:47:04.861 CEST: ldp: bytes_written = 34, at offset = 160
033336: Jun 13 09:47:04.861 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033337: Jun 13 09:47:09.145 CEST: ldp: bytes_written = 34, at offset = 160
033338: Jun 13 09:47:09.145 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
033339: Jun 13 09:47:13.633 CEST: ldp: Scan listening TCBs

Attached the pcap trace taken from C6504E Te2/1

Status MPLS LDP




ASR1004#sh mpls ldp discovery

 Local LDP Identifier:

    172.16.217.254:0

    Discovery Sources:

    Interfaces:

        TenGigabitEthernet1/0/0 (ldp): xmit




ASR1004#sh mpls ldp neighbor te1/0/0

ASR1004#




C6504E#sh mpls ldp discovery | begin 2/1

TenGigabitEthernet2/1 (ldp): xmit

C6504E#sh mpls ldp neighbor te2/1

C6504E#

 

I keep away the config and debug and focus in wiresharke capture you share 
I found that one side send

window = 0

and this problem, that explain why we get TCP notify in debug,  I think it not LDP close the session but the TCP
can you config

ip tcp window-size

in both side and check 

Hello,

do you refer to the

[RST,ACK]

packets ?

I think TCP window size is always zero in packets with the RST flag set.

Steve_0-1686660970597.png

 

 

you are correct I check the capture it for

RST/ACK not for ACK/SYNC


anyway 
debug

ip tcp

transactions <<- share this please 

>I would not wonder too much about a potential TCP connection .....

i have done a simple test in my lab here with the same hardware (ASR1004 and a C6504E) . A simple

access-list

to block the

mpls ldp hello

packets from on side is enough to put both interfaces into

xmit

mode and LDP session stucks.

LABOR / SET ACCESSLIST ON C6504E-SIM Te2/2

 

C6504E-SIM#sh ip access-lists ACL-BLOCK-MPLS-LDP-HELLO

Extended IP access list ACL-BLOCK-MPLS-LDP-HELLO

    10 deny udp any eq 646 any eq 646 (146 matches)

    20 permit ip any any (11321 matches)




C6504E-SIM#

interface TenGigabitEthernet2/2

 ip access-group ACL-BLOCK-MPLS-LDP-HELLO in

!


C6504E-SIM#sh mpls ldp neighbor

C6504E-SIM#




C6504E-SIM#sh mpls ldp discovery | begin 2/2

        TenGigabitEthernet2/2 (ldp): xmit




C6504E-SIM#sh mpls ldp neighbor te2/2
C6504E-SIM#




ASR1004-SIM#sh mpls ldp discovery
Local LDP Identifier:
172.16.214.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet1/0/0.1 (ldp): xmit




ASR1004-SIM#sh mpls ldp neighbor
ASR1004-SIM#




LABOR / REMOVE ACCESSLIST ON C6504E-SIM Te2/2




C6504E-SIM#sh mpls ldp discovery
Local LDP Identifier:
172.16.212.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet2/2 (ldp): xmit/recv
LDP Id: 172.16.214.254:0




C6504E-SIM#sh mpls ldp neighbor te2/2
Peer LDP Ident: 172.16.214.254:0; Local LDP Ident 172.16.212.254:0
TCP connection: 172.16.214.254.19176 - 172.16.212.254.646
State: Oper; Msgs sent/rcvd: 12/17; Downstream
Up time: 00:00:01




ASR1004-SIM#sh mpls ldp discovery
Local LDP Identifier:
172.16.214.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet1/0/0.1 (ldp): xmit/recv
LDP Id: 172.16.212.254:0

ASR1004-SIM#sh mpls ldp neighbor
Peer LDP Ident: 172.16.212.254:0; Local LDP Ident 172.16.214.254:0
TCP connection: 172.16.212.254.646 - 172.16.214.254.19176
State: Oper; Msgs sent/rcvd: 28/21; Downstream
Up time: 00:02:47
LDP discovery sources:
Targeted Hello 172.16.214.254 -> 172.16.212.254, active, passive
TenGigabitEthernet1/0/0.1, Src IP addr: 172.16.220.30

 

Review Cisco Networking for a $25 gift card