MPLS LDP Session Establishment failed

[Steve]

Hello,

i have tried to connect an

ASR1004(RP2,ESP20,SIP10) and a C6504E(VS-S720-10G)

over 10G direct links.

10G port on C6504E

is located on a

WS-X6708-10GE ( with WS-F6700-DFC3C)

linecard. Both interfaces are

MPLS enabled via mpls ip

IP connectivity between the

mpls loopbacks

of the routers is working fine, can ping with max

MTU 9216 and df-bit set

The tcpdump (via monitor session) from the interface on the

C6504E

shows "LDP Hello Messages" from both routers.

ASR1004

which has the higher

LSR-ID

initiates the LDP TCP Connection to

C6504E

but connection is refused on

C6504E

by sending  TCP (RST,ACK).

What could be the reason for dropping the connection ? i have attached a zip with the pcap trace from interface traffic

ASR1004	loopback1	172.16.217.254/32
C6504E	loopback1	172.16.211.254/32
ASR1004	te1/0/0	172.16.220.14/30
C6504E	te2/1	172.16.220.13/30

Global setting on both routers:

mpls ldp router-id Loopback1 force

C6504E#ping 172.16.220.14 size 9216 df-bit source lo1 repeat 10
Type escape sequence to abort.
Sending 10, 9216-byte ICMP Echos to 172.16.220.14, timeout is 2 seconds:
Packet sent with a source address of 172.16.211.254
Packet sent with the DF bit set
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 1/2/4 ms

ASR1004#ping 172.16.220.13 source lo1 size 9216 df-bit repeat 10
Type escape sequence to abort.
Sending 10, 9216-byte ICMP Echos to 172.16.220.13, timeout is 2 seconds:
Packet sent with a source address of 172.16.217.254
Packet sent with the DF bit set
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 2/2/3 ms

[MHM Cisco World]

show mpls ldp neighbor 

I need to see this 

[Steve]

on

C6504E

there are other LDP Sessions active which i won't post here - so i have selected the affected interface

C6504E#sh mpls ldp neighbor te2/1
C6504E#

ASR1004#sh mpls ldp neighbor
ASR1004#

As there is no LDP Session established we won't see here anything (i think)

Some more infos

ASR1004#sh mpls ldp discovery detail
Local LDP Identifier:
172.16.217.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet1/0/0 (ldp): xmit
Enabled: Interface config
Hello interval: 5000 ms; Transport IP addr: 172.16.217.254
Clients: IPv4, mLDP

C6504E#sh mpls ldp discovery detail
Local LDP Identifier:
172.16.211.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet2/1 (ldp): xmit
Enabled: Interface config
Hello interval: 5000 ms; Transport IP addr: 172.16.211.254
Clients: IPv4, mLDP

[MHM Cisco World]

Udp port 646 is open or not ?

[Steve]

Yes

 no access-lists

on interfaces

ASR1004#telnet 172.16.211.254 646 /source-interface lo1
Trying 172.16.211.254, 646 ...
% Connection refused by remote host

As you can see TCP connection is refused - so port should be open

In the meanwhile i have done some more connectivity tests and it seems there is packetloss on the link. 10 pings was not enough to loose packets

ASR1004#ping 172.16.211.254 size 1500 df-bit repeat 100 source lo1
Type escape sequence to abort.
Sending 100, 1500-byte ICMP Echos to 172.16.211.254, timeout is 2 seconds:
Packet sent with a source address of 172.16.217.254
Packet sent with the DF bit set
!!!!!!!!!!!.!!!!!!!!!!.!!!!!!!!!!!.!!!!!!!!!!.!!!!!!!!!!!.!!!!!!!!!!!.
!!!!!!!!!!!.!!!!!!!!!!!.!!!!!!
Success rate is 92 percent (92/100), round-trip min/avg/max = 1/1/1 ms

i will have to fix this before we can follow up - will take some time (until next week) as this is a remote site  - thanks for your help

[MHM Cisco World]

I think I found issue here 

[MHM Cisco World]

I see in ldp discovery thst you use multiPoint Ldp (mLDP) not LDP

This change source destination of ldp packet.

Disbale mldp under interface.

[Steve]

This doesn't change anything

interface TenGigabitEthernet2/1
mtu 9216
ip address 172.16.220.13 255.255.255.252
no ip redirects
no ip proxy-arp
no mpls mldp
mpls ip

C6504E#sh mpl ldp discovery detail | begin 2/1
TenGigabitEthernet2/1 (ldp): xmit
Enabled: Interface config
Hello interval: 5000 ms; Transport IP addr: 172.16.211.254
Clients: IPv4, mLDP 

Maybe this indicates only interface is capable of mLDP or it refers to this

Directly Connected MPLS LDP Sessions

If a label switch router (LSR) is one hop from its neighbor, it is directly connected to its neighbor. The LSR sends out Label Distribution Protocol (LDP) link Hello messages as User Datagram Protocol (UDP) packets to all the devices on the subnet (multicast). A neighboring LSR may respond to the link Hello message, allowing the two devices to establish an LDP session. This is called basic discovery.

[MHM Cisco World]

you disable it in C6000 side, disable it in ASR1K side 

[Steve]

Sorry my answer was too inaccurate - it was changed on both sides

ASR1004#sh run int te1/0/0
Building configuration...

Current configuration : 302 bytes
!
interface TenGigabitEthernet1/0/0
mtu 9216
ip address 172.16.220.14 255.255.255.252
no ip redirects
no ip proxy-arp
load-interval 30
mpls ip
no mpls mldp
!
ASR1004#sh mpls ldp discovery detail
Local LDP Identifier:
172.16.217.254:0
Discovery Sources:
Interfaces:
TenGigabitEthernet1/0/0 (ldp): xmit
Enabled: Interface config
Hello interval: 5000 ms; Transport IP addr: 172.16.217.254
Clients: IPv4, mLDP

[Harold Ritter]

Hi @Steve ,

For the output you provided, it looks like neither router receive the LDP hellos from the neighbour, as both of them show as

xmit

 when they should as

xmit/recv

Regards, 

[Steve]

Hello Harold,

i think until LDP session isn't established interface will always be

xmit

 If you check the the pcap file you can see hellos from both routers. i have enabled debug on

ASR1004

and there you can see hello is working and

ASR1004

(Router with the higher LSR-ID) tries to establish a TCP connection (LDP Session) to

C6504E

On

ASR1004

MPLS ldp:
LDP transport events debugging is on for interface

TenGigabitEthernet1/0/0

LDP transport connection events debugging is on for interface

TenGigabitEthernet1/0/0

*Jun 7 22:11:31.010: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 7 22:11:34.993: ldp: Rcvd ldp hello; TenGigabitEthernet1/0/0, from 172.16.220.13 (172.16.211.254:0), intf_id 0, opt 0xC
*Jun 7 22:11:34.993: ldp: ldp Hello from 172.16.220.13 (172.16.211.254:0) to 224.0.0.2, opt 0xC
*Jun 7 22:11:34.993: ldp: New adj 0x7F19D75FA358 for 172.16.211.254:0, TenGigabitEthernet1/0/0
*Jun 7 22:11:34.993: ldp: adj_addr/xport_addr 172.16.220.13/172.16.211.254
*Jun 7 22:11:34.993: ldp: Request adj send hello back on TenGigabitEthernet1/0/0 to (xport addr 172.16.211.254) in 50 msec
*Jun 7 22:11:34.993: ldp: local interface = TenGigabitEthernet1/0/0, holdtime = 15000, peer 172.16.220.13 holdtime = 15000
*Jun 7 22:11:34.993: ldp: Link intvl min cnt 2, intvl 5000, interface TenGigabitEthernet1/0/0
*Jun 7 22:11:34.993: ldp: Opening ldp conn; adj 0x7F19D75FA358, 172.16.217.254 <-> 172.16.211.254; with normal priority
*Jun 7 22:11:34.993: ldp: :{ldp conn 172.16.217.254:64033=>172.16.211.254:646}: Registered tcb 0x7F19CA59D528 [key 48199] with LDP TCB database, total 1
*Jun 7 22:11:34.994: ldp: Conn failed (TCP connect notify)!; adj 0x7F19D75FA358, 172.16.220.13
*Jun 7 22:11:34.994: ldp: : rx_notify tcp_rc 5, adj_state 3
*Jun 7 22:11:34.994: ldp: {ldp conn 172.16.217.254:64033=>172.16.211.254:646} (Te1/0/0) (adj 0x7F19D75FA358): processing transport close request
*Jun 7 22:11:34.994: ldp: Unregistered from LDP TCB database tcb 0x7F19CA59D528 [key 48199], total 0
*Jun 7 22:11:34.994: ldp: Close tcp connection to 172.16.211.254
*Jun 7 22:11:34.994: ldpx_tcp: {ldp conn 172.16.217.254:64033=>172.16.211.254:646}: client requesting close of tcb
*Jun 7 22:11:34.994: ldp: Adj 0x7F19D75FA358; state set to closed
*Jun 7 22:11:35.043: ldp: Send ldp hello; TenGigabitEthernet1/0/0, src/dst 172.16.220.14/224.0.0.2, inst_id 0
*Jun 7 22:11:35.553: ldp: Scan listening TCBs

[Harold Ritter]

Hi @Steve ,

> i think until LDP session isn't established interface will always be

xmit

This

xmit/rcvd

state only pertains to LDP hellos. So the

show mpls ldp dis det

output you provided clearly state that the routers did not consider to have two way reachability.

Regards,

[Harold Ritter]

Hi @Steve ,

You probably want to run the same debug command on the 6500 side to see what it says.

Regards,

[Steve]

Hi @Harold Ritter 

this is a little bit strange now - the debug on

C6504E

shows no receiving hello messages but if i check the pcap trace i can see it. The trace is taken from

C6504E

with a monitor session (source is te2/1 in both directions).

As @MHM Cisco World noted earlier 'is port udp 646 open'  - is there a way to check this ? ( i only have verified tcp)

Below the captured hello packet on

C6504E

which was sent from ASR1004 and the debug log from

C6504E

DEBUG LOG from C6504E (no received ldp hello packets)

031445: Jun 8 11:14:47.821 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
031446: Jun 8 11:14:51.617 CEST: ldp: bytes_written = 34, at offset = 160
031447: Jun 8 11:14:51.617 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
031448: Jun 8 11:14:55.474 CEST: ldp: bytes_written = 34, at offset = 160
031449: Jun 8 11:14:55.474 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0
031450: Jun 8 11:14:59.430 CEST: ldp: bytes_written = 34, at offset = 160
031451: Jun 8 11:14:59.430 CEST: ldp: Send ldp hello; TenGigabitEthernet2/1, src/dst 172.16.220.13/224.0.0.2, inst_id 0

i have cross checked the debug command on a different interface (on the same Linecard) which has a working LDP Session and there i can see received and send hello packets.

So it seems C6504E is receiving the hello packets from ASR1004 but it drops it