ASR9001 w/ nvEdge: IRL links not establishing

[Simon Sigre]

Has anyone else experienced issues with the ASR9001 and IRL links not coming up when trying to commission nvEdge.

The control plane is fine;

RP/0/RSP0/CPU0:asr9001s-prod-01(admin)#sh dsc
Sun Feb 28 09:35:52.857 UTC
---------------------------------------------------------
           Node  (     Seq)     Role       Serial State
---------------------------------------------------------
    0/RSP0/CPU0  (       0)   ACTIVE  FOCxxxxxxxx PRIMARY-DSC
    1/RSP0/CPU0  (  831809)   ACTIVE  FOCyyyyyy BACKUP-DSC

The 2x routers are correctly clustered and the control planes are established however the 2x10Gb IRL links wont come up as an nv edge interface.;

rack0[TenGigE0/0/2/2] <--> (directly connected) <--> rack1[TenGigE1/0/2/2]

rack0[TenGigE0/0/2/3] <--> (directly connected) <--> rack1[TenGigE1/0/2/3]

An 'show nv edge data forwarding location all' shows the following;

nV Edge Data interfaces in configured state: 4

TenGigE1_0_2_2                
TenGigE0_0_2_2                
TenGigE1_0_2_3                
TenGigE0_0_2_3      

          
The documentation tells me that I should be seeing a visual representation of the state and reference to interfaces being in a forwarding state.

..however the interfaces themselves are in a down state;

RP/0/RSP0/CPU0:asr9001s-prod-01#sh int brie | include Te
Sun Feb 28 09:34:20.588 UTC
          Te0/0/2/0  admin-down  admin-down               ARPA  1514   10000000
          Te0/0/2/1  admin-down  admin-down               ARPA  1514   10000000
*here*          Te0/0/2/2        down        down               ARPA  1514   10000000
*here*          Te0/0/2/3        down        down               ARPA  1514   10000000
          Te1/0/2/0  admin-down  admin-down               ARPA  1514   10000000
          Te1/0/2/1  admin-down  admin-down               ARPA  1514   10000000
*here*          Te1/0/2/2        down        down               ARPA  1514   10000000
*here*          Te1/0/2/3        down        down               ARPA  1514   10000000

And the interfaces are saying down due to link-loss

RP/0/RSP0/CPU0:asr9001s-prod-01#sh controllers tenGigE 1/0/2/3
Sun Feb 28 09:40:23.895 UTC
Operational data for interface TenGigE1/0/2/3:

State:
    Administrative state: enabled
    Operational state: Down (Reason: Link loss or low light, no loopback)
    LED state: Red On

The interfaces are 'no shut' .

I also noted these errors in logs;

LC/1/0/CPU0:Feb 26 11:13:33.618 : cluster_dlm_lc[148]: %PLATFORM-UDLD-4-DETECTION_DISABLED : UDLD protocol Unknown link on TenGigE1_0_2_2, UDLD Detection Disabled
LC/1/0/CPU0:Feb 26 11:13:33.633 : cluster_dlm_lc[148]: %PLATFORM-UDLD-4-DETECTION_DISABLED : UDLD protocol Unknown link on TenGigE1_0_2_2, UDLD Detection Disabled
LC/1/0/CPU0:Feb 26 11:13:33.643 : cluster_dlm_lc[148]: %PLATFORM-UDLD-4-DETECTION_DISABLED : UDLD protocol Unknown link on TenGigE1_0_2_3, UDLD Detection Disabled
LC/1/0/CPU0:Feb 26 11:13:33.647 : cluster_dlm_lc[148]: %PLATFORM-UDLD-4-DETECTION_DISABLED : UDLD protocol Unknown link on TenGigE1_0_2_3, UDLD Detection Disabled

Platform state reports fine also

RP/0/RSP0/CPU0:asr9001s-prod-01#sh platform
Sun Feb 28 09:52:29.595 UTC
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     ASR9001-RP(Active)        IOS XR RUN       PWR,NSHUT,MON
0/0/CPU0        ASR9001-LC-S              IOS XR RUN       PWR,NSHUT,MON
0/0/0           A9K-MPA-20X1GE            OK               PWR,NSHUT,MON
1/RSP0/CPU0     ASR9001-RP(Active)        IOS XR RUN       PWR,NSHUT,MON
1/0/CPU0        ASR9001-LC-S              IOS XR RUN       PWR,NSHUT,MON
1/0/0           A9K-MPA-20X1GE            OK               PWR,NSHUT,MON

[Eddie Chami]

Have you configured the IRL - can you display the IRL config?

[Simon Sigre]

IRL config? There isn't much to it...
"nv edge interface" on each of the interfaces.

I actually logged this with TAC yesterday and they got back to me straight away and the current running theory is actually a certain iteration of the SFP (the serial numbers look slightly odd compared to others) .. same 'type' and a genuine Cisco part . It appears the optics themselves (all 4) just are not firing.

Might just be a L1 issue!

[Simon Sigre]

A visual inspection of the router;

A little bit of a caveat if you are running ASR9001 units... you require 2x ports for data link and 2 of them are locked via a 120G upgrade license lock.. yeah not super happy.

Also this doesn't help when performing a remote build; https://quickview.cloudapps.cisco.com/quickview/bug/CSCux68035

[Eddie Chami · Accepted solution]

You can still do nv edge with one IRL port if you wish. 

Eddie.

[Simon Sigre]

Looks like I will have to.. lucky its A/A and ill wind up with 20Gb/s .. still a shame though to produce a device with on-board ports that are locked out day-1.

[Eddie Chami]

Simon,

The chassis you have is discounted, with an upgrade lic to enable all ports, so its pay as you grow. You could have bought a 9001 with all ports active, but you do pay more up front.

Eddie. 

[Simon Sigre]

I guess we will have to agree to disagree on that one, I love the product set but if it comes pre-discounted then surely that just becomes the normal price? I guess discussing/ debating with a Cisco employee (Principle Engineer at Cisco?).

[Eddie Chami]

Yes i work at Cisco Simon, i should probably rephrase the discounted to repriced lower, with an upgrade lic that would equal to what you had bought if all ports are active. 

Some customers want XR, with that small RU factor, but don't need all the ports, a good example is a HW XR BGP Route-Reflector.

I'm not trying to convince you its good this way, but we have customer segment that wants that.