ASA EIGRP NEIGHBOR PROBLEM

[CiscoMedMed]

I currently have as ASA attached to a pair of NX6K and exchanging EIGRP. I'm attempting to migrate from the NX6K pair to a pair of NX9K. The EIGRP active EIGRP peering interface on the new 9K my workstation was no longer able to reach the inside interface of the ASA. The new peer the 9K in the example below is at 10.1.0.8 and it has an asterisk next to it. Does the * signify that it is the preferred path to 10.1.48.0? The 9K can see the arp entry of for my workstation mac address.

ASA01/pri/act# sho route 10.1.48.0

Routing entry for 10.1.48.0 255.255.254.0
Known via "eigrp 10", distance 90, metric 3072, type internal
Redistributing via eigrp 10
Last update from 10.1.0.8 on INSIDE, 0:00:04 ago
Routing Descriptor Blocks:
* 10.1.0.8, from 10.1.0.8, 0:00:04 ago, via INSIDE
Route metric is 3072, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 1000000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
10.1.0.3, from 10.1.0.3, 0:00:04 ago, via INSIDE
Route metric is 3072, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 1000000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
10.1.0.2, from 10.1.0.2, 0:00:04 ago, via INSIDE
Route metric is 3072, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 1000000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1

[Giuseppe Larosa]

Hello @@CiscoMedMed,

all three EIGRP neighbors are out of interface INSIDE and the ASA is performing ECMP Equal Cost Multi Path. This is a supported design.

You need to ensure that the destination subnet is present on all three Nexus and that there is a L2 path between them for the corresponding VLAN

Whatever next-hop is chosen by ASA the device must be able to reach at Layer 2 each device in the subnet.

Who is the default gateway of your test PC in 10.1.48.0/23  ?

The ASA cannot know that the PC is behind the new N9K only, it can send its packet to everyone of the three next-hops.

So you need to ensure end to end L2 connectivty between the new N9K and the two N6k on the user VLAN.

Check if the two N6K can resolve at ARP level the IP address of the test PC, if they cannot your user VLAN is divided in two parts and this explains the issue that you see.

Edit:

to fix a disjointed VLAN is enough to have an 80.21.Q trunk between N9k and one of the N6K carrying the user VLAN(s)

Hope to help

Giuseppe

[CiscoMedMed]

Thank you very much Giuseppe for your thoughts! My workstation is off of the NX6K and the gateway is the HSRP VIP which is active of the NX6Ks. 

NX6K01# sho hsrp brief
Interface Grp Prio P State Active addr Standby addr Group addr
Vlan48 48 200 P Active local 10.1.48.3 10.1.48.1 (conf)

NX9K01# sho hsrp brief
Vlan48 48 70 Listen 10.1.48.2 10.1.48.3 10.1.48.1 (conf)

Both the 6K and the 9K see the mac address of my workstation..

NX6K01# sho ip arp | i 10.1.48.28
10.1.48.28 00:17:44 0050.568b.a4be Vlan48

NX9K01# sho ip arp | i 10.1.48.28
10.1.48.28 00:00:31 0050.568b.a4be Vlan48

Since the 6K and the 9K can see my mac address this shows that there is L2 communication between the 6Ks and the 9Ks (two 40Gbps links VPC port channeled and trunking). 

From the workstation, arp -a can see the mac of the HSRP VIP 48.1, the SVI IPs of the 6Ks and the SVI of the 9K. HSRP VIP is active on the 6K01.

H:\>arp -a

Interface: 10.1.48.28 --- 0x5
Internet Address Physical Address Type
10.1.48.1 00-00-0c-07-ac-30 dynamic
10.1.48.2 00-2a-6a-a1-4b-01 dynamic
10.1.48.3 00-2a-6a-a1-4e-7c dynamic
10.1.48.4 cc-16-7e-ba-91-db dynamic

The ASA hangs off of the NX6Ks at 10.1.0.4. The inside gateway is 10.1.0.1. 
And it is this VLAN/Subnet which is active for EIGRP (not passive like the PC

VLAN). NX6K01 has SVI 1000 10.1.0.2 NX6K02 10.1.0.3 and and NX9K01

has 10.1.0.8. 

One surprising bit was that the workstation could still ping the Internet during 

the turn-up of the EIGRP active interface. But I could no longer ping the

inside of the ASA. If I had through of it I should have run an arp -a on the

workstation during the change window. Any further ideas appreciated. 
Do you agree that my VLAN for the workstation is not disjointed? 

[Giuseppe Larosa]

Hello,

I agree that the VLAN is not disjointed .

This point you need to verify if there are any other L3 links between the ASA and the N6K01.

>> One surprising bit was that the workstation could still ping the Internet during

the turn-up of the EIGRP active interface. But I could no longer ping the

inside of the ASA

Firewalls break asymmetric routing they need to see both directions of a flow even an ICMP flow coming and going from/to the same interface.

Check who is sending the best default route in EIGRP (it is the ASA ?) and verify if this can create an asymmetric routing on the ASA.

If there is no chance of asymmetric routing and your N9K is connected to both N6K that I imagine they  are a vPC pair the issue is strange.

Hope to help

Giuseppe

[CiscoMedMed]

Yes - the default route emanates from the ASA makes it to the 6K and the 9K. And actually the default route appeared to be working as pinging google dns recursively kept working during the test when eigrp to the 9K was enabled. Pinging the inside of the ASA failed at those times however. Also another failure was in being able to reach the remote DMVPN sites when the 6k-9k eigrp interface was enabled. It seems like some type of asymmetry issue. But in both case the traffic is flowing through the inside interface of the ASA. 

[CiscoMedMed]

How about this possibility NX6K01 NX6K02 and NX9K01 all had the EIGRP route table. NX9K02 is running and has all VLANs available for layer 2. Most but not all of the SVIs are shut on NX9K02 and including the VLAN that would carry the EIGRP traffic. 

I was just turning up the routed interface on 9K01 to gently make the change. But perhaps having L2 and L3 on three nexus and only L2 on the fourth could have caused an issue.?

[Giuseppe Larosa]

Hello @CiscoMedMed ,

>> I was just turning up the routed interface on 9K01 to gently make the change. But perhaps having L2 and L3 on three nexus and only L2 on the fourth could have caused an issue.?

Yes, if you have already deployed N9k02 and N9k02 and N9k01 form a L2 vPC you probably need to enable ip routing on N9K02, specially if there is a chance that the flow will go through it.

Hope to help

Giuseppe

[paul driver]

Hellos
Can you post  a topology diagram please.

Also when you say migrate are you using this same addressing from the old 6k on the new 9k?