Solved: Re: ASA port-channel not coming up (port-channel between ASA & vPC)

23nick · ‎09-04-2020

Hello,

I have ASA connected to vPC pairs like below. I have read cisco documentation and found that it supports below design by adding few commands. I have formed ospf neighborship between switches in vPC domain. But my ASA interface connected to vPC switches are not coming up. I Tried lacp active & passive mode on asa po48 interface but it did not work. i want to form OSPF neighborship between vPC and ASA. Any sugggestion whats wrong with my ASA or its a bug ?

Thankyou

Diagram: https://i.imgur.com/369RP8E.png

Only required configs pasted below.

NX-S1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Enabled
Virtual-peerlink mode : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po1 up 1,48

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
48 Po48 down* success success -

NX-S1# sh run vpc

!Command: show running-config vpc
!Running configuration last done at: Fri Sep 4 13:42:57 2020
!Time: Fri Sep 4 14:34:13 2020

version 9.3(4) Bios:version
feature vpc

vpc domain 1
role priority 1
peer-keepalive destination 1.1.1.2 source 1.1.1.1 vrf peer-keepalive
peer-gateway
layer3 peer-router

interface port-channel1
vpc peer-link

interface port-channel48
vpc 48

NX-S1# sh run int eth1/7

!Command: show running-config interface Ethernet1/7
!Running configuration last done at: Fri Sep 4 13:42:57 2020
!Time: Fri Sep 4 14:52:01 2020

version 9.3(4) Bios:version

interface Ethernet1/7
switchport mode trunk
switchport trunk allowed vlan 48

NX-S2# sh run vpc

!Command: show running-config vpc
!Running configuration last done at: Fri Sep 4 13:41:53 2020
!Time: Fri Sep 4 14:41:14 2020

version 9.3(4) Bios:version
feature vpc

vpc domain 1
peer-keepalive destination 1.1.1.1 source 1.1.1.2 vrf peer-keepalive
peer-gateway
layer3 peer-router

interface port-channel1
vpc peer-link

interface port-channel48
vpc 48

NX-S2# sh run int eth1/7

!Command: show running-config interface Ethernet1/7
!Running configuration last done at: Fri Sep 4 13:41:53 2020
!Time: Fri Sep 4 14:52:45 2020

version 9.3(4) Bios:version

interface Ethernet1/7
switchport mode trunk
switchport trunk allowed vlan 48

On asa

!
router ospf 1
network 10.1.1.0 255.255.255.248 area 0
log-adj-changes

ciscoasa(config)# int po48
ciscoasa(config-if)# no shut
ciscoasa(config-if)#
ciscoasa(config-if)# sh int ip br
Interface IP-Address OK? Method Status Protocol
Ethernet0 unassigned YES unset administratively down up
Ethernet1 unassigned YES unset administratively down up
Ethernet2 unassigned YES unset up up
Ethernet3 unassigned YES unset up up
Port-channel48 10.1.1.3 YES manual down down
ciscoasa(config-if)#
ciscoasa(config-if)# sh run int po48
!
interface Port-channel48
nameif inside
security-level 100
ip address 10.1.1.3 255.255.255.248
ciscoasa(config-if)#

Christopher Hart · ‎09-05-2020

Hi Nick!

Your vPC and OSPF configuration look good, except it looks like you removed the layer3 peer-router command from the vPC domain of both Nexus switches. This is required to form dynamic routing protocol adjacencies through a vPC VLAN (that is, a VLAN that is trunked across the vPC Peer-Link) with a vPC-connected device (which your ASA is, since it's connected via a vPC).

My previous post stated that you only needed to remove the Eth1/7 link between the two Nexus switches carrying VLAN 48 (which it looks like you've done). We do still need layer3 peer-router in this particular scenario.

Can you re-apply the layer3 peer-router command under the vPC domain of both switches and see if your OSPF adjacencies come up?

Thank you!

-Christopher

View solution in original post

23nick · ‎09-05-2020

@Christopher Hart Yes i reapplied that command and there was also one command missing on ASA. vPC switches are sending vlan 48 as tagged frame, so i configured int po48.48 on asa and put it in vlan 48. And it did the trick

Also want to know one thing. I would be adding one more ASA to vPC .

And connecting links between them to act as active/standby. Would this topology have any issues ?

Thankyou so much for help

View solution in original post

Christopher Hart · ‎09-05-2020

Hi Nick!

Connecting an active/standby ASA pair to Nexus switches via vPC is supported. However, a unique vPC would need to be connected to each ASA firewall - you cannot "extend" vPC Po48 to have member interfaces connected to the second/standby ASA. You would need to create a new vPC (let's call it Po49) and connect its member interfaces to the second/standby ASA. You can still trunk VLAN 48 across vPC Po49, of course - you just need a unique vPC connected to the second/standby ASA.

The "Configuring Cisco ASA Service Appliance in Routed Mode with vPC" section of the Design and Configuration Guide: Best Practices for Virtual Port Channels (vPC) on Cisco Nexus 7000 Series Switches document has a topology that demonstrates this:

In the above topology, the vPC connected to ASA-1 is a different number (let's say vPC Po10) than the vPC connected to ASA-2 (let's say vPC Po20).

Also, note that the above document was written for the Nexus 7000 series switch - however, the same information is applicable to all Nexus platforms, including your Nexus 3000/9000 switches.

I hope this helps - thank you!

-Christopher

View solution in original post

venkateshwarlu.t · ‎09-04-2020

In the Nexus Switches interface E1/7 connecting to ASA ? If yes, looks it is missing Channel-group command to make into port-channel.

23nick · ‎09-04-2020

Eth1/7 on NX-SW1 & NX-SW2 are connected to each other not to ASA. I have this seperate link for OSPF only.

NX-SW1 eth1/2 & NX-SW2 eth1/3 connected to ASA, i forgot to add this in diagram.

venkateshwarlu.t · ‎09-04-2020

share config of NX-SW1 eth1/2 & NX-SW2 eth1/3

23nick · ‎09-04-2020

@venkateshwarlu.t

Below is the config

NX-S1# sh run int eth1/2

!Command: show running-config interface Ethernet1/2
!Running configuration last done at: Fri Sep 4 17:34:13 2020
!Time: Fri Sep 4 17:35:09 2020

version 9.3(4) Bios:version

interface Ethernet1/2
switchport mode trunk
switchport trunk allowed vlan 48

NX-S1# sh run int po48

!Command: show running-config interface port-channel48
!Running configuration last done at: Fri Sep 4 17:34:13 2020
!Time: Fri Sep 4 17:36:07 2020

version 9.3(4) Bios:version

interface port-channel48
description ASA-FW
switchport mode trunk
switchport trunk allowed vlan 48
vpc 48

NX-S1#

interface Vlan48
description ASA-FW
no shutdown
no ip redirects
ip address 10.1.1.1/29
no ipv6 redirects
ip router ospf ASA area 0.0.0.0

router ospf ASA
router-id 1.1.1.1
area 0.0.0.0 range 10.1.1.0/29

NX-S1# sh vlan id 48

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
48 ASA active Po1, Po48, Eth1/2, Eth1/3
Eth1/4, Eth1/5, Eth1/7

NX-S1# sh ip ospf nei
OSPF Process ID ASA VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
1.1.1.2 1 FULL/DR 00:00:10 10.1.1.2 Vlan48

NX-S2# sh run int eth1/3

!Command: show running-config interface Ethernet1/3
!Running configuration last done at: Fri Sep 4 17:34:52 2020
!Time: Fri Sep 4 17:35:44 2020

version 9.3(4) Bios:version

interface Ethernet1/3
description Member_of_Po48_ASA-FW
switchport mode trunk
switchport trunk allowed vlan 48

NX-S2# sh run int po48

!Command: show running-config interface port-channel48
!Running configuration last done at: Fri Sep 4 17:34:52 2020
!Time: Fri Sep 4 17:36:00 2020

version 9.3(4) Bios:version

interface port-channel48
description ASA-FW
switchport mode trunk
switchport trunk allowed vlan 48
vpc 48

NX-S2#

interface Vlan48
description ASA-FW
no shutdown
no ip redirects
ip address 10.1.1.2/29
no ipv6 redirects
ip router ospf ASA area 0.0.0.0

router ospf ASA
router-id 1.1.1.2
area 0.0.0.0 range 10.1.1.0/29

NX-S2# sh ip ospf nei
OSPF Process ID ASA VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
1.1.1.1 1 FULL/BDR 00:00:24 10.1.1.1 Vlan48

NX-S2# sh vlan id 48

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
48 ASA active Po1, Po48, Eth1/2, Eth1/3
Eth1/4, Eth1/5, Eth1/7

Christopher Hart · ‎09-04-2020

Hi Nick!

If I am understanding your topology and current configuration correctly, you have two Nexus switches running NX-OS 9.3(4) that are in a vPC domain. Po1 consists of Eth1/4-5 and is the vPC Peer-Link, while Eth1/1 is a dedicated Layer 3 interface for the vPC Peer-Keepalive. Eth1/7 of both switches is also connected, which carries VLAN 48 (which is also trunked over the vPC Peer-Link, making it a vPC VLAN). As I understand it, you want to form an OSPF adjacency over VLAN 48 between both of the Nexus switches and the ASA.

Let's simplify the topology a bit - first, since you have layer3 peer-router configuration applied under the vPC domain, there is no need for a separate trunk link between both vPC peers to carry VLAN 48. In other words, the link connecting Eth1/7 on both switches can be completely removed. Some older Nexus configuration guides or documentation may suggest that this is required - however, with modern NX-OS software releases, the layer3 peer-router configuration allows dynamic routing protocol adjacencies to form over vPC VLANs with vPC-connected devices.

Next, let's use the same interface on both switches to connect to the ASA firewall. Right now, it looks like Eth1/2 of NX-S1 connects to one interface of the ASA, while Eth1/3 of NX-S2 connects to the other interface of the ASA. This isn't necessarily invalid (meaning, it can work), but it's not best practice as generally the interfaces used as part of a vPC connecting a device are identical on both switches. To fix this, let's move the cable in Eth1/3 of NX-S2 to Eth1/2 of NX-S2, if possible.

Finally, we need to configure Eth1/2 of both NX-S1 and NX-S2 to be a member of vPC Po48. Po48 is already configured correctly as a vPC - however, Eth1/2 is not a member of Po48 yet. We can do this by executing the channel-group 48 force mode active command on both switches, as shown here.

NX-S1# configure terminal
NX-S1(config)# interface Ethernet1/2
NX-S1(config-if)# channel-group 48 force mode active
NX-S1(config-if)# end
NX-S1#

NX-S2# configure terminal
NX-S2(config)# interface Ethernet1/2
NX-S2(config-if)# channel-group 48 force mode active
NX-S2(config-if)# end
NX-S2#

So long as the ASA's interfaces are configured for either LACP Active or Passive modes, the vPC interface should then come up. Assuming that OSPF is correctly configured on all interfaces, your OSPF adjacencies should then come up between NX-S1, NX-S2, and the ASA.

I hope this helps - thank you!

-Christopher

23nick · ‎09-04-2020

@Christopher Hart @venkateshwarlu.t Thanks.

While revieving configs after posting here, i noticed that fault. Now interface on ASA has come up and vpc 48 is also up. But ospf neighborship is still not coming up between ASA & vPC. I will check my configs once again tomorrow and troubleshoot.

@Christopher HartI read cisco blogs and applied that commands. I will try doing what you suggested. Many thanks

23nick · ‎09-05-2020

@Christopher HartI removed link eth1/7 between vPC switches. and layer3 peer-router command.

But still ospf not coming up between ASA & vPC. I checked configs and it looks fine to me. I have SVI configured on both vPC switches and advertised that subnet in OSPF. On ASA i configured IP on Po48 interface and advertised subnet in OSPF. What else am i missing?

Topology & configs: https://i.imgur.com/31lUxNQ.png

NX-S1# sh ip ospf nei
OSPF Process ID ASA VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
1.1.1.2 1 FULL/DR 00:43:12 10.1.1.2 Vlan48

NX-S2# sh ip ospf nei
OSPF Process ID ASA VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
1.1.1.1 1 FULL/BDR 00:43:33 10.1.1.1 Vlan48

On ASA

router ospf 1
network 10.1.1.0 255.255.255.248 area 0
log-adj-changes

!

interface Ethernet2
channel-group 48 mode passive
no nameif
no security-level
no ip address
!
interface Ethernet3
channel-group 48 mode passive
no nameif
no security-level
no ip address
!
interface Port-channel48
nameif inside
security-level 100
ip address 10.1.1.3 255.255.255.248
!
ciscoasa# sh ospf nei
ciscoasa#

NX-S1# sh run vpc
!Command: show running-config vpc
!Running configuration last done at: Sat Sep 5 06:23:21 2020
!Time: Sat Sep 5 07:09:03 2020
version 9.3(4) Bios:version
feature vpc
vpc domain 1
role priority 1
peer-keepalive destination 1.1.1.2 source 1.1.1.1 vrf peer-keepalive
peer-gateway
interface port-channel1
vpc peer-link
interface port-channel48
vpc 48
(Same vpc config on NX-S2)

!

interface Vlan48
description ASA-FW
no shutdown
no ip redirects
ip address 10.1.1.1/29
no ipv6 redirects
ip router ospf ASA area 0.0.0.0
!
router ospf ASA
router-id 1.1.1.1
area 0.0.0.0 range 10.1.1.0/29

--------------------
On NX-S2
interface Vlan48
description ASA-FW
no shutdown
no ip redirects
ip address 10.1.1.2/29
no ipv6 redirects
ip router ospf ASA area 0.0.0.0
!
router ospf ASA
router-id 1.1.1.2
area 0.0.0.0 range 10.1.1.0/29

Thanks.

Christopher Hart · ‎09-05-2020

Hi Nick!

Your vPC and OSPF configuration look good, except it looks like you removed the layer3 peer-router command from the vPC domain of both Nexus switches. This is required to form dynamic routing protocol adjacencies through a vPC VLAN (that is, a VLAN that is trunked across the vPC Peer-Link) with a vPC-connected device (which your ASA is, since it's connected via a vPC).

My previous post stated that you only needed to remove the Eth1/7 link between the two Nexus switches carrying VLAN 48 (which it looks like you've done). We do still need layer3 peer-router in this particular scenario.

Can you re-apply the layer3 peer-router command under the vPC domain of both switches and see if your OSPF adjacencies come up?

Thank you!

-Christopher

23nick · ‎09-05-2020

@Christopher Hart Yes i reapplied that command and there was also one command missing on ASA. vPC switches are sending vlan 48 as tagged frame, so i configured int po48.48 on asa and put it in vlan 48. And it did the trick

Also want to know one thing. I would be adding one more ASA to vPC .

And connecting links between them to act as active/standby. Would this topology have any issues ?

Thankyou so much for help

Christopher Hart · ‎09-05-2020

Hi Nick!

Connecting an active/standby ASA pair to Nexus switches via vPC is supported. However, a unique vPC would need to be connected to each ASA firewall - you cannot "extend" vPC Po48 to have member interfaces connected to the second/standby ASA. You would need to create a new vPC (let's call it Po49) and connect its member interfaces to the second/standby ASA. You can still trunk VLAN 48 across vPC Po49, of course - you just need a unique vPC connected to the second/standby ASA.

The "Configuring Cisco ASA Service Appliance in Routed Mode with vPC" section of the Design and Configuration Guide: Best Practices for Virtual Port Channels (vPC) on Cisco Nexus 7000 Series Switches document has a topology that demonstrates this:

In the above topology, the vPC connected to ASA-1 is a different number (let's say vPC Po10) than the vPC connected to ASA-2 (let's say vPC Po20).

Also, note that the above document was written for the Nexus 7000 series switch - however, the same information is applicable to all Nexus platforms, including your Nexus 3000/9000 switches.

I hope this helps - thank you!

-Christopher

23nick · ‎09-05-2020

@Christopher Hart Yes i am planning for same design and playing with routing protocols. The only thing i would be having is connecting each ASA to different ISP/WAN routers for Internet/VPN connectivity.

Thanks a lot for you help. The people in this community are awesome & good for learning stuffs.

venkateshwarlu.t · ‎09-04-2020

NX-SW1 eth1/2 config missing Channel-group command to make into port-channel 48 & NX-SW2 eth1/3 missing Channel-group 48 command to make into port-channel 48.

NX-SW1#

interface Ethernet1/2
switchport mode trunk
switchport trunk allowed vlan 48

channel-group 48 mode active

NX-SW2#

interface Ethernet1/3
switchport mode trunk
switchport trunk allowed vlan 48

channel-group 48 mode active

Hope it will helps.