07-09-2020 08:21 AM
Dear Cisco Support Community ,
Thank you all in advance .
I am new to Cisco nexus switch and as of now i have simple question for connection with layer 2 uplink (firewall):
our Scenario is we have two c9000 series and we have two fortigate firewall.
the Scenario is we need to connect a server with two links (one link to each switch "9000 series") and then the server Shall ping to fortigate (through the trunk port connected between the switch and Fortigate ).
Here i have attached the schame for the network .
I had also posted this issue before (i got it till some stage and then after that we are unable to complete what ever appeared later) .
I will share also the link of the previous post.
https://community.cisco.com/t5/data-center-switches/need-help-with-nexus-switch/m-p/4105322#M5991
Note The VPC seems to be working (show VPC shows everything is good , however we can't ping from / to switch and fortgiate .
Eagerly waiting your help.
Config for Link between fortigate and nexus switch :
Nexus 1
===========
interface port-channel17
description ***To-FW-1***
switchport
switchport mode trunk
vpc 17
interface port-channel18
description ***To-FW-2****
switchport
switchport mode trunk
vpc 18
interface Ethernet1/17
switchport
switchport mode trunk
channel-group 17
no shutdown
interface Ethernet1/18
description ***To-FW-2***
switchport
switchport mode trunk
channel-group 18
no shutdown
Nexus SW 2
==========
!
interface port-channel17
description ***To-FW-1***
switchport
switchport mode trunk
vpc 17
!
interface port-channel18
description ***To-FW-2****
switchport
switchport mode trunk
vpc 18
!
interface Ethernet1/17
description ***To-FW-1***
switchport
switchport mode trunk
channel-group 17
no shutdown
!
interface Ethernet1/18
description ***To-FW-2****
switchport
switchport mode trunk
channel-group 18
no shutdown
Thanks
Solved! Go to Solution.
08-05-2020 11:22 AM
So your choices are either one subnet/VLAN or two ...
(1) Single channel group and subnet on the Fortigate, connecting to a VPC on Nexus 1 and 2. Nexus 3 and 4 configured as a second VPC domain, connected to 1 and 2 as a double ended VPC. Devices connected to 3/4 are not going to see any detectable performance hit as a result of passing through 1/2 on their way to the Fortigate.
(2) Two separate channel groups and subnets on the Fortigate, one connects to Nexus 1/2 and one to Nexus 3/4. Create a Layer 3 path between Nexus 1/2 and 3/4.
07-14-2020 05:01 AM
Sounds like something is wrong in the IP configuration somewhere, but you don't show any of this in your post. Where are you pinging from, how is it configured and where is it connected?
07-17-2020 09:25 AM
Dear Tony ,,,,,
Thanks for your reply .....
Our Scenario is we have two Fortigate ( Active / Passive ) with up-link to each nexus switch (we have 4 fortigate).
Here are configuration for first two nexus switches :
Nexus 1:
feature vpc
feature lacp
feature inter-vlan
conf t
interface eth 1/48(for keep alive)
switchport
switchport mode access
switchport access vlan 10
no shut
exit
interface vlan 10
no shut
ip add 10.10.10.1 / 30
exit
vpc domain 1
peer-keep alive dest 10.10.10.2 source 10.10.10.1 vrf def
exit
interface eth 1/49
channel-group 1 mode actvie
exit
interface port-channel 1
switchport
switchport mode trunk
switchport trunk allowed vlan all
peer-link
exit
interface eth1/1-2
switchport
channel-group 20 mode active
exit
interface port-chnnel 20
switchport
switchport mode trunk
switchport trunk allowed vlan all
vpc 10
exit
same configuration for Nexus 2 :
different is vpc domain 1
peer-keep alive dest 10.10.10.1 source 10.10.10.2 vrf defualt
rest is same .
Now with enabling the port mirroing at fortigate these two switches works fine (connected to servers using one link from each switch to the server) .
Problem :
we can't introduce the other two nexus (all switches are 9000 series ) to the fortigate as it flaps all the links and start dropping the packets .
note:
fortigate uses channel group between all the ports (single channel group , trunk port from passive and active).
We dont want to change the hirarchy of fortigate (as less as possible if needed)
how do you think we can link these other two switches throught the trunk port of fortigate (they will be connected with different end devices "servers") also we really want to have link from eatch fortigate to each switch in the network (all 4 f them )
Thanks in advnace
07-20-2020 08:41 AM
OK it may not be relevant but it's not good practice to use operational SVIs as your VPC keepalive. You should configure your E1/48 as no switch port, give it an IP address and use a separate VRF. However that's by the by.
In terms of your design, at Layer 2 do the two Fortigates act as two separate devices, or do they share some sort of entity? Ideally you want each Fortigate to have a "normal" port channel configuration, they don't need to know it's actually a VPC but they do need to know it's a port channel and therefore they should expect inbound on either of the ports. Their load balancing algorithm doesn't necessarily have to match that of the VPC.
Can you draw how you'd ideally like it to look like when you have four Nexuses? You are aware the two pairs of Nexuses need to use different VPC domain IDs if they are linked at L2.
07-20-2020 02:56 PM
07-21-2020 05:49 AM
OK so the other two Nexuses will be a separate VPC pair, with VPC port channels to the two firewalls. Is that correct?
If that's the case then each Fortigate needs to have two separate port channels configured, two ports in each. One port channel goes to your original two Nexuses, and one goes to the 3rd and 4th.
Can you post up "sh vpc brief" from the VPC Primary of each pair?
07-21-2020 09:55 PM
Good morning Dear Toney
I have seen the vpc status its working well and forming adjacency but switches are at the site not with me currently, however whenever possible i would show you the vpc status.
So, you mean that each fortigate will have two port channels trunk with different vlan tag.
In that scnario do we need also to connect the two vpc domain devices togather, if needed how the connection will be and what are the configuration for this connection a side from the two vpc domain config.
One last question, vpc domain can bundle two devices only right?
Because if they requested to keep the fortigate config downstream port as one channel group for all 4 ports, is it possible?
07-22-2020 06:56 AM
A VPC domain comprises two Nexuses. If you have four, they need to be two VPC domains, with different domain IDs if they have any L2 connectivity between them.
A channel group on an external device, like your Fortigate, must go to a single switch or a single VPC pair. So your four ports could be in one channel group with two ports going to Nexus 1 and two to Nexus 2. Or two channel groups one going to each VPC pair.
Why do they want four ports to go as one channel group to two separate sets of switches? I can't see any logical reason.
07-24-2020 03:32 AM
From you post you mentioned that , With 2 VPC domain we can create one channel group at two fortigates (all downstream link ) could be linked with one channel group (trunk port with Vlan tagged at fortigate ) .
I have checked the scenario as below :
- Created two VPC domain (1 pair of UTP & 1 pair of Fiber ).
- Didn't link the two VPC pairs together (as that the end devices connected to them are total separate either fiber link or UTP link).
- Assigned the downstream ports at each VPC domain as one channel group , trunk port with VPC no 15,16 respectively .
Result :
- The ports (at each fortigates were not working normaly it goes up/down and it was abnormal .
- I thought this scenario is not possible as the fortigate will think each VPC pair as standalone device which couldn't be possible to use one channel group with multiple devices (as there are multiple VPC pairs ).
Can you explain please how to use multiple VPC domains and connect all of them in one port channel at the pairs of fortigate devices ?
Thanks
07-24-2020 05:41 AM
You need to stick to the principles. A VPC Domain has exactly two Nexus switches as members. A VPC has member interfaces on each of the two VPC Domain members. Incoming frames can be received on any member port, and outgoing frames sent on any member port.
So your four Nexuses can't be all one VPC domain, they will be two pairs. VPC Domain IDs must be different.
On the Fortigate a channel group must have all it's member interfaces connected to the same VPC pair.
What I can't understand is the desire to have one channel group on the Fortigate connected to different VPC pairs. Inherently in a channel group the device is free to use any of the member ports. So how would it be expected to work if a frame was received on say port 1, from Nexus one, but the Fortigate chooses to send its reply out of port 4 going to Nexus 4?
Why can't you have all traffic from the Fortigate going to Nexus 1 and 2?
What else is connected? Presumably you have other devices and maybe a whole network connected to the Nexuses in some way.
07-24-2020 06:26 AM
Yes i understand that but i juat wanted to clarify.
So, what are the other options to keep the network configuration same as its.
Does interconnecting two vpc domain with each other and then connecting them to firewalls using links to nexus 1 & links 2 works fine (all 4 nexus switches has one vlan for member ports).
Kindly advice.
07-24-2020 08:08 AM
I'm afraid I really don't follow what function you're trying to achieve.
Let's start with the Nexuses, and forget the connection to the Fortigates. How is the LAN configured with the Nexuses and other devices. With that defined, what functions are the connections to the Fortigates providing?
I think there's a danger of getting bogged down in "can I connect this to that" without knowing exactly what function is wanted.
07-31-2020 03:11 AM
Thanks Tony & sorry for the late reply as i was traveling & couldn't check the Community .
In reference to the main function wanted from the VPC connection it is simple as this :
- We have a single Vlan for all devices connected to the Nexus switches.
- Each end device connected to nexus has dual link (channel - group).
- Each Nexus shall has two connection with gateway (Fortigate "Active/Passive").
- All uplinks and member ports are assigned to single same Vlan .
- Some Servers connected through UTP switches & some connected through Fiber switches .
- Now two nexus switches (UTP ports ) function properly without any issue between the uplinks & member ports.
This is our simple scenario and I think the easy way is to change the network architecture by creating another ether channel to connect to the other two nexus fiber switches (with different vlan id as SVI ).
If this is the only possible scenario (to change port channel configuration at firewall) , Then (correct me if i am wrong ) we don't need to do any physical connection between the two multiple VPC domain as the downstream devices are either connected using fiber or UTP connection !
Do you think there is any other possible solution with the said scenario ?.
Please advice.
Thanks
07-31-2020 06:54 AM
@mabuzaid1 wrote:This is our simple scenario and I think the easy way is to change the network architecture by creating another ether channel to connect to the other two nexus fiber switches (with different vlan id as SVI ).
If this is the only possible scenario (to change port channel configuration at firewall) , Then (correct me if i am wrong ) we don't need to do any physical connection between the two multiple VPC domain as the downstream devices are either connected using fiber or UTP connection !
Do you think there is any other possible solution with the said scenario ?.
Please advice.
Thanks
Let's see if I have this correct. What you're suggesting is that you'll create a separate channel group on the Fortigates, with a different subnet and VLAN ID, and connect this to a VPC channel group on the other two Nexuses. No Layer 2 connection between the first VPC pair and the second. Communications between devices on Nexus 1&2, and devices on Nexus 3&4 will go via the Fortigate, these being the only devices with connections to both Nexus pairs.
All correct so far?
If so then depending on how much internal traffic you have that could be inefficient, considering the Nexuses can do L3 switching and routing.
What is the reason for the second two Nexuses? Why do they have to connect to the Fortigates rather than having let's say Fortigates connecting Nexus 1&2, then Nexus 3&4 connecting to 1&2 with a double ended VPC. Depending on the models you could use direct attach cables and connect the Nexus to Nexus links at 40 or even 100gig.
08-05-2020 10:57 AM
Hi Tony ,
Yes you got the idea , The reason why he have multiple nexus switches is because some end devices has only
Fiber NIC And some has only UTP , with that said we have multiple VPC devices .
The reason not to interconnect multiple VPCs together and connect them through fortigate is that we would like
To utilize The full speed of each firewall port (10GB) with a single devices (in this case channel group).
In case of interconnection between multiple VPC (we will need to create multiple VPC domain and configure it
Thanks
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide