cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5277
Views
0
Helpful
3
Replies

Nexus Private VLANs

leesutcliffe
Level 1
Level 1

Hi

We have a requirement for private VLANS for DMZ hosting within one of our datacentres.

I just want to query how private VLANs would work in our environment.

We have physical servers connected to fex ports (2 fex per rack for each 5k) of a 5548UP switch, virtual servers using the nexus 1000v (vmware hosts connected to fex ports)

Out firewalls and load balancers are connected to an upstream pair of nexus 7ks using vPCs

My question is this, ordinarily the firewall would be in a promiscuous port but as these reside on a physically separate switch will the normal vPC trunk still be sufficient or would the "switchport mode private-vlan trunk promiscuous" be required on the vPC up to the northbound 7k.

As these connections are already in production I do not want to affect the existing traffic (that doesn’t use private VLANs)

Thanks

1 Accepted Solution

Accepted Solutions

Hello Lee,

I would say the security enforcement provided by a PVLAN configuration is local indeed. But to allow an end-to-end communication the seconday pvlans has to be created and allowed on the intermediate switches trunks => The flows will be tagged just like any regular VLAN.

In order for [secondary] pvlan hosts :

  • To be able to communicate with the outside/promiscuous port (from the VM to the N7k)
  • To be able to communicate between them (ex : two hosts from the same community pvlan but on different ESX).
  • And for the other PVLAN switches of the domain to be able to know if a communication is allowed or not.

For all of these conditions, the secondary tag has to be extended throughout the switches (or at least to the other switches using PVLANs where communication need to go through).

We have also to note the PVLAN traffic flow is considered as unidirectional => the tag depends on the traffic direction =>

Upstream direction :

From the end device (VM in pvlan) to the exit device (where the promiscuous port is located), the traffic travels with its associated secondary vlan tag.

Once it reaches the promiscuous port (on the N7k) :

=> If it is a “normal” promiscuous port, the secondary tag is stripped and the packet sent untagged to the FW or the L3 device (untagged / primary pvlan).

=>  If it is a “promiscuous trunk” port, the secondary tag is re-written with the primary vlan dot1q tag.

Ex : VM in isolated PVLAN 121 needs to communicate with the outside => The flow are tagged with pvid 121 between all the switches until it reaches the exit promiscuous interface .

This is why the secondary pvlan has to be created and allowed on the upstream trunks.

Downstream direction :

From the promiscuous port switch to the end devices, the traffic flow is tagged with the primary pvlan tag. When it reaches the end local switch, the flow is then forwarded through the associated private-vlan host interface.  

The primary pvlan has to be created and allowed on the downstream trunks.

===

We have also to consider the communication between two hosts in the same community pvlan and

hosted in two different ESX => The communication (within the pvlan domain / no need to reach the promiscuous port) is tagged with the community pvlan tag through the intermediate physical switches. 

===

I suppose it could work by tagging asymmetrically (secondaries on upstream trunk / primary + community on downstream) depending on the direction but I do not do that => I am used to create and allow all the pvlans on each side of the trunks.

So the ultimate answer is yes => We need to create the secondary vlans and allow them on intermediate switches trunks just like any regular vlan. The security enforcment occurs on the end devices locally as you said. 

I also think there is different ways to reach an end-to-end PVLAN secure communication especially depending on the restrictions and conditions (for ex. vlan defined as pvlan on the N1k and the N7k where the promiscuous port sits - But as regular vlan on intermediate switches...). Anyway here is what the documentation states :

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter6.html#con_1344030

You can extend private VLANs across multiple devices by trunking the primary, isolated, and community VLANs to other devices that support private VLANs. To maintain the security of your private VLAN configuration and to avoid other uses of the VLANs configured to be private VLANs, configure private VLANs on all intermediate devices, including devices that have no private VLAN ports.

Hope that helps.

Kind regards.

Karim

View solution in original post

3 Replies 3

krahmani323
Level 3
Level 3

Hello Lee,

If I have correctly understood the architecture could be represented as follows (if not please let me know) =>

[VM ---(link-a)---N1K]----(link_b)------N2K_FEX----(link_c)----N5548UP----(link_d)---2x_N7k---(link_e)---FW

====================================

If this is case, please find some comments about each link in the chain =>  

  • Link_a => The vEthernet associated to the VM is a “switchport private-vlan host” – isolated or community.

  • Link_b => Is to be configured as a regular trunk between the N1k and FEX (allowing regular vlans and pvlans towards the ESX).

We have also to note that by default a PVLAN is not supported on a FEX trunk. ie if you add a pvlan in the allowed list of a trunk it will not be forwarded through the trunk.

To allow pvlan to be forwared over a FEX trunk a special feature (PVLAN on FEX Trunk) has to be activated [available since 5.1(3)N2(1)]=>  http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/layer2/513_N2_1/b_Cisco_n5k_layer2_config_gd_rel_513_N2_1_chapter_0101.html#task_16C0869F1B0C4A68AFC3452721909705

  • Link c => fex-fabric connection between the N5k & the N2k.    

  • Link d => The Vpc between the N5k and N7k is a just regular trunk where the pvlans has just to be in the allowed list.

Just like for link_b, the pvlans are “transported” throughout the trunks just like regular vlans (tagged with dot1q depending on the communication direction) – the main idea being that pvlan security enforcement is applied locally on the end devices where ‘private-vlan hosts’ and ‘promiscuous ports’ are defined.

  • Link_e

=> You can use a dedicated link and configure the interface as a normal promiscusous port between the N7K and the FW.

=> A PVLAN promiscuous trunk port (supported since 5.0(2)) can be defined as well, if it is wanted to trunk both regular vlans AND PVLAN to the FW within the same trunk link. 

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter6.html#task_DF6EB3D08C3B45F68F0210030E355B47

====================================

I Hope it answers the questions...If not feel free to ask.

Best regards.

Karim

Hello Karim,

Many thanks for your reply and apologies for taking so long to get back to you, it's been fairly manic here!

You've understood the architechture correctly, I'm in the process of gather the requirements so I can begin this work

I query I have is the trunk between the 5k and 7k.

As I understand it, private vlans are locally significant to the switch, so do the secondary vlans need to be extended up to the 7ks?

Or will the the secondary vlan tag get stripped and only the primary vlan tag will be associated with that frame?

Kind regards

Lee

Hello Lee,

I would say the security enforcement provided by a PVLAN configuration is local indeed. But to allow an end-to-end communication the seconday pvlans has to be created and allowed on the intermediate switches trunks => The flows will be tagged just like any regular VLAN.

In order for [secondary] pvlan hosts :

  • To be able to communicate with the outside/promiscuous port (from the VM to the N7k)
  • To be able to communicate between them (ex : two hosts from the same community pvlan but on different ESX).
  • And for the other PVLAN switches of the domain to be able to know if a communication is allowed or not.

For all of these conditions, the secondary tag has to be extended throughout the switches (or at least to the other switches using PVLANs where communication need to go through).

We have also to note the PVLAN traffic flow is considered as unidirectional => the tag depends on the traffic direction =>

Upstream direction :

From the end device (VM in pvlan) to the exit device (where the promiscuous port is located), the traffic travels with its associated secondary vlan tag.

Once it reaches the promiscuous port (on the N7k) :

=> If it is a “normal” promiscuous port, the secondary tag is stripped and the packet sent untagged to the FW or the L3 device (untagged / primary pvlan).

=>  If it is a “promiscuous trunk” port, the secondary tag is re-written with the primary vlan dot1q tag.

Ex : VM in isolated PVLAN 121 needs to communicate with the outside => The flow are tagged with pvid 121 between all the switches until it reaches the exit promiscuous interface .

This is why the secondary pvlan has to be created and allowed on the upstream trunks.

Downstream direction :

From the promiscuous port switch to the end devices, the traffic flow is tagged with the primary pvlan tag. When it reaches the end local switch, the flow is then forwarded through the associated private-vlan host interface.  

The primary pvlan has to be created and allowed on the downstream trunks.

===

We have also to consider the communication between two hosts in the same community pvlan and

hosted in two different ESX => The communication (within the pvlan domain / no need to reach the promiscuous port) is tagged with the community pvlan tag through the intermediate physical switches. 

===

I suppose it could work by tagging asymmetrically (secondaries on upstream trunk / primary + community on downstream) depending on the direction but I do not do that => I am used to create and allow all the pvlans on each side of the trunks.

So the ultimate answer is yes => We need to create the secondary vlans and allow them on intermediate switches trunks just like any regular vlan. The security enforcment occurs on the end devices locally as you said. 

I also think there is different ways to reach an end-to-end PVLAN secure communication especially depending on the restrictions and conditions (for ex. vlan defined as pvlan on the N1k and the N7k where the promiscuous port sits - But as regular vlan on intermediate switches...). Anyway here is what the documentation states :

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter6.html#con_1344030

You can extend private VLANs across multiple devices by trunking the primary, isolated, and community VLANs to other devices that support private VLANs. To maintain the security of your private VLAN configuration and to avoid other uses of the VLANs configured to be private VLANs, configure private VLANs on all intermediate devices, including devices that have no private VLAN ports.

Hope that helps.

Kind regards.

Karim

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card