Can VMs on server connected to a routed port talk to VMs tagged with VLAN on server connected to trunk port

[CiscoPurpleBelt]

Can VMs on server connected to a routed port talk to VMs tagged with VLAN (same exact subnet as VMs on routed port that are not tagged) on server connected to trunk port?

Basically, I can easily build SVIs and have them all ping the associated sub-interface on the router or each other all throughout the LAN. I CAN't however ping the routed interface that is in the same subnet nor ping any of the VMs (still same subnet) from any other device in the LAN. The routed port is on a switch module on the 3800 series router. Assuming all my configs are right (once again SVIs can all ping each other excluding the routed port) would this ever be possible? Doesn't the VMs all in the same subnet all need to be tagged with the same VLAN?

[Joseph W. Doherty]

"Can VMs on server connected to a routed port talk to VMs tagged with VLAN (same exact subnet as VMs on routed port that are not tagged) on server connected to trunk port?"

Don't believe you can, at least as you've described. The problem would be to associate the VLAN with the routed port that (directly) connects to a server.

However, rather than using a routed port, if you made the port a host port, for the same VLAN, and configured a SVI, that should work.

"The routed port is on a switch module on the 3800 series router."

Which module? This a L3 switch service module? If so, believe you don't want to use a "routed" port.

[CiscoPurpleBelt]

yes it is as I can create SVIs. Do you know why someone would have configured the port that connects to these VMs as a routed port?

[Joseph W. Doherty]

No, normally I would expect a trunk or access port to a VM.

[nixpengu1n · Accepted solution]

Hello,

I've never tried that before, however from theoretical point of view it could work. If you configure a port as routed port and then assign a NIC on ESXi host to VM network without any tag. Then you create a VM in this VM network and assign an IP address form the same subnet as routed port. Default gateway should be an IP address on a switch of this routed port. 

As it is said before it is not a best practice from VMware point of view: port in general should be either an access or trunk port.

However with small or particular environments such setup could take place.

[CiscoPurpleBelt]

Yes that is how it is now. I advised in order to migrate their VMs which all are in the same subnet, one group of VMs can't be tagged with a VLAN and the other not. I need to changed all ports to trunk and tag accordingly on the network devices and the VMs all must be tagged as well. Am I missing anything?

[CiscoPurpleBelt]

Yes it is a layer 3 module. I understand how most of the time we configure port which connects to servers for trunk or access vlan but this case it is not, and I am trying to figure out why. My best theory is it was done for isolation. I advised that we should change this port to trunk, I create a sub-interface for VLAN that all the VMs in this same network group be assigned to (so they need to tag everything on their end to the same VLAN that is in this subnet). Now for isolation reasons, let's say they don't want this VLAN to talk to anything else, you think I should just create an ACL or VACL?

[nixpengu1n]

Hello,

It is hugely depending on your network setup and what you want to achieve. If you want this VLAN not to be propagated on Layer 2 than VACL. If you are OK with broadcast and Multicast traffic to be sent inside of a LAN from that VLAN than ACL. 

If you want a complete isolation of this VLAN inside of a LAN than do not define SVI on a switch. Just create a VLAN, assign necessary ports and tag this VLAN on ESXi end for VMs. Then you will need to figure out how to route a traffic within a LAN for this VMs of course, but from security point of view - it is a most robust method to separate traffic.

[CiscoPurpleBelt]

Awesome thanks. Ok let's say they want complete isolation (I don't believe that is the case because this VLAN will still need to talk to users on a different VLAN and at remote sites). If this is the case, given the current VMs are on the switch module of the router, and the other VMs that will be in same VLAN are on another switch which connects to the router through one more switch, what would the GW be for the VMs? Would it just be the sub-interface on the router since you are saying to not use any SVIs on the switches?

[nixpengu1n]

Hello,

Well, I am a bit confused. Please provide a network diagram to understand you situation and what you are trying to achieve.

[CiscoPurpleBelt]

Sorry I know I am making it more confusing. See attachment I made real quick. All connections are trunks and not pruned.

It is basically just a VM hosts on same subnet however one group of hosts is tagged and other isn't. I am used to making configs for hosts by tagging everything or nothing at all - the normal way I would say. I plan on having all VM hosts tagged, as well as VLANs with the SVIs but wanted to know if it would work without tagging everything - it does not right now.

[nixpengu1n]

Hello,

You can't have two interfaces on a router in the same subnet. If you want to have a complete isolation of VM traffic, please use separate subnet and do not expose it to Router or Switch as SVI or routed port. Just a VLAN.

However, as I said, then you will need to figure out how to access this VMs. In general such scenario is often used for testing purposes.