MPLS and SVI issues.

[Emperor2000]

Hello i have a strange (in my mind) problem.

I have the design outlayed below in pdf document.

When i only have mpls links up everything is as it should.

But as soon as i take up one of the routed links that isnt an xconnect i get a loop.

Dosent matter if i bring it up in l2 network l3 network.

I can kill the loop either by killing the switchport in l2, l3 or kill one of the xconnects.

I imagine ive done some fatal thinking misstake but i cannot understand what.

Why would Vlan 315 descide to loop when i bring it up ?

If it wants to loop on the Xconnects i wouldnt need to bring the svi up in core for it to do so?

How can a routed interface trigger a spanning-tree loop?

[Vaibhava Varma]

Hi Gustav

We are doing here QinQ Transport across the MPLS Domain using Psuedowires for transparently creating a Layer 2 Switched Domain between the Customer Sites.Is that correct ?

Regarding the logical setup of the network in question can you please clarify that :

1. Do the Access Switches on the respective sites have a Trunk Link between them or not ?

2. The Cross_Links coming onto the Dist Switches from the Access Switches is for what purpose and is in what mode ie from the Access Side is it a trunk Port same as the straight Links to Dist Switches. or from Access_to_Dist its Access Vlan 315 and from Dist_to_Access its Access Vlan 60.

3. For the Straight-Links coming on from the Access Switches to Dist Switches is Vlan 315 allowed on that Trunk Link as Well ?

Regards

Varma

[Emperor2000]

Hello  Varma

Yes we are doing exactly what your are suggesting.

1. No not at this moment. Design isnt really finnished though. Cablign isnt done for theese links.

2. It is access ports on both sides. Vlan 315 on access side and vlan 60 on dist side.

3. Yes Vlan 315 is allowed.

Some history.

Vlan 315 is this specific customers vlan that transports there public connectivity to their firewalls.

Firewall is a cluster existing on both sides of the core. Therefore we are required to transport vlan 315 as a layer2 between sites.

Vlan 60 houses customers public connectivity in our core network.

2 vlan databases exist. Core vlans that are "ours"

Customer Vlans that the customer is free to use as they see fit.

[Vaibhava Varma · Accepted solution]

Hi Gustav

What it appears to me there is a looping of frames for the Po1.60 on C1 and C2 becuae of using VLAN 315 both on the cross-link (access-mode) and straigh-links ( trunk mode)

Lets Start from Access-1 and say a frame is sent out of Access-1 Cross-Link to Dist-2 and enters dist 2 with Vlan-tag 60 and exits of Dist-2 Uplink to C2 with Vlan-tag-60 and hits dot1q-60. Now when C2 sends a reply out of Po1.60 it again sends out frame with Vlan tag 60 out of the cross-link and enters Access-1 with Vlan tag 315 and if its a broadcast for unknown mac then will exit out of access-1 straight link to Dist-1 and enter the Dist-1 QinQ port with Tag 61 and hit the dot1q-61. Again on reverse path the dot1q61 will send out frame to Dist-1 with Tag 61 and will exit out of Dist-1 with respective tag 315 and entrer the Access-1 with tag 315 and again exit out of Access-1 cross-link to Dist-2 and enters Dist-2 with tag-60 and goes to dot1q-60. So even though this is not s definite loop but certainly potential point of looping and generating unnecessary and unwanted  traffic here.

If Vlan 315 has to be transported as a L2 Mode only across to the remote side then why are we using it as Acess on the Cross-Links.It means that the IP configured on VLAN 315 will communicate to remote sites as well as the Po1.60 on the Core Switches which again does not seems fine to me. As mentioned above Vlan 60 is for the Public connectivity and is IP routed across Core Backbone so we should be using a separate Vlan-ID apart from those being trunked on the core links for the L3 connectivity with Core Routers. That would definitely not generate the unwanted traffic flow in case of L2 Broadcasts.

In summary from my understanding I see a potential issue in using the VLAN 315 on both the Cross-Links as Access and Straight Links as Trunk in case of broadcast and since the requirement for the Straight Link is L2 Transport and Cross-Link is L3 Transport we should use separate VLANs for those.

Hope this helps to provide some insight into this issue from my viewpoint.

Regards

Varma

[Emperor2000]

Hello Varma.

Yes excluding vlan 315 from qinq trunks seems to have solved the issue.

Problem in our world is then that ASA firewall cluster will talk to 1 of 4 possible routing points on external network.

if firewall fails over to other side of core it will not be able to reach that routing point that it is choosing from that side. since that L2 segment isnt transported.

Iam after a scenario whre the firewalls fail over but not core device. So firewall on 1/2 side is talking to SVI on c1 then fails over to 3,4 side it cant talk to svi on C1 anymore.

Even an HSRP solution in core wouldnt solve the issue?

[Vaibhava Varma]

Hi Gustav

Sorry but I am not able to visualize fully the end to end network setup from above description..getting little confused

Still I will put my understanding of the network setup and requirement...Please correct me wherever I am wrong:

1. Firewall at Site-1 behind C1/C2 and Firewall at Site-2 behind C3/C4 are running in cluster mode and for that they need an L2 connectivity which was being provided using Vlan 315 across the QinQ PWE3 circuit across MPLS Backbone via Po.61 and Po.62

2. Same Time the Site-1/Site-2 needed Public connectivity also which was provided using Po.60 in L3 Mode on Dist Switch and Vlan 315 on Firewall Side.

Now the issue was that Vlan315 was being used for dual purpose at the same time to provide L2 and L3 connectivty and being passed onto both the Links-Straight and Cross which cuased the first issue in question and later solved by discarding vlan 315 on the straight trunk link but that breaks the L2 connectivity between the Cluster.

( I am still wondering how we were using same Vlan 315 for connecting the FW Cluster and also connecting to MPLS Core in L3 Mode ? )

I think it would have been better to still pass the Vlan315 over the Straight Trunk to Dist Switch and change the Access Vlan on the Cross-Link to some other vlan..In that way we would have been having both L2 and L3 connctivity at same time.

Now thinking of the Firewall-Failure at Site-1 and Site-2 FW taking over what it would is to have a Public Connectivity which would still be there via Po.60 on C3/C4. Running HSRP on C1/C2 for Po.60 and C3/C4 for Po.60 would be a good solution for HA of the Public Connectivitt but how would we transport the HSRP Hellos. Do C1/C2 or C3/C4 have an L2 Link to Pass HSRP Hellos

Sorry but I might still not be able to comprehend the requirement  fully in the above post and you might find some disconnect in my thought process.If you can elaborate more on the Firewall Failover Scenario and the requirement of Public Connectivity to SVI on C1. I believe if the FW Fails to 3/4 then the SVI on C3.C4 should serve the Physical Connectivity.

Regards

Varma

[Emperor2000]

Hello Varma.

Well the Firewall has a number of networks connected to it.

External network is vlan 315.

And some other vlans for internal zones.

Some DMZ and so forth.

On Vlan 315 the firewall has 2 interfaces that is connected to vlan 315 and one virtual interface.

The cluster then talks to each other on all vlans that it has.

When the cluster looses connectivity to each other on a number of networks it fails over.

So it would be nice to have l2 connectivity even on Vlan 315 as with the other networks.

A second reason would be that if FW1 behind C1/C2 is primary then the FW will talk on its virtual external interface (which in this case is FW1 physical interface) to C1 for instance. In a scenario where the FW1 dies or becomes unresponsive FW2 will take over. It will resume all sessions and such. Since FW2 is taking over every session to it will also take over the external sessions on vlan 315. When that occurs FW2 will try to talk to SVI 315 on C1. Since there is no L2 connectivity between sites it will fail to do so.

So in conclusion Vlan 315 is not a sync vlan but rather the way we transport L3 to the firewalls.  Since they are a cluster it is logicaly one FW on 2 sites and should therefore have connectivity to same interfaces to talk to for Session failover to work.