L3 or L2 Link between Core Switches

[Joselgil17]

Hello,

I am working on a project to re-design the network architecture of a client. This client has the typical network architecture with a pair of 6500s acting as CORE switches and the rest of access switches directly connected to these devices - COLLAPSED DISTRIBUTION/CORE - END-to-END vlans.

We are planning to introduce distribution switches to migrate the L2 boundary to those switches instead of the CORE ones, so the vlans will be expanded to those distribution switches (L2 spanning tree & L3 HSRP deployment). This will avoid any broadcast problem reaching the CORE switches from the access switches and eliminate posible spanning-tree loops.

Those new distribution switches will have L3 redundant connections to the CORE switches running EIGRP so this will provide us high availability and load balacing. The connection between these distribution switches is going to be a L3 link (Cisco recommendation) in order to summarize our networks to the CORE switches, so in case of an access-distribution link failure on one distrituion switch, this would re-route the traffic over the other distribution switch that has another connection to that access switch (triangle topology).

The problem comes when we need to decide what kind of LINK do we configure between the CORE switches.....we have two options:

1) L2 link between the 2 CORE switches:

ADVANTAGE: This kind of connection will allow us to use firewalls as a distribution layer if we don't want to use L3 switches (for example, perimetral firewalls facing internet). We could connect one firewall to each CORE switch via an access port, configuring VRRP between the two firewalls so they can talk across that L2 link.We would route directly to the VIP VRRP ip address from the CORE switches, providing us the distribution layer the we need....

DISADVANTAGE: Any broadcast traffic would affect both CORE switches as the vlans would be expanded between both.

2) L3 link between the 2 CORE switches:

ADVANTAGE: Broadcast traffic is isolated to one CORE switch...... I cannot see any other.... maybe summarization if we need to apply it here.....

DISADVANTAGE: We could not implement the topology explained above as there would not be L2 connectinty between the 2 CORE switches so if we attach a couple of firewalls to deploy VRRP, it would not work.....

I can think of a mix OPTION that would be to configure a L2 link and then establish an EIGRP session between the two CORE switches using L3 vlan interfaces so we could apply summarization.....

These are my thoughts so I would like to know if I am wrong or if there is other options that I am not evaluating.....

Many thanks for you help,

[Alessio Andreoli]

The Core siwtches should work on a layer 3 fashion to limit the broadcast domain that i am understanding you want to keep in the distribution switches and to have a much bigger control. Not to mention about the possibility to implement BGP and MPLS. One thing more, with layer 3 links you will have also a better possibility to control your QoS and to redistribute traffic if required.

Layer 3 for president

Alessio

[Joselgil17]

Hi,

Thanks for the reply but I would like to know if it's possible to configure the CORE-2-CORE link as L2 and form those rounting protocols sessions (EIGRP, OSPF or BGP) between the CORE switches with L3  vlan interfaces. This will provide us the posibility of connection a cluster of devices to those CORE switches so they can talk (vrrp) through that L2 link.

Regards,

[Peter Paluch · Accepted solution]

Hi Jose,

It is not common to connect devices, even a cluster, to core switches directly. A more proper design would be to put them into a data center, as the data center has its own access and distribution (sometimes called aggregation) layer switches, and the distribution switches would then in turn be connected to core switches.

Nevertheless, if you need to connect your cluster directly to the core switches, you can do it just fine. You simply create a VLAN for this cluster, create appropriate SVIs on the necessary core switches and run VRRP if you so desire. Routing between the core switches will then be through this L2 interconnection and through the SVIs.

What is even possible, and what I would recommend, is to actually use a combination of interconnections between your core routers. You can keep a routed interconnection between your core routers to carry the core-to-core traffic, and at the same time, you can have the VLAN and SVIs created for your cluster. In addition, assuming that you run some routing protocol between your core routers, define the SVIs as passive interfaces in that routing protocol. This will allow you to avoid routing the core-to-core traffic through the VLAN in which the cluster devices are located.

Best regards,

Peter

[Joselgil17]

Hi Peter,

Many thanks for your reply, I agree with you that the ideal design would be to connect that firewall cluster directly into a couple of distribution switches, and the distribution switches would then in turn be connected to core switches. But as firewalls are able to route, is more cheaper to connect those firewalls to the CORE performing distribution funtions instead of buying a couple of L3 switches to the connect the firewalls to them....

Apart from that, I cannot see any disadvantage on using this topology on the Internet access block for example, as we are not spreading any vlan to the CORE and the firewalls is going to block ay broadcast. Those firewalls connected to the CORE running VRRP and acting as a default-gateway for the internal traffic, a couple a L2 switches connecting the OUTSIDE interface of those fws to a couple of boundary Internet routers running BGP to an ISP, sounds good to me..... What do you think?

About your last recommendation for the CORE-CORE interconnection, you mean that you agree on using a L2 link between the CORE switches to allow the cluster to talk VRRP and at the same time establish a L3 routing protocol session (EIGRP,OSPF or BGP) between the 2 core switches using a dedicated SVI, right?....

Many thanks again.

KR

[Peter Paluch]

Hello Jose,

I am sorry to be responding later ... please accept my apology.

About your last recommendation for the CORE-CORE interconnection, you  mean that you agree on using a L2 link between the CORE switches to  allow the cluster to talk VRRP and at the same time establish a L3  routing protocol session (EIGRP,OSPF or BGP) between the 2 core switches  using a dedicated SVI, right?....

Basically, yes. What I want to achieve is to have a L2 link between these switches that will carry the traffic for the cluster and VRRP will be run in it (or HSRP/GLBP - is your cluster actually going to participate in any of these protocols? It should not...). However, this L2 link should not be also used for core-to-core transit traffic. So you should

• either allocate another L2 interconnect and use a dedicated SVI to perform inter-core routing
• or allocate another interconnect and configure it as a routed link (the no switchport on physical ports)

Does this make it more clear? Please feel welcome to comment and ask further!

Best regards,

Peter

[Joselgil17]

Hello Peter,

Many thanks for your reply and sorry to bother you with all these questions....

I understand it, you want to avoid using that L2 link for the L3 adjacency between the CORE switches. But I don't understand why?, I can only think on one posible reason; If that L2 link gets saturated the VRRP messages are going start failing, so the cluster is going to become ACTIVE-ACTIVE... Correct me if I am wrong.....

About the design option, cannot see any disadvantage on using this topology on the Internet  access block for example, as we are not spreading any vlan to the CORE  and the firewalls is going to block ay broadcast. Those firewalls  connected to the CORE running VRRP and acting as a default-gateway for  the internal traffic, a couple a L2 switches connecting the OUTSIDE  interface of those fws to a couple of boundary Internet routers running  BGP to an ISP, sounds good to me..... What do you think? What are the disadvantages that you see on this model?...

Thanks again,

[Peter Paluch]

Hi Jose,

I understand it, you want to avoid using that L2 link for the L3  adjacency between the CORE switches. But I don't understand why?

It's more of a design issue that a decisive technical reason. Look, you want to have a VLAN that contains your cluster. It is easier to have this VLAN separate and reserved just for the function of connecting the cluster to the network, rather than allow all kinds of transit traffic flow through. You may end up solving issues with security, isolation, even unwanted interaction of the transit traffic with the cluster operation. As a simple example, think of creating ACLs to protect this VLAN. They will be much easier if they just need to deal with the traffic from and to the cluster. However, if the VLAN becomes a transit VLAN, dealing with all kinds of locally originated/locally targeted/transit traffic will become much more fussy.

Sure, you can use this VLAN that spans both core switches to also provide a transit path for core-to-core traffic, and in the basic approach, it would run just fine. But from a design perspective (call it even a network aesthetics if you will), it is not appropriate - we use VLANs to contain traffic of similar kind.

Do you have any diagram of the intended state of the network, at least about the physical connections of the core switches and the cluster?

Best regards,

Peter

[Leo Laohoo]

2) L3 link between the 2 CORE switches:
ADVANTAGE: Broadcast traffic is isolated to one CORE switch...... I cannot see any other.... maybe summarization if we need to apply it here.....
DISADVANTAGE: We could not implement the topology explained above as there would not be L2 connectinty between the 2 CORE switches so if we attach a couple of firewalls to deploy VRRP, it would not work.....

Then use routing "to the floor".  For example, if you have a five storey building, then each floor is routing it's own subnet.  Any broadcast will affect this floor (and this floor alone).