Which is better in Layer 2 or 3 connection

[interfacedy]

Hi We have four office building at each four sites like below connections. Each link represents two fiber cables, for example, e1/0 to e1/0 between Site A to B has two cables. The four site are at one city. Do you think which one is better if using layer 2 or 3? Thank you

[Georg Pauwen]

Hello,

layer 2 is usually faster than layer 3. It obviously depends on the IP addressing, if you have a flat network (all in the same subnet) you could implement layer 2 links, if you have different subnets, you need some sort of layer 3 configuration. What does the IP addressing for these four sites look like ?

[Joseph W. Doherty]

"layer 2 is usually faster than layer 3."

BTW, generally on L3 switches, L3 is just about as fast a L2.

[MHM Cisco World]

These four connect SW form l2 loop,

To break it stp will blk one link, that ok but it add more work in one sw, 

Let explain , assume D-B link is blk by stp, and D need to forward traffic to A, that meaning that all traffic must go to C then go to A.

You can little make load balance by using pvst and change priority for each sw for each vlan in such away that not all vlan in trunk is blk 

If you use l3 you can easily get load balance via two path but also some server need l2 to connect to other servers or storage.

You have two choose and i prefer always use l3.

[interfacedy]

Thanks, The four sites has same subnet ip address. so if we setup L3, the four sites need different subnet? L2 can need to think about stp issue, which would cause almost half link down. How about we add aditional linke like A---D or C---B? 

[MHM Cisco World]

That will be good, the traffic will take direct cross link and if it fialed then there is two redundacy link.

[Joseph W. Doherty]

In modern networks, and if you already have L3 switches, L3 is often a better choice.

To your question in your later post, would you need different subnets, the answer would be yes.  BTW, with L3 switches, you can run L2 and L3, across your inter-site links, concurrently, making it pretty easy to convert/transition to L3.

[interfacedy]

@Joseph W. Doherty  If L2, half cable is blocked due to stp issue, but if L3, it also need primary and secondary routes, which also means half of cable is in backup status, right?

You mentioned "BTW, with L3 switches, you can run L2 and L3, across your inter-site links, concurrently, making it pretty easy to convert/transition to L3." 

Do you mean using L2 for connection among four sites? and then convert to L3

[Joseph W. Doherty]

"If L2, half cable is blocked due to stp issue, but if L3, it also need primary and secondary routes, which also means half of cable is in backup status, right?"

Maybe, maybe not, depends on how you're doing routing.  Cisco routers often support ECMP, e.g. A<traffic between>D might use both paths (i.e. A<>B<>D and A<>C<>D), equally and concurrently.

"Do you mean using L2 for connection among four sites? and then convert to L3"

Yes, as from your later posting it seems you might already have, or plan, L2 between the sites (especially as you note "The four sites has same subnet ip address.").

[MHM Cisco World]

for L3 
if you use OSPF or eigrp both support equal cost multi path and hence there is no link not use as one router primary and other backup. 
but again it depend on your network you want L2 in all SW or L3 between all SW. 
for which is faster 
it also depend 
if for L3 you config SVI as Peer IP for routing protocol between SW then it same as use L2 why 
becuase this depend on STP. 
so when I talk about L3 I talk about using router port between SW not using SVI of VLAN for peering. 

[Joseph W. Doherty]

"so when I talk about L3 I talk about using router port between SW not using SVI of VLAN for peering."

Yes, that's what's normally done.  However, as I mentioned doing L2 and L3 concurrently . . .

What's you can also do is assign a SVI for what would be a routed port.  I.e. such a VLAN, between the L3 switches, is only between two switches, no STP issue.  It can then "share" the trunk link and, for routing, it performs much like a routed port would.

e.g. R1 SVI 900 <VLAN 900> SVI 900 R2 SVI 901 <VLAN 901> SVI 901 R3 SVI 902 <VLAN 902> SVI 902 R4 SVI 903 <VLAN 903> SVI 903 R1

One other advantage of using a dedicated VLAN for the p2p link, if you have spare ports, you can define another port, on the L3 switch that might "survive" if the original port's hardware fails.  To recover quickly, just reconnect the failed port link to the "backup" port.

[MHM Cisco World]

with respect @Joseph W. Doherty  I dont agree in point of using SVI according to recovery time, as I mention before I prefer L3 router port if we talk about fast recover and for multi port you can use port-channel and config it as L3. 

below is ciscolive slide show why we need use router port for L3 in Data Center or enterprise 

150-200 msec compare to 8 msec is huge different. 

[Joseph W. Doherty]

Ah, a factoid of which I was unaware!  Wonder if that delta holds across all Cisco's L3 switches?

"150-200 msec compare to 8 msec is huge different."

Yes it is.  However, in the "real world" it generally wouldn't matter.

Why in another posting, you (@MHM Cisco World) posted that using BFD with a 900 ms failure detection time is generally acceptable, and probably often is.

Consider "usual" failure detection times for a FHRP (i.e. HSRP, GLBP, VRRP) or routing protocols.  So, again, 200 ms delay, usually doesn't matter.

If it does matter, we're likely in the world of networking using cut through switching, tuning routing protocol timers, adjusting interface bounce timers, etc.

BTW, the recovery time I was referring to was getting the down link, on a down port, back up by moving it to another port.  I.e. avoiding the need to "find" another such port on the device, and configure it for usage, rather than just moving the port connection to a known standby port.  (BTW, I do also like using port-channels for redundancy, but with MAN/WAN links, often cost prohibits using such.)

That aside, the SVI for p2p are great for transition from L2 on same trunk.  After transition, you can certainly migrate to the usual routed port setup.

[MHM Cisco World]

I already mention that this for DataCenter Or enterprise 
""below is ciscolive slide show why we need use router port for L3 in Data Center or enterprise""

for BFD we was talking about connect your Network to Service Provider not inside DataCenter.

[Joseph W. Doherty]

"I already mention that this for DataCenter Or enterprise"

In that other thread, it was I who mentioned a (WAN) Service Provider, an assumption on my part.  The OP only described using DWDM links between their sites.  Even if it's actually across a Service Provider's WAN cloud, OP appears to be doing BFD directly with their other sites.  I.e. it seems to me that thread is discussing Enterprise connectivity.

Further, if indeed there's a WAN Service Provider involved, not an ISP (also not mentioned, or implied, I believe), when I worked with WAN Service Provider connections, I considered them part of the "Enterprise".

Regardless, for most Enterprises, again, I doubt "real-world" it matters little if link failure time is 8 ms or 300 ms or perhaps even 900 ms.  Probably ditto for most DCs too.

BTW, as some Cisco devices appear to use a default of 100 ms for their debounce timer (at least on some of their Nexus 5000 and 7000 platforms - very much DC switches, right?), one would assume the debounce timer was off at least for the 8 ms result.  But, if some Cisco equipment uses a default debounce of 100 ms, that sort of implies they consider that delay more useful then detecting an 8 ms link failure?

Again, to be clear, I'm not saying using a p2p SVI is (overall) better than a routed port.  The former has some advantages over the latter, while the latter has some advantages over the former.

For example, another advantage of the latter, the routed port, it's what most network engineers expect to be used.  That is a plus for it especially as some/few network engineers have some difficulty working with L3 switches, as they have both L2 switch and router properties.