L3 switch configured as router

[Jan Rolny]

Hi all,

I really appreciate this forum :-) And I would like to share my question regarding L3 switch routing function.

I have exactly this situation (please see picture) There are 5 L3 switches 3750-X series. Every switch is in different subnet. Let's say 192.168.1.0, 192.168.2.0, 192.168.3.0, 192.168.4.0 and 192.168.5.0 mask 255.255.255.0. All L3 swicthes are connected together to L2 switch 2960 series. What i need is to route between all subnets (something like full mesh) So for example server 1 can ping (communicate) with all other server server 2 can ping with all other servers and so on.

Is it possible? I think yes, but I am not sure how to configure it :-)

Thank you,

Jan.

[Shashank Singh]

Hi Jan,

Not sure what model of 2960 switch you have and what is the IOS license running on it. Normally 2960 is a layer 2 switch and hence it will not be able to perform intervlan routing (2960-S switches can do intervlan routing though).

To confirm on this, try configure 'ip routing' on your 2960 switch in global config mode and see if the command is supported. If it is, your switch can route packets between 3750-X subnets.

On the contrary, 3750X is a L3 switch capable of intervlan routing. In case there was a 3750-X in place of 2960, that would definitely have routed packets between vlans. We just need to config ip routing on the switch to make it L3.

Hppe that helps,

Shashank

Please rate helpful posts.

[kapathak]

Hi Jan,

So if I got this right, you want all servers to be able to ping each other. Yes, it is possible with the current setup you have.

We can assign the role of intervlan routing to one of the 3750-x switches.

So here is what the configuration would look like:

On the main 3750-x

- Make sure that we have an SVI for all subnets/vlans i.e 192.168.2-5.0 is present on it

For example, the IP of SVI for vlan a would be:

ip add 192.168.2.1 255.255.255.0

for vlan b:

ip add 192.168.3.1 255.255.255.0

and so on

- Create the necessary vlans (vlan a, b,c..etc)

- Ensure that the port where the server connects to is mapped to the right vlan

- Port which connects to the 2960 switch is a trunk port allowing all the necessary vlans

- Make sure from show ip int br that all the vlan interfaces are up/up

On the 'other' 3750-x's

- Create the necessary vlans

- Create an SVI for the subnet/vlan which should be present on this. So if we vlan a with subnet 192.168.2.0 is supposed to reside on this switch, the IP of SVI for vlan a would be

ip add 192.168.2.2 255.255.255.0 (anything apart from 192.168.2.1 since that belongs to the main 3750-x)

- Ensure that the port where server connects to is mapped to right vlan

- Port which connects to 2960 should be a trunk port

On the 2960 switch

- Create necessary vlans

- Ports that connect to all 3750-x's should be trunk and allow all vlans

- ip default-gateway should be the SVI of any vlan, say vlan a of the main 3750-x

Default gateway on Servers should be the SVI IP of the 'main' 3750-x for the respective vlan

Please let me know if this helps.

[Richard Burts]

Jan

I am not sure how important it may be but the switches in your drawing at 3560s but your post talks about 3750s. Which is correct?

I read your post and your requirements differently than Shashank does. I assume that your switch in the middle is just a layer 2 switch and that you want the routing done on the layer 3 switches. That should be quite possible to accomplish.

Kapil suggests a solution and it should work. But I would suggest a somewhat different approach. Your post talks about having 5 IP networks/subnets. I am suggesting that these be configured on the layer 3 switches in a VLAN which includes the switch interface which connects to its local server. Your post does not talk about how the switch ports of the layer 3 switches that connect to the layer 2 switch. I am suggesting that these interfaces that connect to the layer 2 switch be configured in a different VLAN. I suggest that you use another network/subnet for this VLAN.

This results in one VLAN that connects all of the layer 3 switches and each layer 3 switch has its own unique VLAN/network that connects to its server. Then I suggest that you configure some dynamic routing protocol on the layer 3 switches (it might be OSPF but even RIP would be an acceptable protocol for this use). The routing protocol would run on each layer 3 switch and would advertise its network/subnet to the other switches. Through the routing protocol each layer 3 switch would learn where the various server addresses were reachable and what to use for the next hop to reach that server. This would assure that every server could successfully ping every other server.

My suggested solution uses a routing protocol and does not need any trunking configured. The solution from Kapil does need trunking but does not need a routing protocol. Kapil's solution has one switch doing all of the inter-vlan forwarding (and potentially becoming a single point of failure) while my solution has switches sharing the work of forwarding. Either solution would work. You can choose which one fits better into your situation.

HTH

Rick

[kapathak]

I agree with Rick. Being a LAN Switching TAC Engineer, I totally overlooked the possibility of using routing protocols to accomplish this. My bad. But yes, now you have two options to choose from. All the best! :-D

[Richard Burts]

Kapil

One of the interesting and helpful things about the forum is that multiple people looking at the questions bring their own perspective to the issue and may suggest different approaches to solving the question. What a good thing that is

I believe that there is a subtle lesson here. Most of us, especially when we were relatively new to networking, look for there to be ONE right answer. As we gain experience we recognize that frequently there are multiple answers that could work and then we need to figure out which of these answers is the one that works best in our particular environment.

HTH

Rick

[Jan Rolny]

Hi Rick,

I tried your suggestion and probably something will be wrong. Routes does not advertise. To clarify my network, you are right. There are different subnets and each subnet is in its VLAN. Each VLAN is configured with VLAN interface with IP address asigned. This IP's act as default GW for particular server. So each server has default GW of appropriate VLAN interface. In each subnet ports are assigned to the same VLAN. One port is connected with server and second port is connected with L2 swicth(with default VLAN1 all ports). You can notice that I do not have configured default GW on L3 switches. But I do not know what should be dafult GW for this swicthes.

show ip route switch0

Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default, U - per-user static route, o - ODR
       P - periodic downloaded static route

Gateway of last resort is not set

     10.0.0.0/24 is subnetted, 1 subnets
C       10.252.50.0 is directly connected, Vlan50

show ip route switch1

Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default, U - per-user static route, o - ODR
       P - periodic downloaded static route

Gateway of last resort is not set

     10.0.0.0/24 is subnetted, 1 subnets
C       10.252.40.0 is directly connected, Vlan40

here is configuration of two L3 switches. L2 switch is configure by default so all ports are in VLAN1 an no IP address is assigned.

SWITCH0

!
!
ip routing
!
!

!
interface FastEthernet0/23
switchport access vlan 50
!
interface FastEthernet0/24
switchport access vlan 50
!interface Vlan1
no ip address
shutdown
!
interface Vlan40
ip address 10.252.50.254 255.255.255.0
!
router rip
version 2
network 10.0.0.0
!
ip classless
!
!
end

SWITCH1

!
!
ip routing
!
!

!
interface FastEthernet0/23
switchport access vlan 40
!
interface FastEthernet0/24
switchport access vlan 40
!interface Vlan1
no ip address
shutdown
!
interface Vlan40
ip address 10.252.40.254 255.255.255.0
!
router rip
version 2
network 10.0.0.0
!
ip classless
!
!
end

Thank you very much,

Jan.

[Richard Burts · Accepted solution]

Jan

I am glad that you have tried to make it work following my suggestion. I believe that I see what is the problem. In my suggestion I suggested that each layer 3 switch should have 2 VLANs. One VLAN would be to connect to its local ports (vlan40, vlan50, etc) and the other VLAN (perhaps vlan1) would be on the interface connecting to they layer 2 switch. In my suggestion all the layer 3 switches would be in a common subnet for their connection through the layer 2 switch. In what you have configured each switch has only a single VLAN. So the layer 3 switch sees it as VLAN40, including its connection to the layer 2 switch. But the layer 2 switch sees all its connections as VLAN 1.

Since all ports are configured as access ports and not as trunk ports then data will flow from one layer 3 switch to the other layer 3 switches (that part is good). But the routing does not flow that way. What is happening is that the layer 3 switch at 10.252.40.254 sees a routing advertisement coming from 10.252.50.254. But the routing logic says this is an error since the source of the routing update is in a different subnet.

So I suggest that you change your configuration of the layer 3 switches. Keep the VLANs as you have configured them (vlan40, vlan50, etc) to connect to the servers, etc. But change the interface that connects to the layer 2 switch. Put that interface into another VLAN (perhaps vlan1) and then configure the VLAN interface. perhaps something like this

interface vlan1

ip address 10.252.1.1 255.255.255.0

By doing this all the connections from layer 3 switches to the layer 2 switch will be in the same VLAN and also in the same subnet. This will allow the routing of RIP to work.

HTH

Rick

[Jan Rolny]

Hi Rick,

thank you so much for help :-)

It works fine now. First time I tried to configure L3 switches with no swicthport command so I had dedicated L3 interface with different IP address connected to VLAN1 on L2 switch and this does not work. (there must be default GW configured and it does not exist).

Then I configured new interface but it is VLAN1 interface on L3 switch connected to L2 switch in the same VLAN as you suggested and it works fine:-)

So if I understand right, so when it is routed via two different VLAN interfaces, then traffic go out say from VLAN1 (outgoing interface) on L3 switch and it is tagged as VLAN1 so L2 swicth knows what to do with this packet (tagged as VLAN1) and send it to right destination. 

But in other case, if I will configure L3 switch with separated interface (no switchport command) without default gateway configured, then L3 switch do not know where to send this packet destined to another network. Is it right?

Best regards,

Jan.

[Richard Burts]

Jan

I am glad that you got it working and that my suggestions were helpful.

There are at least 2 ways that you could configure this on the layer 3 switches and it would work fine.

One approach is to configure the layer 3 switch interface that connects to the layer 2 switch with no switchport. This makes it a layer 3 interface, there is no VLAN associated with the interface, and you configure the IP address directly on the interface. All of the layer 3 switches need their interface connecting to the layer 2 switch to be in the same subnet. It is not required to configure a default gateway on the layer 3 switch for routing to work. (But it would be necessary to configure a default gateway if you want the management interface of the switch to be able to reach other networks.)

The other approach is to let the layer 3 switch interface remain switchport which makes it a layer 2 interface in VLAN 1. You then need to configure the interface vlan 1 and assign the IP address on the VLAN interface. All of the layer 3 switches need to have their VLAN 1 interface in the same subnet.

You do not need to configure trunking on the layer 2 switch in either of these approaches. And if there is no trunking then there is no tagging of VLANs.

You said: "So if I understand right, so when it is routed via two different VLAN interfaces, then traffic go out say from VLAN1 (outgoing interface) on L3 switch and it is tagged as VLAN1 so L2 switch knows what to do with this packet (tagged as VLAN1) and send it to right destination."

That is almost but not quite right. As I explain above there is no tagging of VLAN because we do not need to configure trunking on the switches. So the layer 3 switch sends traffic out the vlan 1 interface, the layer 2 switch receives the traffic on an interface in vlan 1, and forwards it out the right interface on vlan 1. It is all in vlan 1 but there is no trunking and no tagging of traffic.

Also you said: "But in other case, if I will configure L3 switch with separated interface (no switchport command) without default gateway configured, then L3 switch do not know where to send this packet destined to another network. Is it right?"

This depends of whether a dynamic routing protocol is defined or not. If no dynamic routing is configured then you would need to configure a default route (which fulfills the function of default gateway) on each layer 3 switch - or you could configure a static route for each of the other networks/subnets present in your network. With a dynamic routing protocol the layer 3 switch learns the next hop for each of the networks/subnets which are in your network. The switch now knows how to forward to all of the other networks/subnets.

HTH

Rick

[bbb bbb]

Hi Rick,

would like to know if when time comes, does the diagram above would scale much..?

btw, i actually tried to make them interconnect too and realized that i jumped into one your approach. did no config on layer 2 switch .. did ospf network to reach other subnet.. see my attachment below.

"One approach is to configure the layer 3 switch interface that connects to the layer 2 switch with no switchport. This makes it a layer 3 interface, there is no VLAN associated with the interface, and you configure the IP address directly on the interface. All of the layer 3 switches need their interface connecting to the layer 2 switch to be in the same subnet. It is not required to configure a default gateway on the layer 3 switch for routing to work. (But it would be necessary to configure a default gateway if you want the management interface of the switch to be able to reach other networks.)

The other approach is to let the layer 3 switch interface remain switchport which makes it a layer 2 interface in VLAN 1. You then need to configure the interface vlan 1 and assign the IP address on the VLAN interface. All of the layer 3 switches need to have their VLAN 1 interface in the same subnet.

You do not need to configure trunking on the layer 2 switch in either of these approaches. And if there is no trunking then there is no tagging of VLANs."

thanks

[Richard Burts]

I would say that the digram in your post does scale fairly well up to 24 layer 3 switches and 24 subnets. (I say scale to 24 since the layer 2 switch in your diagram seems to be a 24 port switch.)

HTH

Rick