mouving from router on stick topologie to switch module sm-x

[Aladdin0z]

I am moving from router on stick to an integrated switch module topologie , 

whats the best way to do it , 

where to put the gateways ?

if i deciede to use svi interfaces inside the switch module , how can we insure the routing between the switch module and the router .

i dont know if the router will have a unified routing table that includes switch svi interfaces ?

[Flavio Miranda]

Hi @Aladdin0z 

 You need to think on the router with switch module as a switch layer3. The module does not required a routing table as it is layer2 only but it will use the routing capacity of the router. 

So, basically when you create interface vlans (SVI) that can be used as gateway for your end devices and if you have a layer3 interface on the routing to the internet, the routing table would between that interface and the SVI. 

[Aladdin0z]

Great , thats the point , i wasnt sure about the router having all the interfaces and the svi in one routing table , that fixes all my issues . 

can you give me a brief descreption of why they mention BDI (bridge domain interface ) 

[Flavio Miranda]

It is the same thing for different platform:

ISR router (for BVI) and ASR1K (for BDI).

[M02@rt37]

Hello @Aladdin0z,

Create VLANs on the switch module and configure SVI interfaces for each VLAN. These SVI interfaces will act as the default gateways for devices in each VLAN.

Connect the switch module to the router using a physical link. You can either connect a dedicated routed link or use a VLAN trunk between the switch and the router.

Then two choices:

Configure static routes on the router and switch module. On the router, set the next-hop IP address for the SVI interfaces on the switch module. On the switch module, set a default route pointing to the router.

Use a dynamic routing protocol (such as OSPF or EIGRP) to dynamically exchange routing information between the router and the switch module. This way, the router will have a unified routing table that includes routes to the SVI interfaces on the switch module.

https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://www.andovercg.com/datasheets/cisco-4451_config.pdf&ved=2ahUKEwivyqeE1PCAAxXzRKQEHR5PBA4QFnoECA0QAQ&usg=AOvVaw3bH0DDuhLap42x_TIsREBM

[Aladdin0z]

The switch is imtegrated , there is no link between the the sm-x and the router 

[M02@rt37]

Yes @Aladdin0z!

I have to check but SM-X module on particular Cisco Router, a dedicated link can be used ! I think it is on 3945 I have seen that.

Check the SM-x dedicated documentation.

[M02@rt37]

@Aladdin0z

Here: https://www.cisco.com/c/en/us/td/docs/routers/access/interfaces/eesm/software/configuration/guide/4451_config.html

[Aladdin0z]

Yes i see your point , 

but the same document talks also about  Configuring SVI interface 

i thinks its a better option with less complixity ! isnt it ?

[M02@rt37]

Yes it is @Aladdin0z 

[Joseph W. Doherty]

"i dont know if the router will have a unified routing table that includes switch svi interfaces ?"

It should be unified (by default - start mucking with VRFs, different story).

"I am moving from router on stick to an integrated switch module topologie , 

whats the best way to do it "

Why are you doing this, at all?

Reason I ask what you goals might be, your expectations might not be met nor align with optimal usage of the hardware.

Keep in mind, what follows is dated information, so perhaps it's now much better on the 4K ISRs.  (Such as SM modules appear to connect to MGF providing a 10g fabric[?].)

In the past, switch modules, for ISRs, were principally provided as a convenience.  A way to avoid needing to install both a router AND a switch at some location.

This worked fine, but like a separate router and switch(es), for LAN routing, an ISR is a major LAN performance bottleneck.

In theory, moving to the switch into the router itself, as a module, allows increasing the bandwidth between the switch and the router.  The reason I say in theory, often router modules don't provide multi-gig bandwidth connections to the router's backplane.  Why?  Because an ISR, might not have the capacity to deal with all its routed ports, running full speed, so there may be no real need for the switch module to have an "internal" connection with more than a gig's bandwidth, if even that.  (The latter, again, because the router might not even be able, again, to deal with its built-in Ethernet ports.)

Also again, even if the switch module has more "internal" bandwidth, the router might be unable to support that rate.

This is the primary reason, switch modules are convenient, they reduce number of devices, but may not improve performance, in fact, some reduce performance relative to an external switch.

Typically, if you want wire-speed LAN routing, you use a L3 switch, which might connect to a router for off-site connectivity, where router features become more important.

Cisco, though, again in past, had some (usually few) router switch modules, which were L3 switches in their own right.  Those did L3 on the module itself, and did (I believe) L3 with the router, with the options a standalone L3 switch provided.

The latter, for heavy LAN routing, was huge improvement, eliminating LAN routing that would have be done on the router, but in overall performance, its usually about the same as a separate router and L3 switch.  Also again, perhaps some more bandwidth between the L3 switch module and the router, or perhaps a bit less.  As before, the switch module cut down on the device count, and cut down, possibly, overall costs too.

To recap, converting a L2 or L3 switch, connected to an ISR, into a L2 or L3 switch module, generally doesn't much change performance, and doesn't make a huge difference in configs, except something like sub interfaces might become SVIs.

"where to put the gateways ?"

Where are they now?  If they are subinterfaces, tied to .Q tags, with a L2 switch module, they will likely become SVIs related to VLANs if module is L2 only.

[Aladdin0z]

Hello @Joseph W. Doherty  thanks for your detailed post , 

from what you said i have understood that , if i mount the sm-x module there will not be a routing issue between the router and the etherswitch module that i added since the routing table will be unified and the SVI will automatically apear in the global vrf .

now my question is , why do we see some recomendations of using a bridg domain and service instance in order to connect the router to the module 

like the following config   ( the refrence for this config is the Cisco SM-X Layer 2/3 EtherSwitch Service
Module Configuration Guide for Cisco 4451-X ISR) in my case i have a cisco 4331 isr serie 

Router:
interface Ethernet-Internal 1/0/0
service instance 1 ethernet
encapsulation dot1q 50
rewrite ingress tag pop 1

interface BDI 1
mtu 9216
ip address 10.0.0.1 255.255.255.0

Switch:

interface Gi0/18

switchport trunk vlan allowed 50
switchport mode trunk

vlan 50
name Egress vlan

interface vlan 50
ip address 10.0.0.2 255.255.255.0

ip route 0.0.0.0 0.0.0.0 10.0.0.1

i want to make the difference between the different approaches available.

[Joseph W. Doherty]

Ah, I believe I understand your question now.  There are two ways to configure?

If so, it appears, like newer L2 switches, SM supports limited L3.

The BDI config allows your SM module to route between SVIs and route to/from the (logical) router.

In this kind of config, your SVIs would be the gateways, and by default, might not all appear in router's route table unless for each there's a BDI for each VLAN.  Without multiple BDIs, you'll need route statements on the (logically) router.

The "dated" L2 router modules did not have any L3 routing capability.

Perhaps this approach is confusing, as it's something you likely wouldn't use other than for this hardware.  Actually, many years ago, I've done something similar between a router and a L3 switch which didn't "share" a dynamic routing protocol.

[Joseph W. Doherty]

Background:

I've skimmed the reference M02@rt37 provided.  BTW, what's the exact/specific SM module you have?

Two factoids that caught my attention, the SM module only (?) uses "internal" gig connections (at least 1, maybe 2?) to the (10g?) MGF fabric.  Also, the SM module, described in the reference, is a variant of the 3560E (when I saw the QoS references, I wondered, as the QoS architecture seemed to be that of a 3560/3750.)

BTW, my "dated" information, was based on 2800 series ISR, and their Ethernet module options.  I assumed (so far, not totally incorrectly [?]) their architectural approach might still apply to 4000 ISRs.

I had used L2 Ethernet modules in the 2800s, but never used one of their L3 Ethernet modules.  So, don't know, without researching, exactly what the interface options, if plural, were between a L3 module and the router.

Skimming the SM L2/L3 module reference, I tried to get a better idea of its architecture, as the config example implies much.  Unclear whether that's the only way for the router and module to work together, or the "best" way.

Your question:  "why do we see some recomendations of using a bridg domain and service instance in order to connect the router to the module"

For starters, as I previously noted, if the module supports L3, for VLAN to VLAN routing, we want to (much) avoid using the router for that.

It appears, from the referenced documentation, the L2/L3 module really may/does much operate as a separate device, from the router, and they are (logically/physically) interconnected via a gig (?) interface.

If such, I would have expected the interconnecting "link" between the router and its module to provide L2 transit, with some form of L3 addresses, for both the router and module (sharing the "link").

Think of it much like a physically separate router and L3 switch, with a connecting link, attached to ports on both the router and the L3 switch.

If that's in fact, what the architecture works like, although you can do what Cisco shows, I'm (currently) baffled why Cisco shows to do it this way.  Why?

Because you've defined the module interface as a trunk, but only allowing a single VLAN.  So why use a trunk?  Perhaps so we can use L2 CoS?

On the router side, since we're connected to a switch trunk, we don't use a subinterface to accept a VLAN tagged frame, instead we "rewrite ingress tag pop 1" which I believe (?) discards the VLAN tag, for ingress.

It doesn't make much sense, unless, perhaps, the bridge interface on the router allows us to use two SM modules, supporting the same VLANs?  Or we need a bridge interface because of the way the router and other hardware, such as a SM module, share the MGF?

With what I know so far, I cannot explain, "why" Cisco is doing what they show.  I can say, functionally, it should work fine, and perhaps optimally or nearly optimally.  (Why "nearly", again the purpose of using tagged frames between the router and module, for a single VLAN while doing L3, unknown to me.)

Lastly, unlike earlier L2 only modules, which "integrated" into the router, sort of making the router a hybrid L3 switch, in the case of SM module, perhaps logically, it remains very much its own device in many respects.  At the moment, I don't know if other SVIs, on the module, are directly visible to the router.

What might be ideal, would be someone to add a reply that's used such modules, where they can tell us, how the router and the module "see" each other.  Otherwise, much slogging through Cisco documentation.

[Joseph W. Doherty]

Reply - some EVC description, they might apply to why Cisco is doing what they are doing.

https://community.cisco.com/t5/networking-knowledge-base/understanding-ethernet-virtual-circuits-evc/ta-p/3108219