Solved: Re: How do I configure an etherswitch?

joelm · ‎06-01-2020

I have a two-port router and external switch. One router interface is configured with the WAN information. The other interface has the LAN information, including dot1Q VLANs, and some firewall rules. Plug the LAN-side cable into a switch, configure that switch port as a trunk and off I go. I've been successfully using it for years. The basic configuration would look something like this:

interface GigabitEthernet0/0
 description Internet facing
 ! static IP address given to me by ISP
 ip address a.b.c.d 255.255.255.X
!
interface GigabitEthernet0/1
 description LAN interface
 no shut
!
interface GigabitEthernet0/1.100
 description VLAN100
 encapsulation dot1Q 100
 ip address 192.168.100.1 255.255.255.0
!
interface GigabitEthernet0/1.200
 description VLAN200
 encapsulation dot1Q 200
 ip address 192.168.200.1 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 ISP_GATEWAY_IP_ADDR

On the switch, the LAN cable from router is plugged into this port:
interface GigabitEthernet0/1
 switchport mode trunk

I would like to replace both the router and the switch with an ISR4451 router, running 16.8.1 (Fuji) that has two SM-X-ES3-24-P switch modules, running 15.0(2)EJ, in it. I believe this setup is called an EtherSwitch. How do I configure the LAN side of things on the router? I tried the following:

Create dot1Q VLAN interfaces on the ethernet-internal1/0/0 or 1/0/1 interfaces (eg: eth-int1/0/1.100). Router says that isn't valid.
Set up BDI using the instructions at this link. Step #3, service instance 1 ethernet is not valid for my IOS version.
Configure SVI using the instructions documented in the link above. Configuration would look something like this:

On the router:
router(config)#interface GigabitEthernet0/0/0
router(config-if)#description Internet facing
 ! static IP address given to me by ISP
router(config-if)#ip address a.b.c.d 255.255.255.X

router(config-if)#interface Vlan100
router(config-if)#ip address 192.168.100.1 255.255.255.0
router(config-if)#interface Vlan200
router(config-if)#ip address 192.168.200.1 255.255.255.0

On the switch:
left-switch(config)#interface Vlan100
left-switch(config-if)#ip address 192.168.100.100 255.255.255.0
left-switch(config)#ip default-gateway 192.168.100.1
!
! PC is plugged into this port with IP 192.168.100.201/24, gw: 192.168.100.1
left-switch(config)#interface GigabitEthernet0/24
left-switch(config-if)#switchport access vlan 100
left-switch(config-if)#switchport mode access

With the SVI configuration, the router can ping a host on the internet. It can ping the left-switch(.100.100), and the PC (.100.201). The left-switch and the PC can ping any interface on the router including the WAN IP address, but can't get out onto the internet.

Can someone offer me documentation or even better, a working configuration example, that will allow me to use this EtherSwitch configuration? It can use SVI, BDI, or anything else.

joelm · ‎06-10-2020

I couldn't get the switch modules working because they were both defective (sometimes buying used doesn't work out). I replaced them with an SM-X-ES3D-48-P, configured SVI, and everything is working as expected.

For those of you looking for configuration tips, here is what I did. On the router, I configured one of the on-board interfaces with my ISP's connection information.

interface GigabitEthernet0/0/0
 description **Internet facing interface**
 ip address A.B.C.D netmask 255.255.255.X
 ip nat outside

ip route 0.0.0.0 0.0.0.0 ISP_GATEWAY_IP

Then, I set up router-on-a-stick for the LAN side:

interface VLAN50
 encapsulation dot1Q 50
 ip address 192.168.50.1 255.255.255.0
 ip nat inside

interface VLAN100
 encapsulation dot1Q 100
 ip address 192.168.100.1 255.255.255.0
 ip nat inside

interface VLAN150
 encapsulation dot1Q 150
 ip address 192.168.150.1 255.255.255.0
 ip nat inside

To set up DHCP on the different VLANs:

ip dhcp pool NAME1       # for VLAN50
 network 192.168.50.0 255.255.255.0
 default-router 192.168.50.1
 <for brevity, the rest of the DHCP config lines not included>
!
ip dhcp pool NAME2      # for VLAN100
 network 192.168.100.0 255.255.255.0
 default-router 192.168.100.1 
!         
ip dhcp pool NAME3      # for VLAN150
 network 192.168.150.0 255.255.255.0
 default-router 192.168.150.1

To get NAT working:

access-list 2 permit 192.168.50.0   0.0.0.255
access-list 2 permit 192.168.100.0  0.0.0.255
access-list 2 permit 192.168.150.0  0.0.0.255

ip nat inside source list 2 interface GigabitEthernet0/0/0 overload

On the switch, each port is configured with:

switchport access vlan 50 or 100 or 150
switchport mode access

When a computer is plugged in, it will get an IP address in the appropriate subnet based on the port's VLAN.

This guide does not include any firewall/ACL protections, so make sure those are configured.

View solution in original post

Reza Sharifi · ‎06-01-2020

Hi,

interface GigabitEthernet0/1.100
 description VLAN100
 encapsulation dot1Q 100
 ip address 192.168.100.1 255.255.255.0

The lan is using private IP space right?

So, can you post the NAT config from the router?

HTH

joelm · ‎06-02-2020

For the existing 2-port router that works, this is the config:

interface GigabitEthernet0/0
 description Internet facing
 ip address a.b.c.d 255.255.255.X
 ip nat outside
!
interface GigabitEthernet0/1
 description LAN-facing interface
 no ip address
!
interface GigabitEthernet0/1.50 
 description static, routeable IP provided by ISP
 encapsulation dot1Q 50
 ip address 50.X.X.X  255.255.255.X
!
interface GigabitEthernet0/1.100
 encapsulation dot1Q 100
 ip address 192.168.100.1 255.255.255.0
 ip nat inside
!
interface GigabitEthernet0/1.200
 encapsulation dot1Q 200
 ip address 192.168.200.1 255.255.255.0
 ip nat inside
!
ip nat inside source list 2 interface GigabitEthernet0/1.50 overload
access-list 2 permit 192.168.100.0 0.0.0.255
access-list 2 permit 192.168.200.0 0.0.0.255

I have since discovered that the left and right switches can't talk to each other using the SVI configuration above. I believe I need to set up a bridge domain interface between the two switch modules and then set up a Bidirectional Forwarding Detection (BFD) protocol on the bridge domain interfaces (BDI) instead. Unfortunately, the Cisco documentation for BFD doesn't work; the commands don't exist.

Richard Burts · ‎06-02-2020

I believe that @Reza Sharifi has asked a key question when he asks about nat. This part of the original post is a classic description of a problem when nat is not correctly configured: "With the SVI configuration, the router can ping a host on the internet. It can ping the left-switch(.100.100), and the PC (.100.201). The left-switch and the PC can ping any interface on the router including the WAN IP address, but can't get out onto the internet."

The additional information that you posted is puzzling to me. Like the original post
Gig0/0 has a public IP, has nat outside, and the description says it is Internet facing. In this post is an interface we did not see before G0/1.50. It seems to have another public IP but does not have nat outside configured. But the nat rule is using this interface for translating traffic. The original post included a static default route using the ISP gateway address which I assumed was the a.b.c.d. Now it is unclear whether it is a.b.c.d or is 50.x.x.x. Perhaps you can provide some clarification?

HTH

Rick

joelm · ‎06-02-2020

I have a working configuration on my 2-port router with an external switch. I changed the original post above to highlight the existing configuration. I'm trying to duplicate this setup in an ISR 4451 running 16.8.1 (Fuji) & two SM-X-ES3-24-P modules both running 15.0(2)EJ.

If I didn't have the switch modules, I would run one port of the ISR 4451 to the WAN side, and one port on the LAN side going to my external switch. Change all references to gi0/0 and gi0/1 in the existing config to gi0/0/0 and gi0/0/1, and maybe a few other minor tweaks and it should just work.

However, I want to get rid of the external switch and use the two SM-X-ES3-24-P modules instead. I thought about just connecting a cable between one of the 4 router ports and the switch modules (that's dumb, I know), but that would only give me connectivity to one of the two modules. The right way to do this is to use the Ethernet-Internal1/0/0 and 2/0/0 interfaces on the router side to route traffic to/from the two switch modules.

I'm thinking (not 100% sure) that I need to set up bridging between the two switch modules, and use BDI on the Ethernet-Internal ports on the router. But, the Cisco instructions for configuring BDI don't work with my router and the switches. Steps #3-6 & 8-11 of those instructions are not valid commands for Ethernet-Internal ports; they only work on the 4 external ports on the router. Here are the instructions, so you don't have to bring up the document:

1. configure terminal
2. interface Ethernet-Internal 1/0/0
3. service instance 1 ethernet
4. encapsulation dot1q 20
5. rewrite ingress tag pop 1 symmetric
6. bridge-domain 1 split-horizon group 0

7. interface Ethernet-Internal 2/0/0
8. service instance 1 ethernet
9. encapsulation dot1q 20
10. rewrite ingress tag pop 1 symmetric
11. bridge-domain 1 split-horizon group 0

12. interface BDI 1
13. mtu 9216
14. ip address 10.0.0.1 255.255.255.0

Any insights on how to set this up would be appreciated.

Reza Sharifi · ‎06-02-2020

Hi,

So, you are trying to connect the switch modules together? Are the modules 3750 or 3850 series? If yes, maybe you want to stack them if possible. "sh ver" on the switch module will tell you the type of image is running on the switches.

HTH

joelm · ‎06-02-2020

left-switch#show version
Cisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 15.0(2)EJ,
RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Fri 13-Sep-13 12:09 by prod_rel_team

ROM: Bootstrap program is C3560E boot loader
BOOTLDR: C3560E Boot Loader (C3560X-HBOOT-M) Version 15.0(2r)EJ1, RELEASE SOFTWARE (fc1)

left-switch uptime is 17 minutes
System returned to ROM by address error at PC 0x27C97E0, address 0x0
System restarted at 15:10:26 EDT Tue Jun 2 2020
System image file is "flash:/c3560e-universalk9-mz.150-2.EJ.bin"

License Level: ipbase
License Type: Permanent Right-To-Use
Next reload license Level: ipbase

cisco SM-X-ES3-24-P (PowerPC405) processor with 262144K bytes of memory.
Processor board ID FOC19262T5X
Last reset from address error
2 Virtual Ethernet interfaces
26 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : F0:B2:E5:74:3D:00
Model number                    : SM-X-ES3-24-P
System serial number            : FOC19262T5X
Hardware Board Revision Number  : 0x00

Switch Ports Model              SW Version            SW Image                 
------ ----- -----              ----------            ----------               
*    1 26    SM-X-ES3-24-P      15.0(2)EJ             C3560E-UNIVERSALK9-M     

Configuration register is 0xF

and

right-switch#show version
Cisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 15.0(2)EJ,
RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Fri 13-Sep-13 12:09 by prod_rel_team

ROM: Bootstrap program is C3560E boot loader
BOOTLDR: C3560E Boot Loader (C3560X-HBOOT-M) Version 15.0(2r)EJ1, RELEASE SOFTWARE (fc1)

right-switch uptime is 18 minutes
System returned to ROM by address error at PC 0x27C97E0, address 0x0
System restarted at 15:10:36 EDT Tue Jun 2 2020
System image file is "flash:/c3560e-universalk9-mz.150-2.EJ.bin"

License Level: ipbase
License Type: Permanent Right-To-Use
Next reload license Level: ipbase

cisco SM-X-ES3-24-P (PowerPC405) processor with 262144K bytes of memory.
Processor board ID FOC19262T3V
Last reset from address error
2 Virtual Ethernet interfaces
26 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : F0:B2:E5:74:41:80
Model number                    : SM-X-ES3-24-P
System serial number            : FOC19262T3V
Hardware Board Revision Number  : 0x00

Switch Ports Model              SW Version            SW Image                 
------ ----- -----              ----------            ----------               
*    1 26    SM-X-ES3-24-P      15.0(2)EJ             C3560E-UNIVERSALK9-M     

Configuration register is 0xF

joelm · ‎06-02-2020

router#show version
Cisco IOS XE Software, Version 16.08.01
Cisco IOS Software [Fuji], ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.8.1, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Tue 27-Mar-18 13:43 by mcpre


ROM: IOS-XE ROMMON

router uptime is 4 hours, 8 minutes
Uptime for this control processor is 4 hours, 10 minutes
System returned to ROM by PowerOn
System restarted at 11:24:56 EDT Tue Jun 2 2020
System image file is "bootflash:isr4400-universalk9.16.08.01.SPA.bin"
Last reload reason: PowerOn

cisco ISR4451-X/K9 (2RU) processor with 1788457K/6147K bytes of memory.
Processor board ID FJC2301A0EH
4 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
6598655K bytes of flash memory at bootflash:.
0K bytes of WebUI ODM Files at webui:.
20971520K bytes of SATA hard disk at harddisk:.

Configuration register is 0x2102

Reza Sharifi · ‎06-02-2020

C3560E-UNIVERSALK9-M

Regular 3650s are stack-able. I am wondering if the stacking command are available and you can stack them. Can you check for stacking commands?

HTH

Reza Sharifi · ‎06-02-2020

Ignore my previous message. Theses are 3560s and are not stackable

Reza Sharifi · ‎06-02-2020

So, I think since they are not stackable, you just have to connect each switch module to the router using the internal links and use the router as a router-on-a- stick.

HTH

joelm · ‎06-10-2020