Configuring a SM-X Layer 2/3 EtherSwitch Service Module on a 4331 router

[Tod Larson]

How do I configure the Cisco 4331 routers' ethernet-internal interface to communicate with a SM-X Layer 2/3 EtherSwitch Service Module?

Neither interface ethernet-internal 1/0/0 nor interface ethernet-internal 1/0/1 have the ip address command available.  They don't have the "no switchport" command either.

I can do the "hw-module session 1/0" command to console into the etherswitch.  Once consoled in I can configure the switch, it just won't communicate with the internal interfaces on the router.
 

Here are the references I've tried to use.

http://www.cisco.com/c/en/us/products/collateral/routers/4000-series-integrated-services-routers-isr/datasheet-c78-730357.html

http://www.cisco.com/c/en/us/td/docs/routers/access/interfaces/eesm/hardware/installation/guide/eesm_HIG.pdf

http://www.cisco.com/c/en/us/td/docs/routers/access/interfaces/eesm/software/configuration/guide/4451_config.pdf

I fee like we are missing something obvious!

[Tod Larson]

I hate to answer my own posts, but here it is.  TAC tells us that there are 2 choices to make this work.  Apparently the way that worked on an ISR and ISRG2 does not work on the 4000 series routers.  I guess that's progress.

Option 1. Use a physical cable to connect one of the router's interfaces to one of the etherswitches interfaces and treat it just like the etherswitch is a seperate physical switch.  I'm sure there is a use case for that but I'll not cover that here.

Option 2. Use the "service instance" feature on the router's internal interface to bind it to a new "BDI" virtual interface on the router.  This is what we'll do.

On our router ethernet-internal 1/0/0 maps to Gi0/18 on the etherswitch, all internal to the box.  The router will be10.0.0.1 and the switch will be 10.0.0.2.

Router:
interface Ethernet-Internal 1/0/0
service instance 1 ethernet
encapsulation dot1q 50
rewrite ingress tag pop 1

interface BDI 1
mtu 9216
ip address 10.0.0.1 255.255.255.0

Switch:

interface Gi0/18

switchport trunk vlan allowed 50
switchport mode trunk

vlan 50
name Egress vlan

interface vlan 50
ip address 10.0.0.2 255.255.255.0

ip route 0.0.0.0 0.0.0.0 10.0.0.1

Then there are a million ways to design and configure the switch as a normal 3560X switch but that's beyond the scope of my question.

[dannytjoe]

Hi Tod,

Thank you for posting this. I am surprised that Cisco provides minimum configuration documentationfor this setup. Maybe I'm not looking at the right places.

Anyhow, I have the Cisco SM-X-ES3-24-P with the ISR 4331 router and according to the documentation, the interface mapping is ethernet-internal x/0/0 (router) to Gig 0/26 (ether switch).

After following your config, my interface BDI 1 stayed in down status. I have tried 'no shutdown' on the BDI interface, ethernet-internal x/0/0 and even ethernet-internal x/0/0/1. Nothing will turn on the BDI interface. 

On the ether switch side, i can see int G0/26 is connected up and in trunk mode. 

What am I missing here?  Thank you again for any help.

[Jamiegprice15]

The switches I received had bad code, I had to upgrade them to arp correctly. Check to see if they are arping correctly. Tod's information helped me, but here's a config that works (if your switch module works) that will help to get you going. If you found this helpful, please rate high.

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.08.07 13:24:09 =~=~=~=~=~=~=~=~=~=~=~=
sho run
Building configuration...

 
Current configuration : 5542 bytes
!
! Last configuration change at 17:23:15 MST Thu Aug 6 2015 by jamie
!
version 15.5
service timestamps debug datetime localtime show-timezone
service timestamps log datetime localtime
service password-encryption
no platform punt-keepalive disable-kernel-core
platform trace runtime slot F0 bay 0 process forwarding-manager module interfaces level info
!
hostname 
!
boot-start-marker
boot system flash bootflash:isr4300-universalk9.03.16.00.S.155-3.S-ext.SPA.bin
boot-end-marker
!
!
vrf definition Mgmt-intf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
logging buffered 10000
!
aaa new-model
!
!
no aaa authentication login default group tacacs+ local
no aaa authentication login console local
no aaa authentication enable default group tacacs+ none
no aaa authorization config-commands
no aaa authorization exec default group tacacs+ none
no aaa authorization commands 15 default group tacacs+ none
no aaa accounting exec default start-stop group tacacs+
no aaa accounting commands 15 default start-stop group tacacs+
!
!
!
!
!
aaa session-id common
clock timezone MST -7 0
clock summer-time MST recurring
!
!
!
!
!
!
!
!
!
!
!

no ip domain lookup
ip domain name

!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
voice-card 0/1
 no watchdog
!
voice-card 0/4
 no watchdog
!
license udi pid ISR4331/K9 sn FDO192010ZA
archive
 log config
  logging enable
  hidekeys
!
!
spanning-tree extend system-id
spanning-tree uplinkfast
!
username TRON password 7 070C285F4D06

!
redundancy
 mode none
!
!
!
!
!
vlan internal allocation policy ascending
!
!
class-map match-any cmVOICETRAFFIC
 match access-group name aclVOIP
!
policy-map pmPRIOVOIPVIDEO
 class cmVOICETRAFFIC
  set ip dscp ef
  priority 5000
policy-map pmShape-50mb-parent
 class class-default
  shape average 50000000  
   service-policy pmPRIOVOIPVIDEO
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0/0
 bandwidth 50000
 no ip address
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/0/0.724
 encapsulation dot1Q 724
 ip address 172.19.254.74 255.255.255.252
 no cdp enable
 service-policy output pmShape-50mb-parent
!
interface GigabitEthernet0/0/1
 description up only for troubleshoot - no ip to follow.
 ip address 172.19.12.10 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0/0/2
 no ip address
 negotiation auto
!
interface Service-Engine0/1/0
!
interface Service-Engine0/4/0
!
interface Ethernet-Internal1/0/0
 shutdown
 no negotiation auto
 no mop enabled
 no mop sysid
 service instance 23 ethernet
  encapsulation dot1q 23
  rewrite ingress tag pop 1 symmetric
  bridge-domain 23
 !
 service instance 123 ethernet
  encapsulation dot1q 123
  rewrite ingress tag pop 1 symmetric
  bridge-domain 123
 !
!
interface Ethernet-Internal1/0/1
 shutdown
 no negotiation auto
 switchport mode trunk
 no mop enabled
 no mop sysid
!
interface GigabitEthernet0
 vrf forwarding Mgmt-intf
 no ip address
 negotiation auto
!
interface Vlan1
 no ip address
 shutdown
!
interface BDI1
 no ip address
 shutdown
!
interface BDI2
 no ip address
 shutdown
!
interface BDI23
 mtu 9216
 ip address 172.19.23.1 255.255.255.0
 bfd interval 100 min_rx 100 multiplier 3
!
interface BDI123
 mtu 9216
 ip address 172.19.123.1 255.255.255.0
 bfd interval 100 min_rx 100 multiplier 3
!
router ospf 100

ip forward-protocol nd
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 172.19.254.73
ip ssh version 2
!
!
!

!
!
!
control-plane
!
!
voice-port 0/1/0
 signal groundStart
 ring frequency 20
 description Fax Machine XXXX
!
voice-port 0/1/1
!
voice-port 0/1/2
!
voice-port 0/1/3
!
voice-port 0/1/4
!
voice-port 0/1/5
 !
 !
 !
 !
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
!
!
!
banner login ^CC
****************WARNING***************************

****************WARNING***************************
^C
!
line con 0
 logging synchronous
 stopbits 1
line aux 0
 stopbits 1
line vty 0
 exec-timeout 0 0
 no activation-character
 logging synchronous
 no exec
 transport preferred none
 transport input ssh
 stopbits 1
line vty 1 4
 exec-timeout 0 0
 logging synchronous
 transport input ssh
line vty 5 15
 exec-timeout 0 0
 logging synchronous
 transport input ssh
!
ntp server 172.18.50.51 prefer
ntp server 172.17.50.11
!
end

 #
 #hw w session 1/0
Establishing session connect to subslot 1/0
To exit, type ^a^q

picocom v1.4

port is        : /dev/ttyDASH0
flowcontrol    : none
baudrate is    : 9600
parity is      : none
databits are   : 8
escape is      : C-a
noinit is      : no
noreset is     : no
nolock is      : yes
send_cmd is    : ascii_xfr -s -v -l10
receive_cmd is : rz -vv

Terminal ready

.(
 -SW>
 -SW>ena

 -SW#term len 0
 -SW#sho run
Building configuration...

Current configuration : 15353 bytes
!
! Last configuration change at 17:01:03 MST Sun Jan 1 2006
!
version 15.2
no service pad
service timestamps debug datetime localtime show-timezone
service timestamps log datetime localtime
service password-encryption
!
hostname  -SW
!
boot-start-marker
boot-end-marker
!
!
aaa new-model
!
!
!
!
!
!
!
!
aaa session-id common
clock timezone MST -7 0
clock summer-time MST recurring
system mtu routing 1500
!
!
vtp domain
vtp mode transparent
!
!
!
!
!
!
mls qos map policed-dscp  24 26 46 to 0
mls qos map cos-dscp 0 8 16 24 32 46 48 56
mls qos srr-queue input bandwidth 90 10
mls qos srr-queue input threshold 1 8 16
mls qos srr-queue input threshold 2 34 66
mls qos srr-queue input buffers 67 33
mls qos srr-queue input cos-map queue 1 threshold 2 1
mls qos srr-queue input cos-map queue 1 threshold 3 0
mls qos srr-queue input cos-map queue 2 threshold 1 2
mls qos srr-queue input cos-map queue 2 threshold 2 4 6 7
mls qos srr-queue input cos-map queue 2 threshold 3 3 5
mls qos srr-queue input dscp-map queue 1 threshold 2 9 10 11 12 13 14 15
mls qos srr-queue input dscp-map queue 1 threshold 3 0 1 2 3 4 5 6 7
mls qos srr-queue input dscp-map queue 1 threshold 3 32
mls qos srr-queue input dscp-map queue 2 threshold 1 16 17 18 19 20 21 22 23
mls qos srr-queue input dscp-map queue 2 threshold 2 33 34 35 36 37 38 39 48
mls qos srr-queue input dscp-map queue 2 threshold 2 49 50 51 52 53 54 55 56
mls qos srr-queue input dscp-map queue 2 threshold 2 57 58 59 60 61 62 63
mls qos srr-queue input dscp-map queue 2 threshold 3 24 25 26 27 28 29 30 31
mls qos srr-queue input dscp-map queue 2 threshold 3 40 41 42 43 44 45 46 47
mls qos srr-queue output cos-map queue 1 threshold 3 5
mls qos srr-queue output cos-map queue 2 threshold 3 3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3 2 4
mls qos srr-queue output cos-map queue 4 threshold 2 1
mls qos srr-queue output cos-map queue 4 threshold 3 0
mls qos srr-queue output dscp-map queue 1 threshold 3 40 41 42 43 44 45 46 47
mls qos srr-queue output dscp-map queue 2 threshold 3 24 25 26 27 28 29 30 31
mls qos srr-queue output dscp-map queue 2 threshold 3 48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3 56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3 16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 3 threshold 3 32 33 34 35 36 37 38 39
mls qos srr-queue output dscp-map queue 4 threshold 1 8
mls qos srr-queue output dscp-map queue 4 threshold 2 9 10 11 12 13 14 15
mls qos srr-queue output dscp-map queue 4 threshold 3 0 1 2 3 4 5 6 7
mls qos queue-set output 1 threshold 1 138 138 92 138
mls qos queue-set output 1 threshold 2 138 138 92 400
mls qos queue-set output 1 threshold 3 36 77 100 318
mls qos queue-set output 1 threshold 4 20 50 67 400
mls qos queue-set output 2 threshold 1 149 149 100 149
mls qos queue-set output 2 threshold 2 118 118 100 235
mls qos queue-set output 2 threshold 3 41 68 100 272
mls qos queue-set output 2 threshold 4 42 72 100 242
mls qos queue-set output 1 buffers 10 10 26 54
mls qos queue-set output 2 buffers 16 6 17 61
mls qos
!
crypto pki trustpoint TP-self-signed-2664206976
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-2664206976
 revocation-check none
 rsakeypair TP-self-signed-2664206976
!
!
crypto pki certificate chain TP-self-signed-2664206976
 certificate self-signed 01
  3082022B 30820194 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
  31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
  69666963 6174652D 32363634 32303639 3736301E 170D3131 30333330 30313238
  34365A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
  4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D32 36363432
  30363937 3630819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
  8100A9A2 9BFBFD8A 03CC45DB 3B35D57B 4A2462D1 4CB9849A 47343CD1 A4AA40CA
  928E4A3D 81BE70F1 39556075 E9401C02 20E85CC4 5B00980E A739B9C4 902D3FA6
  EFE106FA 85BA20E1 2BFBA2E7 94936644 93ADEB06 E2481231 7FEE1565 7BCDD8EC
  0FCBF6DB 7A9079CA 694AA2A4 D000C1FE 650B7062 4057EB4F 4C3AB56A 227560B5
  1FF50203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF 301F0603
  551D2304 18301680 145AF498 02865168 DA60128B DFC78159 2E4E9881 65301D06
  03551D0E 04160414 5AF49802 865168DA 60128BDF C781592E 4E988165 300D0609
  2A864886 F70D0101 05050003 8181008C 091F7811 2518A88C D5D3DA1A B5DECA89
  5CDA765D 3E13038D 00402E92 8DEBFCE1 F9E89419 70B95DD3 89B87012 EFDF19A9
  97669AF3 37D18C30 9032767D E2F626F0 5FAD3E74 A2F7B7A8 FBF30338 4BEDF68C
  39102EA7 B7AD2328 C46F4530 C46DB732 BBEF40A2 AAD924A9 154E72E5 1DE08DCD
  0C649189 D74CC16E 8D9A473E F162EF
  quit
spanning-tree mode pvst
spanning-tree extend system-id
errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause sfp-config-mismatch
errdisable recovery cause gbic-invalid
errdisable recovery cause l2ptguard
errdisable recovery cause psecure-violation
errdisable recovery cause port-mode-failure
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause mac-limit
errdisable recovery cause vmps
errdisable recovery cause storm-control
errdisable recovery cause inline-power
errdisable recovery cause arp-inspection
errdisable recovery cause loopback
errdisable recovery cause small-frame
!
vlan internal allocation policy ascending
!
vlan 23
 name Egress_VLAN
!
vlan 123
 name VoIP
!
!
class-map match-all AUTOQOS_VOIP_DATA_CLASS
 match ip dscp ef
class-map match-all AUTOQOS_DEFAULT_CLASS
 match access-group name AUTOQOS-ACL-DEFAULT
class-map match-all AutoQoS-VoIP-RTP-Trust
 match ip dscp ef
class-map match-all AUTOQOS_VOIP_SIGNAL_CLASS
 match ip dscp cs3
class-map match-all AutoQoS-VoIP-Control-Trust
 match ip dscp cs3  af31
!
policy-map AUTOQOS-SRND4-CISCOPHONE-POLICY
 class AUTOQOS_VOIP_DATA_CLASS
  set dscp ef
  police 128000 8000 exceed-action policed-dscp-transmit
 class AUTOQOS_VOIP_SIGNAL_CLASS
  set dscp cs3
  police 32000 8000 exceed-action policed-dscp-transmit
 class AUTOQOS_DEFAULT_CLASS
  set dscp default
  police 10000000 8000 exceed-action policed-dscp-transmit
policy-map AutoQoS-Police-CiscoPhone
 class AutoQoS-VoIP-RTP-Trust
  set dscp ef
  police 320000 8000 exceed-action policed-dscp-transmit
 class AutoQoS-VoIP-Control-Trust
  set dscp cs3
  police 32000 8000 exceed-action policed-dscp-transmit
!
!
!
!
!
!
interface GigabitEthernet0/1
 switchport access vlan 23
 switchport voice vlan 123
 srr-queue bandwidth share 10 10 60 20
 queue-set 2
 priority-queue out
 mls qos trust device cisco-phone
 mls qos trust cos
 auto qos voip cisco-phone
 spanning-tree portfast
 service-policy input AutoQoS-Police-CiscoPhone
!
interface GigabitEthernet0/2
 switchport access vlan 23
 switchport voice vlan 123
 srr-queue bandwidth share 10 10 60 20
 queue-set 2
 priority-queue out
 mls qos trust device cisco-phone
 mls qos trust cos
 auto qos voip cisco-phone
 spanning-tree portfast
 service-policy input AutoQoS-Police-CiscoPhone
!
...
!
interface GigabitEthernet0/24
 switchport access vlan 23
 switchport voice vlan 123
 srr-queue bandwidth share 10 10 60 20
 queue-set 2
 priority-queue out
 mls qos trust device cisco-phone
 mls qos trust cos
 auto qos voip cisco-phone
 spanning-tree portfast
 service-policy input AutoQoS-Police-CiscoPhone
!
interface GigabitEthernet0/25
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport protected
!
interface GigabitEthernet0/26
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport protected
!
interface Vlan1
 no ip address
 shutdown
!
interface Vlan23
 ip address 172.19.23.2 255.255.255.0
!
interface Vlan123
 ip address 172.19.123.2 255.255.255.0
!
ip http server
ip http secure-server
ip route 0.0.0.0 0.0.0.0 172.19.23.1
!
ip access-list extended AUTOQOS-ACL-DEFAULT
 permit ip any any

!
!
!
banner login ^CC
****************WARNING***************************

****************WARNING***************************
^C
!
line con 0
 flowcontrol software
line vty 0
 length 0
 transport input ssh
line vty 1
 no exec
 transport preferred none
 transport input ssh
 stopbits 1
line vty 2 4
 transport input ssh
line vty 5 15
 transport input ssh
!
!
end

 -SW#
 -SW#

[pratikbhai patel]

Hi,

I've implemented BDIs and it seems up but still how can I check its working or not? I want to configure 3 interface  vlans and want to make them work on router 4331?

Any suggesstions?

[Jamiegprice15]

With CDP turned on, can you see your neighbors?

[Tod Larson]

Yes. For our setup, On the 4331 we see the etherswitch as a device on int s1/0/0.

When ssh'ing to the etherswitch itself we see the 4331 as a device on int g0/18.

[Jamiegprice15]

If I understand correctly I think you need to trunk on the switch side and call your service instances on the router side. Per the "s1/0/0." It looks like you aren't using the internal Ethernet shown below. I show this is shutdown below, I would need to no shut (as in prod).

interface Ethernet-Internal1/0/0
 shutdown
 no negotiation auto
 no mop enabled
 no mop sysid
 service instance 23 ethernet
  encapsulation dot1q 23
  rewrite ingress tag pop 1 symmetric
  bridge-domain 23
 !
 service instance 123 ethernet
  encapsulation dot1q 123
  rewrite ingress tag pop 1 symmetric
  bridge-domain 123

[S Anderson]

Does anyone have any more information on how to configure a 4331 with SM-X 18. I have performed the following:

Router:
interface Ethernet-Internal 1/0/0
service instance 1 ethernet
encapsulation dot1q 50
rewrite ingress tag pop 1

interface BDI 1
mtu 9216
ip address 10.0.0.1 255.255.255.0

Switch:

interface Gi0/18

switchport trunk vlan allowed 50
switchport mode trunk

vlan 50
name Egress vlan

interface vlan 50
ip address 10.0.0.2 255.255.255.0

ip route 0.0.0.0 0.0.0.0 10.0.0.1

I can't get the new BDI interface to come up.

[izozulya]

Hello Guys,

Can someone explain if there is way to use second Ethernet-Internal together with the 1st one interface?  I have 16P SM-X  where internal Ge0/18 is connected to Ethernet-Internal2/0/0 and Ge 0/17 to Ethernet-Internal2/0/1. Port-channel config seems to be not an option :(... are there any other options?

Regards

Igor

[charleswatkins]

You need to map the service instance to the bdi interface.  Add the command bridge-domain 1 to the service instance configuration.

[charleswatkins]

You need to map the service instance to the bdi interface.  Add the command bridge-domain 1 to the service instance configuration.

[sarterburn1]

I did 1 step more and my BDI's came up on the router side. I have not gotten them to work on the switch side yet, but I can ping through the router, switch, and to a connected device.

interface Ethernet-Internal1/0/0
no negotiation auto
service instance 791 ethernet
encapsulation dot1q 791
rewrite ingress tag pop 1 symmetric
bridge-domain 791
!
service instance 891 ethernet
encapsulation dot1q 891
rewrite ingress tag pop 1 symmetric
bridge-domain 891
!
service instance 910 ethernet
encapsulation dot1q 910
rewrite ingress tag pop 1 symmetric
bridge-domain 910
!
service instance 911 ethernet
encapsulation dot1q 911
rewrite ingress tag pop 1 symmetric
bridge-domain 911

Only difference from yours to mine was the "bridge-domain xxx" command.

[pratikbhai patel]

Hi Tod,

I've configured BVI on router and Vlans on the switch but still it not working yet. Do I need to give default routes per BVIs?

[jegede1@yahoo.com]

Hi Todd, How did you figure this out : "On our router ethernet-internal 1/0/0 maps to Gi0/18 on the etherswitch". How can I know which port on the switch module SM-X-ES3-24 maps to the Ethernet_Internal port?

Please advise.

Thank you.

Wale (jegede1@yahoo.com)