This how-to is a step-by-step quick start guide to help you in setting up IOx on IE3400 industrial ethernet switches and running a simple docker container via CLI.
With the introduction of IOS-XE 17.2.1 all variants of IE3400 and IE3400H support application hosting with IOx and running of vanilla docker containers. Much in the same way as Catalyst 9300 does.
No additional license is needed, as IOx on IE3400 is supported in the Network Essentials feature set.
IE3400(H) provides the following Hardware Resources:
IE3400(H) HW Spec
Available for IOx
ARM Cortex-A53, 64bit, 4cores @ 1.2GHz (aarch64)
2cores @ 1.2GHz (1400cpu units*)
*Artificial benchmark number for comparison of different IOx platforms
**SD-Card is an optional component but mandatory for IOx
For network connectivity to IOx applications, IE3400 provides an internal "virtual" switch interface called "AppGigabitEthernet1/1" which can be configured mostly like a regular switch port. This interface connects to on internal Linux bridge to which the different IOx apps are connected. Apps can have one or more interfaces and be placed in any VLAN.
Only L2 mode ("switchport") is supported for AppGig1/1 currently
AppGig1/1 must be in mode trunk.
IE3400 or IE3400H switch
4GB SD-Card (SD-IE-4GB=)
IOS-XE version 17.2.1 installed on flash: (not on the SD-CARD!), see release notes for installation instructions
Connection to the CLI, either via console or network (i.e. telnet, ssh, ...)
A docker container image with binaries compiled for aarch64 (a sample container image is attached to this document)
Step-by-step instructions to setup IOx on IE3400
1. Formatting the SD-CARD
4GB SD-CARD needs to be inserted and mustn’t be removed once IOxisenabled
SD-CARD needs to be re-formatted with ext4 filesystem
Caveat: IE3400 can’t boot from ext4 formatted SD-CARD
!SD-CARD needs to be formatted with ext4, else iox will fail with obscure error message
!IMPORTANT: Make sure unit boots from flash: and has its config also stored on flash: IE3400#show boot
Current Boot Variables: BOOT variable = flash:ie3x00-universalk9.17.02.01.SPA.bin; Boot Variables on next reload: BOOT variable = flash:ie3x00-universalk9.17.02.01.SPA.bin; Config file = flash:/nvram_config ENABLE_FLASH_PRIMARY_BOOT = no MANUAL_BOOT variable = no ENABLE_BREAK variable = yes IE3400#
!IMPORTANT: All data on sdflash: will be lost after formatting. IE3400#format sdflash: ext4 Format operation may take a while. Continue? [confirm] Format operation will destroy all data in "sdflash:". Continue? [confirm] format completed with no errors
Format of sdflash: complete IE3400#
2. Starting IOx
IOx is by default disabled and needs to be enabled first
Wait until IOx has fully started and then check that CAF, IOxman, Libvirtd and Dockerd are in “Running” state
(Please refer to the IOx configuration guide for complete coverage of all commands and possible limitations)
! Enabling IOX is a simple one line command. Be patient and wait until it is ready IE3400#term mon IE3400#conf t IE3400(config)#iox Warning: Do not remove SD flash card when IOx is enabled or errors on SD device could occur.
*Apr 9 00:50:38.145: %UICFGEXP-6-SERVER_NOTIFIED_START: R0/0: psd: Server iox has been notified to start
*Apr 9 00:51:44.766: %IM-6-IOX_ENABLEMENT: R0/0: ioxman: IOX is ready.
! Verify via show command that the services are running IE3400(config)#end IE3400#show iox
IOx Infrastructure Summary: --------------------------- IOx service (CAF) 184.108.40.206 : Running IOx service (HA) : Not Supported IOx service (IOxman) : Running IOx service (Sec storage) : Not Supported Libvirtd 1.3.4 : Running Dockerd 18.03.0 : Running
3. Configuring Interface AppGigabitEthernet
Interface AppGigabitEthernet needs to be configured as trunk
Allow VLANs needed for IOx Apps (optional, but best practice)
“mode access” is not supported, all frames leaving the interface will always be dot1q tagged.
IE3400#conf t Enter configuration commands, one per line. End with CNTL/Z. IE3400(config)#interface AppGigabitEthernet1/1 IE3400(config-if)#switchport mode trunk IE3400(config-if)#switchport trunk allowed vlan 1-4 IE3400(config-if)#end IE3400#
4. App Network & Resources
Before an IOx app can be installed, the “appid” has to be configured
"appid" is a user chosen string, like “my_app” or “iperf3”, to identify the application
Also, VLAN(s) and IP address(es) need to be configured
One App can be configured with several interfaces (e.g. one formanagement and one for monitoring)
Network interfaces are represented as eth0, eth1… inside the app
Configurable app resources include CPU, Memory, VCPU(s) andpersistent disk
IE3400#conf t Enter configuration commands, one per line. End with CNTL/Z. IE3400(config)#app-hosting appid iperf3 IE3400(config-app-hosting)#app-vnicAppGigabitEthernet trunk IE3400(config-config-app-hosting-trunk)#vlan 1 guest-interface 0 !place eth0 in Vlan 1 IE3400(config-config-app-hosting-vlan-access-ip)#guest-ipaddress 100.64.1.11 netmask 255.255.255.0 IE3400(config-config-app-hosting-vlan-access-ip)#exit IE3400(config-config-app-hosting-trunk)#vlan 3 guest-interface 1 !place eth1 in Vlan 2 IE3400(config-config-app-hosting-vlan-access-ip)#guest-ipaddress 192.168.3.2 netmask 255.255.255.0 IE3400(config-config-app-hosting-vlan-access-ip)#exit IE3400(config-config-app-hosting-trunk)#exit IE3400(config-app-hosting)#app-default-gateway 100.64.1.1 guest-interface 0 IE3400(config-app-hosting)#app-resource profile custom IE3400(config-app-resource-profile-custom)#cpu 1400 IE3400(config-app-resource-profile-custom)#memory 256 IE3400(config-app-resource-profile-custom)#vcpu 2 IE3400(config-app-resource-profile-custom)#end IE3400#
5. Install, Activate & Start App
Copy the app package to either flash: orsdflash:
Next, the app needs to be installed
After successful installation, the app needs to be activated
Finally, the app can be started
If changes are made to the app’s (network) configuration, it needs to bestopped, deactivated and again activated and started
Sample app from this example is attached to this document
IE3400#term mon IE3400#copy http://192.168.3.3/tftp/iperf3_eft_dockerimage_aarch64.tar flash: Destination filename [iperf3_eft_dockerimage_aarch64.tar]? Accessing http://192.168.3.3/tftp/iperf3_eft_dockerimage_aarch64.tar... Loading http://192.168.3.3/tftp/iperf3_eft_dockerimage_aarch64.tar !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 7281152 bytes copied in 3.901 secs (1866195 bytes/sec) IE3400#app-hosting install appid iperf3 package flash:iperf3_eft_dockerimage_aarch64.tar Installing package 'flash:iperf3_eft_dockerimage_aarch64.tar' for 'iperf3'. Use 'show app-hosting list' for progress.
IE3400# *Apr 9 05:28:59.605: %IM-6-INSTALL_MSG: R0/0: ioxman: app-hosting: Install succeeded: iperf3 installed successfully Current state is DEPLOYED
IE3400#app-hosting activate appid iperf3 iperf3 activated successfully Current state is: ACTIVATED
IE3400# *Apr 9 05:30:32.088: %IM-6-ACTIVATE_MSG: R0/0: ioxman: app-hosting: Activate succeeded: iperf3 activated successfully Current state is in ACTIVATED IE3400#app-hosting start appid iperf3 iperf3 started successfully Current state is: RUNNING
Congratulations, you are running a docker container on your IE3400 switch!