Introduction Overview of CGv6 on VSM Software Different Installation / Upgrade Scenarios Information about Linux CGv6 OVA Package for each IOS-XR Release Installation / Upgrade of Different Images Installation of IOS-XR Images Upgrade FPGA Images Installation and Activation of Application (Linux) Image Trouble-shooting Issues related to Installation / Upgrade Tips / Things to Remember for Installation / Upgrade How do I know CGv6 Application S/w Version on VSM ? How can I know CGv6 Application Software Version from the CGv6 OVA Package without installing it ? How do I find out MD5 Checksum of a Linux CGv6 OVA Package ? Is there any command to check the Status of Installation / Activation while it is going on ? What files are included within the OVA package ? What are the different states / status a Virtual Service can have ? Related Information   Introduction This document talks about how to troubleshoot issues related to installation and upgrade of CGv6 software on VSM service card. It contains installation / upgrade steps to create the context and not to replace the official guide. For official CGv6 on VSM installation guide, please select "Cisco ASR 9000 Series Aggregation Router CGv6 Configuration Guide" for the specific IOS-XR release and use the appropriate section from VSM specific guide. Also, it does not cover VSM hardware installation.   Overview of CGv6 on VSM Software   ASR9K chassis can have different types of cards, like - Route Processors (RP) or Route Switch Processor (RSP) Line Card (LC) Service Card (SC) like, Virtualized Services Module (VSM)   Following diagram shows what type of Software runs on which cards.     As shown above, VSM card runs 2 types of Software - IOS-XR and Linux.   CGv6 software functionality is distrubuted across all types of cards (RP, VSM as well as LC) as well as both types of Software (IOS-XR and Linux).   Following images are mandatory to install on ASR9K chassis to avail CGv6 functionality.   IOS-XR Images asr9k-mini-px.vm or asr9k-mini-px.pie (Base IOS-XR Image) asr9k-services-infra.pie (VSM Services Infra Pie) asr9k-services-px.pie (CGv6 Services Pie) asr9k-fpd-px.pie (FPGA Image IOS-XR Pie) Linux Image asr9k-vsm-cgv6-<version>.ova (Linux Open Virtual Appliance or OVA package, a tar file)   NOTES: - For IOS-XR, bug fixes are available as SMU (Software Maintenance Update) as well. - For Linux images, there is NO SMU available. A new version of OVA package needs to be installed. - Size of Linux OVA package is approximately 250+ MB. Back to top 1. Different Installation / Upgrade Scenarios Following table captures different Installation / Upgrade Scenarios and which software needs to be upgraded.   Scenario Software to be Installed / Upgraded (in what sequence) Additional Comments Install (fresh) CGv6 software on VSM on existing IOS-XR release image   Assumptions: - Services pies are not yet installed - CGv6 OVA package is not yet installed Steps:   - Install services-infra pie - Install services pie (CGv6) - Upgrade FPGA images (if needed) - Install CGv6 OVA Package It is recommended to install IOS-XR binaries first and then Linux CGv6 OVA package. Multiple IOS-XR pies can be installed using single command as well. Upgrade CGv6 software on VSM to a new IOS-XR release   Assumptions: - Services pies are already installed (with existing IOS-XR s/w) - Old Linux OVA Package is installed Steps:   - Upgrade IOS-XR images - Upgrade services-infra pie (with new IOS-XR version) - Upgrade services (CGv6) pie (with new IOS-XR version) - Upgrade FPGA images (with new IOS-XR version), if needed. - de-activate, uninstall old Linux CGv6 OVA package. - Upgrade to new Linux CGv6 OVA package. It is recommended to install IOS-XR binaries first and then Linux CGv6 OVA package. Multiple IOS-XR pies can be upgraded using single command as well. IOS-XR bug fix / enhancement feature - Install IOS-XR SMU Some bug fix may need XR SMU as well as Linux CGv6 OVA package installation. In that case, it is recommended to install IOS-XR SMUs first and then new CGv6 OVA package. Linux bug fix / enhancement feature - Upgrade to New Linux CGv6 OVA package Some bug fix may need XR SMU as well as Linux CGv6 OVA package installation. In that case, it is recommended to install IOS-XR SMUs first and then new CGv6 OVA package.     Back to top 2. Information about Linux CGv6 OVA Package for each IOS-XR Release CGv6 Application Image is delivered via Linux CGv6 OVA Package binaries.   Following table lists Linux CGv6 OVA Package (latest) for each IOS-XR Release. Appropriate Linux CGv6 OVA Package can be downloaded from the "download link" shown in the table. Download Linux CGv6 OVA Package needs to be saved on RP disk (prior to installation).   Please use the compatible and preferably latest Linux CGv6 OVA Package binary corresponding to the IOS-XR image. Otherwise, CGv6 Application software may not work or work as expected.   IOS-XR Version Linux CGv6 OVA Package Version (with Download Link) OVA Filename, Date of Posting, MD5 Checksum Additional Comments 5.1.1 5.1.1.00 asr9k-vsm-cgv6-5.1.1.00.ova - Posted on: Feb 5, 2014 - md5: 6735e598f29984aaa48f173ad16d8309     NOTE: While downloading Linux CGv6 OVA Package, please make sure - you are downloading the right binary from right location - MD5 Checksum and (approximate) size matches after download     Installation / Upgrade of Different Images   Back to top 3. Installation of IOS-XR Images IOS-XR Images can be downloaded from Download Software Center.   Please follow the standard procedure to install / upgrade IOS-XR images.   As an example, following steps can be followed to install CGv6 Services pie. RP/0/RSP0/CPU0#admin (admin)#install add t ftp:// <tftp_addr>/<image_location>/asr9k-services-px.pie         synchronous activate . . . . . . . . . . . (admin)#exit RP/0/RSP0/CPU0# NOTES: - "install add ..." command needs to be executed from "admin" mode - "add" and "activate" commands are to be preferably given together - Pie can first be copied to RP disk and then installed from RP disk as well - Unlike ISM, "hw-module service cgn location <>" CLI is NOT needed for VSM Back to top   4. Upgrade FPGA Images An important aspect is to upgrade the FPGA images. At times, FPGA images may get updated because of some bug fix. If the FPGA is not updated, certain functionality may not work. Hence, please upgrade FPGA images as well, if needed.   As a first step, please capture output of  "show hw-module fpd location <VSM_location>" command (in "admin" mode) for all VSM cards. The output will show like the following.   RP/0/RSP0/CPU0:CRANE#admin show hw-module fpd location 0/0/CPU0 ===================================== =========================================== Existing Field Programmable Devices =========================================== HW Current SW Upg/ Location Card Type Version Type Subtype Inst Version Dng? ============ ======================== ======= ==== ======= ==== =========== ==== 0/0/CPU0 A9K-VSM-500 1.0 lc cbc 0 33.02 No lc fpga2 0 1.25 No lc bios 0 3.00 Yes lc ibmc 0 5.07 No -------------------------------------------------------------------------------- NOTES: 1.  One or more FPD needs an upgrade or a downgrade.  This can be accomplished     using the "admin> upgrade hw-module fpd <fpd> location <loc>" CLI. RP/0/RSP0/CPU0:CRANE#   "S/w Upg / Dng" column will indicate if any FPGA would need upgrade ("Yes") or not ("No").   For those FPGAs, issue "upgrade hw-module fpd <Subtype> location <VSM_location>" command to upgrade the image(s).   Once the FPGAs are upgraded, please ensure that the VSM card is reloaded and the FPGA version shows correctly (via "show hw-module fpd location <VSM_location>" command, in "admin" mode).   NOTE: You can enable auto FPD upgrade by configuring "fpd auto-upgrade" command in admin config mode (available from release 4.2.3 onwards).   Back to top 5. Installation and Activation of Application (Linux) Image Before executing any VM (Virtual Service) related commands, virtual service needs to be enabled on ASR9K router, using the "virtual-service enable" CLI.   RP/ 0 /RSP0/CPU0:ios(config)#virtual-service enable RP/ 0 /RSP0/CPU0:ios(config)#commit   NOTE: - It needs to be executed only once for ASR9K, NOT for each VSM card.   As mentioned earlier, as a first step, Linux CGv6 OVA Package needs to be copied to RP disk, as shown below.   RP/0/RSP0/CPU0:CRANE#copy t ftp://200.123.114.26/auto/tftp-blr-users/foobar/asr9k-vsm-cgv6.ova disk1:/ Destination filename [/disk1://asr9k-vsm-cgv6.ova]? Copy : Destination exists, overwrite ?[confirm] Accessing t ftp://200.123.114.26/auto/tftp-blr-users/foobar/asr9k-vsm-cgv6.ova CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC 393290312 bytes copied in 816 sec ( 481946)bytes/sec RP/0/RSP0/CPU0:CRANE#   Now that the CGv6 OVA package is copied to RP's disk, it can be installed on multiple VSM service cards on same chassis.   NOTE: - You do NOT need to remove any XR (say, CGv6) configuration before installing the Linux CGv6 OVA Package. Once the installation is over, existing XR configuration will automatically be applied. - When you are upgrading CGv6 OVA Package, you need to first de-activate, uninstall old CGv6 OVA package and then install, activate new CGv6 OVA Package.   Once, the CGv6 OVA is copied into RP disk, "virtual-service install name <service/VM name> package <OVA package name> node <VSM_location>" command can be issued, to install it on a specific VSM card. Logs are shown below.   RP/0/RSP1/CPU0:SERVICE_PE2_1#dir disk0:*ova* Tue Dec 24 15:23:24.360 UTC Directory of disk0: 28770063 -rwx 257996800 Mon Dec 23 20:59:41 2013 vsmcgv6.ova 12101599232 bytes total (9311405056 bytes free) RP/0/RSP1/CPU0:SERVICE_PE2_1#virtual-service install name cgn1 package disk0:/vsmcgv6.ova node 0/1/CPU0 Tue Dec 24 15:23:43.000 UTC RP/0/RSP1/CPU0:SERVICE_PE2_1#RP/0/RSP1/CPU0:Dec 24 15:23:43.018 : service_mgr[1122]: %OS-SMGR-6-VM_INSTALL_STATUS : 'Install' of service VM 'cgn1' is in progress and will continue asynchronously. Please use 'show virtual-service list' for viewing the status RP/0/RSP1/CPU0:SERVICE_PE2_1#sh virtual-service list Tue Dec 24 15:23:49.453 UTC Virtual Service List: Name Status Package Name Node Name ______________________________________________________________________________ cgn1 Installing vsmcgv6.ova 0/1/CPU0 RP/0/RSP1/CPU0:SERVICE_PE2_1# RP/0/RSP1/CPU0:Dec 24 15:24:35.988 : licmgr[306]: %LICENSE-LICMGR-7-CLIENT_NORESPONSE : License client "pkg/bin/service_mgr" failed to respond RP/0/RSP1/CPU0:Dec 24 15:24:53.636 : service_mgr[1122]: %OS-LIB_VMAN-5-UNSIGNED_PACKAGE : Unsigned software package vsmcgv6.ova loaded for service container cgn1. RP/0/RSP1/CPU0:SERVICE_PE2_1#sh virtual-service list Tue Dec 24 15:25:07.671 UTC Virtual Service List: Name Status Package Name Node Name ______________________________________________________________________________ cgn1 Installing vsmcgv6.ova 0/1/CPU0 RP/0/RSP1/CPU0:SERVICE_PE2_1# RP/0/RSP1/CPU0:Dec 24 15:30:26.050 : licmgr[306]: %LICENSE-LICMGR-7-CLIENT_NORESPONSE : License client "pkg/bin/service_mgr" failed to respond RP/0/RSP1/CPU0:Dec 24 15:30:46.520 : service_mgr[1122]: %OS-SMGR-6-VM_INSTALL_STATUS : 'Install' of service VM 'cgn1' completed successfully RP/0/RSP1/CPU0:SERVICE_PE2_1# RP/0/RSP1/CPU0:SERVICE_PE2_1#sh virtual-service list Tue Dec 24 15:34:15.736 UTC Virtual Service List: Name Status Package Name Node Name ______________________________________________________________________________ cgn1 Installed vsmcgv6.ova 0/1/CPU0 RP/0/RSP1/CPU0:SERVICE_PE2_1#   NOTE: - It may take approximately 7-8 minutes for the installation to be completed. So, please be patient. Once, the "Status" is shown as "Installed", installation process is completed. Now, it is time to activate the OVA package.   NOTE: - Activation is done in "config" mode. - As part of Activation, XR interfaces associated with the CGv6 VM MUST be specified.   Activation command is shown below. RP/0/RSP1/CPU0:SERVICE_PE2_1#conf Tue Dec 24 15:34:20.385 UTC RP/0/RSP1/CPU0:SERVICE_PE2_1(config)#virtual-service cgn1 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/0 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/1 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/2 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/3 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/4 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/5 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/6 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/7 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/8 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/9 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/10 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# vnic interface TenGigE0/1/1/11 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# activate RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)#commit Tue Dec 24 15:34:38.080 UTC RP/0/RSP1/CPU0:Dec 24 15:34:38.236 : service_mgr[1122]: %OS-SMGR-3-VM_LICENSING_FAIL : VM (cgn1) Licensing Failed with error: ('License Manager' detected the 'warning' condition 'Licensed feature does not exist') LC/0/1/CPU0:Dec 24 15:35:29.096 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Client MSG process: Message from Server of type REGISTER LC/0/1/CPU0:Dec 24 15:35:29.096 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Server MSG Version 1.0, Client API version 1.0 LC/0/1/CPU0:Dec 24 15:35:29.096 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Notify register response for tid 1: descr=VM activated event=1 vm=cgn1 RP/0/RSP1/CPU0:Dec 24 15:35:29.265 : config[65881]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'lab'. Use 'show configuration commit changes 1000001545' to view the changes. RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)# LC/0/1/CPU0:Dec 24 15:35:30.048 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Client MSG process: Message from Server of type REGISTER LC/0/1/CPU0:Dec 24 15:35:30.048 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Server MSG Version 1.0, Client API version 1.0 LC/0/1/CPU0:Dec 24 15:35:30.048 : forge_cgn_ha_agent[190]: %LIB-VMANCLNT-7-VMAN_DEBUG : Notify register response for tid 1: descr=VM up event=4 vm=cgn1 RP/0/RSP1/CPU0:SERVICE_PE2_1(config-virt-service)#end RP/0/RSP1/CPU0:Dec 24 15:35:47.349 : config[65881]: %MGBL-SYS-5-CONFIG_I : Configured from console by lab RP/0/RSP1/CPU0:SERVICE_PE2_1#sh virtual-service list Tue Dec 24 15:35:50.161 UTC Virtual Service List: Name Status Package Name Node Name ______________________________________________________________________________ cgn1 Activated vsmcgv6.ova 0/1/CPU0 RP/0/RSP1/CPU0:SERVICE_PE2_1#   NOTE: - It may take approximately 1 minute for the activation to be completed. So, please be patient. Once, the "Status" is shown as "Activated", activation process is completed. Now, you can proceed with CGv6 configuration (if not done earlier).     Trouble-shooting Issues related to Installation / Upgrade   Back to top 6. Tips / Things to Remember for Installation / Upgrade   Following are different things to remember during Installation / Upgrade:   Once CGv6 Services Pie is installed, IOS-XR software on all VSM cards is upgraded. "virtual-service enable" CLI MUST be executed once before configuring any CGv6 VM. "hw-module service cgn location <>" CLI is NOT needed for VSM. To install/upgrade Linux CGv6 OVA Package, "virtual-service ..." CLIs MUST be executed once for each VSM location. Multiple CGv6 VMs / services cannot be installed / activated on same VSM service card. Make sure that you are using right (matching with IOS-XR image) and uncorrupted (valid MD5 checksum and size) Linux CGv6 OVA Package. Installation and activation of Linux CGv6 OVA Package takes approximately 8-10 minutes. So, please be patient. Do NOT forget to upgrade FPGAs (if needed). Software installation / upgrade will cause some traffic outage. Hence, it is better to do it in a Maintenance Window (MW) or off-hours.   Back to top 7. How do I know CGv6 Application S/w Version on VSM ?   Version name will usually be part of the OVA package filename. In case the file is renamed, please issue "show virtual-service detail name <>" IOS-XR CLI command for the specific CGv6 VM / service (as shown below) which will provide the CGv6 Application version information. If there are multiple VSM cards in one ASR9K Chassis, same command needs to be executed for all VSM locations.   "Installed Version:" field (under "Application") will provide the CGv6 Application S/w version on VSM (e.g. - "5.1.1.0" in the following sample output).   RP/0/RSP1/CPU0:SERVICE_PE2_1#show virtual-service detail name cgn1 Tue Dec 24 15:38:20.951 UTC Virtual Service cgn1 Detail State : Activated Node name : 0/1/CPU0 Node status : Install Mgr Ready, SDR Mgr Ready Package information Name : vsmcgv6.ova Path : disk0:/vsmcgv6.ova Application Name : CGv6 Installed version : 5.1.1.0 Description : Carrier Grade IPv6 (CGv6) Signing Key type : Unknown Package Method : SHA1 Licensing Name : Not Available Version : Not Available Activated profile name : None Resource reservation Disk : 10000MB Memory : 32768MB CPU : 75 (system CPU %) VCPU : 60 Attached devices # Type Name Alias 1 Watchdog None None 2 CDROM hdc ide0-1-0 3 HDD hda DD_10GB_UM_local 4 Serial/aux None serial1 5 Serial/shell None serial0 6 NIC net1 net1 7 NIC net1 net1 8 NIC net1 net1 9 NIC net1 net1 10 NIC net1 net1 11 NIC net1 net1 12 NIC net1 net1 13 NIC net1 net1 14 NIC net1 net1 15 NIC net1 net1 16 NIC net1 net1 17 NIC net1 net1 Network interfaces: Name TenGigE0/1/1/0 TenGigE0/1/1/1 TenGigE0/1/1/2 TenGigE0/1/1/3 TenGigE0/1/1/4 TenGigE0/1/1/5 TenGigE0/1/1/6 TenGigE0/1/1/7 TenGigE0/1/1/8 TenGigE0/1/1/9 TenGigE0/1/1/10 TenGigE0/1/1/11 Resource admission (without profile) Disk space : 10000MB Memory : 32768MB CPU : 100% system CPU VCPUs : 60 (sockets:3 cores:10 threads:2) RP/0/RSP1/CPU0:SERVICE_PE2_1#   Back to top 8. How can I know CGv6 Application Software Version from the CGv6 OVA Package without installing it ?   Usually, the CGv6 OVA Package filename will contain the version in its name itself.   Back to top 9. How do I find out MD5 Checksum of a Linux CGv6 OVA Package ?   MD5 (Message Digest 5) Checksum of Linux install-kit (for that matter, any file) can be obtained using "md5sum" utility on Linux and "md5" utility on IOS-XR, as shown below. NOTE: It takes about few seconds to calculate MD5 checksum on Linux, but takes about 2 minutes on IOS-XR (RP).   On Linux Machine: linux_host# md5sum ./asr9k-vsm-cgv6-5.1.1.0.ova 47c05d10745da32c7b4f791adeeef618 ./asr9k-vsm-cgv6-5.1.1.0.ova   On IOS-XR (RP): RP/0/RSP0/CPU0:CRANE#run Fri Nov 29 23:00:24.362 UTC # md5 /disk1:/asr9k-vsm-cgv6-5.1.1.0.ova 47c05d10745da32c7b4f791adeeef618 # exit   Back to top 10. Is there any command to check the Status of Installation / Activation while it is going on ?   Yes, you can use "show virtual-service list <>" CLI to see the status of Installation / Activation from time to time, as shown below.   RP/0/RSP1/CPU0:SERVICE_PE2_1#show virtual-service list Tue Dec 24 15:23:49.453 UTC Virtual Service List: Name Status Package Name Node Name ______________________________________________________________________________ cgn1 Installing vsmcgv6.ova 0/1/CPU0 RP/0/RSP1/CPU0:SERVICE_PE2_1#   Back to top 11. What files are included within the OVA package ?   Following table lists and describes the files located inside the OVA package.   Sl. No. Filename Description / Purpose 1. vsmcgv6_ism-os-5.3-x86_64.iso ISO Image for CGv6 Application Binary 2. vsmcgv6.xml Contains h/w resource information about the CGv6 service VM 3. vsmcgv6.ver Virtual Manager Infra-structure version (set to 1.5 for 5.1.1 release) 4. vsmcgv6.cert X.509 Certificate 5. vsmcgv6.mf Manifest file containing SHA1 message digests 6. vsmcgv6.mf2 Manifest file containing SHA512 message digets 7. vsmcgv6.env OVA Metadata file     Back to top 12. What are the different states / status a Virtual Service can have ?   A Virtual Service or VM can have one of the following states / status.   Virtual Service State / Status Meaning of State / Status Installing Virtual Service or VM is being installed (which includes verification and copying of OVA package into the specified location) Installed Virtual Service or VM is installed (which includes verification and copying of OVA package into the specified location) Install Failed Virtual Service or VM installation has failed as part of reload or recovery action. For initial installation, this state / status is not used. Activating Virtual Service or VM is being activated / instantiated. Activated Virtual Service or VM is activated / instantiated. Once the VM is activated, CGv6 Application processes get started. It takes little more time for the CGv6 Application processes to be up and running and ready to process packets. Deactivating Virtual Service or VM which was activated earlier is being deactivated now (via "no activate" CLI). Deactivated Virtual Service or VM which was activated earlier is deactivated now (via "no activate" CLI).       Related Information   IOS-XR Software Maintenance Updates (SMUs) Guide Download Software Center CGv6 on VSM: Features Supported Across different IOS-XR Release (DOC-38475) CGv6 Configuration Guides for different IOS-XR Releases Cisco ASR9000 Upgrade Procedure for Release 4.3.2     Back to top ... View more

Introduction Introduction to NAT44 Why NAT44 ? NAT44 Overview Basic Terminologies Standard Compliance Basic Deployment Scenario: NAT444 Network Diagram NAT44 on VSM Configuration I2O Packet Flow on VSM O2I Packet Flow on VSM Parameters to be considered before Deployment Performance and Scale Address Pool and Mapping VRF (Inside and Outside) Port Limit (Per-User) External Logging (Netflow and Syslog) Destination-Based Logging (DBL) Bulk Port Allocation (BPA) NAT Session Timeouts (TCP, UDP, ICMP) Endpoint Independent and Dependent Filtering (EIF / EDF) Traffic Diversion (to VSM) Default Route Static Route ACL-based Forwarding (ABF) Static Port Forwarding Application Layer Gateway (ALG) Redundancy N:1 ABF-Based Redundancy NAT44 Licenses Summary of parameter values NAT44 Configuration / Feature Differences with ISM MTU and Fragmentation Considerations Specific Deployment Scenarios / Use-Cases Sample Applications over NAT44 Multiple Outside ServiceApps sharing same Outside VRF NAT Bypass MPLS (Layer 3) VPN and NAT Interworking NAT Hairpining Handling of Asymmetric I2O and O2I traffic with better performance Validation of Configuration show run show virtual-service list show virtual-service detail show cgn nat44 <> statistics show cgn nat44 <> inside-translation show cgn nat44 <> outside-translation Monitoring CGN Using SNMP Useful Deployment Tips Related Information Introduction Cisco's CGv6 (Carrier Grade IPv6) solution helps customer transitioning from IPv4 to IPv6 gradually, maintaining business continuity. VSM (Virtualized Services Module) on ASR9K is a Service Card which provides different CGv6 Applications. This is the next generation Service Card after ISM (Integrated Services Module). This document talks about how to deploy NAT44 (Network Address Translation - from IPv4 to IPv4), one of the applications fall under CGv6 suite of applications.   This document does NOT intend to cover details on VSM, how to install CGv6 software on VSM, how to trouble-shoot CGv6 issues, CGv6 Applications other than NAT44.   DISCLAIMER: This document contains some Public IP addresses to explain different deployment use-cases and features related to NAT44. The Public IP addresses are picked up randomly and matching of these addresses with any specific customer's or organization's public IP is purely coincidental. Please DO NOT use these public addresses in your NAT setup / deployment, unless the public IP addresses are not owned by you / your organization. Cisco is not responsible for any issues arising out of this.     Introduction to NAT44 This section introduces NAT44 and terminologies related to it.   Back to top 1. Why NAT44 ?   IPv4 address is 32-bit long which means there can be 4,294,967,296 (i.e., 4 billions) unique addresses. However, population on earth has crossed 7 billions and is still growing. Also, nowadays, a single person can own multiple IP devices, each having its own IP address. Thus, we are running short of IPv4 addresses. In 2 of the 5 RIR (Regional Internet Registries), IPv4 address exhaustion is declared (in its last /8 stage).   Apat from moving to IPv6 (which uses 128-bit long IP address), an alternate mechanism is to use NAT44 where Private IP addresses can be re-used in different Private networks, but can be translated to a unique Public IP address when the packet enters Internet or Public network.   If you are a Service Provider and running short of Public IP addresses, NAT44 should be of great usefulness to you as an intermediate solution before you slowly convert your IPv4 network into IPv6.   Back to top 2. NAT44 Overview   NAT44 literally means translating one IPv4 address to another IPv4 address. "Typically", first IPv4 address is "Private" IPv4 Address (defined by RFC 1918) and second IPv4 address is "Public" (non-Private) IPv4 address. As defined by RFC 1918, Private IPv4 addresses fall in the following blocks.   Start IP Address End IP Address Comments 10.0.0.0 10.255.255.255 10.0.0.0/8 prefix - contains 16,777,216 addresses 172.16.0.0 172.31.255.255 172.16.0.0/12 prefix - contains 1,048,576 addresses 192.168.0.0 192.168.255.255 192.168.0.0/16 prefix - contains 65,536 address   However, it is to be noted that not always "Private" IPv4 address is to be translated to "Public" IPv4 address. There are deployment cases where NAT44 is used to convert "Private" IPv4 address (say, 192.168.x.x range) to another "Private" IPv4 address (say, 10.x.x.x range) as well.   NOTE: In this document, however, we will refer "Private" side as the pre-NAT side of the NAT device and "Public" side as the post-NAT side of the NAT device, irrespective of the actual IP address being used.   One more important point, although NAT indicates Network Address Translation, it is actually NAPT (Network Address and Port Translation). What it means is - in addition to IPv4 address, Layer 4 (UDP/TCP) port is also translated by NAT44.   NOTE: Unless mentioned otherwise, in this document, NAT44 will indicate translation of both IPv4 address and Port.   Following Layer 4 Protocols are supported by NAT44 implementation on VSM. UDP (IP Protocol 17) - UDP Port number (2 bytes) is translated along with IPv4 address. TCP (IP Protocol 6) - TCP Port number (2 bytes) is translated along with IPv4 address. ICMP (IP Protocol 1) - ICMP Identifier field (2 bytes) in Echo Request and Echo Response messages is used as (dummy) Layer 4 port. GRE (used with PPTP ALG, IP Protocol 47) - Call ID field (2 bytes) is used as (dummy) Layer 4 port.   Another useful property of NAT44 is multiple Private IP addresses can be mapped to one Public IP address. This is possible as all of the 65,536 ports (0 to 65,535) on any Private IP address are typically not used by Applications.   Following diagram shows how NAT44 works.     As shown above, the Private IP addresses 192.168.1.1 and 192.168.1.5 are translated to same Public IP address 20.1.1.1 (along with specific Layer 4 ports) by the NAT44 device (VSM on ASR9K). The translation entries are maintained in NAT DB at the NAT44 device.   Back to top 3. Basic Terminologies   In this section, let us get introduced with different terminologies related to NAT44 deployment on VSM.   Terminology Definition / What it means Inside / Subscriber Side / Access Side / Private Side Private side of the NAT44 device (which usually contains Private IPv4 addresses) Outside / Network Side / Core Side Public side of the NAT44 device (which usually contains Public IPv4 addresses) Inside-to-Outside (I2O) Traffic flowing from Inside to Outside across NAT44 device Outside-to-Inside (O2I) Traffic flowing from Outside to Inside across NAT44 device Stateful (SF) Which preserves State or Database. E.g. - NAT44 is Stateful protocol. Endpoint Independent Filtering (EIF) Any device (IP + Port) can send O2I traffic to a Public IP + Port which exists in NAT DB. VSM implements this behavior. It is also termed as "Full-Cone NAT". Endpoint Dependent Filtering (EDF) Only the device (IP + Port) which has received I2O traffic can send O2I  traffic to a public IP + Port which exists in NAT DB. Destination IP and  port information is also maintained in NAT DB in this case. VSM implements this behavior (Address-Dependent as well as Address and Port-Dependent) Port Parity Preservation Odd port should be translated to Odd port and Even port should be translated to Even port. VSM supports this.     Back to top 4. Standard Compliance   Following are the important RFCs that the NAT44 solution on VSM complies (most of the cases, fully) to.   RFC Number / IETF Draft Name Title of the Specification Additional Comments RFC 768 User Datagram Protocol (UDP)   RFC 792 Internet Control Message Protocol (ICMP)   RFC 793 Transmission Control Protocol (TCP)   RFC 959 File Transfer Protocol (FTP)   RFC 1812 Requirements for IP Version 4 Routers   RFC 3954 Cisco System Netflow Services Export Version 9   RFC 4787 Network Address Translation (NAT) Behavioral Requirements for Unicast UDP   RFC 5382 NAT Behavioral Requirements for TCP   RFC 5424 The Syslog Protocol   RFC 5508 NAT Behavioral Requirements for ICMP   RFC 6888 Common Requirements for Carrier Grade NATs (CGN) Evolved from draft-nishitani-cgn     Basic Deployment Scenario: NAT444   This section talks about one of the basic deployment scenario for NAT44 which is called "Double NAT44" or "NAT444".   Back to top 5. Network Diagram   Following diagram depicts NAT444.     In the above diagram, there are 2 NAT44 instances shown (hence, it is termed as "Dual NAT44" or "NAT444").   CPE (Customer Premise Equipment) NAT44 - This typically happens within customer premise. Each customer / end user is given one (or, more) IPv4 address (10.8.1.11 in the above example) by the Service Provider. There could be multiple devices within customer premise which will have IPv4 addresses in 192.168.1.x network (as shown in the above diagram). As an example, CPE NAT44 device translates IPv4 address/Port of 192.168.1.1:4128 to 10.8.1.11:6593, while sending the packet out to provider network. A NAT table is maintained at CPE NAT level as well. But, the scale of this is less (as there won't be too many end devices). As you can see, both Private-side and Public-side IP addresses used here are from RFC 1918 (i.e., Private IPv4 addresses).   CGN (Carrier Grade NAT) or LSN (Large Scale NAT) - This happens within the Service Provider network. CGN device (as per the diagram) translates Private IPv4 address/Port of 10.8.1.11:6593 to a public IPv4 address/Port of 170.0.0.1:9185 before sending the packet to Internet. A relatively huge NAT table is maintained at CGN level as it may need to deal with millions of CPEs.   VSM with ASR9K will be positioned as CGN in Service Provider network.   Back to top 6. NAT44 on VSM Configuration   Before we start configuring NAT44 on VSM, let us understand the following concepts related to NAT44 implementation on VSM.   CGN Instance - On each VSM card, we can configure one CGN instance (<cgn1> in the diagram below). NAT44 Instance - On each VSM card, we can configure one NAT44 instance (<nat1> in the diagram below). ServiceInfra interface - ServiceInfra interface is a virtual interface which is used by the internal (to ASR9K) management / control traffic. We need 1 ServiceInfra interface per VSM card. ServiceInfra interface cannot be in any VRF (needs to be in default VRF). ServiceApp interface - ServiceApp interface is a virtual interface which is used to bring CGv6 Application (NAT44) data traffic to VSM card. We need 1 pair of ServiceApp interface for each VRF instance - 1 Inside ServiceApp interface (ServiceApp <x> in the diagram below) to bring Inside traffic to VSM (I2O direction) and 1 Outside ServiceApp interface (ServiceApp <y> in the diagram below) to bring Outside traffic to VSM (O2I direction).   NOTE: ServiceInfra and ServiceApp interfaces are used within the ASR9K router only and do not need to be announced/redistributed into IGP.   Inside VRF instance - Within one NAT44 instance, we can have multiple Inside VRF instances. For each Inside VRF instance, we can have different NAT44 parameters configured. Inside ServiceApp must be within one unique Inside VRF. Outside ServiceApp may or may not be within a (Outside) VRF.   NOTEs: - We cannot have multiple Inside ServiceApps within same Inside VRF. - We can, however, have multiple Outside ServiceApps within same Outside VRF or default VRF. - Inside and Outside VRF cannot be same.   Logical Partitioning inside VSM - Internally, VSM is divided into 2 Logic Partitions. Hence, in order to achive full throughput and scale limit (per VSM), you should have minimal of 2 Inside VRFs, 4 ServiceApp interfaces (2 in each partition), 2 Public IP address pool, etc. Traffic / subscribers which needs to avail NAT44 service also needs to be divided into 2 groups (each one associated to each Inside VRF). Following table shows the partitioning of ServiceApp interfaces between these 2 groups (which needs to be kept in mind during configuration).   Logical Partition  / Group 1 Logical Partition / Group 2 ServiceApp 1, ServiceApp 2 ServiceApp 3, ServicApp 4 ServiceApp 5, ServiceApp 6 ServiceApp 7, ServiceApp 8 ServiceApp 9, ServicApp 10 ServiceApp 11, ServiceApp 12 ... ... ServiceApp 253, ServiceApp 254 ServiceApp 255, ServiceApp 256       NOTE: ServiceApp interafce pair needs to be selected from same Logical Partition (as per the above table). Otherwise, packets will be dropped. Using consecutive numbers for ServiceApp interface pair will be recommended (as shown above).   Following diagram provides summary / overview of NAT44 configuration for VSM.     Below are sample configurations for each of the above steps.   Step 1: Install and Activate CGv6 VM Configure on VSM Card at Slot 1 ! Install CGv6 VM's OVA package ! ----------------------------- RP/0/RSP0/CPU0:ios# virtual-service install name cgn123 package /disk0a:/usr/vsmcgv6.ova node 0/1/CPU0 ! Configure and Activate CGv6 VM ! ------------------------------ RP/0/RSP0/CPU0:ios#conf t Fri Aug 2 18:10:25.525 UTC RP/0/RSP0/CPU0:ios(config)#virtual-service cgn123 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/0 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/1 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/2 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/3 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/4 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/5 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/6 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/7 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/8 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/9 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/10 RP/0/RSP0/CPU0:ios(config-virt-service)#vnic interface tenGigE 0/1/1/11 RP/0/RSP0/CPU0:ios(config-virt-service)#commit Fri Aug 2 18:11:34.285 UTC RP/0/RSP0/CPU0:ios(config-virt-service)#activate RP/0/RSP0/CPU0:ios(config-virt-service)#commit       Step 2: Configure ServiceInfra interface ! Bring up 10-GE interfaces on VSM ! Must before configuring any ServiceInfra/App interfaces ! ------------------------------------------------------- interface TenGigE 0/1/1/0 no shut interface TenGigE 0/1/1/1 no shut interface TenGigE 0/1/1/2 no shut interface TenGigE 0/1/1/3 no shut interface TenGigE 0/1/1/4 no shut interface TenGigE 0/1/1/5 no shut interface TenGigE 0/1/1/6 no shut interface TenGigE 0/1/1/7 no shut interface TenGigE 0/1/1/8 no shut interface TenGigE 0/1/1/9 no shut interface TenGigE 0/1/1/10 no shut interface TenGigE 0/1/1/11 no shut ! Define Service Infra interface ! ------------------------------ interface ServiceInfra 1 ! Assign IPv4 address and netmask ipv4 address 100.1.1.1 255.255.255.252 ! Define service location service-location 0/1/CPU0       Step 3 and 4: Configure Ingress and Egress LC Interfaces ! Configure Ingress interfaces ! ---------------------------- interface TenGigE0/0/0/0 ! Add it to Inside VRF vrf ivrf1 ! Assign IP address to it ipv4 address 10.10.10.1 255.255.255.0 ! interface TenGigE0/0/0/1 vrf ivrf2 ipv4 address 10.10.11.1 255.255.255.0 ! Configure Egress interfaces ! -------------------------- interface TenGigE0/2/0/0 vrf ovrf1 ipv4 address 20.20.20.1 255.255.255.0 interface TenGigE0/2/0/1 vrf ovrf2 ipv4 address 20.20.21.1 255.255.255.0   Step 5: Configure CGN and NAT44 Service parameters (Important ones are shown) ! Define CGN Instance ! ------------------- service cgn cgn1  ! Define location service-location preferred-active 0/1/CPU0  ! Define NAT44 Service Type and instance (one per VSM card) service-type nat44 nat1   ! Define per-user Port limit for NAT44 instance portlimit 250   ! Enable Active FTP ALG   alg ActiveFTP   ! Define UDP protocol session timeout values   protocol udp     session initial timeout 240     session active timeout 600   ! Define inside VRF   inside-vrf ivrf1     ! Define Public IP address Pool     map outside-vrf ovrf1 address-pool 100.1.1.0/24   ! Define inside VRF   inside-vrf ivrf2     ! Define Public IP address Pool     map outside-vrf ovrf2 address-pool 100.1.2.0/24     Step 6 and 7: Configure Inside and Outside ServiceApp interfaces ! Define Inside Service App interfaces ! ------------------------------------ interface ServiceApp 1  ! Define VRF where it should belong to (optional)  vrf ivrf1   ! Assign IPv4 address and netmask   ipv4 address 1.1.1.1/30   ! Define CGN instance and service type in which it belongs to   service cgn cgn1 service-type nat44 interface ServiceApp 3 vrf ivrf2   ipv4 address 1.1.2.1/30   service cgn cgn1 service-type nat44 ! Define Outside Service App interfaces ! ------------------------------------- interface ServiceApp 2  vrf ovrf1   ipv4 address 2.1.1.1/30   service cgn cgn1 service-type nat44  interface ServiceApp 4  vrf ovrf2   ipv4 address 2.1.2.1/30   service cgn cgn1 service-type nat44     Step 8 and 9: Configure Traffic Diversion Rules ! Define static route for Inside-to-outside traffic ! ------------------------------------------------- router static  ! Specify the Inside VRF  vrf ivrf1   ! Define address family   address-family ipv4 unicast    ! Re-direct all traffic to Inside Service App    0.0.0.0/0 ServiceApp 1 router static  vrf ivrf2   address-family ipv4 unicast    0.0.0.0/0 ServiceApp 3 ! Define static route for Outside-to-inside traffic ! ------------------------------------------------- router static  vrf ovrf1   address-family ipv4 unicast    100.1.1.0/24 ServiceApp 2 router static  vrf ovrf2   address-family ipv4 unicast    100.1.2.0/24 ServiceApp 4     Back to top 7. I2O Packet Flow on VSM   Following diagram shows the packet flow within ASR9K (including VSM) for Inside to Outside NAT44 traffic at a high-level.     Step 1: Packet from Inside / Private side reaches ASR9K Ingress Line Card.   Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to VSM card, via Inside ServiceApp 1 interface. Inside VRF (ivrf1) will be used during the forwarding lookup.   Step 3: NAT44 Application performs necessary action for the packet. As it is Inside-to-Outside packet, it first checks if there is a NAT entry exists for the source IP (10.10.10.2) and port (5000). If it does not exists, it allocates a Public IP (from the Public IP pool configured) and port and creates a NAT entry. From the NAT entry, it finds the corresponding Public IP (100.2.0.192) and port (23156) and replaces the existing Source IP and port with the same.   Step 4: After NAT translation is done, a forwarding lookup is done in Outside VRF (ovrf1) for the destination IP address (5.5.5.2) which forwards the packet to egress LC interface.   Step 5: Egress Line Card interface sends the packet out to Outside / Public side.   Back to top 8. O2I Packet Flow on VSM   Following diagram shows the packet flow within ASR9K (including VSM) for Outside to Inside NAT44 traffic at a high-level.     Step 1: Packet from Outside / Public side reaches ASR9K Ingress Line Card.   Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to VSM card, via Outside ServiceApp 2 interface. Outside VRF (ovrf1), if being configured, will be used during the forwarding lookup.   Step 3: NAT44 Application performs necessary action for the packet. As it is Outside-to-Inside packet, it first checks if there is a NAT entry exists for the Destination IP (100.2.0.192) and port (23156). If it does not exists, packet is dropped. Otherwise, from the NAT entry, it finds the corresponding Private IP (10.10.10.2) and port (5000) and replaces the existing Destination IP and port with the same.   Step 4: After (reverse) NAT translation is done, a forwarding lookup is done in Inside VRF (ivrf1) for the destination IP address (10.10.10.2) which forwards the packet to egress LC interface.   Step 5: Egress Line Card interface sends the packet out to Inside / Private side.   Parameters to be considered before Deployment   Before deploying NAT44 solution with VSM, you need to analyze the deployment scenario/requirement and come up with the appropriate values of different configurable parameters related to NAT44. Improper selection of the parameters may not address the deployment goals accurately.   Back to top 9. Performance and Scale   These are possibly the most important parameters to be considered during deployment. These are not configurable parameters, but these will determine how many VSM cards will be needed for the NAT44 solution.   Identify the performance and scale need for NAT44 solution Please check "CGv6 on VSM Feature Support" page, to find out the supported scale and performance values.   Based on the above two, find out how many VSM cards will be needed for the solution.   NOTE: For better NAT44 performance, Cisco recommends keeping the default physical interface MTU size on the customer-facing interfaces in NAT44 deployment.   Back to top 10. Address Pool and Mapping   Identify the following for the deployment scenario.   How many Public IPv4 addresses the customer has to form the Public IP pool ? How many (approximately) Private (Pre-NAT) IPv4 addresses the customer has ? Do you need to limit the ratio of Private to Public IP addresses ?   Based on the same, number of inside VRFs, address pools, many-to-one ratio can be configured as follows.   Address Mapping related parameter configuration service cgn cgn1   service-type nat44 nat1    inside-vrf ivrf1    ! Define Public IP address Pool    map outside-vrf ovrf1 address-pool 100.1.1.0/20    ! Define Many to one IP addresses maping    map ip many-to-one 6       NOTE: Network address and Broadcast address in the subnet (as configuredby address-pool parameter) are also used as NAT'ed Public / Outside IP address.   Back to top 11. VRF (Inside and Outside)   Identify the following VRF related information from deployment requirement.   Whether Private side traffic comes / can / should come in VRF or not ? Whenther Public side traffic can / should go in a VRF or not ? Is there a overlap of Private IP addresses or not ? Are there different values of certain parameters (which are configured per-VRF) needed or not ? What is the Scale and Bandwidth requirement for each VRF   Above information will determine, How many Inside and Outside VRFs will be needed in the solution Which per-Inside VRF parameters need to be set How the Public IP pool needs to be divided   Back to top 12. Port Limit (Per-User)   As NAT DB is shared among all users, there could be a situation where few users (Private IP owners) uses lots of NAT entries (in thousands) and because of the same, new users cannot create NAT entries. In order to keep control on creation of NAT entries by a single user, Port Limit parameter is introduced. Port limit indicates how many port numbers are allowed per Private / Inside IP address. If any flow arrives after Port Limit is reached, no NAT entry will be created for the same and the packets for the new flows will be dropped. The Port limit value will be same for all Private IP addresses.     NOTE: Port Limit is specified per Private / Inside IP address and NOT per Public / Outside IP address.   Find out the following:   How many ports will be needed by a Private IP address holder ? How many Public IP addresses are available ?   Based on the above Port Limit can be configured as follows.     Configuration of Port Limit service cgn cgn1   ! Define NAT44 Service Type and instance (one per VSM card)  service-type nat44 nat1   ! Define per-user Port limit for NAT44 instance   portlimit 250     Back to top 13. External Logging (Netflow and Syslog)   As you have seen above, a NAT DB entry is created for each new flow (Private IP address and port) which keeps the mapping of Private / Inside IP address + Port and Public / Outside IP address + Port. The NAT DB entry is deleted when the flow terminates (because of timeout and any user intervention).   On VSM, we support millions of NAT translations with high (in thousands and millions) setup and deletion rate (Please refer to CGv6 on VSM Feature List for scale numbers). This produces huge amount of NAT translation entries / records which cannot be stored / saved on VSM or ASR9K because of limited disk space. Hence, there is a need of having external servers where these translations can be saved.   The creation and deletion of NAT transactions (also called "Logging Records") are very important to be saved to get association of different IP addresses. In almost all countries, it is mandated by the Lawful Agencies / Authorities that the Service Provide must provide these records accurately so that the agencies can find out the owner of these APIs for different reasons. Thus it becomes a very important feature for deployment of NAT44.   On VSM, we support the 2 formats of logging records - Netflow (version 9) and Syslog. Table below compares the two.   Items for Comparison Netflow Version 9 (NFv9) Syslog Standard Document   RFC 3954 RFC 5424 Text / Binary format Binary Text Underlying Layer 4 protocol UDP (Server usually uses Port 2055, 2056, 4432, 4739, 9995, 9996, 6343) UDP (Server typically uses Port 514) Average Logging Recod Size 20 Bytes (with DBL and BPA disabled) 70 Bytes (with DBL and BPA disabled) Usage of Template Uses Netflow Template Does not use any template   Logging records are sent to Logging Server (IP address and Port for which is configurable) either when the Logging packet is complete or timeout occurs. Logging packets are sent with ServiceInfra interface's remote IP (for e.g., 100.1.1.2 is the remote IP of 100.1.1.1) as Source address.   NOTE: External Logging Server IP address should be reachable via any ASR9K LC interface (NOT via Management Ethernet interface), in default VRF.   Destination-Based Logging (DBL)   In some countries, Lawful Agencies also require the Service Provider to provide the Destination IP address and Port (along with Source IP address and Port information) as part of Logging Record. This is termed as Destination-Based Logging (DBL).   NOTE: Including Destination (IP + Port) information in the Logging Record will increase the size of the Record.   DBL can be supported with Netflow v9 as well as with Syslog.   Bulk Port Allocation (BPA)   In order to reduce the size of the logging record, a feature called Bulk Port Allocation (BPA) is introduced.   Without BPA, for every new flow, a Port is allocated for the Public IP and a new logging record is created with Private IP + Port and Public IP + Port. Similarly, another Logging record is created when a flow or NAT entry is deleted.   Following table captures the working of it (without BPA) with some sample IP addresses and port numbers.   Event Inside Source IP + Port Outside Source IP + Port Logging Record Additional Comments A new flow arrives 10.1.1.1:1000 170.0.0.5:8175 10.1.1.1:1000 <-> 170.0.0.5:8175 New port (8175) allocation happens. New logging record is created Existing flow arrives 10.1.1.1:1000 170.0.0.5:8175 N/A No new Port allocation happens. No logging record is created A new flow arrives (with new port, but same IP) 10.1.1.1:2000 170.0.0.5:13591 10.1.1.1:2000 <-> 170.0.0.5:13591 New port (13591) allocation happens. New logging record is created.   With BPA, for the first new flow, a bulk (group) of Ports are allocated (but only one port from the group is assigned for the new flow) for the Public IP and a logging record is created with Private IP and Public IP + Start Port + End Port. Next time, a new flow comes, a new port from the already allocated pool is taken and no logging record is created. Only when all the ports in the group are used, a new bulk is allocated and logging record is created. Similarly, when all the ports from a group is released, a (deletion) logging record is generated.   Following table captures the working of it (with BPA enabled) with sample IP addresses and port numbers.   Event Inside Source + Port Bulk Port Allocation (size 64) Outside Source IP + Port Logging Record Additional Comments A new flow arrives 10.1.1.1:1000 Allocates Bulk Port range (8192 - 8255) for 170.0.0.5 170.0.0.5:8201 10.1.1.1 <-> 170.0.0.5:8192-8255 Bulk port allocation and new port (8201) assignment happens. New logging record is created. Existing flow arrives 10.1.1.1:1000 N/A 170.0.0.5:8201 N/A No new Port allocation happens. No logging record is created. A new flow arrives (with new port, but same IP) 10.1.1.1:2000 N/A 170.0.0.5:8199 N/A New Port (8199) allocation happens (from same port range). No logging record is created. A new flow arrives (with new port, but same IP) 10.1.1.1:5000 N/A 170.0.0.5:8236 N/A New Port (8236) allocation happens (from same port range). No logging record is created.     Following table illustrates the amount stoage space needed per second with varying translation creation + deletion rate for both Netflow and Syslog (with DBL disabled). It also shows the impact of BPA on the storage.   NAT translation creation + deletion rate per second Netflow without BPA Syslog without BPA Netflow with Bulk size 256 and 50% utilization Syslog with Bulk size 256 and 50% utilization 20,000 0.8 MB 2.8 MB 6.25 KB 21.875 KB 50,000 2 MB 7 MB 15.625 KB 54.6875 KB 100,000 4 MB 14 MB 31.25 KB 109.375 KB 500,000 20 MB 70 MB 156.25 KB 546.875 KB 1,000,000 40 MB 140 MB 312.5 KB 1.093 MB       NOTE: Both DBL and BPA cannot be enabled at the same  time. This is because BPA requires logging record to be generated for a  set of NAT entries whereas DBL requires logging record to be generated  for each NAT entry. These are conflicting requirements.   Please collect the following information and accordingly determine External Logging related parameter configuration.   What setup rate needs to be supported ? Do the Lawful Agencies mandate DBL ? What protocol (netflow or syslog) the external logging server support ? How many packets per second the logging server can support ? How many logging servers need to be configured ? How much space would be needed for storing logging record ? How much would the cost of disk space ?   Following table shows how to configure different external logging parameters.   Configuration of External Logging Parameters service cgn cgn1  service-type nat44 nat1   inside-vrf ivrf1    ! Define Netflow V9 logging configuration    external-logging netflow version 9     ! Define netflow server configuration     server      ! Define logging server address and port      address 10.10.0.0 port 2055      ! Define Path MTU size      path-mtu 1200      ! Define refresh rate      refresh-rate 50      ! Define timeout      timeout 50      ! Define Syslog logging configuration     external-logging syslog      server       address 10.10.0.1 port 514     Back to top 14. NAT Session Timeouts (TCP, UDP, ICMP)   As the new flows creates NAT entries, we need a way to delete those NAT entries as well. Protocols like UDP is connection-less and hence, there is no way to identify when a session / connection has ended by inspecting the packets. Also, in some cases, if no packet flows between the applications for a long period of time, there is no point in holding on to the NAT entries, as that will disallow creation of new NAT sessions.   Due the above reasons, NAT entries are deleted when session timeout happens.   For every protocol (TCP, UDP, ICMP), we have different types and values of session timeouts, as depicted below.   TCP - It has two timeouts.                                                                                        Initial Timeout - It kicks in before the connection moves into Active state. Default: 120 sec. Active Timeout - It kicks in when the connection is in Active state. Default: 30 Minutes. UDP - It has two timeouts as well.                                                                                        Initial Timeout - It kicks in before the connection moves into Active state. Default: 30 sec. Active Timeout - It kicks in when the connection is in Active state. Default: 120 sec. ICMP - It has one timeout. Default: 60 sec.   If the timeout is configured to be too less, NAT sessions will be deleted too soon, resulting in changing of Public IP addresses. This may cause issues with the Applications.   If the timeout is configured to be too high, NAT sessions will remain in the system for long time (even though there is no data is flowing using the session) and it will restrict new sessions to be created as well (resulting in dropping of those packets).   Hence, timeout value needs to be set appropriately.   NOTE: These timeouts are set for the NAT44 instance (per VSM card) and cannot be configured differently for each Inside VRF.   You need to have some knowledge about the following, to determine the timeout values.   Application traffic that is going through the network                                                                                          How sensitive it is with respect to change in IP address ? How periodically, it would generate packets How many NAT sessions are expected ?   Following table shows how to configure timeout values for different protocols.   Configuration of Session Timeout (TCP, UDP, ICMP) service cgn cgn1   service-type nat44 nat1   ! Define UDP protocol session timeout values   protocol udp    session initial timeout 65535    session active timeout 65535   ! Define TCP protocol session timeout values   protocol tcp    session initial timeout 65535    session active timeout 65535   ! Define ICMP protocol session timeout value   protocol icmp    timeout 908     Back to top 15. Endpoint Independent and Dependent Filtering (EIF / EDF)   Filtering behavior of NAT is explained in Section 5 of RFC 4787.   When an internal endpoint opens an outgoing session through a NAT,    the NAT assigns a filtering rule for the mapping between an internal    IP:port (X:x) and external IP:port (Y:y) tuple.     The key behavior to describe is what criteria are used by the NAT to    filter (Outside-to-Inside, O2I) packets originating from specific external endpoints. We have the following 3 options here:   Endpoint-Independent Filtering (EIF) - NAT forwards any packet destined to X:x, regardless of the external IP address and port source. Endpoint-Dependent Filtering (EDF) Address-Dependent Filtering - NAT forwards any packet destined to X:x, provided it is coming from the external IP to which I2O packets were sent earlier. Address and Port-Dependent Filtering - NAT forwards any packet destined to X:x, provided it is coming from the external IP:port to which I2O packets were sent earlier.   All the behavior are explained in the following diagram with an example scenario.     As shown above, depending on the filtering policy and NAT session DB, O2I packets coming from specific IP and port will be either allowed to cross NAT device (VSM card) or dropped by CGv6 Application on VSM.   Table below shows how to configure CGv6 application on VSM to support different filtering policy.   Configuration for Endpoint-Independent and Endpoint-Dependent Filtering ! To configure Endpoint-Independent Filtering ! ------------------------------------------- service cgn cgn1  service-type nat44 nat1   inside-vrf ivrf1    ! Do not specify 'filter-policy' CLI for Endpoint-Independent Filtering ! To configure Address Dependent Filtering ! ---------------------------------------- service cgn cgn1  service-type nat44 nat1   inside-vrf ivrf1    ! Filtering policy for Address Dependent Filtering    filter-policy ignore-port ! To configure Address and Port Dependent Filtering ! ------------------------------------------------- service cgn cgn1  service-type nat44 nat1   inside-vrf ivrf1    ! Filtering policy for Address and Port Dependent Filtering    filter-policy       Back to top 16. Traffic Diversion (to VSM)   We can use different mechanisms to divert the traffic to VSM card (to specific ServiceApp interface).   Default Route   If the inside traffic is arriving on a (Inside) VRF and ALL traffic needs to undergo NAT44 operation, we can add a default route under the VRF to divert traffic to the Inside ServiceApp (as shown below).   Configuration for Traffic Diversion using Default route ! Define static route for Inside-to-outside traffic ! ------------------------------------------------- router static  ! Specify the Inside VRF  vrf insidevrf1   ! Define address family   address-family ipv4 unicast     ! Re-direct all traffic to Inside Service App    0.0.0.0/0 serviceapp 1     Static Route   In some cases, Outside ServiceApp interfaces reside in same / default VRF. Hence, in order to divert O2I traffic to specific Outside ServiceApp interfaces, static route can be used in the following way.   Configuration for Traffic Diversion using Static Route router static  address-family ipv4 unicast   ! Diverting traffic for one Public IP pool to one Outside ServiceApp   100.1.1.0/24 serviceapp 2   ! Diverting traffic for another Public IP pool to another Outside ServiceApp   200.1.1.0/24 serviceapp 4     ACL-based Forwarding (ABF)   In certain deployment cases, default route may not work.   Deployment Use-case 1: Customer wants to divert traffic coming from specific subscriber network (containing Private IP address) to VSM (for NAT44) and the remaining traffic (which may already contain Public IP address) need not go to VSM.   Deployment Use-case 2: Customer wants to distribute subscriber traffic into different Inside VRFs which may go to same VSM card or different VSM cards (but, will certainly using different Public IP pool).   Deployment Use-case 3: Subscriber / Access / Inside traffic comes in default / global VRF and we need to distribute the traffic to different Inside ServiceApps which are in different Inside VRFs.   NOTE: Typically, ABF is to be used in Private side / Subscriber side and NOT towards Public / Core / Network side.     In the above cases, ABF provides a very good solution. Below is sample configuration on how to use ABF for traffic diversion.   Configuration for Traffic Diversion using ABF ! Define IPv4 Access List ipv4 access-list nat44_abf  ! Divert to ServiceApp 1  10 permit ipv4 10.0.100.0/24 any nexthop1 vrf ivrf1 ipv4 5.1.1.2  ! Divert to ServiceApp 3  20 permit ipv4 10.0.200.0/24 any nexthop1 vrf ivrf2 ipv4 6.1.1.2 100 permit ipv4 any any ! Configure Ingress Interface interface TenGigE0/5/0/2 vrf subscriber  ipv4 address 10.1.1.1 255.255.255.0  ! Apply ABF on Ingress Interface  ipv4 access-group nat44_abf ingress ! Define one Inside ServiceApp interface interface ServiceApp1 vrf ivrf1  ipv4 address 5.1.1.1/30  service cgn cgn1 service-type nat44 ! Define another Inside ServiceApp interface interface ServiceApp3  vrf ivrf2  ipv4 address 6.1.1.1/30  service cgn cgn1 service-type nat44   Back to top 17. Static Port Forwarding   There is a very important point about Dynamic NAT44. All the NAT translations / sessions are created by I2O traffic. If O2I traffic reaches VSM for which there is no NAT entry, the packets will be dropped. This works fine for most of the cases as the session / traffic is initiated by Applications which reside on Private side.   However, there are cases, where a server resides on the Private side of the network. In this case, there are 2 issues.   Clients (on Public side) cannot initiate session to this Server, as those (O2I) packets will be dropped by VSM. Clients do not know the Public IP of the Server (and cannot use Private IP to communicate either)   To address these problems related to Server sitting on Private side of the network, a new feature called "Static Port Forwarding" is introduced. Following diagram explains "Static Port Forwarding" feature functionality.     Using Static Port Forwarding, user can provide the Private IP address and Port (via CLI) and NAT44 Application on VSM generates (dynamically) the corresponding Public IP and Port. Also, these mapping are made permanent - i.e., they do not get timed out and get deleted. The Public IP and port generated by the Application can be added to DNS database and thus made available to all.   With this design, any device on Public side of network can now send traffic to the known Public IP and port (to reach the Server sitting inside Private network). Also, as the NAT entry is permanent, O2I traffic always finds the entry and using the same can translate the packet (without dropping it).   Staic Port Forwarding entries can be created separately for each protocol - TCP, UDP and ICMP. To know the maximum number of such entries, please check CGv6 on VSM Feature page.     NOTES: - User cannot provide the Outside / Public IP and port while configuring Static Port Forwarding. - Outside / Public IP is generated by the NAT44 Application and hence, can be random. - Outside port is usually assigned same as Inside Port, unless the Outside port is already being used   As these are used for Servers located in Private network, it is desirable that the Public IP address remain constant across RP failover.   As Outside / Public IP address for a Static Port Forwarding entry is allocated by NAT44 application based on the instant situation of Public IP availability stock as well as the allocation algorithm's state, it is highly probable that if the Static Port Forwarding entry is deleted and re-added or, if the VSM card is reloaded), it may allocate a different Public IP address.   In order to keep the same Outside IP address, it is recommended to add the Static Port Forwarding configuration as a first thing after VSM card comes up and when there is no NAT traffic on the router / VSM card (could be in a "maintenance window"). That way, every time, VSM is reloaded, the Public IP addresses assigned for Static Port Forwarding entries will remain the same.   Try to find out the following, before using Static Port Forwarding.   Will there be any Servers sitting in the Private side of the deployment network ? How many of those Servers will be ? Can the Public IP address of the Servers be random and can be published via DNS or similar mechanism ?   A sample configuration for Static Port Forwarding is shown below.   Configuration for Static Port Forwarding service cgn cgn1   service type nat44 nat1    inside-vrf ivrf1    map address-pool 180.0.0.0/24    ! Define TCP related configuration    protocol tcp     ! Define Static Port Forwarding configuration     static-forward inside address 10.0.0.50 port 6050     static-forward inside address 10.0.0.50 port 80    ! Define UDP related configuration    protocol udp     ! Define Static Port Forwarding configuration     static-forward inside address 10.0.0.61 port 550    ! Define ICMP related configuration    protocol icmp     ! Define Static Port Forwarding configuration     ! For ICMP, 'identifier' field in ICMP header is used for 'port'     static-forward inside address 10.0.0.189 port 750     Following table shows the NAT entries corresponding to the above Static Port Forwarding entries (for TCP protocol).   "show cgn nat44 <> inside--translation" output for Static Port Forwarding   RP/0/RSP0/CPU0:VSM-Lab#show cgn nat44 nat44-1 inside-translation protocol tcp inside-vrf vrf1 inside-address 10.0.0.50 port start 25 end 8000

 Fri Nov  1 13:44:44.874 UTC 
Inside-translation details 
---------------------------
 NAT44 instance : nat44-1
 Inside-VRF     : vrf1 
--------------------------------------------------------------------------------------------
    Outside         Protocol  Inside       Outside       Translation   Inside      Outside
    Address                   Source       Source        Type          to          to 
                             Port         Port                        Outside     Inside
                                                                      Packets     Packets 
-------------------------------------------------------------------------------------------- 
  180.0.9.0          tcp       80           80            static       1218         45 
  180.0.9.0          tcp     6050         6050            static        516       7162       NOTE: - Outside IP (180.0.9.0) and Ports are assigned by the NAT44 Application when user supplies Inside IP (10.0.0.50) and Ports (80, 6025). - "Translation Type" is shown as "static" (as opposed to "dynamic") - Outside port is same as Inside port (usually, unless it is not available)   Back to top 18. Application Layer Gateway (ALG)   Application Layer Gateway (ALG) is a device which has the capability of inspecting into a packet beyond layer 4 and can change Application layer content to meet certain need.   NAT44 changes IP address and Layer 4 (UDP/TCP/ICMP etc.) port. There are certain applications which embeds IP address and/or Port information in Application content. Now, the issue is while the packet traverses across a NAT44 device, layer 3 and 4 addresses get changed. However, in the Application layer, the content still contains old layer 3 and / or layer 4 addresses. This causes an issue in the Application as this is unexpected. Hence, the need of ALG functionality within a NAT device which while changing Layer 3 and 4 packet content, also changes Application content where Layer 3 and 4 addresses were embedded. With this change, end Application will be happy as it does not need to deal with 2 sets of addresses any more.   However, there are 2 ways this problem can be solved.   Adding ALG functionality to NAT device - ALG functionality needs to be added to NAT device for each such Applicaton which embeds Layer 3 and/or 4 addresses in Application content. Changing the Application to deal with NAT device in the network - Application needs to assume that the Layer 3 and/or 4 address can get changed on the way (as there could be NAT device in the network) and hence, either avoid embedding Layer 3/4 addresses in Application content or find other mechanism to handle 2 sets of addresses.   The main challenges in adding ALG functionality to NAT device are:   Every time a new Application is written which embeds Layer 3/4 address in Application content, all NAT devices need to be changed to address the new Application. This is not scalable as lot of new Applications are written everyday. In order to implement ALG for an Application, NAT device needs to understand the Application protocol as well as packet formt/content. Many Applications implement proprietary protocol and hence, do not want to make the same as public information. Hence, those Application packets will face issues with NAT device. There are applications (like, crypto) which do not want any intermediate device to understand / peek into the Application content and modify it. These packets should only be encoded/decoded by end Applications. These application packets would not be able to deal with intermediate NAT device.   On the contrary,   Application can be written in such a way not to use Layer 3/4 addresses in Application content and to use Application-level ID, instead. There can be other mechanisms, e.g. - using STUN (Session Traversal Utilities for NAT) that applications can take to handle intermediate NAT device.   NOTE: Due to the above reasons, our solution does not encourage to support ALG within NAT44 functionality.   However, few basic ALG functionality is implemented with NAT44 Application. These are:   Active FTP (File Transfer Protocol) - Passive FTP does not need ALG. RTSP (Real Time Streaming Protocol) PPTP (Point-to-Point Tunneling Protocol)   Following table show how to configure / enable different ALGs on VSM.   Configuration of ALGs service cgn cgn1   service-type nat44 nat1    ! Enable Active FTP ALG    alg ActiveFTP     ! Enable RTSP ALG     alg rtsp    ! Enable PPTP ALG    alg pptpAlg     Back to top 19. Redundancy   Redundancy is an integral part of CGN (Carrier Grade NAT). Majority of the Service Providers want Redundancy in their network to make it more resilient. It helps in minimizing the traffic outage when one device fails.   On VSM, we support 2 types of redundancy. N:1 ABF-based redundancy (N can be 1 as well) 1:1 Warm Stand-by Redundancy (starting from XR release 5.2.2)   N:1 ABF-Based Redundancy   In this case, there needs to be N Primary VSM card and 1 Backup VSM card. Value of N can be 1 as well, but typically would be 2 or 3. Following are different characteristics of this solution.   Each VSM card needs to be configured independently                                                                                                 Configuration (usually) should be similar (w.r.to parameters, number of ServiceApp interfaces, etc.) In ABF configuration, For each Primary VSM,                    ServiceApp interface on Primary VSM becomes 1st Next-Hop and ServiceApp interface on Back-up VSM becomes 2nd Next-hop Each VSM will have different Public IP address pools (Address Pool is not same in Primary and Backup) When one Primary VSM card fails, Backup VSM card receives all the traffic of Primary VSM card. When Primary VSM card comes up, all traffic again switches back to it from Backup VSM card (it causes, another traffic outage) Each VSM card can support full scale and bandwidth   Following table shows how the ABF Next-Hop configuration would look like for N:1 (N = 3) ABF-based Redundancy.   ServiceApp interfaces ABF Next-Hop 1 ABF Next-Hop 2 ServiceApp interfaces on Primary VSM 1 ServiceApp interfaces on Primary VSM 1 ServiceApp interfaces on Backup VSM ServiceApp interfaces on Primary VSM 2 ServiceApp interfaces on Primary VSM 2 ServiceApp interfaces on Backup VSM ServiceApp interfaces on Primary VSM 3 ServiceApp interfaces on Primary VSM 3 ServiceApp interfaces on Backup VSM     Following diagram depicts N:1 (N = 2) configuration with 1 set of ServiceApps.     As shown in the diagram, before 1st failure, traffic coming from 10.2.0.0/24 subnet will go via Primary 1 VSM (Inside ServiceApp 1) which will use Public IP pool of 151.0.1.0/24. if and when Primary 1 VSM fails, traffic coming from 10.2.0.0/24 subnet will go via Backup VSM (Inside ServiceApp 5) which will use Public IP pool of 151.0.3.0/24. Instead of Primary 1 VSM, if Primary 2 VSM would have failed, traffic from 10.2.1.0/24 (not 10.2.0.0/24) should have switched to Backup VSM.   Following table captures the relevant ABF configuration for this sample setup.   Sample Configuration for N:1 Redundancy using ABF     ! ABF rule for Inside to Outside traffic ipv4 access-list abf_2_1   10 permit ipv4 10.2.0.0/24 any nexthop1 vrf inside1 ipv4 1.1.1.2                                  nexthop2 vrf ibackup ipv4 1.1.5.2   20 permit ipv4 10.2.1.0/24 any nexthop1 vrf inside2 ipv4 1.1.3.2                                  nexthop2 vrf ibackup ipv4 1.1.5.2   30 permit ipv4 any any   ! Static route for Outside to Inside traffic router static   address-family ipv4 unicast     151.0.1.0/24 ServiceApp 2     151.0.2.0/24 ServiceApp 4     151.0.3.0/24 ServiceApp 6   1:1 Warm Stand-by Redundancy In this mode, one VSM card acts as Primary and another as backup. Following are the properties of 1:1 Warm Stand-by redundancy. Easy to configure (Active and Stand-by locations are specified once) A single Public IP address pool is used Configuration is sent to both VSM cards NAT Application data traffic reaches to Active card only During switchover (usually takes few seconds), dynamic NAT entries get deleted (on failed VSM cards) and re-created on newly Active VSM card. Hence, there would be some traffic drop. Also, Private to Public IP/Port mapping gets changed during Switchover. When the previously Active VSM comes up, it stays as Stand-by and the traffic does not come back to it (until the Active VSM card fails) Each VSM can support full scale and throughput Following diagram explains the working of 1:1 Redundancy for NAT44. Please note that both Primary and Backup VSM cards share same ServiceApp interfaces and NAT44 configuration (including Public IP pool). Following tables shows how to configure 1:1 VSM redundancy. Configuration for 1:1 Warm Standby Redundancy   ! Configure preferred-active and preferred-standby VSM service cgn cgn1   service-location preferred-active 0/1/cpu0 preferred-standby 0/2/cpu0   service-type nat44 nat1     inside-vrf inside1       map address-pool 151.0.1.0/24       ! Remaining configurration       ...   To check the health of the VSM card and trigger redundancy switchover, in case any failure occurs, following HA-specific test can be enabled: Data Path Test - It checks if the packet path from CGv6 Application to Fabric works fine or not. If the test is enabled and any failure is detected by the test, the VSM card will be reloaded. NOTE: By default, these tests run in the back-ground however, no acton is taken in case failure is detected. Using the following CLI, the test is enabled. Configuration of HA  Data-Path tests ! To enable Data Path Test service-cgv6-ha location 0/1/cpu0 datapath-test     Back to top 20. NAT44 Licenses   In order to use NAT44 functionality on ASR9K, Licenses needs to be purchased. Depending on the number of Stateful Translations needed by the solution, required number of NAT44 specific licenses can be purchased. This fits nicely with "Pay as you grow" model as well.   Following licenses need to be purchased for NAT44 and other CGv6 applications.   License Part Number License Description Additional Comments A9K-VM-LIC Cisco ASR 9000 Application VM License For launching (activating) any Application VM, this license is needed. When CGN software is ordered, one free A9K-VM-LIC will be provided along with it. A9K-XLAT-LIC-5M Cisco ASR 9000 CGN / NAT44 License (1 per 5 million translations) For 60 Millions translations, 12 of these licenses will be needed. Same license is applicable for both ISM as well as VSM cards.     Details of the Licensing (how to install, etc.) can be obtained from ASR9K S/w License Operations White Paper.   Back to top 21. Summary of parameter values   In the following table, all the useful parameters are listed along with their default and valid range of values. You can consult this table to come up with the parameter value for your deployment case.     Parameter (CLI) Defined at what level Default Value Range of Values Supported from XR Release Additional Comments alg ActiveFTP per NAT44 instance Disabled Enable / Disable 5.1.1 Active FTP ALG alg pptpAlg per NAT44 instance Disabled Enable / Disable 5.1.1 PPTP ALG alg rtsp per NAT44 instance Disabled Enable / Disable 5.1.1 RTSP ALG bulk-port-alloc size per Inside VRF Disabled 16, 32, 64, 128, 256, 512, 1024, 2048, 4096 5.1.1   dynamic-port-range start per NAT44 instance 1024 1 - 65535 5.1.1 Starting port number (for Public IP address) to be used for Dynamic NAT entries. external-logging netflow v9 server path-mtu per Inside VRF 1500 bytes 100 - 2000 5.1.1   external-logging netflow v9 server refresh-rate per Inside VRF 500 packets 1 - 600 5.1.1 For sending NFv9 template external-logging netflow v9 server timeout per Inside VRF 30 seconds 1 - 3600 5.1.1 For sending NFv9 template external-logging syslog server path-mtu per Inside VRF 1500 bytes 100 - 2000 5.1.1   map ip one-to-one per Inside VRF Disabled 1 - 65535 5.1.1   portlimit per NAT44 instance 100 1 - 65535 5.1.1   protocol udp session initial timeout per NAT44 instance 30 sec 1 - 65535 5.1.1   protocol udp session active timeout per NAT44 instance 120 sec 1 - 65535 5.1.1   protocol tcp session initial timeout per NAT44 instance 120 sec 1 - 65535 5.1.1   protocol tcp session active timeout per NAT44 instance 1800 sec 1 - 65535 5.1.1   protocol tcp mss per Inside VRF   28 - 1500 bytes 5.1.1       Back to top 22. NAT44 Configuration / Feature Differences with ISM     Parameter / Criteria ISM Value VSM Value Additional Comments Number of ServiceApp Interfaces 244 256   Number of NAT44 Translations 20 Millions 60 Millions   End-point Dependent Filtering Not Supported Supported   TCP Sequence Check Not Supported Supported   CLI: map ip many-to-one Supported Not Supported Will be supported on VSM in future release Minimum number of Public IPv4 addresses in a Pool 4 (i.e., /30) 32 (i.e., /27)     Back to top 23. MTU and Fragmentation Considerations Cisco recommends keeping the default physical interface MTU size on the customer-facing interfaces in NAT44 deployment. Configure the MTU on physical VSM interfaces to 1518.   Specific Deployment Scenarios / Use-Cases   In this section, specific deployment scenarion / use-cases and presented which can be useful for some deployment.   Back to top 24. Sample Applications over NAT44   Following table lists some of the important / well-known applications and their working status over NAT44 (on VSM).     NOTE: - This list is not a complete/exhaustive list of applications that would work with NAT44. - It also does not mean that any application not listed here will not work with NAT44.   Application Details of Application (Protocol / Port) Working Status with NAT44 HTTP Typically HTTP server runs over TCP Port 80 Works with NAT44 E-mail Typically uses TCP, server ports: SMTP (25, 587), POP3 (110), IMAP (143) Works with NAT44 FTP Typically uses TCP port 20 (data) and port 21 (control) Works with NAT44 Telnet Typically uses TCP port 23 Works with NAT44 SSH Typically uses TCP port 22 Works with NAT44 VNC Typically uses TCP port 5900, 5800, 5500 Works with NAT44 VPN Client (SSL) Typically uses TCP Works with NAT44 VPN Client (PPTP+GRE) Typically uses TCP and GRE Works with NAT44 VPN Client (IPSec) Typically uses IPSec (via NAT-T) Works with NAT44 (as NAT-T uses UDP) Xbox Gaming   Works with NAT44     Back to top 25. Multiple Outside ServiceApps sharing same Outside VRF   It is very common to have default VRF as Outside VRF and when there are multiple Inside VRFs mapped to the same. In the following table, a sample ServiceApp and VRF configuration is shown. Please note that in this case, multiple Outside ServiceApps share share a comon Outside VRF.   Inside VRF Inside ServiceApp Outside VRF Outside ServiceApp ivrf1 ServiceApp 1 default or ovrf ServiceApp 2 ivrf2 ServiceApp 3 default or ovrf ServiceApp 4   To handle this scenario, we need to specify "outsideServiceApp" parameter configuration to indicate what is the Outside ServiceApp to be used with the mapping. Static route rule for specific Public IP address pool should guide the traffic to specific Outside ServiceApp. Only these configurations are shown below. Rest of the configuration remain the same.   Configuration for Shared / Common Outside VRF service cgn cgn1   service-type nat44 nat1     ! 1st Inside VRF     inside-vrf ivrf1       ! MAP command using outsideServiceApp       map [outside-vrf ovrf] outsideServiceApp ServiceApp2 address-pool 100.0.1.0/24     ! 2nd Inside VRF     inside-vrf ivrf2       ! MAP command using outsideServiceApp       map [outside-vrf ovrf] outsideServiceApp ServiceApp4 address-pool 100.0.2.0/24 router static [vrf ovrf] address-family ipv4 unicast ! Divert traffic to 1st Outside ServiceApp 100.0.1.0/24 ServiceApp 2     ! Divert traffic to 2nd Outside ServiceApp     100.0.2.0/24 ServiceApp 4     Back to top 26. NAT Bypass   There is an use-case where traffic selected traffic coming onto ASR9K via a Private side interface needs to go via NAT44 application, whereas remaining traffic can bypass NAT44 and should get forwarded via regular forwarding rule. This is called NAT bypass.   To implement NAT bypass, let's consider this situation: Private Traffic enters ASR9K via one VRF (say, RED) Selected traffic gets diverted to VSM card's Inside ServiceApp which is in different VRF (say, BLUE) and after translation comes out of Outside ServiceApp and eventually goes out of ASR9K via core-facing interface which is in RED VRF. Rest of the traffic (which bypasses NAT44) goes out of ASR9K via same core-facing i/f in RED VRF.   Please note that other than NAT bypass, there could be other deployment scenario where preserving VRF (across VSM) becomes a necessity. But, the solution would be same / very similar.   The above scenario can be achieved using VSM with some additional ABF and Static Route and appropriate VRF configuration.   Let us take a look at the following diagram as a sample scenario.     In the above diagram, traffic enters and exits out of ASR9K in RED VRF. Also, please note that Inside ServiceApp is in BLUE vrf whereas Outside ServiceApp is in RED VRF.   Following table shows the specific or additional configuration that is needed for appropriate traffic diversion.   Additional Configuration to support NAT Bypass   ! Define ACL to divert traffic to ServiceApp 1 (RED to BLUE VRF) ipv4 access-list cgn_abf     ! Divert traffic to ServiceApp 1 in BLUE VRF   10 permit ipv4 10.10.10.0/24 any nexthop1 vrf BLUE ipv4 1.1.1.2     20 permit ipv4 any any   ! Apply ABF on Ingress LC interface interface TenGigE 0/0/0/0.1   vrf RED   ipv4 address 10.10.10.1/24     ! Apply Ingress ABF   ipv4 access-group cgn_abf ingress   ! We need a special static route to divert O2I traffic (after reverse NAT) ! to the right interface router static     ! After reverse NAT, look-up is done in BLUE (Inside) VRF   vrf BLUE       address-family ipv4 unicast         ! Divert to TenGigE 0/0/0/0.1 interface       10.10.10.0/24 vrf RED 10.10.10.2       Back to top 27. MPLS (Layer 3) VPN and NAT Interworking   An Multiprotocol Label Switching (MPLS) Layer 3 Virtual Private Network (L3VPN) consists of a set of sites that are interconnected by means of an MPLS provider core network. At each customer site, one or more customer edge (CE) routers attach to one or more provider edge (PE) routers.   MPLS VPNs are easier to manage and expand than conventional VPNs. When a new site is added to an MPLS VPN, only the edge router of the service provider that provides services to the customer site needs to be updated. The components of the MPLS VPN are described as follows:   Provider (P) router — Router in the core of the provider network. PE routers run MPLS switching and do not attach VPN labels to routed packets. VPN labels are used to direct data packets to the correct private network or customer edge router. Provider Edge (PE) router — Router that attaches the VPN label to incoming packets based on the interface or subinterface on which they are received, and also attaches the MPLS core labels. A PE router attaches directly to a CE router. Customer edge (CE) router — Edge router on the network of the ISP that connects to the PE router on the network. A CE router must interface with a PE router.   Details on L3VPN can be obtained from ASR9K MPLS Layer 3 VPN Configuration Guide.   Service providers and enterprises that have network application services they want to offer or share with customers and partners will want to      minimize any connectivity burden placed on the user of the service. It is desirable, even mandatory, to extend the offering to as many potential users as needed to achieve the desired goals or return.   By deploying NAT within the common MPLS VPN infrastructure, communications service providers can relieve some of the connectivity burden on      customers and accelerate their ability to link more shared application services to more consumers of those services   There are two options for NAT deployment within the MPLS provider edge: Distributed with ingress NAT PEs Centralized with egress NAT PEs   Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs.   With this design, scalability is maintained to a large extent while performance is optimized by distributing the NAT function over many edge devices. Each NAT PE handles traffic for sites locally connected to that PE. NAT rules and access control lists or route maps control which packets require translation.     In the above case, at ingress PE router, customer private IP addresses are NAT'ed to public IP address by VSM card and then MPLS labels are added by it before the packet goes out of ASR9K. On the return path, MPLS label(s) are removed by MPLS-facing LC and IT packets are sent to VSM. VSM performs reverse NAT and sends the packet to LC which then sends it back to CE router.   Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs.   With this design, scalability is reduced to some degree since the central PE must maintain routes for all customer networks that access the shared service. The application performance requirements must also be considered so that the traffic does not overburden the router that must translate the IP addresses of the packets. Because NAT occurs centrally for all customers using this path, IP address pools can be shared; thus, the total number of subnets required is reduced.     In the above case, in the egress PE router, first MPLS labels are removed by ingress LC and the IP packet is sent to VSM card. Customer private IP addresses are NAT'ed to public IP address by VSM card and the packets are forwarded to shared service network. In the reverse path, VSM translates Public destination IP to private IP addresses and then adds MPLS labels before sending it to egress LC. Egress LC swaps one label and sends the packet out of ASR9K towards MPLS network.   NOTE: MPLS labels should be removed by LC before sending the packet to VSM as IP packet. However, VSM can add the required MPLS labels before sending the packet to egress LC.   For sample MPLS VPN configuration, please refer to "Configuration Example: MPLS VPN on XR" document.   Back to top 28. NAT Hairpining   Host behind a NAT device (on Private side) communicating with another host behind the same NAT device (on Private side) is called "NAT Hairpinning". Let us consider the following deployment use-case.   We have a server behind a NAT device which has a Public IP as well, created using Static Port Forwarding configuration. Now, this server can be accessed from hosts on public side of NAT device as well as Private side of the NAT device. When a host on Private side of the NAT device tries to communicate with the Server, it uses NAT Hairpinning. Following diagram depicts how NAT hairpinning works with NAT44 on VSM.     In this case, we first add a Static Port Forwarding entry for a service offered by the Server at 10.100.1.2:2055. Let's say, the Public IP pool configured is 200.2.0.0/24. NAT44 application allocates Public IP and port as 200.2.0.210:2055 (Please note that the port is preserved here). This Public IP and port is published / made available to the hosts. Let's the packet flow now when a host with Private IP 10.10.10.2 tries to access the service.   Step 1: Host sends a packet (with Source IP 10.10.10.2 and port 3000) to Server's Public IP and port and the packet reaches ASR9K on TenGigE 0/0/0/0.1 which is in VRF ivrf. Step 2: Because of Static Route configuration in VRF ivrf, the packet is forwarded to VSM card via (Inside) ServiceApp 1 interface. Step 3: Application treats this as I2O packet. A new dynamic NAT entry is created by NAT44 application and it translates source IP/Port from (Private) 10.10.10.2:3000 to (Public) 200.2.0.192:23516. Please note that the Destination IP address and port are not touched here. The packet is sent out and because of Static route definition for Public IP pool, the packet comes back to VSM card via (Outside) ServiceApp 2. Step 4: Application treats this as O2I packet. As there is already a (Static) NAT entry exists for Destination IP and port, NAT44 application translates Destination IP address/Port from (Public) 200.2.0.210:2055 to (Private) 10.100.1.2:2055. Please note that the Source IP address and port are not touched here. The packet is sent out and via regular forwarding rule, it reaches egress i/f TenGigE 0/0/0/0.2. Step 5: The packet is forwarded to the Server with Source IP/Port as 200.2.0.192:23516 and Destination IP/Port as 10.100.1.2:2055 and the server consumes it.   Response from the Server will take the reverse path and it will use Dynamic NAT rule in first reverse NAT and then Static NAT rule in second NAT processing, to reach the host.   In this case, Static Port Forwarding configuration is used for Server as an use-case. But, it can be another host as well for which there is no Static Port Forwarding configuration. Only important thing is sender needs to know the Public IP and port of the receiver.   Please note that there is no specific NAT44 configuration is needed for this. Regular / basic NAT44 configuration will work here. Hence, no configuration is provided here.   Back to top 29. Handling of Asymmetric I2O and O2I traffic with better performance   Unlike ISM card, we do NOT need to make any special configuration (like, dividing traffic into 2 sub-groups) for VSM. As long as I2O + O2I traffic do not exceed the capacity of the CGv6 Application instances, direction of traffic does not matter i.e., I2O and O2I traffic do not need to be equally balanced.     Validation of Configuration   Once NAT44 on VSM is configured, we need different ways (mainly, via "show" CLIs) to validate those. In this section, important NAT44 related CLIs are listed and also explained how those can be used for validation.   Back to top 30. show run   As a first step, different "show run" output should be verified to ensure that configuration commands are rightly applied. Following commands can be used.   'show run' commands Usage of it show running-configuration  or show run - Displays all the configuration on ASR9K. - If you have less configuration, this is better way to get all NAT44 configuration at one shot. - Otherwise (if you have lot of configuration, this may not be the best way to get all NAT44 configuration show run service cgn * - Displays only CGN related information show run interface service* - Displays all ServiceInfra and ServiceApp interface configuration - Also includes Service-Mgmt and Service-Engine interface configuration (which can be ignored) show run router static - Displays all static route information.     Back to top 31. show virtual-service list   This CLI will list down the services that are installed and running on ASR9K chassis.     Sample "show virtual-service list" CLI Output     RP/0/RSP1/CPU0:SERVICE_PE2_1#show virtual-service list Tue Dec 24 15:35:50.161 UTC   Virtual Service List:   Name                   Status            Package Name             Node Name    ______________________________________________________________________________ cgn1                   Activated         vsmcgv6.ova               0/1/CPU0     RP/0/RSP1/CPU0:SERVICE_PE2_1#     Back to top 32. show virtual-service detail   This CLI shows detail information about the CGv6 service VM (mainly, what h/w resources are given to it, etc.), as shown below.     Sample "show virtual-service detail name <>" Output   RP/0/RSP1/CPU0:SERVICE_PE2_1#show virtual-service detail name cgn1 Tue Dec 24 15:38:20.951 UTC   Virtual Service cgn1 Detail   State                   : Activated   Node name               : 0/1/CPU0   Node status             : Install Mgr Ready, SDR Mgr Ready   Package information     Name                  : vsmcgv6_ivybridge.ova     Path                  : disk0:/vsmcgv6.ova     Application       Name                : CGv6       Installed version   : 1.0       Description         : Carrier Grade NAT     Signing       Key type            : Unknown Package       Method              : SHA1     Licensing       Name                : Not Available       Version             : Not Available     Activated profile name  : None     Resource reservation    Disk   : 10000MB    Memory : 32768MB    CPU    : 75 (system CPU %)    VCPU   : 60     Attached devices   #             Type Name      Alias   1         Watchdog None      None   2            CDROM hdc       ide0-1-0   3              HDD hda       DD_10GB_UM_local   4       Serial/aux None      serial1   5     Serial/shell None      serial0   6              NIC net1      net1   7              NIC net1      net1   8              NIC net1      net1   9              NIC net1      net1   10             NIC net1      net1   11             NIC net1      net1   12             NIC net1      net1   13             NIC net1      net1   14             NIC net1      net1   15             NIC net1      net1   16             NIC net1      net1   17             NIC net1      net1   Network interfaces:     Name     TenGigE0/1/1/0     TenGigE0/1/1/1     TenGigE0/1/1/2     TenGigE0/1/1/3     TenGigE0/1/1/4     TenGigE0/1/1/5     TenGigE0/1/1/6     TenGigE0/1/1/7     TenGigE0/1/1/8     TenGigE0/1/1/9     TenGigE0/1/1/10     TenGigE0/1/1/11     Resource admission (without profile)     Disk space            : 10000MB     Memory                : 32768MB     CPU                   : 100% system CPU     VCPUs                 : 60 (sockets:3 cores:10 threads:2)     RP/0/RSP1/CPU0:SERVICE_PE2_1#       Back to top 33. show cgn nat44 <> statistics   This is a very important CLI related to NAT44. A sample output of the CLI is shown below.   Sample "show cgn nat44 <> statistics" CLI Output     RP/0/RSP0/CPU0:#Show cgn nat44 nat statistics Fri Sep 16 23:12:40.018 UTC   Statistics summary of NAT44 instance: 'nat' Number of active translations: 92400 Number of sessions: 30706 Translations create rate: 542 Translations delete rate: 1206 Inside to outside forward rate: 147827 Outside to inside forward rate: 204847 Inside to outside drops port limit exceeded: 10167470 Inside to outside drops system limit reached: 0 Inside to outside drops resource depletion: 0 No translation entry drops: 9453431 PPTP active tunnels: 3 PPTP active channels: 2 PPTP ctrl message drops: 1246 Number of subscribers: 0 Drops due to session db limit exceeded: 0 Pool address totally free: 499 Pool address used: 1549 Pool address usage: -------------------------------------------------   External Address       Ports Used -------------------------------------------------   200.0.9.0           79   200.0.9.2           16   200.0.9.8           57   200.0.9.12         36   200.0.9.16         78   200.0.9.20         52   200.0.9.24         65   ...   200.0.10.1        2   200.0.10.24      1   200.0.10.26    1 -------------------------------------------------     Following table explains the impotant fields from the above CLI and how it can be used for validation.   Please note that the values displayed by the CLI is valid at that instant for that particular VSM card. It may change at a later point of time.   Field Name Description Additional Comments for Validation Number of active translations Number of NAT DB (Static + Dynamic) Entries Monitoring this over a period of time will provide the NAT44 translation scale used in the VSM card. Number of sessions Number of Session DB (Static + Dynamic) Entries Monitoring this over a period of time will provide the NAT44 session  scale used in the VSM card. Translations create rate Number of NAT44 translations created in last 1 second Monitoring this over a period of time will provide the NAT44 session  setup rate on the VSM card. Translations delete rate Number of NAT44 translations deleted  in last 1 second Monitoring this over a period of time will provide the NAT44 session  teardown rate on the VSM card. Inside to outside forward rate Number of packets gone through NAT44 translation in I2O direction in last 1 second Monitoring this over a period of time will provide the NAT44 packet processing rate on VSM card (PPS) in I2O direction Outside to inside forward rate Number of packets gone through NAT44 translation in O2I direction in last 1 second Monitoring this over a period of time will provide the NAT44 packet processing rate on VSM card (PPS) in O2I direction Inside to outside drops port limit exceeded Number of I2O packets dropped because no NAT entries cannot be created for the specific users which has reached port limit. If you have configured port limit value too less, you may hit this errors very often. You may need to consider to increase the port limit configuration in that case. Inside to outside drops system limit reached Number of I2O packets dropped because NAT DB or User DB is full. In case you are hitting this very often, you may need to check if you're hitting the scale limit and if so, you may need to go for another VSM card. Inside to outside drops resource depletion Number of I2O packets dropped because Layer 4 port could not be allocated.   No translation entry drops Number of O2I packets dropped because there was no NAT entry existing.   Number of subscribers Number of NAT Users   Drops due to session db limit exceeded Number of I2O packets dropped because Session DB is full.       Back to top 34. show cgn nat44 <> inside-translation   This command is helpful in seeing the list of all Outside / Public IP address + Port mapping for a specific Inside / Private IP address (and a Port range).   Following is a sample output of the command.   Sample "show cgn nat44 <> inside-translation" CLI Output   RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 inside-translation protocol udp inside-vrf red                         inside-address 10.2.5.134 port start 1 end 4580 Wed May 15 14:25:09.713 UTC Inside-translation details --------------------------- NAT44 instance : nat1 Inside-VRF     : red --------------------------------------------------------------------------------------------    Outside         Protocol  Inside       Outside       Translation   Inside      Outside    Address                   Source       Source        Type          to          to                              Port         Port                        Outside     Inside                                                                       Packets     Packets --------------------------------------------------------------------------------------------   100.200.2.75       udp     4500         65024         dynamic       9236        2213   100.200.2.75       udp     4501         65153         dynamic       2236        0   100.200.2.75       udp     4502         65152         dynamic       4232        1218   100.200.2.75       udp     4503         65155         dynamic       2236        7128   100.200.2.75       udp     4504         65154         dynamic       2235        1821   100.200.2.75       udp     4505         65157         dynamic       5236        3129   100.200.2.75       udp     4506         65156         dynamic       2238        8172   100.200.2.75       udp     4507         65159         dynamic       7238        2189   100.200.2.75       udp     4508         65158         dynamic       2236        3119   ...   100.200.2.75       udp     4576         65226         dynamic       1239        2178   100.200.2.75       udp     4577         65229         dynamic       8235        4171   100.200.2.75       udp     4578         65228         dynamic       2235        0   100.200.2.75       udp     4579         65231         dynamic       4230        4127   100.200.2.75       udp     4580         65230         dynamic       2235        0 RP/0/RSP0/CPU0:#   Above table shows all Outside IP (100.200.2.75) and Port combination for Inside IP (10.2.5.134) and Port Range (1 through 4580).     NOTE: - As one Inside address can only map to one Outside address, there are not multiple Outside addresses in 1st column. - The number of entries in this table should not exceed Port Limit value.   Back to top 35. show cgn nat44 <> outside-translation   This command is helpful in seeing the list of all Inside / Private IP address + Port mapping for a specific Outside / Public IP address (and a Port range).   Following is a sample output of the command.   Sample "show cgn nat44 <> outside-translation" CLI Output   RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 outside-translation protocol udp                         outside-address 100.200.2.75 port start 1 end 65535 Wed May 15 14:17:44.640 UTC Outside-translation details --------------------------- NAT44 instance : nat1 Outside-VRF    : default --------------------------------------------------------------------------------------------    Inside          Protocol  Outside      Inside         Translation  Inside      Outside    Address                   Destination  Destination    Type         to          to                              Port         Port                        Outside     Inside                                                                       Packets     Packets --------------------------------------------------------------------------------------------   10.2.5.134         udp     65024        4500          dynamic       4236        1216   10.2.5.134         udp     65025        4885          dynamic       9233        5161   10.2.5.134         udp     65026        4886          dynamic       4244        3491   10.2.5.134         udp     65027        4887          dynamic       1281        6018   10.2.5.134         udp     65028        4888          dynamic       2237        4187   ...   10.2.5.142         udp     65532        4882          dynamic       6261        3178   10.2.5.142         udp     65533        4881          dynamic       7282        0   10.2.5.142         udp     65534        4884          dynamic       1211        4178   10.2.5.143         udp     65535        4883          dynamic       1261        5189 RP/0/RSP0/CPU0:#   Above table shows all Inside IPs (10.2.5.134, 10.2.5.142) and Port combination for Inside IP (100.200.2.75) and Port Range (1 through 65535). In this case, there are multiple Inside IP addresses are mapped to one Public IP address.   Monitoring CGN Using SNMP The attached CGN_MIB_Parameters.xlsx sheet lists the parameters supported in the CGN implementation on VSM/ISM.   Useful Deployment Tips   Go through this Deployment Guide to clearly understand                                      how different feature works what parameters are configurable and need to be considered for specific deployment what are the different deployment use cases. Go through "CGv6 on VSM: Features Supported Across IOS-XR Release" page  to exactly know from which release specific features that you're  interested in are supported. This will help you in selecting the right  IOS-XR release.                     Select the configuration parameter values wisely, considering                                  pros and cons, best-case and worst-case scenarios side effects Draw network diagram for deployment use-case including interface names (with VRF and IP address/subnet) and review for correctness. Try to find out what type of application traffic (VPN, Games, SIP, Video, etc.) may undergo NAT44 and whether there could be issues or not (not all applications are NAT friendly)                                     If possible, try it in a lab setup, before actual deployment Estimate traffic throughput and nature of it (burstiness, average, etc.), scale of sessions/users and accordingly use correct number of VSM cards and related configuration. If possible, deploy in phases. First, send selected traffic (you can use ABF) through VSM and see if there are any issues. If not, you can send more and more traffic through VSM.     Related Information CGv6 on VSM: Features Supported Across IOS-XR Releases CGv6 on ISM: Features Supported Across IOS-XR Releases CGv6 on ISM: CGN / NAT44 Deployment Guide ASR9K Configuration Guides ASR9K Command Reference Guides ASR9K BNG Deployment Guide IANA Service Name and Transport Protocol Port Port Number Registry Configuration Example: MPLS VPN on IOS-XR Cisco ASR9000 S/w License Operations White Paper   - 0 - Back to top ... View more

Introduction Supported CGv6 features on VSM Q. What all CGv6 features are supported on VSM and from which IOS-XR releases they are available ? Q. What are the supported scale numbers related to CGv6 features on VSM ? Q. What is the roadmap of CGv6 features on VSM? Q. When a specific feature will be supported on VSM? Related Information Introduction   This document answers questions related to CGv6 Features supported on VSM card across different IOS-XR Releases. This document tries to list all the CGv6 on VSM related features in one place in a consolidated manner to help CGv6 on VSM customers. It does not intend to replace official IOS-XR release notes. For official release note document, please select "Release Notes for Cisco ASR 9000 Series Aggregation Services Routers for Cisco IOS-XR Software Release X.Y.Z" link for appropriate IOS-XR release.     Supported CGv6 features on VSM Back to top 1. Q. What all CGv6 features are supported on VSM and from which IOS-XR releases they are available ? A. Following table captures different CGv6 features supported on VSM and also indicates from which IOS-XR release they are supported.   Feature Group Feature Description Supported on VSM from IOS-XR Release Additional Comments Generic (Common to All Applications) Multiple CGv6 Applications on same VSM card 5.1.1 At present only 1 Application (NAT44) is supported, but the Architecture supports multiple CGv6 Applications (will be added in future release). NAT44 Generic / Base support 5.1.1     Active FTP ALG 5.1.1 Configurable per NAT44 instance.   RTSP ALG 5.1.1 Configurable per NAT44 instance.   PPTP ALG 5.1.1 Configurable per NAT44 instance.   Port limit 5.1.1 Configurable per NAT44 instance.   Netflow version 9 (NFv9) logging 5.1.1 Configurable per inside-VRF.   Syslog logging 5.1.1 Configurable per inside-VRF.   Bulk Port Allocation (BPA) 5.1.1 Configurable per inside-VRF.   Destination Based Logging (DBL) 5.1.1 Configurable per inside-VRF.   NFv9 logging with BPA 5.1.1 Configurable per inside-VRF.   NFv9 logging with DBL 5.1.1 Configurable per inside-VRF   Syslog loggging with BPA 5.1.1 Configurable per inside-VRF   Syslog logging with DBL 5.1.1 Configurable per inside-VRF   One-to-One mapping 5.1.1 Configurable per inside-VRF. One Public IP address is associated with maximum of one Private IP address. Multiple Private IP addresses are not mapped to single Public IP address, if this option is enabled.   OutsideServiceApp mapping for inside-VRF 5.1.1 Configurable per inside-VRF. It is a MUST when you have multiple Outside ServiceApp interfaces inside same / single Outside VRF.   Static Port Forwarding 5.1.1 Configurable per inside-VRF. User can specify a particular Inside/Private IP address, IP protocol and Port number which will be statically mapped to a Public IP address and Port, selected by the CGv6 Application. Usually, port number is preserved, unless there is a conflict (Port number is already in use).   Endpoint Dependent Filtering (EDF) 5.1.1 Configurable per inside-VRF. Endpoint Independent Filtering (EIF) is also supported (default behaviour) along with this.   TCP Sequence Check 5.1.1 Configurable per inside-VRF. . Back to top 2. Q. What are the supported scale numbers related to CGv6 features on VSM ? A. Following table captures different supported scale numbers related to CGv6 features on VSM.   Parameter Name Parameter Value per VSM Parameter Value per ASR9K Chassis Additional Comments Number of CGN/CGv6 Instances 1 2 (5.1.1) Under one CGN/CGv6 instance, multiple CGv6 Applications (like, NAT44, DS-Lite, NAT64) are supported. Only NAT44 is supported now. Others will be supported in future. In future, we will support more number of CGv6 instances per ASR9K chassis Number of ServiceInfra Interfaces 1 2 (5.1.1) ServiceInfra interface is used to send Management / Control traffic related to CGv6. Hence, only 1 ServiceInfra interface per VSM card is needed. In future, we will support more number of ServiceInfra interfaces per ASR9K chassis Number of ServiceApp interfaces 256 256 (5.1.1) For running a CGv6 Application instance (like, NAT44, DS-Lite, etc.), you need a pair of ServiceApp interfaces. Hence, you can run at the most 128 CGv6 Application instances (of different types) per ASR9K. Number of NAT44 instances 1 2 (5.1.1) Within 1 NAT44 instance, multiple Inside VRFs are possible. Several parameters can be configured on per Inside VRF. In future, we will support more number of NAT44 instances per ASR9K chassis Number of Stateful NAT Translations 60 Millions 120 Millions (5.1.1) NAT DB is shared across NAT44, Stateful NAT64 and DS-Lite Applications. In future, we will support more number NAT translations per ASR9K chassis Number of NAT Sessions 60 Millions 120 Millions (5.1.1) NAT Session DB is shared across NAT44, Stateful NAT64 and DS-Lite Applications. In future, we will support more number of NAT sessions per ASR9K chassis Number of NAT Users 1 Million 2 Millions (5.1.1) NAT User DB is shared across NAT44, Stateful NAT64 and DS-Lite Applications. In future, we will support more number of NAT users per ASR9K chassis Number of Static Port Forwarding Entries 6,000 12,000 (5.1.1) In future, we will support more number of Static Port Forwarding entries per ASR9K chassis Number of Public IPv4 addresses 65536 or /16     Number of VRFs 128 (Inside) + 128 (Outside) > 128 (Inside) + > 128 (Outside ), as per ASR9K VRF scale - As we support maximum of 256 ServiceApp interfaces per VSM card, at the most, we need to have 128 Inside and 128 Outside VRF. - You can have less number of Outside VRFs as well. - Please note that ASR9K supports higher number of VRFs in the chassis. This is only with respect to CGv6 Application on VSM. Number of IPv4 Prefixes Same as supported by any Typhoon-based LC.   Related to forwarding. Number of IPv6 Prefixes Same as supported by any Typhoon-based LC.   Related to forwarding. Number of MPLS labels Same as supported by any Typhoon-based LC.   Related to forwarding.     Back to top 3. Q. What is the roadmap of CGv6 features on VSM? A. Please contact your Cisco Account Team or ASR9K Product Marketing team (whichever you have access to).   Back to top 4. Q. When a specific feature will be supported on VSM? A. Please contact your Cisco Account Team or ASR9K Product Marketing team (whichever you have access to).       Note:     Related Information ASR9000 Release Notes ASR9000 Configuration Guides ASR9000 Command Reference Guides CGv6 on VSM: Software Installation / Upgrade Guide CGv6 on VSM: CGN / NAT44 Deployment Guide CGv6 on ISM: Features Supported Across IOS-XR Releases ASR9000/XR Understanding Route Scale Back to top ... View more

Introduction Overview of CGv6 on ISM Software Different Installation / Upgrade Scenarios Information about Linux Install-Kit for each IOS-XR Release Installation / Upgrade of Different Images Installation of IOS-XR Images Upgrade FPGA Images Installation of Application (Linux) Image Trouble-shooting Issues related to Installation / Upgrade Tips / Things to Remember for Installation / Upgrade How do I know CGv6 Application S/w Version on ISM ? How can I know CGv6 Application Software Version from the Install-Kit without installing it ? How do I find out MD5 Checksum of a Linux Install-kit ? Is there any command to check the Status of Installation while it is going on ? What do I do when installation of Install-Kit failed with "vkg_get_services_role UNDEFINED role" error ? Is there any sample Install-Kit Download log output for successful scenario ? Related Information Introduction This document talks about how to troubleshoot issues related to installation and upgrade of CGv6 software on ISM service card. It contains installation / upgrade steps to create the context and not to replace the official guide. For official CGv6 on ISM installation guide, please select "Cisco ASR 9000 Series Aggregation Router CGv6 Configuration Guide" for the specific IOS-XR release and use the appropriate section from ISM specific guide. Also, it does not cover ISM hardware installation. Overview of CGv6 on ISM Software ASR9K chassis can have different types of cards, like - Route Processors (RP) or Route Switch Processor (RSP) Line Card (LC) Service Card (SC) like, Integrated Services Module (ISM) Following diagram shows what type of Software runs on which cards. As shown above, ISM card runs 2 types of Software - IOS-XR and Linux. CGv6 software functionality is distrubuted across all types of cards (RP, ISM as well as LC) as well as both types of Software (IOS-XR and Linux). Following images are mandatory to install on ASR9K chassis to avail CGv6 functionality. IOS-XR Images asr9k-mini-px.vm or asr9k-mini-px.pie (Base IOS-XR Image) asr9k-services-px.pie (CGv6 Services Pie) asr9k-fpd-px.pie (FPGA Image IOS-XR Pie) Linux Image asr9k-ism-cgv6-install-kit-<version>.sh (Linux Install-Kit binary, within a shell script) NOTES: - For IOS-XR, bug fixes are available as SMU (Software Maintenance Update) as well. - For Linux images, there is NO SMU available. A new version of Install-Kit needs to be installed. - Size of Linux Install-Kit is approximately 375 MB. Back to top 1. Different Installation / Upgrade Scenarios Following table captures different Installation / Upgrade Scenarios and which software needs to be upgraded. Scenario Software to be Installed / Upgraded (in what sequence) Additional Comments Install (fresh) CGv6 software on ISM on existing IOS-XR release image Assumptions: - Services pie is not yet installed - Role 'cgn' is not yet configured Steps: - Install Services pie - Configure role as 'cgn' - Upgrade FPGA images (if needed) - Install Linux Install-Kit In order to install Linux CGv6 Install-Kit, role of ISM card needs to be set to 'cgn' first which can be done only when Services pie is installed. Hence, please install IOS-XR binaries first and then Linux Install-Kit. Upgrade CGv6 software on ISM to a new IOS-XR release Assumptions: - Services pie is already installed (with existing IOS-XR s/w) - Role 'cgn' is already configured (with existing IOS-XR s/w) - Old Linux Install-Kit is installed Steps: - Install new Linux Install-Kit - Upgrade IOS-XR images - Upgrade Services pie (with new IOS-XR version) - Upgrade FPGA images (with new IOS-XR version), if needed. As 'role' is already configured as 'cgn', we can go ahead and install new Linux Install-Kit in this case. Please install Linux Install-Kit first and then IOS-XR binaries. IOS-XR bug fix / enhancement feature - Install IOS-XR SMU Some bug fix may need XR SMU as well as Linux Install-kit installation. In that case, please install Linux Install-Kit first and then IOS-XR SMUs. Linux bug fix / enhancement feature - Install Linux Install-Kit Some bug fix may need XR SMU as well as Linux Install-kit installation. In that case, please install Linux Install-Kit first and then IOS-XR SMUs. Back to top 2. Information about Linux Install-Kit for each IOS-XR Release CGv6 Application Image is delivered via Linux Install-Kit binaries. Following table lists Linux Install-Kit (latest) for each IOS-XR Release. Appropriate Linux Install-Kit can be downloaded from the "download link" shown in the table. Download Linux Install-Kit needs to be saved on RP disk (prior to installation). Please use the compatible and preferably latest Linux Install-Kit binary corresponding to the IOS-XR image. Otherwise, CGv6 Application software may not work or work as expected. IOS-XR Version Linux Install-Kit Version (with Download Link) Install-Kit Filename, Date of Posting, MD5 Checksum CGv6 Application S/w Version Additional Comments 5.3.2 5.3.2.01 asr9k-ism-cgv6-install-kit.5.3.2.01.sh - Posted Sep 4, 2015 - md5 b46991f9c7eec9e2430e7a4cbcc49fc6 5.3.2.01 5.3.1 5.3.1.02 asr9k-ism-cgv6-install-kit-5.3.1.02.sh - Posted 04-AUG-2015 - MD5 c0fb4f778d2c5522fa84d0e31e4f64f0 5.3.1.2 5.3.0 5.3.0.00 asr9k-ism-cgv6-install-kit-5.3.0.00.sh - MD5 5b4a88d0b6745ec8af18269ebdf9efd0 - Posted 23-JAN-2015 2.3.0.0 5.2.4 5.2.4.02 asr9k-ism-cgv6-install-kit-5.2.4.02.sh - MD5 54ff5617430c4c056dcaabf10bc524bf - Posted 29-JUL-2015 2.2.4.2 5.2.2 5.2.2.00 asr9k-ism-cgv6-install-kit-5.2.2.00.sh - Posted 28-OCT-2014 - MD5 a332b242af2e3babd648bfe1a3cd35d4 2.2.2.0 5.2.0 5.2.0.00 asr9k-ism-cgv6-install-kit-5.2.0.00.sh - MD5 ea3e6b55ee57d37ab5cb9406ac7f9d29 - Posted 05-JUL-2014 2.2.2.0 5.1.3 5.1.3.00 asr9k-ism-cgv6-install-kit-5.1.3.00.sh - MD5 b3735be3d31caf65577cee203587a3fb - Posted 23-SEP-2014 2.1.3.0 5.1.2 5.1.2.00 asr9k-ism-cgv6-install-kit-5.1.2.00.sh - MD5 766f4bd42ae127a7a5423d42a98afb09 - Posted May 07 2014 2.1.2.0 5.1.1 5.1.1.00 asr9k-ism-cgv6-install-kit-5.1.1.00.sh - Posted on: Feb 5, 2014 - md5: f038d4f73ea00deed15edbedae2d0acb 2.1.1.0 5.1.0 5.1.0.01 asr9k-ism-cgv6-install-kit-5.1.0.01.sh - Posted on: Oct 17, 2013 - md5: 105fb985fd533f394a1ca8119aa55801 2.1.0.1 4.3.4 4.3.4.00 asr9k-ism-cgv6-install-kit-4.3.4.00.sh - Posted on: Dec 24, 2013 - md5: 34c204517ffc835ec0aa8feb7c53d901 1.1.4.0 4.3.2 4.3.2.03 asr9k-ism-cgv6-install-kit-4.3.2.03.sh - Posted on: Dec 20, 2013 - md5: 8cc2f98799652e0a4e56331fd56c9f87 1.1.2.3 4.3.1 4.3.1.01 asr9k-ism-cgv6-install-kit-4.3.1.01.sh - Posted on: Oct 4, 2013 - md5: 63ef3b9fea605e3e89d57e15f307a5e7 1.1.1.1 4.3.0 4.3.0.01 asr9k-ism-cgv6-install-kit-4.3.0.01.sh - Posted on: Dec 14, 2012 1.1.0.1 4.2.3 4.2.3.01 asr9k-ism-cgv6-install-kit-4.2.3.01.sh - Posted on: Apr 30, 2013 1.0.3.1 NOTE: While downloading Linux Install-Kit, please make sure - you are downloading the right binary from right location - MD5 Checksum and (approximate) size matches after download Installation / Upgrade of Different Images Back to top 3. Installation of IOS-XR Images IOS-XR Images can be downloaded from Download Software Center. Please follow the standard procedure to install / upgrade IOS-XR images. As an example, following steps can be followed to install CGv6 Services pie. RP/0/RSP0/CPU0#admin (admin)#install add tftp://<tftp_addr>/<image_location>/asr9k-services-px.pie         synchronous activate . . . . . . . . . . . (admin)#exit RP/0/RSP0/CPU0# NOTES: - "install add ..." command needs to be executed from "admin" mode - "add" and "activate" commands are to be preferably given together - Pie can first be copied to RP disk and then installed from RP disk as well Immediately after CGv6 Services is installed, please configure the "role" as "CGN" for all ISM locations, using "hw-module service cgn location <>", as shown below. RP/0/RSP0/CPU0#config (config)#hw-module service cgn location <ism_node_location> (config)#commit (config)#exit Back to top 4. Upgrade FPGA Images A very important aspect is to upgrade the FPGA images (specifically on ISM). At times, FPGA images may get updated because of some bug fix. If the FPGA is not updated, certain functionality may not work. Hence, please upgrade FPGA images as well before Linux Install-Kit is installed. As a first step, please capture output of  "show hw-module fpd location <ISM_location>" command (in "admin" mode) for all ISM cards. The output will show like the following. RP/0/RSP0/CPU0:CRANE#admin show hw-module fpd location 0/0/CPU0 ===================================== ====================================================================                                                      Existing Field Programmable Devices                                       ====================================================================                                         HW                                                 Current SW Upg/ Location     Card Type                Version       Description        Type Subtype Inst   Version   Dng? ============ ======================== ======= ======================== ==== ======= ==== =========== ==== 0/0/CPU0     A9K-ISM-100                0.20  Amistad LC6                lc   fpga1   0       0.33     No                                         1.0   Can Bus Ctrl (CBC) LC6     lc   cbc     0      18.06     Yes                                         0.19  CPUCtrl LC6                lc   cpld1   0       0.01     No                                         0.20  ROMMONB LC6                lc   rommon  0       1.02     No                                         0.19  ISM BIOS                   lc   fpga7   0       0.17     No                                         0.19  ISM WESSON CPLD            lc   cpld3   0       0.10     Yes                                         0.1   Maintenance LC6            lc   fpga2   0       2.13     No -------------------------------------------------------------------------------- 0/0/CPU0     A9K-ISM-100                0.20  Amistad LC6                lc   fpga1   1       0.33     No -------------------------------------------------------------------------------- NOTES: 1.  One or more FPD needs an upgrade or a downgrade.  This can be accomplished     using the "admin> upgrade hw-module fpd <fpd> location <loc>" CLI. RP/0/RSP0/CPU0:CRANE# "S/w Upg / Dng" column will indicate if any FPGA would need upgrade ("Yes") or not ("No"). For those FPGAs, issue "upgrade hw-module fpd <Subtype> location <ISM_location>" command to upgrade the image(s). Once the FPGAs are upgraded, please ensure that the ISM card is reloaded and the FPGA version shows correctly (via "show hw-module fpd location <ISM_location>" command, in "admin" mode). NOTE: You can enable auto FPD upgrade by configuring "fpd auto-upgrade" command in admin config mode (available from release 4.2.3 onwards). Back to top 5. Installation of Application (Linux) Image As mentioned earlier, as a first step, Linux Install-Kit needs to be copied to RP disk, as shown below. RP/0/RSP0/CPU0:CRANE#copy t ftp://200.123.114.26/auto/tftp-blr-users/foobar/                      asr9k-ism-cgv6-install-kit.sh disk1:/ Destination filename [/disk1://asr9k-ism-cgv6-install-kit.sh]? Copy : Destination exists, overwrite ?[confirm] Accessing t ftp://200.123.114.26/auto/tftp-blr-users/foobar/asr9k-ism-cgv6-install-kit.sh CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC 393290312 bytes copied in    816 sec (   481946)bytes/sec RP/0/RSP0/CPU0:CRANE# NOTE: - You do NOT need to remove any XR (say, CGv6) configuration before installing the Linux Install-Kit. Once the installation is over, existing XR configuration will automatically be applied. Once, the Install-Kit is copied into RP disk, "download install-image <> to <ISM_location>" command can be issued. Starting set of logs are shown below. RP/0/RSP0/CPU0:CRANE(admin)#download install-image disk1://asr9k-ism-cgv6-install-kit.sh                             to 0/0/CPU0 RP/0/RSP0/CPU0:Nov 28 00:15:47.185 : sim_download_recovery_image[65851]: %PLATFORM-SIM_IPC-6-SW_INFO : You are going to install new application on LC. Card will be reloaded automatically once installation is completed. Do you want to proceed Installation?[confirm]RP/0/RSP0/CPU0:Nov 28 00:15:49.906 : sim_download_recovery_image[65851]: %PLATFORM-SIM_IPC-6-SW_INFO : main Installation is in progress, Please do not reload the LC  LC/0/0/CPU0:Nov 28 00:15:49.914 : avsm_se_sm[125]: %PLATFORM-SIM_SE-5-HA_INFO : ism_service_switchover: OFFLINE msg to se controller success LC/0/0/CPU0:Nov 28 00:15:49.926 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in shutting down state. NOTE: - It may take approximately 15-20 minutes for the installation to be completed. So, please be patient. On successful installation, ISM card will be reloaded. Once it comes up, Service Engine will go into "ready" state. Following logs will indicate the same. LC/0/0/CPU0:Nov 27 10:25:21.033 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in ready state. LC/0/0/CPU0:Nov 27 10:25:21.034 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Received SE Operation Ready Notification..... LC/0/0/CPU0:Nov 27 10:25:21.037 : avsm_se_sm[125]: %PLATFORM-SIM_SE-5-SW_INFO : Service Engine heartbeat started on A9K-ISM-100 If you issue "show platform" command at this time, you will notice SE will be in SEOS-READY state, as shown below. RP/0/RSP0/CPU0:CRANE(admin)#show plat Node            Type                      State            Config State ----------------------------------------------------------------------------- 0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON 0/FT0/SP        FAN TRAY                  READY            0/FT1/SP        FAN TRAY                  READY            0/0/CPU0        A9K-ISM-100(LCP)          IOS XR RUN       PWR,NSHUT,MON 0/0/CPU1        A9K-ISM-100(SE)           SEOS-READY       0/1/CPU0        A9K-2T20GE-L              IOS XR RUN       PWR,NSHUT,MON 0/PM0/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON 0/PM1/SP        A9K-3KW-AC                FAILED           PWR,NSHUT,MON RP/0/RSP0/CPU0:CRANE(admin)# If ServiceInfra interface was already configured with IP address, Service Engine will get into app_ready state, as shown by the logs below. LC/0/0/CPU0:Nov 28 00:34:11.262 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_7 (CPU 22): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:13.106 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_7 (CPU 22): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:23.269 : avsm_agent[124]: %ISM_SIA-6-INFO : Sent app_ready signal  LC/0/0/CPU0:Nov 28 00:34:25.031 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in app_ready state. LC/0/0/CPU0:Nov 28 00:34:27.200 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 0, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.171 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 0, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.185 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.189 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.191 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  If you issue "show platform" command at this time, you will notice SE will be in APP-READY state, as shown below. RP/0/RSP0/CPU0:CRANE(admin)#show plat Node            Type                      State            Config State ----------------------------------------------------------------------------- 0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON 0/FT0/SP        FAN TRAY                  READY            0/FT1/SP        FAN TRAY                  READY            0/0/CPU0        A9K-ISM-100(LCP)          IOS XR RUN       PWR,NSHUT,MON 0/0/CPU1        A9K-ISM-100(SE)           APP-READY       0/1/CPU0        A9K-2T20GE-L              IOS XR RUN       PWR,NSHUT,MON 0/PM0/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON 0/PM1/SP        A9K-3KW-AC                FAILED           PWR,NSHUT,MON RP/0/RSP0/CPU0:CRANE(admin)# Now, the installation of Linux Install-Kit is complete and the card is ready to use. Trouble-shooting Issues related to Installation / Upgrade Back to top 6. Tips / Things to Remember for Installation / Upgrade Following are different things to remember during Installation / Upgrade: Once CGv6 Services Pie is installed, IOS-XR software on all ISM cards is upgraded. To upgrade Linux Install-Kit, "download install-image ..." CLI MUST be executed once for each ISM location. When both IOS-XR and Linux software needs to be upgraded, upgrade Linux Install-Kit first and then IOS-XR software. Make sure that you are using right (matching with IOS-XR image) and uncorrupted (valid MD5 checksum and size) Linux Install-Kit. Role MUST be configured as "cgn" before executing "download install-image ..." CLI. Installation of Linux Install-Kit takes approximately 15-20 minutes. So, please be patient. Do NOT forget to upgrade FPGAs (specifically on ISM). Software installation / upgrade will cause some traffic outage. Hence, it is better to do it in a Maintenance Window (MW) or off-hours. Back to top 7. How do I know CGv6 Application S/w Version on ISM ? Please issue "show platform summary location <>" IOS-XR command for the location where ISM is located (as shown below). If there are multiple ISM cards in one ASR9K Chassis, same command needs to be executed for all ISM locations. "SW Ver :" field (for CPU1) will provide the CGv6 Application S/w version on ISM (e.g. - "1.1.2.1" in the following sample output). RP/0/RSP1/CPU0:ASR9000#show platform Tue Nov 12 14:24:07.327 UTC Node            Type                      State            Config State ----------------------------------------------------------------------------- 0/RSP0/CPU0     A9K-RSP-4G(Standby)       IOS XR RUN       PWR,NSHUT,MON 0/RSP1/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON 0/2/CPU0        A9K-8T-L                  IOS XR RUN       PWR,NSHUT,MON 0/4/CPU0        A9K-ISM-100(LCP)          IOS XR RUN       PWR,NSHUT,MON 0/4/CPU1        A9K-ISM-100(SE)           APP-READY 0/9/CPU0        A9K-8T-L                  IOS XR RUN       PWR,NSHUT,MON RP/0/RSP1/CPU0:ASR9000# RP/0/RSP1/CPU0:ASR9000#show platform summary location 0/4/CPU0 Tue Nov 12 19:57:46.522 UTC -------------------------------------------------------------------------------      Platform Node : 0/4/CPU0 (slot 6)                PID : A9K-ISM-100          Card Type : Integrated Services Module             VID/SN : V01 / FOC1710N5P7         Oper State : IOS XR RUN         Last Reset : Card is reset by user                    : Thu Nov  7 01:04:31 2013      Configuration : Power is enabled                      Bootup enabled.                       Monitoring enabled         Rommon Ver : Version 1.2(20091201:235620)         IOS SW Ver : 4.3.2         Main Power : Power state Enabled. Estimate power 405 Watts of power required.             Faults : N/A -------------------------------------------------------------------------------      Platform Node : 0/4/CPU1 (slot 6)          Card Type : Integrated Services Module (Service Engine)         Oper State : APP-READY    Last Start Time : Mon Nov 11 07:07:08 2013    Last Ready Time : Mon Nov 11 07:09:24 2013             Uptime : 36:48:22           BIOS Ver : 0.17 (Thurley.3.60.18.0033)             SW Ver : 1.1.2.1 (Built on Sep 30 2013, from services/cgn@r432-support/5)         App Status : 8 CGv6 Application instance(s) is/are running ------------------------------------------------------------------------------- Back to top 8. How can I know CGv6 Application Software Version from the Install-Kit without installing it ? Linux Install-Kit is a shell-script which embeds Application binary. You can save the Install-Kit on a Linux/Unix machine, add the executable permission mode to it and execute it with "-v" option (as shown below). It will provide details on CGv6 Application Softwate version. bgl-ads-1766:4.3.2> ./asr9k-ism-cgv6-install-kit-4.3.2.02.sh -v ASR9K infra version:  4.3.2.2 (Built from cds/infra@cgv6_ism432_rb_t_dev_br/7)                               (Built on Sat Nov 23 21:49:27 IST 2013) ASR9K application version:  1.1.2.2 (Built on Nov 21 2013, from services/cgn@r432-support/11) Back to top 9. How do I find out MD5 Checksum of a Linux Install-kit ? MD5 (Message Digest 5) Checksum of Linux install-kit (for that matter, any file) can be obtained using "md5sum" utility on Linux and "md5" utility on IOS-XR, as shown below. NOTE: It takes about few seconds to calculate MD5 checksum on Linux, but takes about 2 minutes on IOS-XR (RP). On Linux Machine: ================= linux_host# md5sum ./asr9k-ism-cgv6-install-kit-4.3.2.01.sh                       47c05d10745da32c7b4f791adeeef618  ./asr9k-ism-cgv6-install-kit-4.3.2.01.sh On IOS-XR (RP): =============== RP/0/RSP0/CPU0:CRANE#run    Fri Nov 29 23:00:24.362 UTC # md5 /disk1:/asr9k-ism-cgv6-install-kit-4.3.2.01.sh 47c05d10745da32c7b4f791adeeef618 # exit Back to top 10. Is there any command to check the Status of Installation while it is going on ? Yes, you can use "show ism install status location <>" CLI to see the status of Installation from time to time. Stage 1: Install-Kit download from RP to ISM (Linux) While the Install-Kit is being downloaded from RP to ISM card, it will indicate the percentage completion of download, as shown below. RP/0/RSP0/CPU0:CRANE(admin)#show ism install status location 0/0/CPU0 Thu Nov 28 00:24:20.343 UTC Downloading install-kit: 97% completed. This operation can still be aborted Previous request status: Successful RP/0/RSP0/CPU0:CRANE(admin)# Stage 2: Install-Kit extraction on Linux When the Linux Install-Kit has been copied to ISM card and installation (extraction) of software is going on, it shows the following message. RP/0/RSP0/CPU0:CRANE(admin)#show ism install status location 0/0/CPU0 Thu Nov 28 00:24:36.903 UTC Install-kit linux installation in progress. This operation can no longer be aborted Previous request status: Successful RP/0/RSP0/CPU0:CRANE(admin)# Stage 3: Rebooting of ISM card Once the installation of software is completed and ISM card is rebooted, following status will be displayed. RP/0/RSP0/CPU0:CRANE(admin)#show ism install status location 0/0/CPU0 Thu Nov 28 00:28:49.476 UTC Line card rebooting. This operation can no longer be aborted Previous request status: Successful RP/0/RSP0/CPU0:CRANE(admin)# Stage 4: ISM card is rebooted and Up (Installation Over) When ISM card is rebooted and it has come up (i.e., installation process is completed), it will display the following status message. RP/0/RSP0/CPU0:CRANE(admin)#show ism install status location 0/0/CPU0 Thu Nov 28 00:36:54.169 UTC There are no install requests in operation. Previous request status: Successful RP/0/RSP0/CPU0:CRANE(admin)# Back to top 11. What do I do when installation of Install-Kit failed with "vkg_get_services_role UNDEFINED role" error ? Please note the "role" MUST be set to "cgn" before executing the "download install-image ..." CLI. Otherwise, you will see the following error messages. RP/0/RSP0/CPU0:(admin)#download install-image                        compactflash:asr9k-ism-cgv6-install-kit-4.3.2.02.sh                        from 0/RSP1/CPU0 to 0/2/CPU0 Wed Nov 27 19:22:33.725 UFA RP/0/RSP0/CPU0:Nov 27 19:22:33.892 UFA: sim_download_recovery_image[65892]: %PLATFORM-SIM_IPC-6-SW_INFO : You are going to install new application on LC. Card will be reloaded automatically once Do you want to proceed Installation?[confirm] installation is completed. RP/0/RSP0/CPU0:Nov 27 19:22:37.960 UFA: sim_download_recovery_image[65892]: %PLATFORM-SIM_IPC-3-SW_ERROR : vkg_get_services_role UNDEFINED role! RP/0/RSP0/CPU0:Nov 27 19:22:37.977 UFA: sim_download_recovery_image[65892]: %PLATFORM-SIM_IPC-3-SW_ERROR : main Please check whether proper service-role is configured on card. Retry the command after some-time, if proper role is already configured  RP/0/RSP0/CPU0:(admin)# When you see the message, please configure the "role" as "cgn" (as shown below) and re-execute "download install-image ..." CLI after that. RP/0/RSP0/CPU0#config (config)#hw-module service cgn location 0/2/cpu0 (config)#commit (config)#exit Back to top 12. Is there any sample Install-Kit Download log output for successful scenario ? Below is a sample Install-Kit Download log output when ServiceInfra interface is already configured on the ISM card. This can be compared as a reference. However, please note the exact output can vary depending on the release, existing CGv6 configuration, etc. Also, lines are broken into multiple lines for better display. RP/0/RSP0/CPU0:CRANE(admin)#download install-image disk1://asr9k-ism-cgv6-install-kit.sh                             to 0/0/CPU0 RP/0/RSP0/CPU0:Nov 28 00:15:47.185 : sim_download_recovery_image[65851]: %PLATFORM-SIM_IPC-6-SW_INFO : You are going to install new application on LC. Card will be reloaded automatically once installation is completed. Do you want to proceed Installation?[confirm] RP/0/RSP0/CPU0:Nov 28 00:15:49.906 : sim_download_recovery_image[65851]: %PLATFORM-SIM_IPC-6-SW_INFO : main Installation is in progress, Please do not reload the LC  LC/0/0/CPU0:Nov 28 00:15:49.914 : avsm_se_sm[125]: %PLATFORM-SIM_SE-5-HA_INFO : ism_service_switchover: OFFLINE msg to se controller success LC/0/0/CPU0:Nov 28 00:15:49.926 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in shutting down state. LC/0/0/CPU0:Nov 28 00:15:49.927 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Starting of SI APIV Deletion ... RP/0/RSP0/CPU0:CRANE(admin)# LC/0/0/CPU0:Nov 28 00:15:54.995 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_5: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 7 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.995 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_5: (PID 25887) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.995 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_4: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 6 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.995 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_6: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 5 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_3: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 4 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_7: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 1 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_0: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 3 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_1: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 0 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_4: (PID 25783) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_6: (PID 25991) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO :    ID_2: * Pending entities to de-allocate SHM * : 4 Dispatcher(s), 2 App(s)  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_0: (PID 25408) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_7: (PID 26072) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.996 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_1: (PID 25512) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.997 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_2: (PID 25616) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:54.997 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_3: (PID 25676) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:55.155 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :    ID_0: * Pending entities to de-allocate SHM * : 3 Dispatcher(s), 0 App(s)  LC/0/0/CPU0:Nov 28 00:15:55.155 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :    ID_2: * Pending entities to de-allocate SHM * : 2 Dispatcher(s), 0 App(s)  LC/0/0/CPU0:Nov 28 00:15:55.155 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   Dispatcher_0: (PID 3516) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:55.155 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   Dispatcher_2: (PID 3534) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:55.176 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :    ID_1: * Pending entities to de-allocate SHM * : 1 Dispatcher(s), 0 App(s)  LC/0/0/CPU0:Nov 28 00:15:55.176 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   Dispatcher_1: (PID 3524) Exiting via Signal (15) Handler ...  LC/0/0/CPU0:Nov 28 00:15:55.784 : avsm_agent[124]: %INFRA-2-CRIT : Core dump generated for 'dispatcher' (core:dispatcher-3544-11-.gz). If the problem persists, contact your technical support representative for assistance. LC/0/0/CPU0:Nov 28 00:16:10.067 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in reset(Manual) state. LC/0/0/CPU0:Nov 28 00:16:10.067 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in powering on state. LC/0/0/CPU0:Nov 28 00:16:12.086 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in reset(Manual) state. LC/0/0/CPU0:Nov 28 00:16:12.086 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in powering on state. LC/0/0/CPU0:Nov 28 00:16:40.870 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in loading SE maintenance image state. LC/0/0/CPU0:Nov 28 00:17:40.259 : online_diag_lc[341]: %DIAG-DIAG-6-GOLDXR_GENERAL : ISMSimLoopback punt path test succeeded, failure cleared: tsec 2 LC/0/0/CPU0:Nov 28 00:17:41.313 : online_diag_lc[341]: %DIAG-DIAG-6-GOLDXR_GENERAL : ISMSimLoopback punt path test succeeded, failure cleared: tsec 3 LC/0/0/CPU0:Nov 28 00:17:45.173 : svc_pm[343]: %PLATFORM-VKG_PM-6-INFO : Service PM info: Starting of APIV version negotiation ... LC/0/0/CPU0:Nov 28 00:17:45.173 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Starting of APIV version negotiation (CREATE) ... LC/0/0/CPU0:Nov 28 00:17:45.177 : external_fpd_agent[167]: %PLATFORM-FPD-7-DEBUG : fpd-debug: Service Module in maintenance mode, external FPD agent not active  LC/0/0/CPU0:Nov 28 00:17:46.178 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: SIS APIV is up LC/0/0/CPU0:Nov 28 00:17:46.750 : inv_agent[205]: %PLATFORM-INV_AGENT-7-INFO : Sending SIM x86 inventory info  LC/0/0/CPU0:Nov 28 00:17:48.409 : avsm_agent[124]: %ISM_L2_UFA-3-ERR : avsm_l2_ufa_conn_open(),550 open_req_msg_send FAILED, APIV trans error 0x1001 (Operation not permitted)(0x1001)  LC/0/0/CPU0:Nov 28 00:17:49.180 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Received SE Operation Ready Notification..... LC/0/0/CPU0:Nov 28 00:17:59.183 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 18 received  LC/0/0/CPU0:Nov 28 00:17:59.183 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x20010000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.184 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 8 received  LC/0/0/CPU0:Nov 28 00:17:59.184 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x60010000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.184 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 19 received  LC/0/0/CPU0:Nov 28 00:17:59.184 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xa0010000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.185 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 12 received  LC/0/0/CPU0:Nov 28 00:17:59.185 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xe0010000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.185 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 13 received  LC/0/0/CPU0:Nov 28 00:17:59.186 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x20020000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.186 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 18 received  LC/0/0/CPU0:Nov 28 00:17:59.186 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x60020000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.186 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 19 received  LC/0/0/CPU0:Nov 28 00:17:59.187 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xa0020000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.187 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 12 received  LC/0/0/CPU0:Nov 28 00:17:59.187 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xe0020000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.187 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 13 received  LC/0/0/CPU0:Nov 28 00:17:59.188 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x20030000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.188 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 18 received  LC/0/0/CPU0:Nov 28 00:17:59.188 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0x60030000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.189 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xa0030000, rc 3  LC/0/0/CPU0:Nov 28 00:17:59.189 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 19 received  LC/0/0/CPU0:Nov 28 00:17:59.189 : avsm_agent[124]: %ISM_SIA-3-ERR : Invalid port index 8 received  LC/0/0/CPU0:Nov 28 00:17:59.190 : avsm_agent[124]: %ISM_SIA-3-ERR : if config failed: if handle 0xe0030000, rc 3  LC/0/0/CPU0:Nov 28 00:26:28.317 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in shutting down state. LC/0/0/CPU0:Nov 28 00:27:28.564 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in reset(Service Engine Software) state. RP/0/RSP0/CPU0:Nov 28 00:27:28.618 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_PRE_RESET_NOTIFICATION : Node 0/0/CPU0 , Power Cycle (0x05000000)  RP/0/RSP0/CPU0:Nov 28 00:27:28.621 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_CPU_RESET : Node 0/0/CPU0 CPU reset detected. RP/0/RSP0/CPU0:Nov 28 00:27:28.624 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-ISM-100 state:BRINGDOWN RP/0/RSP0/CPU0:Nov 28 00:27:28.642 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/0/CPU0 , Power Cycle (0x05000000)  RP/0/RSP0/CPU0:Nov 28 00:27:28.673 : mibd_entity[316]: %HA-HA_EM-7-FMFD_CONNECTION_FAIL : Could not connect to /dev/fm/fd_wdsysmon.d/node0_0_CPU0 : No such file or directory RP/0/RSP0/CPU0:Nov 28 00:27:28.700 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-ISM-100 state:ROMMON RP/0/RSP0/CPU0:Nov 28 00:27:28.722 : invmgr[252]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/0/CPU0, state: BRINGDOWN RP/0/RSP0/CPU0:Nov 28 00:27:29.677 : mibd_entity[316]: %HA-HA_EM-7-FMFD_CONNECTION_FAIL : Could not connect to /dev/fm/fd_wdsysmon.d/node0_0_CPU0 : No such file or directory RP/0/RSP0/CPU0:Nov 28 00:27:31.679 : mibd_entity[316]: %HA-HA_EM-7-FMFD_CONNECTION_FAIL : Could not connect to /dev/fm/fd_wdsysmon.d/node0_0_CPU0 : No such file or directory RP/0/RSP0/CPU0:Nov 28 00:27:35.681 : mibd_entity[316]: %HA-HA_EM-7-FMFD_CONNECTION_FAIL : Could not connect to /dev/fm/fd_wdsysmon.d/node0_0_CPU0 : No such file or directory RP/0/RSP0/CPU0:Nov 28 00:28:03.675 : shelfmgr[378]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/0/CPU0, RomMon Version: 1.2 RP/0/RSP0/CPU0:Nov 28 00:28:03.681 : shelfmgr[378]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/0/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-4.3.4.12I/lc/mbiasr9k-lc.vm validated RP/0/RSP0/CPU0:Nov 28 00:28:03.685 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-ISM-100 state:MBI-BOOTING RP/0/RSP0/CPU0:Nov 28 00:29:03.132 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-ISM-100 state:MBI-RUNNING LC/0/0/CPU0:12: init[65540]: %OS-INIT-7-MBI_STARTED : total time 4.228 seconds LC/0/0/CPU0:Nov 28 00:29:25.311 : sysmgr[91]: %OS-SYSMGR-5-NOTICE : Card is COLD started  LC/0/0/CPU0:Nov 28 00:29:25.590 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 27.574 seconds LC/0/0/CPU0:Nov 28 00:29:26.085 : obfl_pd_driver[269]: %PLATFORM-OBFL-3-ERR_INIT : Failed to access OBFL storage device bootflash:: No such file or directory, exiting... LC/0/0/CPU0:Nov 28 00:29:26.209 : sysmgr[330]: %OS-SYSMGR-6-INFO : Backup system manager is ready  LC/0/0/CPU0:Nov 28 00:29:28.893 : cgv6_services[140]: %PLATFORM-CGV6_SERVICES-7-GENERIC_INFO : Info: Starting Cgv6 Services Driver on location 0/0/CPU0 LC/0/0/CPU0:Nov 28 00:29:29.197 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Initializing APIV modules ... LC/0/0/CPU0:Nov 28 00:29:29.277 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Registering for SysDB notifications ... LC/0/0/CPU0:Nov 28 00:29:29.289 : cgv6_services[140]: %PLATFORM-CGV6_SERVICES-7-GENERIC_INFO : Info: HA:sysdb_bind success LC/0/0/CPU0:Nov 28 00:29:29.291 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Registering for SysDB SE READY notifications ... LC/0/0/CPU0:Nov 28 00:29:29.291 : cgv6_services[140]: %PLATFORM-CGV6_SERVICES-6-INFO : HA:sysdb_register_notification success LC/0/0/CPU0:Nov 28 00:29:29.294 : cgv6_services[140]: %PLATFORM-CGV6_SERVICES-6-INFO : HA:sysdb_register_notification success LC/0/0/CPU0:Nov 28 00:29:29.379 : cgv6_services[140]: %PLATFORM-CGV6_SERVICES-7-GENERIC_INFO : Info: HA:Verification registration success LC/0/0/CPU0:Nov 28 00:29:32.298 : avsm_se_sm[125]: %PLATFORM-SIM_SE-4-HA_WARN : avsm_ha_service_monitor: Diag test result info is not updated since last 3 tries LC/0/0/CPU0:Nov 28 00:29:41.574 : sysmgr[91]: %OS-SYSMGR-7-DEBUG : sysmgr_admin_plane_check:SYSMGR_PLANE_ADMIN Notification sent.   LC/0/0/CPU0:Nov 28 00:29:41.911 : sync_agent[325]: %PLATFORM-FSYNC-6-UNKNOWN_BOARD_TYPE : Frequency Synchronization feature does not recognize hardware board type 0x1000305 LC/0/0/CPU0:Nov 28 00:29:41.961 : sync_agent[325]: %PLATFORM-FSYNC-6-NOT_SUPPORTED : Frequency Synchronization is not supported on this line card. LC/0/0/CPU0:Nov 28 00:29:42.542 : syslog_dev[89]: hsbi_fpd_agent[194]: FPD ltrace_file_name => fpd-agent/hsbi_fpd_agent LC/0/0/CPU0:Nov 28 00:29:44.798 : fib_mgr[172]: %PLATFORM-PLAT_FIB-6-INFO : Svcs Mem mgmt init OK for restart LC/0/0/CPU0:Nov 28 00:29:45.034 : fib_mgr[172]: %PLATFORM-PLAT_FIB-6-INFO : Initializing APIV modules ... LC/0/0/CPU0:Nov 28 00:29:45.044 : fib_mgr[172]: %PLATFORM-PLAT_FIB-6-INFO : FIB registering for SysDB notifications ... LC/0/0/CPU0:Nov 28 00:29:45.160 : svc_pm[343]: %PLATFORM-VKG_PM-6-INFO : Service PM info: Initializing APIV modules ... LC/0/0/CPU0:Nov 28 00:29:50.563 : envmon_lc[161]: %PLATFORM-ENVMON-3-OPERATION_FAIL : Failed in init_obfl_card_temp.545, error code OBFL temperature initialization failed  LC/0/0/CPU0:Nov 28 00:29:56.809 : syslog_dev[89]: sesvi_ea[302]: CREATING SESVI_SI_SHM LC/0/0/CPU0:Nov 28 00:30:00.532 : umass-enum[92]: USB: /dev/lcdisk00: device not available, continuing LC/0/0/CPU0:Nov 28 00:30:10.585 : envmon_lc[161]: %PLATFORM-ENVMON-3-OPERATION_FAIL : Failed in init_obfl_card_volt.792, error code OBFL voltage initialization failed  LC/0/0/CPU0:Nov 28 00:30:30.676 : obfl_agent-lc[267]: %PLATFORM-OBFL-3-ERR_CREATE : Failed to create the OBFL environment record LC/0/0/CPU0:Nov 28 00:31:12.130 : sysmgr[91]: %OS-SYSMGR-7-DEBUG : fib_mgr(1) (jid 172) did not signal end of initialization   LC/0/0/CPU0:Nov 28 00:31:12.635 : vkg_l2fib_mac_cache[339]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: initializing L2 FIB MAC CAHE APIV modules. LC/0/0/CPU0:Nov 28 00:31:12.715 : vkg_l2fib_mac_cache[339]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: registering for SysDB notifications ... LC/0/0/CPU0:Nov 28 00:31:14.772 : canb_upg_agt[132]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : cbc instance 0 is down-rev (V18.06), upgrade to (V18.08). Use the "upgrade hw-module fpd" CLI in admin mode. LC/0/0/CPU0:Nov 28 00:31:15.293 : l2fib[239]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: Initializing L2 FIB APIV modules. LC/0/0/CPU0:Nov 28 00:31:15.346 : l2fib[239]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: registering for SysDB notifications. LC/0/0/CPU0:Nov 28 00:31:16.876 : /pkg/sbin/sysmgr_log[65660]: %OS-SYSMGR-7-CHECK_LOG : /pkg/bin/sysmgr_debug_script invoked for: (fib_mgr) process did not signal EOI and for level:100 . Output is in harddisk:/sysmgr_debug/debug.node0_0_CPU0.163955 LC/0/0/CPU0:Nov 28 00:31:30.760 : obfl_agent-lc[267]: %PLATFORM-OBFL-3-ERR_CREATE : Failed to create the OBFL uptime record RP/0/RSP0/CPU0:Nov 28 00:31:35.536 : shelfmgr[378]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-ISM-100 state:IOS XR RUN RP/0/RSP0/CPU0:Nov 28 00:31:35.544 : invmgr[252]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/0/CPU0, state: IOS XR RUN LC/0/0/CPU0:Nov 28 00:33:09.184 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in checking filesystems state. LC/0/0/CPU0:Nov 28 00:33:09.195 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_RX_ALIGN_LOSS_0 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 Rx alignment loss - 0 LC/0/0/CPU0:Nov 28 00:33:09.495 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_STAT_VEC_0 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 state error on bridge-0 LC/0/0/CPU0:Nov 28 00:33:09.495 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_1_STAT_VEC_0 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-1 state error on bridge-0 LC/0/0/CPU0:Nov 28 00:33:09.495 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_RX_ALIGN_LOSS_1 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 Rx alignment loss - 1 LC/0/0/CPU0:Nov 28 00:33:09.801 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_STAT_VEC_1 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 state error on bridge-1 LC/0/0/CPU0:Nov 28 00:33:09.801 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_1_STAT_VEC_1 : Set|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-1 state error on bridge-1 LC/0/0/CPU0:Nov 28 00:33:11.803 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_RX_ALIGN_LOSS_0 : Clear|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 Rx alignment loss - 0 LC/0/0/CPU0:Nov 28 00:33:11.803 : pfm_node_lc[277]: %PLATFORM-BRIDGE-1-XAUI_0_RX_ALIGN_LOSS_1 : Clear|fialc[135242]|NPU Crossbar Fabric Interface Bridge(0x1025000)| XAUI to NPU-0 Rx alignment loss - 1 LC/0/0/CPU0:Nov 28 00:33:34.263 : inv_agent[205]: %PLATFORM-INV_AGENT-7-INFO : Sending SIM x86 inventory info  LC/0/0/CPU0:Nov 28 00:33:40.403 : l2fib[239]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: starting of APIV version negotiation. LC/0/0/CPU0:Nov 28 00:33:40.404 : vkg_l2fib_mac_cache[339]: %PLATFORM-PLAT_L2FIB-6-GENERIC : avsm l2fib: Starting of L2FIB MAC CACHE APIV version negotiation. LC/0/0/CPU0:Nov 28 00:33:40.404 : svc_pm[343]: %PLATFORM-VKG_PM-6-INFO : Service PM info: Starting of APIV version negotiation ... LC/0/0/CPU0:Nov 28 00:33:40.405 : fib_mgr[172]: %PLATFORM-PLAT_FIB-6-INFO : Starting of APIV version negotiation ... LC/0/0/CPU0:Nov 28 00:33:40.406 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Starting of APIV version negotiation (CREATE) ... LC/0/0/CPU0:Nov 28 00:33:41.739 : fib_mgr[172]: %PLATFORM-PLAT_FIB-6-INFO : CGN L3 FIB APIV is up LC/0/0/CPU0:Nov 28 00:33:41.854 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: SIS APIV is up LC/0/0/CPU0:Nov 28 00:33:42.145 : external_fpd_agent[167]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : cpld3 instance 0 is down-rev (V0.10), upgrade to (V0.16). Use the "upgrade hw-module fpd" CLI in admin mode. LC/0/0/CPU0:Nov 28 00:33:43.009 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Engine0/0/0/3, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.014 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Engine0/0/0/2, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.014 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Engine0/0/0/1, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.014 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Engine0/0/0/0, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.018 : cfgmgr-lc[136]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/0/0' has been restored. LC/0/0/CPU0:Nov 28 00:33:43.734 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Mgmt0/0/0/3, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.734 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Mgmt0/0/0/2, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.734 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Mgmt0/0/0/1, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:43.734 : ifmgr[196]: %PKT_INFRA-LINK-5-CHANGED : Interface Service-Mgmt0/0/0/0, changed state to Administratively Down LC/0/0/CPU0:Nov 28 00:33:44.901 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in ready state. LC/0/0/CPU0:Nov 28 00:33:44.901 : sis[307]: %PLATFORM-VKG_SIS-6-GENERIC_EVENT : sis event: Received SE Operation Ready Notification..... LC/0/0/CPU0:Nov 28 00:33:44.903 : avsm_se_sm[125]: %PLATFORM-SIM_SE-5-SW_INFO : Service Engine heartbeat started on A9K-ISM-100 LC/0/0/CPU0:Nov 28 00:33:57.193 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_0 (CPU 3): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:33:58.359 : avsm_agent[124]: %CGV6_APP-3-APP_ERR :   stat(/var/run/cgv6_app) returned error -1 !  LC/0/0/CPU0:Nov 28 00:33:58.360 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_0 (CPU 3): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:33:59.194 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_1 (CPU 4): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:00.355 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_1 (CPU 4): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:01.206 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_2 (CPU 15): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:02.404 : online_diag_lc[341]: %DIAG-DIAG-6-GOLDXR_GENERAL : ISMSimLoopback punt path test succeeded, failure cleared: tsec 2 LC/0/0/CPU0:Nov 28 00:34:02.560 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_2 (CPU 15): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:03.217 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_3 (CPU 16): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:04.535 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_3 (CPU 16): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:05.228 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_4 (CPU 9): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:06.909 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_4 (CPU 9): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:07.136 : online_diag_lc[341]: %DIAG-DIAG-6-GOLDXR_GENERAL : ISMSimLoopback punt path test succeeded, failure cleared: tsec 3 LC/0/0/CPU0:Nov 28 00:34:07.240 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_5 (CPU 10): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:08.912 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_5 (CPU 10): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:09.252 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_6 (CPU 21): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:10.488 : online_diag_lc[341]: %DIAG-DIAG-6-GOLDXR_GENERAL : ISMSimLoopback punt path test succeeded, failure cleared: tsec 3 LC/0/0/CPU0:Nov 28 00:34:11.136 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_6 (CPU 21): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:11.262 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_7 (CPU 22): Using Local (x86) IP 100.1.1.2, Remote (XR) IP 100.1.1.1 from SHM  LC/0/0/CPU0:Nov 28 00:34:13.106 : avsm_agent[124]: %CGV6_APP-6-APP_INFO : CGV6_APP_7 (CPU 22): Ready to receive data ...  LC/0/0/CPU0:Nov 28 00:34:23.269 : avsm_agent[124]: %ISM_SIA-6-INFO : Sent app_ready signal  LC/0/0/CPU0:Nov 28 00:34:25.031 : avsm_se_sm[125]: %PLATFORM-SIM_SE-6-STATE_CHANGE : Service Engine is in app_ready state. LC/0/0/CPU0:Nov 28 00:34:27.200 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 0, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.171 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 0, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.185 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.189 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  LC/0/0/CPU0:Nov 28 00:34:32.191 : avsm_agent[124]: %DISPATCHER-6-DISPAT_INFO :   DISPATCHER_0_RXC0: Rcvd CONFIG_SVI_PARAMS: UIDB 8, VRF_TBL_ID 0, App Type 5, ifhandle 0x000003e0, IPv4 Addr 0x64010101, IPv6 Addr 0x00000000:00000000:00000000:00000000  RP/0/RSP0/CPU0:CRANE(admin)#show plat Node            Type                      State            Config State ----------------------------------------------------------------------------- 0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON 0/FT0/SP        FAN TRAY                  READY            0/FT1/SP        FAN TRAY                  READY            0/0/CPU0        A9K-ISM-100(LCP)          IOS XR RUN       PWR,NSHUT,MON 0/0/CPU1        A9K-ISM-100(SE)           APP-READY       0/1/CPU0        A9K-2T20GE-L              IOS XR RUN       PWR,NSHUT,MON 0/PM0/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON 0/PM1/SP        A9K-3KW-AC                FAILED           PWR,NSHUT,MON RP/0/RSP0/CPU0:CRANE(admin)# Related Information IOS-XR Software Maintenance Updates (SMUs) Guide Download Software Center CGv6 on ISM: Features Supported Across different IOS-XR Release (DOC-37444) CGv6 Configuration Guides for different IOS-XR Releases Cisco ASR9000 Upgarde Procedure for Release 4.3.2   Back to top ... View more

show services redundancy Introduction Introduction to NAT44 Why NAT44 ? NAT44 Overview Basic Terminologies Standard Compliance Basic Deployment Scenario: NAT444 Network Diagram NAT44 on ISM Configuration I2O Packet Flow on ISM O2I Packet Flow on ISM Parameters to be considered before Deployment Performance and Scale Address Pool and Mapping VRF (Inside and Outside) Port Limit (Per-User) External Logging (Netflow and Syslog) Destination-Based Logging (DBL) Bulk Port Allocation (BPA) NAT Session Timeouts (TCP, UDP, ICMP) Traffic Diversion (to ISM) Default Route Static Route ACL-based Forwarding (ABF) Static Port Forwarding Application Layer Gateway (ALG) Redundancy 1:1 Warm Stand-by Redundancy N:1 ABF-Based Redundancy NAT44 Licenses Summary of parameter values Specific Deployment Scenarios / Use-Cases Sample Applications over NAT44 Multiple Outside ServiceApps sharing same Outside VRF NAT Bypass Interworking of BNG and NAT44 MPLS (Layer 3) VPN and NAT Interworking NAT Hairpining Handling of Asymmetric I2O and O2I traffic with better performance Validation of Configuration show run show cgn nat44 <> statistics show cgn nat44 <> inside-translation show cgn nat44 <> outside-translation show services redundancy Useful Deployment Tips Related Information Introduction Cisco's CGv6 (Carrier Grade IPv6) solution helps customer transitioning from IPv4 to IPv6 gradually, maintaining business continuity. ISM (Integrated Services Module) on ASR9K is a Service Card which provides different CGv6 Applications. This document talks about how to deploy NAT44 (Network Address Translation - from IPv4 to IPv4), one of the applications fall under CGv6 suite of applications. This document does NOT intend to cover details on ISM, how to install CGv6 software on ISM, how to trouble-shoot CGv6 issues, CGv6 Applications other than NAT44.   DISCLAIMER: This document contains some Public IP addresses to explain different deployment use-cases and features related to NAT44. The Public IP addresses are picked up randomly and matching of these addresses with any specific customer's or organization's public IP is purely coincidental. Please DO NOT use these public addresses in your NAT setup / deployment, unless the public IP addresses are not owned by you / your organization. Cisco is not responsible for any issues arising out of this. Introduction to NAT44 This section introduces NAT44 and terminologies related to it. Back to top 1. Why NAT44 ? IPv4 address is 32-bit long which means there can be 4,294,967,296 (i.e., 4 billions) unique addresses. However, population on earth has crossed 7 billions and is still growing. Also, nowadays, a single person can own multiple IP devices, each having its own IP address. Thus, we are running short of IPv4 addresses. In 2 of the 5 RIR (Regional Internet Registries), IPv4 address exhaustion is declared (in its last /8 stage). Apat from moving to IPv6 (which uses 128-bit long IP address), an alternate mechanism is to use NAT44 where Private IP addresses can be re-used in different Private networks, but can be translated to a unique Public IP address when the packet enters Internet or Public network. If you are a Service Provider and running short of Public IP addresses, NAT44 should be of great usefulness to you as an intermediate solution before you slowly convert your IPv4 network into IPv6. Back to top 2. NAT44 Overview NAT44 literally means translating one IPv4 address to another IPv4 address. "Typically", first IPv4 address is "Private" IPv4 Address (defined by RFC 1918) and second IPv4 address is "Public" (non-Private) IPv4 address. As defined by RFC 1918, Private IPv4 addresses fall in the following blocks. Start IP Address End IP Address Comments 10.0.0.0 10.255.255.255 10.0.0.0/8 prefix - contains 16,777,216 addresses 172.16.0.0 172.31.255.255 172.16.0.0/12 prefix - contains 1,048,576 addresses 192.168.0.0 192.168.255.255 192.168.0.0/16 prefix - contains 65,536 address However, it is to be noted that not always "Private" IPv4 address is to be translated to "Public" IPv4 address. There are deployment cases where NAT44 is used to convert "Private" IPv4 address (say, 192.168.x.x range) to another "Private" IPv4 address (say, 10.x.x.x range) as well. NOTE: In this document, however, we will refer "Private" side as the pre-NAT side of the NAT device and "Public" side as the post-NAT side of the NAT device, irrespective of the actual IP address being used.   One more important point, although NAT indicates Network Address Translation, it is actually NAPT (Network Address and Port Translation). What it means is - in addition to IPv4 address, Layer 4 (UDP/TCP) port is also translated by NAT44. NOTE: Unless mentioned otherwise, in this document, NAT44 will indicate translation of both IPv4 address and Port.   Following Layer 4 Protocols are supported by NAT44 implementation on ISM. UDP (IP Protocol 17) - UDP Port number (2 bytes) is translated along with IPv4 address. TCP (IP Protocol 6) - TCP Port number (2 bytes) is translated along with IPv4 address. ICMP (IP Protocol 1) - ICMP Identifier field (2 bytes) in Echo Request and Echo Response messages is used as (dummy) Layer 4 port. GRE (used with PPTP ALG, IP Protocol 47) - Call ID field (2 bytes) is used as (dummy) Layer 4 port. Another useful property of NAT44 is multiple Private IP addresses can be mapped to one Public IP address. This is possible as all of the 65,536 ports (0 to 65,535) on any Private IP address are typically not used by Applications. Following diagram shows how NAT44 works. As shown above, the Private IP addresses 192.168.1.1 and 192.168.1.5 are translated to same Public IP address 20.1.1.1 (along with specific Layer 4 ports) by the NAT44 device (ISM on ASR9K). The translation entries are maintained in NAT DB at the NAT44 device. Back to top 3. Basic Terminologies In this section, let us get introduced with different terminologies related to NAT44 deployment on ISM. Terminology Definition / What it means Inside / Subscriber Side / Access Side / Private Side Private side of the NAT44 device (which usually contains Private IPv4 addresses) Outside / Network Side / Core Side Public side of the NAT44 device (which usually contains Public IPv4 addresses) Inside-to-Outside (I2O) Traffic flowing from Inside to Outside across NAT44 device Outside-to-Inside (O2I) Traffic flowing from Outside to Inside across NAT44 device Stateful (SF) Which preserves State or Database. E.g. - NAT44 is Stateful protocol. Endpoint Independent Mapping (EIM) / Filtering (EIF) Any device (IP + Port) can send O2I traffic to a Public IP + Port which exists in NAT DB. ISM implements this behavior. It is also termed as "Full-Cone NAT". Endpoint Dependent Mapping (EDM) / Filtering (EDF) Only the device (IP + Port) which has received I2O traffic can send O2I  traffic to a public IP + Port which exists in NAT DB. Destination IP and  port information is also maintained in NAT DB in this case. ISM does  not implement this behavior. Port Parity Preservation Odd port should be translated to Odd port and Even port should be translated to Even port. ISM supports this. Back to top 4. Standard Compliance Following are the important RFCs that the NAT44 solution on ISM complies (most of the cases, fully) to. RFC Number / IETF Draft Name Title of the Specification Additional Comments RFC 768 User Datagram Protocol (UDP)   RFC 792 Internet Control Message Protocol (ICMP)   RFC 793 Transmission Control Protocol (TCP)   RFC 959 File Transfer Protocol (FTP)   RFC 1812 Requirements for IP Version 4 Routers   RFC 3954 Cisco System Netflow Services Export Version 9   RFC 4787 Network Address Translation (NAT) Behavioral Requirements for Unicast UDP   RFC 5382 NAT Behavioral Requirements for TCP   RFC 5424 The Syslog Protocol   RFC 5508 NAT Behavioral Requirements for ICMP   RFC 6888 Common Requirements for Carrier Grade NATs (CGN) Evolved from draft-nishitani-cgn Basic Deployment Scenario: NAT444 This section talks about one of the basic deployment scenario for NAT44 which is called "Double NAT44" or "NAT444". Back to top 5. Network Diagram Following diagram depicts NAT444. In the above diagram, there are 2 NAT44 instances shown (hence, it is termed as "Dual NAT44" or "NAT444"). CPE (Customer Premise Equipment) NAT44 - This typically happens within customer premise. Each customer / end user is given one (or, more) IPv4 address (10.8.1.11 in the above example) by the Service Provider. There could be multiple devices within customer premise which will have IPv4 addresses in 192.168.1.x network (as shown in the above diagram). As an example, CPE NAT44 device translates IPv4 address/Port of 192.168.1.1:4128 to 10.8.1.11:6593, while sending the packet out to provider network. A NAT table is maintained at CPE NAT level as well. But, the scale of this is less (as there won't be too many end devices). As you can see, both Private-side and Public-side IP addresses used here are from RFC 1918 (i.e., Private IPv4 addresses). CGN (Carrier Grade NAT) or LSN (Large Scale NAT) - This happens within the Service Provider network. CGN device (as per the diagram) translates Private IPv4 address/Port of 10.8.1.11:6593 to a public IPv4 address/Port of 170.0.0.1:9185 before sending the packet to Internet. A relatively huge NAT table is maintained at CGN level as it may need to deal with millions of CPEs. ISM with ASR9K will be positioned as CGN in Service Provider network. Back to top 6. NAT44 on ISM Configuration Before we start configuring NAT44 on ISM, let us understand the following concepts related to NAT44 implementation on ISM. CGN Instance - On each ISM card, we can configure one CGN instance (<cgn1> in the diagram below). NAT44 Instance - On each ISM card, we can configure one NAT44 instance (<nat1> in the diagram below). ServiceInfra interface - ServiceInfra interface is a virtual interface which is used by the internal (to ASR9K) management / control traffic. We need 1 ServiceInfra interface per ISM card. ServiceInfra interface cannot be in any VRF (needs to be in default VRF). ServiceApp interface - ServiceApp interface is a virtual interface which is used to bring CGv6 Application (NAT44) data traffic to ISM card. We need 1 pair of ServiceApp interface for each VRF instance - 1 Inside ServiceApp interface (ServiceApp <x> in the diagram below) to bring Inside traffic to ISM (I2O direction) and 1 Outside ServiceApp interface (ServiceApp <y> in the diagram below) to bring Outside traffic to ISM (O2I direction). NOTE: ServiceInfra and ServiceApp interfaces are used within the ASR9K router only and do not need to be announced/redistributed into IGP. Inside VRF instance - Within one NAT44 instance, we can have multiple Inside VRF instances. For each Inside VRF instance, we can have different NAT44 parameters configured. Inside ServiceApp must be within one unique Inside VRF. Outside ServiceApp may or may not be within a (Outside) VRF. NOTEs: - We cannot have multiple Inside ServiceApps within same Inside VRF. - We can, however, have multiple Outside ServiceApps within same Outside VRF or default VRF. - Inside and Outside VRF cannot be same. Logical Partitioning inside ISM - Internally, ISM is divided into 2 Logic Partitions. Hence, in order to achive full throughput and scale limit (per ISM), you should have minimal of 2 Inside VRFs, 4 ServiceApp interfaces (2 in each partition), 2 Public IP address pool, etc. Traffic / subscribers which needs to avail NAT44 service also needs to be divided into 2 groups (each one associated to each Inside VRF). Following table shows the partitioning of ServiceApp interfaces between these 2 groups (which needs to be kept in mind during configuration). Logical Partition  / Group 1 Logical Partition / Group 2 ServiceApp 1, ServiceApp 2 ServiceApp 3, ServicApp 4 ServiceApp 5, ServiceApp 6 ServiceApp 7, ServiceApp 8 ServiceApp 9, ServicApp 10 ServiceApp 11, ServiceApp 12 ... ... ServiceApp 241, ServiceApp 242 ServiceApp 243, ServiceApp 244   NOTE: ServiceApp interafce pair needs to be selected from same Logical Partition (as per the above table). Otherwise, packets will be dropped. Using consecutive numbers for ServiceApp interface pair will be recommended (as shown above). Following diagram provides summary / overview of NAT44 configuration for ISM. Below are sample configurations for each of the above steps. Step 1: Configure CGN Instance on ISM Card at Slot 1   ! Define location where CGN will run ! ---------------------------------- hw-module service cgn location 0 / 1 /cpu0 Step 2: Configure ServiceInfra interface   ! Define Service Infra interface ! ------------------------------ interface ServiceInfra 1   ! Assign IPv4 address and netmask   ipv4 address 100 .1 . 1.1 255.255 . 255.252   ! Define service location   service-location 0 / 1 /CPU0 Step 3 and 4: Configure Ingress and Egress LC Interfaces   ! Configure Ingress interfaces ! ---------------------------- interface TenGigE0/ 0 / 0 / 0   ! Add it to Inside VRF   vrf ivrf1     ! Assign IP address to it     ipv4 address 10.10 . 10.1 255.255 . 255.0   interface TenGigE0/ 0 / 0 / 1 vrf ivrf2 ipv4 address 10.10.11.1 255.255.255.0 ! Configure Egress interface ! -------------------------- interface TenGigE0/2 / 0 / 0   vrf ovrf1     ipv4 address 20.20 . 20.1 255.255 . 255.0 interface TenGigE0/2 / 0 / 1 vrf ovrf2 ipv4 address 20.20.21.1 255.255.255.0 Step 5: Configure CGN and NAT44 Service parameters (Important ones are shown)   ! Define CGN Instance ! ------------------- service cgn cgn1 ! Define location   service-location preferred-active 0 / 1 /CPU0   ! Define NAT44 Service Type and instance (one per ISM card)   service-type nat44 nat1   ! Define per-user Port limit for NAT44 instance   portlimit 250   ! Enable Active FTP ALG   alg ActiveFTP   ! Define UDP protocol session timeout values   protocol udp     session initial timeout 240     session active timeout 600   ! Define inside VRF   inside-vrf ivrf1     ! Define Public IP address Pool     map outside-vrf ovrf1 address-pool 100.1 . 1.0 / 24   ! Define inside VRF   inside-vrf ivrf2     ! Define Public IP address Pool     map outside-vrf ovrf2 address-pool 100.1 .2 .0 / 24 Step 6 and 7: Configure Inside and Outside ServiceApp interfaces   ! Define Inside Service App interfaces ! ------------------------------------ interface ServiceApp 1   ! Define VRF where it should belong to (optional)   vrf ivrf1     ! Assign IPv4 address and netmask     ipv4 address 1 .1 . 1.1/30     ! Define CGN instance and service type in which it belongs to     service cgn cgn1 service-type nat44 interface ServiceApp 3   vrf ivrf2     ipv4 address 1 .1 .2 .1/30     service cgn cgn1 service-type nat44 ! Define Outside Service App interfaces ! ------------------------------------- interface ServiceApp 2   vrf ovrf1     ipv4 address 2.1 . 1.1/30     service cgn cgn1 service-type nat44 interface ServiceApp 4   vrf ovrf2     ipv4 address 2.1 .2 .1/30     service cgn cgn1 service-type nat44 Step 8 and 9: Configure Traffic Diversion Rules   ! Define static route for Inside-to-outside traffic ! ------------------------------------------------- router static     ! Specify the Inside VRF   vrf ivrf1       ! Define address family     address-family ipv4 unicast         ! Re-direct all traffic to Inside Service App       0.0.0.0/0 ServiceApp 1 router static   vrf ivrf2     address-family ipv4 unicast       0.0 . 0.0 / 0 S erviceApp 3 ! Define static route for Outside-to-inside traffic ! ------------------------------------------------- router static   vrf ovrf1     address-family ipv4 unicast       100.1 . 1.0 / 24 S erviceApp 2 router static   vrf ovrf2     address-family ipv4 unicast       100.1 .2 .0 / 24 S erviceApp 4 Back to top 7. I2O Packet Flow on ISM Following diagram shows the packet flow within ASR9K (including ISM) for Inside to Outside NAT44 traffic at a high-level. Step 1: Packet from Inside / Private side reaches ASR9K Ingress Line Card. Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to ISM card, via Inside ServiceApp 1 interface. Inside VRF (ivrf1) will be used during the forwarding lookup. Step 3: NAT44 Application performs necessary action for the packet. As it is Inside-to-Outside packet, it first checks if there is a NAT entry exists for the source IP (10.10.10.2) and port (5000). If it does not exists, it allocates a Public IP (from the Public IP pool configured) and port and creates a NAT entry. From the NAT entry, it finds the corresponding Public IP (100.2.0.192) and port (23156) and replaces the existing Source IP and port with the same. Step 4: After NAT translation is done, a forwarding lookup is done in Outside VRF (ovrf1) for the destination IP address (5.5.5.2) which forwards the packet to egress LC interface. Step 5: Egress Line Card interface sends the packet out to Outside / Public side. Back to top 8. O2I Packet Flow on ISM Following diagram shows the packet flow within ASR9K (including ISM) for Outside to Inside NAT44 traffic at a high-level. Step 1: Packet from Outside / Public side reaches ASR9K Ingress Line Card. Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to ISM card, via Outside ServiceApp 2 interface. Outside VRF (ovrf1), if being configured, will be used during the forwarding lookup. Step 3: NAT44 Application performs necessary action for the packet. As it is Outside-to-Inside packet, it first checks if there is a NAT entry exists for the Destination IP (100.2.0.192) and port (23156). If it does not exists, packet is dropped. Otherwise, from the NAT entry, it finds the corresponding Private IP (10.10.10.2) and port (5000) and replaces the existing Destination IP and port with the same. Step 4: After (reverse) NAT translation is done, a forwarding lookup is done in Inside VRF (ivrf1) for the destination IP address (10.10.10.2) which forwards the packet to egress LC interface. Step 5: Egress Line Card interface sends the packet out to Inside / Private side. Parameters to be considered before Deployment Before deploying NAT44 solution with ISM, you need to analyze the deployment scenario/requirement and come up with the appropriate values of different configurable parameters related to NAT44. Improper selection of the parameters may not address the deployment goals accurately. Back to top 9. Performance and Scale These are possibly the most important parameters to be considered during deployment. These are not configurable parameters, but these will determine how many ISM cards will be needed for the NAT44 solution. Identify the performance and scale need for NAT44 solution Please check "CGv6 on ISM Feature Support" page, to find out the supported scale and performance values. Based on the above two, find out how many ISM cards will be needed for the solution. Back to top 10. Address Pool and Mapping Identify the following for the deployment scenario. How many Public IPv4 addresses the customer has to form the Public IP pool ? How many (approximately) Private (Pre-NAT) IPv4 addresses the customer has ? Do you need to limit the ratio of Private to Public IP addresses ? Based on the same, number of inside VRFs, address pools, many-to-one ratio can be configured as follows. Address Mapping related parameter configuration   service cgn cgn1   service-type nat44 nat1         inside-vrf ivrf1           ! Define Public IP address Pool       map outside-vrf ovrf1 address-pool 100.1 . 1.0 / 20       ! Define Many to one IP addresses maping       map ip many-to-one 6   NOTE: Network address and Broadcast address in the subnet (as configuredby address-pool parameter) are also used as NAT'ed Public / Outside IP address. Back to top 11. VRF (Inside and Outside) Identify the following VRF related information from deployment requirement. Whether Private side traffic comes / can / should come in VRF or not ? Whenther Public side traffic can / should go in a VRF or not ? Is there a overlap of Private IP addresses or not ? Are there different values of certain parameters (which are configured per-VRF) needed or not ? What is the Scale and Bandwidth requirement for each VRF Above information will determine, How many Inside and Outside VRFs will be needed in the solution Which per-Inside VRF parameters need to be set How the Public IP pool needs to be divided Back to top 12. Port Limit (Per-User) As NAT DB is shared among all users, there could be a situation where few users (Private IP owners) uses lots of NAT entries (in thousands) and because of the same, new users cannot create NAT entries. In order to keep control on creation of NAT entries by a single user, Port Limit parameter is introduced. Port limit indicates how many port numbers are allowed per Private / Inside IP address. If any flow arrives after Port Limit is reached, no NAT entry will be created for the same and the packets for the new flows will be dropped. The Port limit value will be same for all Private IP addresses.   NOTE: Port Limit is specified per Private / Inside IP address and NOT per Public / Outside IP address. Find out the following: How many ports will be needed by a Private IP address holder ? How many Public IP addresses are available ? Based on the above Port Limit can be configured as follows. Configuration of Port Limit   service cgn cgn1   ! Define NAT44 Service Type and instance (one per ISM card)   service-type nat44 nat1     ! Define per-user Port limit for NAT44 instance     portlimit 250 Back to top 13. External Logging (Netflow and Syslog) As you have seen above, a NAT DB entry is created for each new flow (Private IP address and port) which keeps the mapping of Private / Inside IP address + Port and Public / Outside IP address + Port. The NAT DB entry is deleted when the flow terminates (because of timeout and any user intervention). On ISM, we support millions of NAT translations with high (in thousands and millions) setup and deletion rate (Please refer to CGv6 on ISM Feature List for scale numbers). This produces huge amount of NAT translation entries / records which cannot be stored / saved on ISM or ASR9K because of limited disk space. Hence, there is a need of having external servers where these translations can be saved. The creation and deletion of NAT transactions (also called "Logging Records") are very important to be saved to get association of different IP addresses. In almost all countries, it is mandated by the Lawful Agencies / Authorities that the Service Provide must provide these records accurately so that the agencies can find out the owner of these APIs for different reasons. Thus it becomes a very important feature for deployment of NAT44. On ISM, we support the 2 formats of logging records - Netflow (version 9) and Syslog. Table below compares the two. Items for Comparison Netflow Version 9 (NFv9) Syslog Standard Document RFC 3954 RFC 5424 Text / Binary format Binary Text Underlying Layer 4 protocol UDP (Server usually uses Port 2055, 2056, 4432, 4739, 9995, 9996, 6343) UDP (Server typically uses Port 514) Average Logging Recod Size 20 Bytes (with DBL and BPA disabled) 70 Bytes (with DBL and BPA disabled) Usage of Template Uses Netflow Template Does not use any template Logging records are sent to Logging Server (IP address and Port for which is configurable) either when the Logging packet is complete or timeout occurs. Logging packets are sent with ServiceInfra interface's remote IP (for e.g., 100.1.1.2 is the remote IP of 100.1.1.1) as Source address. NOTE: External Logging Server IP address should be reachable via any ASR9K LC interface (NOT via Management Ethernet interface), in default VRF. Destination-Based Logging (DBL) In some countries, Lawful Agencies also require the Service Provider to provide the Destination IP address and Port (along with Source IP address and Port information) as part of Logging Record. This is termed as Destination-Based Logging (DBL). NOTE: Including Destination information in the Logging Record will increase the size of the Record. DBL can be supported with Netflow v9 as well as with Syslog. Bulk Port Allocation (BPA) In order to reduce the size of the logging record, a feature called Bulk Port Allocation (BPA) is introduced. Without BPA, for every new flow, a Port is allocated for the Public IP and a new logging record is created with Private IP + Port and Public IP + Port. Similarly, another Logging record is created when a flow or NAT entry is deleted. Following table captures the working of it (without BPA) with some sample IP addresses and port numbers. Event Inside Source IP + Port Outside Source IP + Port Logging Record Additional Comments A new flow arrives 10.1.1.1:1000 170.0.0.5:8175 10.1.1.1:1000 <-> 170.0.0.5:8175 New port (8175) allocation happens. New logging record is created Existing flow arrives 10.1.1.1:1000 170.0.0.5:8175 N/A No new Port allocation happens. No logging record is created A new flow arrives (with new port, but same IP) 10.1.1.1:2000 170.0.0.5:13591 10.1.1.1:2000 <-> 170.0.0.5:13591 New port (13591) allocation happens. New logging record is created. Without BPA, for the first new flow, a bulk (group) of Ports are allocated (but only one port from the group is assigned for the new flow) for the Public IP and a logging record is created with Private IP and Public IP + Start Port + End Port. Next time, a new flow comes, a new port from the already allocated pool is taken and no logging record is created. Only when all the ports in the group are used, a new bulk is allocated and logging record is created. Similarly, when all the ports from a group is released, a (deletion) logging record is generated. Following table captures the working of it (with BPA enabled) with sample IP addresses and port numbers. Event Inside Source + Port Bulk Port Allocation (size 64) Outside Source IP + Port Logging Record Additional Comments A new flow arrives 10.1.1.1:1000 Allocates Bulk Port range (8192 - 8255) for 170.0.0.5 170.0.0.5:8201 10.1.1.1 <-> 170.0.0.5:8192-8255 Bulk port allocation and new port (8201) assignment happens. New logging record is created. Existing flow arrives 10.1.1.1:1000 N/A 170.0.0.5:8201 N/A No new Port allocation happens. No logging record is created. A new flow arrives (with new port, but same IP) 10.1.1.1:2000 N/A 170.0.0.5:8199 N/A New Port (8199) allocation happens (from same port range). No logging record is created. A new flow arrives (with new port, but same IP) 10.1.1.1:5000 N/A 170.0.0.5:8236 N/A New Port (8236) allocation happens (from same port range). No logging record is created. Following table illustrates the amount stoage space needed per second with varying translation creation + deletion rate for both Netflow and Syslog (with DBL disabled). It also shows the impact of BPA on the storage. NAT translation creation + deletion rate per second Netflow without BPA Syslog without BPA Netflow with Bulk size 256 and 50% utilization Syslog with Bulk size 256 and 50% utilization 20,000 0.8 MB 2.8 MB 6.25 KB 21.875 KB 50,000 2 MB 7 MB 15.625 KB 54.6875 KB 100,000 4 MB 14 MB 31.25 KB 109.375 KB 500,000 (maximum) 20 MB 70 MB 156.25 KB 546.875 KB   NOTE: Both DBL and BPA cannot be enabled at the same  time. This is because BPA requires logging record to be generated for a  set of NAT entries whereas DBL requires logging record to be generated  for each NAT entry. These are conflicting requirements. Please collect the following information and accordingly determine External Logging related parameter configuration. What setup rate needs to be supported ? Do the Lawful Agencies mandate DBL ? What protocol (netflow or syslog) the external logging server support ? How many packets per second the logging server can support ? How many logging servers need to be configured ? How much space would be needed for storing logging record ? How much would the cost of disk space ? Following table shows how to configure different external logging parameters. Configuration of External Logging Parameters   service cgn cgn1   service-type nat44 nat1     inside-vrf ivrf1         ! Define Netflow V9 logging configuration       external-logging netflow version 9         ! Define netflow server configuration         server           ! Define logging server address and port           address 10.10 . 0.0 port 2055           ! Define Path MTU size           path-mtu 12 00           ! Define refresh rate           refresh-rate 50           ! Define timeout           timeout 50       ! Define Syslog logging configuration       external-logging syslog         server           address 10.10 . 0.1 port 514 Back to top 14. NAT Session Timeouts (TCP, UDP, ICMP) As the new flows creates NAT entries, we need a way to delete those NAT entries as well. Protocols like UDP is connection-less and hence, there is no way to identify when a session / connection has ended by inspecting the packets. Also, in some cases, if no packet flows between the applications for a long period of time, there is no point in holding on to the NAT entries, as that will disallow creation of new NAT sessions. Due the above reasons, NAT entries are deleted when session timeout happens. For every protocol (TCP, UDP, ICMP), we have different types and values of session timeouts, as depicted below. TCP - It has two timeouts.                                                                Initial Timeout - It kicks in before the connection moves into Active state. Default: 120 sec. Active Timeout - It kicks in when the connection is in Active state. Default: 30 Minutes. UDP - It has two timeouts as well.                                                                Initial Timeout - It kicks in before the connection moves into Active state. Default: 30 sec. Active Timeout - It kicks in when the connection is in Active state. Default: 120 sec. ICMP - It has one timeout. Default: 60 sec. If the timeout is configured to be too less, NAT sessions will be deleted too soon, resulting in changing of Public IP addresses. This may cause issues with the Applications. If the timeout is configured to be too high, NAT sessions will remain in the system for long time (even though there is no data is flowing using the session) and it will restrict new sessions to be created as well (resulting in dropping of those packets). Hence, timeout value needs to be set appropriately. NOTE: These timeouts are set for the NAT44 instance (per ISM card) and cannot be configured differently for each Inside VRF. You need to have some knowledge about the following, to determine the timeout values. Application traffic that is going through the network                                                                 How sensitive it is with respect to change in IP address ? How periodically, it would generate packets How many NAT sessions are expected ? Following table shows how to configure timeout values for different protocols. Configuration of Session Timeout (TCP, UDP, ICMP)   service cgn cgn1   service-type nat44 nat1       ! Define UDP protocol session timeout values     protocol udp       session initial timeout 65535       session active timeout 65535     ! Define TCP protocol session timeout values     protocol tcp       session initial timeout 65535       session active timeout 65535     ! Define ICMP protocol session timeout value     protocol icmp       timeout 908 Back to top 15. Traffic Diversion (to ISM) We can use different mechanisms to divert the traffic to ISM card (to specific ServiceApp interface). Default Route If the inside traffic is arriving on a (Inside) VRF and ALL traffic needs to undergo NAT44 operation, we can add a default route under the VRF to divert traffic to the Inside ServiceApp (as shown below). Configuration for Traffic Diversion using Default route   ! Define static route for Inside-to-outside traffic ! ------------------------------------------------- router static   ! Specify the Inside VRF   vrf insidevrf1     ! Define address family     address-family ipv4 unicast       ! Re-direct all traffic to Inside Service App       0.0 . 0.0 / 0 serviceapp 1 Static Route In some cases, Outside ServiceApp interfaces reside in same / default VRF. Hence, in order to divert O2I traffic to specific Outside ServiceApp interfaces, static route can be used in the following way. Configuration for Traffic Diversion using Static Route   router static     address-family ipv4 unicast     ! Diverting traffic for one Public IP pool to one Outside ServiceApp     100.1 . 1.0 / 24 serviceapp 2     ! Diverting traffic for another Public IP pool to another Outside ServiceApp     200.1 . 1.0 / 24 serviceapp 4 ACL-based Forwarding (ABF) In certain deployment cases, default route may not work. Deployment Use-case 1: Customer wants to divert traffic coming from specific subscriber network (containing Private IP address) to ISM (for NAT44) and the remaining traffic (which may already contain Public IP address) need not go to ISM. Deployment Use-case 2: Customer wants to distribute subscriber traffic into different Inside VRFs which may go to same ISM card or different ISM cards (but, will certainly using different Public IP pool). Deployment Use-case 3: Subscriber / Access / Inside traffic comes in default / global VRF and we need to distribute the traffic to different Inside ServiceApps which are in different Inside VRFs. NOTE: Typically, ABF is to be used in Private side / Subscriber side and NOT towards Public / Core / Network side.   In the above cases, ABF provides a very good solution. Below is sample configuration on how to use ABF for traffic diversion. Configuration for Traffic Diversion using ABF   ! Define IPv4 Access List ipv4 access-list nat44_abf   ! Divert to ServiceApp 1   10 permit ipv4 10 .0 .100 .0 / 24 any nexthop1 vrf ivrf1 ipv4 5.1 . 1.2   ! Divert to ServiceApp 3   20 permit ipv4 10 .0 .200 .0 / 24 any nexthop1 vrf ivrf2 ipv4 6 .1 . 1.2 100 permit ipv4 any any ! Configure Ingress Interface interface TenGigE0/ 5 / 0 / 2   vrf subscriber     ipv4 address 10.1 . 1.1 255.255 . 255.0     ! Apply ABF on Ingress Interface    ipv4 access-group nat44_abf ingress ! Define one Inside ServiceApp interface interface ServiceApp1 vrf ivrf1     ipv4 address 5.1 . 1.1/30     service cgn cgn1 service-type nat44 ! Define another Inside ServiceApp interface interface ServiceApp3   vrf ivrf3     ipv4 address 6 .1 . 1.1/30     service cgn cgn1 service-type nat44 Back to top 16. Static Port Forwarding There is a very important point about Dynamic NAT44. All the NAT translations / sessions are created by I2O traffic. If O2I traffic reaches ISM for which there is no NAT entry, the packets will be dropped. This works fine for most of the cases as the session / traffic is initiated by Applications which reside on Private side. However, there are cases, where a server resides on the Private side of the network. In this case, there are 2 issues. Clients (on Public side) cannot initiate session to this Server, as those (O2I) packets will be dropped by ISM. Clients do not know the Public IP of the Server (and cannot use Private IP to communicate either) To address these problems related to Server sitting on Private side of the network, a new feature called "Static Port Forwarding" is introduced. Following diagram explains "Static Port Forwarding" feature functionality. Using Static Port Forwarding, user can provide the Private IP address and Port (via CLI) and NAT44 Application on ISM generates (dynamically) the corresponding Public IP and Port. Also, these mapping are made permanent - i.e., they do not get timed out and get deleted. The Public IP and port generated by the Application can be added to DNS database and thus made available to all. With this design, any device on Public side of network can now send traffic to the known Public IP and port (to reach the Server sitting inside Private network). Also, as the NAT entry is permanent, O2I traffic always finds the entry and using the same can translate the packet (without dropping it). Staic Port Forwarding entries can be created separately for each protocol - TCP, UDP and ICMP. To know the maximum number of such entries, please check CGv6 on ISM Feature page.   NOTES: - User cannot provide the Outside / Public IP and port while configuring Static Port Forwarding. - Outside / Public IP is generated by the NAT44 Application and hence, can be random. - Outside port is usually assigned same as Inside Port, unless the Outside port is already being used As these are used for Servers located in Private network, it is desirable that the Public IP address remain constant across RP / ISM failover. As Outside / Public IP address for a Static Port Forwarding entry is allocated by NAT44 application based on the instant situation of Public IP availability stock as well as the allocation algorithm's state, it is highly probable that if the Static Port Forwarding entry is deleted and re-added or, if the ISM card is reloaded), it may allocate a different Public IP address. In order to keep the same Outside IP address, it is recommended to add the Static Port Forwarding configuration as a first thing after ISM card comes up and when there is no NAT traffic on the router/ISM card (could be in a "maintenance window"). That way, every time, ISM is reloaded, the Public IP addresses assigned for Static Port Forwarding entries will remain the same. If you are using ISM 1:1 redundancy, Backup ISM will assign the same Public IP as assigned in Active ISM (irrespective when the entries are added). Try to find out the following, before using Static Port Forwarding. Will there be any Servers sitting in the Private side of the deployment network ? How many of those Servers will be ? Can the Public IP address of the Servers be random and can be published via DNS or similar mechanism ? A sample configuration for Static Port Forwarding is shown below. Configuration for Static Port Forwarding   service cgn cgn1   service type nat44 nat1     inside-vrf ivrf1     map address-pool 180.0.0.0/24     ! Define TCP related configuration     protocol tcp       ! Define Static Port Forwarding configuration       static -forward inside address 10.0 . 0.50 port 6050       static-forward inside address 10.0.0.50 port 80     ! Define UDP related configuration     protocol udp       ! Define Static Port Forwarding configuration       static -forward inside address 10.0 . 0.61 port 550     ! Define ICMP related configuration     protocol icmp       ! Define Static Port Forwarding configuration       ! For ICMP, 'identifier' field in ICMP header is used for 'port'       static -forward inside address 10.0 . 0.189 port 750 Following table shows the NAT entries corresponding to the above Static Port Forwarding entries (for TCP protocol). "show cgn nat44 <> inside--translation" output for Static Port Forwarding     RP/0/RSP0/CPU0:VSM-Lab#show cgn nat44 nat44-1 inside-translation protocol tcp inside-vrf vrf1                        inside-address 10.0.0.50 port start 25 end 8000 Fri Nov  1 13:44:44.874 UTC Inside-translation details --------------------------- NAT44 instance : nat44-1 Inside-VRF     : vrf1 --------------------------------------------------------------------------------------------    Outside         Protocol  Inside       Outside       Translation   Inside      Outside    Address                   Source       Source        Type          to          to                              Port         Port                        Outside     Inside                                                                       Packets     Packets --------------------------------------------------------------------------------------------   180.0.9.0          tcp       80           80            static       1218         45   180.0.9.0          tcp     6050         6050            static        516       7162     NOTE: - Outside IP (180.0.9.0) and Ports are assigned by the NAT44 Application when user supplies Inside IP (10.0.0.50) and Ports (80, 6025). - "Translation Type" is shown as "static" (as opposed to "dynamic") - Outside port is same as Inside port (usually, unless it is not available) Back to top 17. Application Layer Gateway (ALG) Application Layer Gateway (ALG) is a device which has the capability of inspecting into a packet beyond layer 4 and can change Application layer content to meet certain need. NAT44 changes IP address and Layer 4 (UDP/TCP/ICMP etc.) port. There are certain applications which embeds IP address and/or Port information in Application content. Now, the issue is while the packet traverses across a NAT44 device, layer 3 and 4 addresses get changed. However, in the Application layer, the content still contains old layer 3 and / or layer 4 addresses. This causes an issue in the Application as this is unexpected. Hence, the need of ALG functionality within a NAT device which while changing Layer 3 and 4 packet content, also changes Application content where Layer 3 and 4 addresses were embedded. With this change, end Application will be happy as it does not need to deal with 2 sets of addresses any more. However, there are 2 ways this problem can be solved. Adding ALG functionality to NAT device - ALG functionality needs to be added to NAT device for each such Applicaton which embeds Layer 3 and/or 4 addresses in Application content. Changing the Application to deal with NAT device in the network - Application needs to assume that the Layer 3 and/or 4 address can get changed on the way (as there could be NAT device in the network) and hence, either avoid embedding Layer 3/4 addresses in Application content or find other mechanism to handle 2 sets of addresses. The main challenges in adding ALG functionality to NAT device are: Every time a new Application is written which embeds Layer 3/4 address in Application content, all NAT devices need to be changed to address the new Application. This is not scalable as lot of new Applications are written everyday. In order to implement ALG for an Application, NAT device needs to understand the Application protocol as well as packet formt/content. Many Applications implement proprietary protocol and hence, do not want to make the same as public information. Hence, those Application packets will face issues with NAT device. There are applications (like, crypto) which do not want any intermediate device to understand / peek into the Application content and modify it. These packets should only be encoded/decoded by end Applications. These application packets would not be able to deal with intermediate NAT device. On the contrary, Application can be written in such a way not to use Layer 3/4 addresses in Application content and to use Application-level ID, instead. There can be other mechanisms, e.g. - using STUN (Session Traversal Utilities for NAT) that applications can take to handle intermediate NAT device. NOTE: Due to the above reasons, our solution does not encourage to support ALG within NAT44 functionality. However, few basic ALG functionality is implemented with NAT44 Application. These are: Active FTP (File Transfer Protocol) - Passive FTP does not need ALG. RTSP (Real Time Streaming Protocol) PPTP (Point-to-Point Tunneling Protocol) Following table show how to configure / enable different ALGs on ISM. Configuration of ALGs   service cgn cgn1     service-type nat44 nat1    ! Enable Active FTP ALG    alg ActiveFTP     ! Enable RTSP ALG (from 4.2 . 1 release onwards)     alg rtsp    ! Enable PPTP ALG (from 4.3 . 1 release onwards)    alg pptpAlg Back to top 18. Redundancy Redundancy is an integral part of CGN (Carrier Grade NAT). Majority of the Service Providers want Redundancy in their network to make it more resilient. It helps in minimizing the traffic outage when one device fails. On ISM, we support 2 types of redundancy. 1:1 warm standby redundancy N:1 ABF-based redundancy 1:1 Warm Stand-by Redundancy In this mode, one ISM card acts as Primary and another as backup. Following are the properties of 1:1 Warm Stand-by redundancy. Easy to configure (Active and Stand-by locations are specified once) A single Public IP address pool is used Configuration is sent to both ISM cards NAT Application data traffic reaches to Active card only During switchover (usually takes few seconds), dynamic NAT entries get deleted (on failed ISM cards) and re-created on newly Active ISM card. Hence, there would be some traffic drop. Also, Private to Public IP/Port mapping gets changed during Switchover. When the previously Active ISM comes up, it stays as Stand-by and the traffic does not come back to it (until the Active ISM card fails) Each ISM can support full scale and throughput Following diagram explains the working of 1:1 Redundancy for NAT44. Please note that both Primary and Backup ISM cards share same ServiceApp interfaces and NAT44 configuration (including Public IP pool). Following tables shows how to configure 1:1 ISM redundancy. Configuration for 1:1 Warm Standby Redundancy   ! Configure role for both Primary and Backup ISMs hw-module service cgn location 0/1/cpu0 hw-module service cgn location 0/2/cpu0 ! Configure ServiceInfra interfaces on both ISMs interface ServiceInfra 1   ipv4 address 1.1.1.1/30   service-location 0/1/cpu0 interface ServiceInfra 2   ipv4 address 2.2.2.1/30   service-location 0/2/cpu0 ! Configure preferred-active and preferred-standby ISM service cgn cgn1   service-location preferred-active 0/1/cpu0 preferred-standby 0/2/cpu0   service-type nat44 nat1     inside-vrf inside1       map address-pool 151.0.1.0/24       ! Remaining configurration       ... To check the health of the ISM card and trigger redundancy switchover, in case any failure occurs, following HA-specific tests are executed. Punt Path Test - It checks if the packet path between ISM LC (PPC) CPU and CGv6 Application works fine or not. Data Path Test - It checks if the packet path from CGv6 Application to Fabric works fine or not. If these tests are enabled and any failure is detected by these, ISM card will be reloaded. NOTE: By default, these tests run in the back-ground however, no acton is taken in case failure is detected. Using the following CLIs, these tests are enabled. Configuration of HA Punt-Path and Data-Path tests ! To enable Punt Path Test service-cgv6-ha location 0/1/cpu0 puntpath-test   ! To enable Data Path Test service-cgv6-ha location 0/1/cpu0 datapath-test N:1 ABF-Based Redundancy In this case, there needs to be N Primary ISM card and 1 Backup ISM card. Value of N can be 1 as well, but typically would be 2 or 3. Following are different characteristics of this solution. Each ISM card needs to be configured independently                                                                        Configuration (usually) should be similar (w.r.to parameters, number of ServiceApp interfaces, etc.) In ABF configuration, For each Primary ISM,                                                                         ServiceApp interface on Primary ISM becomes 1st Next-Hop and ServiceApp interface on Back-up ISM becomes 2nd Next-hop Each ISM will have different Public IP address pools (Address Pool is not same in Primary and Backup) When one Primary ISM card fails, Backup ISM card receives all the traffic of Primary ISM card. When Primary ISM card comes up, all traffic again switches back to it from Backup ISM card (it causes, another traffic outage) Each ISM card can support full scale and bandwidth Following table shows how the ABF Next-Hop configuration would look like for N:1 (N = 3) ABF-based Redundancy. ServiceApp interfaces ABF Next-Hop 1 ABF Next-Hop 2 ServiceApp interfaces on Primary ISM 1 ServiceApp interfaces on Primary ISM 1 ServiceApp interfaces on Backup ISM ServiceApp interfaces on Primary ISM 2 ServiceApp interfaces on Primary ISM 2 ServiceApp interfaces on Backup ISM ServiceApp interfaces on Primary ISM 3 ServiceApp interfaces on Primary ISM 3 ServiceApp interfaces on Backup ISM Following diagram depicts N:1 (N = 2) configuration with 1 set of ServiceApps. As shown in the diagram, before 1st failure, traffic coming from 10.2.0.0/24 subnet will go via Primary 1 ISM (Inside ServiceApp 1) which will use Public IP pool of 151.0.1.0/24. if and when Primary 1 ISM fails, traffic coming from 10.2.0.0/24 subnet will go via Backup ISM (Inside ServiceApp 5) which will use Public IP pool of 151.0.3.0/24. Instead of Primary 1 ISM, if Primary 2 ISM would have failed, traffic from 10.2.1.0/24 (not 10.2.0.0/24) should have switched to Backup ISM. Following table captures the relevant ABF configuration for this sample setup. Sample Configuration for N:1 Redundancy using ABF     ! ABF rule for Inside to Outside traffic ipv4 access-list abf_2_1   10 permit ipv4 10.2.0.0/24 any nexthop1 vrf inside1 ipv4 1.1.1.2                                  nexthop2 vrf ibackup ipv4 1.1.5.2   20 permit ipv4 10.2.1.0/24 any nexthop1 vrf inside2 ipv4 1.1.3.2                                  nexthop2 vrf ibackup ipv4 1.1.5.2   30 permit ipv4 any any ! Static route for Outside to Inside traffic router static   address-family ipv4 unicast     151.0.1.0/24 ServiceApp 2     151.0.2.0/24 ServiceApp 4     151.0.3.0/24 ServiceApp 6 Back to top 19. NAT44 Licenses In order to use NAT44 functionality on ASR9K, Licenses needs to be purchased. Depending on the number of Stateful Translations needed by the solution, required number of NAT44 specific licenses can be purchased. This fits nicely with "Pay as you grow" model as well. Following licenses need to be purchased for NAT44 and other CGv6 applications. License Part Number License Description Additional Comments A9K-CGN-LIC-5M Cisco ASR 9000 CGN / NAT44 License (1 per 5 million translations) For 20 Millions translations, 4 of these licenses will be needed. A9K-DSLT-LIC-5M Cisco ASR 9000 DS-Lite License (1 per 5 million translations) Similar logic as above. Not needed for NAT44 / CGN solution. A9K-NAT64-LIC-5M Cisco ASR 9000 Stateful NAT64 License (1 per 5 million translations) Similar logic as above. Not needed for NAT44 / CGN solution. Details of the Licensing (how to install, etc.) can be obtained from ASR9K S/w License Operations White Paper. Back to top 20. Summary of parameter values In the following table, all the useful parameters are listed along with their default and valid range of values. You can consult this table to come up with the parameter value for your deployment case. Parameter (CLI) Defined at what level Default Value Range of Values Supported from XR Release Additional Comments alg ActiveFTP per NAT44 instance Disabled Enable / Disable 4.2.0 Active FTP ALG alg pptpAlg per NAT44 instance Disabled Enable / Disable 4.3.1 PPTP ALG alg rtsp per NAT44 instance Disabled Enable / Disable 4.2.1 RTSP ALG bulk-port-alloc size per Inside VRF Disabled 16, 32, 64, 128, 256, 512, 1024, 2048, 4096 4.2.1   dynamic-port-range start per NAT44 instance 1024 1 - 65535 4.2.0 Starting port number (for Public IP address) to be used for Dynamic NAT entries. external-logging netflow v9 server path-mtu per Inside VRF 1500 bytes 100 - 2000 4.2.0   external-logging netflow v9 server refresh-rate per Inside VRF 500 packets 1 - 600 4.2.0 For sending NFv9 template external-logging netflow v9 server timeout per Inside VRF 30 seconds 1 - 3600 4.2.0 For sending NFv9 template external-logging syslog server path-mtu per Inside VRF 1500 bytes 100 - 2000 4.2.1   map ip many-to-one per Inside VRF Disabled Enable / Disable 4.3.2   map ip one-to-one per Inside VRF Disabled 1 - 65535 4.2.3   portlimit per NAT44 instance 100 1 - 65535 4.2.0   protocol udp session initial timeout per NAT44 instance 30 sec 1 - 65535 4.2.0   protocol udp session active timeout per NAT44 instance 120 sec 1 - 65535 4.2.0   protocol tcp session initial timeout per NAT44 instance 120 sec 1 - 65535 4.2.0   protocol tcp session active timeout per NAT44 instance 1800 sec 1 - 65535 4.2.0   protocol tcp mss per Inside VRF   28 - 1500 bytes 4.2.0   Specific Deployment Scenarios / Use-Cases In this section, specific deployment scenarion / use-cases and presented which can be useful for some deployment. Back to top 21. Sample Applications over NAT44 Following table lists some of the important / well-known applications and their working status over NAT44 (on ISM).   NOTE: - This list is not a complete/exhaustive list of applications that would work with NAT44. - It also does not mean that any application not listed here will not work with NAT44. Application Details of Application (Protocol / Port) Working Status with NAT44 HTTP Typically HTTP server runs over TCP Port 80 Works with NAT44 E-mail Typically uses TCP, server ports: SMTP (25, 587), POP3 (110), IMAP (143) Works with NAT44 FTP Typically uses TCP port 20 (data) and port 21 (control) Works with NAT44 Telnet Typically uses TCP port 23 Works with NAT44 SSH Typically uses TCP port 22 Works with NAT44 VNC Typically uses TCP port 5900, 5800, 5500 Works with NAT44 VPN Client (SSL) Typically uses TCP Works with NAT44 VPN Client (PPTP+GRE) Typically uses TCP and GRE Works with NAT44 VPN Client (IPSec) Typically uses IPSec (via NAT-T) Works with NAT44 (as NAT-T uses UDP) Xbox Gaming   Works with NAT44 Back to top 22. Multiple Outside ServiceApps sharing same Outside VRF It is very common to have default VRF as Outside VRF and when there are multiple Inside VRFs mapped to the same. In the following table, a sample ServiceApp and VRF configuration is shown. Please note that in this case, multiple Outside ServiceApps share share a comon Outside VRF. Inside VRF Inside ServiceApp Outside VRF Outside ServiceApp ivrf1 ServiceApp 1 default or ovrf ServiceApp 2 ivrf2 ServiceApp 3 default or ovrf ServiceApp 4 To handle this scenario, we need to specify "outsideServiceApp" parameter configuration to indicate what is the Outside ServiceApp to be used with the mapping. Static route rule for specific Public IP address pool should guide the traffic to specific Outside ServiceApp. Only these configurations are shown below. Rest of the configuration remain the same. Configuration for Shared / Common Outside VRF service cgn cgn1   service-type nat44 nat1     ! 1st Inside VRF     inside-vrf ivrf1       ! MAP command using outsideServiceApp       map [outside-vrf ovrf] outsideServiceApp ServiceApp2 address-pool 100.0.1.0/24     ! 2nd Inside VRF     inside-vrf ivrf2       ! MAP command using outsideServiceApp       map [outside-vrf ovrf] outsideServiceApp ServiceApp4 address-pool 100.0.2.0/24 router static [vrf ovrf] address-family ipv4 unicast ! Divert traffic to 1st Outside ServiceApp 100.0.1.0/24 ServiceApp 2     ! Divert traffic to 2nd Outside ServiceApp     100.0.2.0/24 ServiceApp 4 Back to top 23. NAT Bypass There is an use-case where traffic selected traffic coming onto ASR9K via a Private side interface needs to go via NAT44 application, whereas remaining traffic can bypass NAT44 and should get forwarded via regular forwarding rule. This is called NAT bypass. To implement NAT bypass, let's consider this situation: Private Traffic enters ASR9K via one VRF (say, RED) Selected traffic gets diverted to ISM card's Inside ServiceApp which is in different VRF (say, BLUE) and after translation comes out of Outside ServiceApp and eventually goes out of ASR9K via core-facing interface which is in RED VRF. Rest of the traffic (which bypasses NAT44) goes out of ASR9K via same core-facing i/f in RED VRF. Please note that other than NAT bypass, there could be other deployment scenario where preserving VRF (across ISM) becomes a necessity. But, the solution would be same / very similar. The above scenario can be achieved using ISM with some additional ABF and Static Route and appropriate VRF configuration. Let us take a look at the following diagram as a sample scenario. In the above diagram, traffic enters and exits out of ASR9K in RED VRF. Also, please note that Inside ServiceApp is in BLUE vrf whereas Outside ServiceApp is in RED VRF. Following table shows the specific or additional configuration that is needed for appropriate traffic diversion. Additional Configuration to support NAT Bypass   ! Define ACL to divert traffic to ServiceApp 1 (RED to BLUE VRF) ipv4 access-list cgn_abf   ! Divert traffic to ServiceApp 1 in BLUE VRF   10 permit ipv4 10.10.10.0/24 any nexthop1 vrf BLUE ipv4 1.1.1.2   20 permit ipv4 any any ! Apply ABF on Ingress LC interface interface TenGigE 0/0/0/0.1   vrf RED   ipv4 address 10.10.10.1/24   ! Apply Ingress ABF   ipv4 access-group cgn_abf ingress ! We need a special static route to divert O2I traffic (after reverse NAT) ! to the right interface router static   ! After reverse NAT, look-up is done in BLUE (Inside) VRF   vrf BLUE     address-family ipv4 unicast       ! Divert to TenGigE 0/0/0/0.1 interface       10.10.10.0/24 vrf RED 10.10.10.2 Back to top 24. Interworking of BNG and NAT44 Broadband Network Gateway (BNG) is the access point for subscribers, through which they connect to the broadband network.  When a connection is established between BNG and Customer Premise Equipment (CPE), the subscriber can access the broadband services provided by the Network Service Provide (NSP) or Internet Service Provider (ISP). ASR9000 supports BNG functionality. The goal of the BNG architecture is to enable the BNG router  to interact with peripheral devices (like CPE) and servers (like AAA and DHCP), in order  to provide broadband connectivity to subscribers and manage subscriber sessions. Each subscriber (or more specifically, an application running on the CPE) connects to the network by a logical session. Based on the protocol used, subscriber sessions are classified into two types: PPoE subscriber session —The PPP over Ethernet (PPPoE) subscriber session is established using the point-to-point (PPP) protocol that runs between the CPE and BNG. PoE subscriber session—The IP over Ethernet (IPoE) subscriber session is established using IP protocol that runs between the CPE and BNG; IP addressing is done using the DHCP protocol. For details on BNG, please refer to BNG Deployment Guide on ASR9K. On the other hand CGN (NAT44) functionality is also supported on ASR9K. Hence, BNG and CGN (NAT44) functionality can interwork to provide NAT'ed (public) IP address to the subscriber traffic (both PPPoE and IPoE). The basic BNG + CGN (NAT44) architecture is shown in the following figure. As shown in the above diagram, subscriber traffic reaches ASR9K router via Bundle Ether interface (in different VLANs), which is in VRF ivrf. It does not necessarily need to arrive in a VRF though. Please note that session establishment happens without the involvement of NAT44. NAT44 comes into picture when Subscriber data traffic reaches ASR9K. After the BNG functionality (like, PPPoE or IPoE termination/de-capsulation) is performed on the Subscriber data traffic (on LC), internal IP packet (containing Private IP address as Source) is sent to ISM card where NAT44 functionality kicks in which converts the Private IP address into Public IP address. Similarly, on the reverse (O2I) path, after ISM performs NAT44 functionality (changes destination IP address to Private), paket is sent to LC where BNG functionality (like, PPPoE or IPoE encapsulation) is peformed before sending the packet to the Subscriber. Following table shows sample snippet of BNG and CGN (NAT44) configuration ( for IPoE subscriber ) to achive the same. For PPPoE subscriber, corresponding BNG configuration needs to be used instead. Please refer to ASR9000 Broadband Network Gateway Configuration Guide for detailed and release-specific BNG configuration. Sample BNG and CGN (NAT44) Configuration for BNG+CGN Interworking     ! ============================================= ! --- BNG Configuration for IPoE Subscriber --- ! ============================================= ! DHCP Configuration dhcp ipv4 profile proxyProfile proxy   helper-address vrf default 13.0.0.2 giaddr 13.0.0.1 ! interface Bundle-Ether1.2 proxy profile proxyProfile ! Dynamic Template Configuration dynamic-template type ipsubscriber dhcp   vrf ivrf   ipv4 unnumbered Loopback1 ! Policy-Map and Class-map configuration policy-map type control subscriber pol1 event session-start match-all   class type control subscriber cls1 do-all    1 activate dynamic-template dhcp    2 authorize aaa list default identifier source-address-mac password cisco   end-policy-map class-map type control subscriber match-all cls1 match protocol dhcpv4 end-class-map ! Interface (Bundle-Ether and Loopback) Configuration interface Bundle-Ether1.2 vrf ivrf ipv4 point-to-point ipv4 unnumbered Loopback1 proxy-arp service-policy type control subscriber pol1 encapsulation dot1q 10 ipsubscriber ipv4 l2-connected   initiator dhcp interface Loopback1 vrf ivrf ipv4 address 10.0.0.1 255.255.0.0 ! ========================== ! CGN (NAT44) Configuration ! ========================== hw-module service cgn location 0/5/CPU0 ! Interface (ServiceInfra and ServiceApp) Configuration interface ServiceApp1 vrf ivrf ipv4 address 1.1.1.1 255.255.255.0 service cgn cgn1 service-type nat44 ! interface ServiceApp2 ipv4 address 2.2.2.2 255.255.255.0 service cgn cgn1 service-type nat44 ! interface ServiceInfra1 ipv4 address 200.1.1.1 255.255.255.0 service-location 0/5/CPU0 ! Service CGN Configuration service cgn cgn1 service-location preferred-active 0/5/CPU0 service-type nat44 nat1   inside-vrf ivrf    map address-pool 100.2.0.0/16 ! Static Route Configuration router static address-family ipv4 unicast     100.2.0.0/16 ServiceApp2 ! vrf ivrf   address-family ipv4 unicast    0.0.0.0/0 ServiceApp1 Back to top 25. MPLS (Layer 3) VPN and NAT Interworking An Multiprotocol Label Switching (MPLS) Layer 3 Virtual Private Network (L3VPN) consists of a set of sites that are interconnected by means of an MPLS provider core network. At each customer site, one or more customer edge (CE) routers attach to one or more provider edge (PE) routers. MPLS VPNs are easier to manage and expand than conventional VPNs. When a new site is added to an MPLS VPN, only the edge router of the service provider that provides services to the customer site needs to be updated. The components of the MPLS VPN are described as follows: Provider (P) router — Router in the core of the provider network. PE routers run MPLS switching and do not attach VPN labels to routed packets. VPN labels are used to direct data packets to the correct private network or customer edge router. Provider Edge (PE) router — Router that attaches the VPN label to incoming packets based on the interface or subinterface on which they are received, and also attaches the MPLS core labels. A PE router attaches directly to a CE router. Customer edge (CE) router — Edge router on the network of the ISP that connects to the PE router on the network. A CE router must interface with a PE router. Details on L3VPN can be obtained from ASR9K MPLS Layer 3 VPN Configuration Guide. Service providers and enterprises that have network application services they want to offer or share with customers and partners will want to      minimize any connectivity burden placed on the user of the service. It is desirable, even mandatory, to extend the offering to as many potential users as needed to achieve the desired goals or return. By deploying NAT within the common MPLS VPN infrastructure, communications service providers can relieve some of the connectivity burden on      customers and accelerate their ability to link more shared application services to more consumers of those services There are two options for NAT deployment within the MPLS provider edge: Distributed with ingress NAT PEs Centralized with egress NAT PEs Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs. With this design, scalability is maintained to a large extent while performance is optimized by distributing the NAT function over many edge devices. Each NAT PE handles traffic for sites locally connected to that PE. NAT rules and access control lists or route maps control which packets require translation. In the above case, at ingress PE router, customer private IP addresses are NAT'ed to public IP address by ISM card and then MPLS labels are added by it before the packet goes out of ASR9K. On the return path, MPLS label(s) are removed by MPLS-facing LC and IT packets are sent to ISM. ISM performs reverse NAT and sends the packet to LC which then sends it back to CE router. Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs. With this design, scalability is reduced to some degree since the central PE must maintain routes for all customer networks that access the shared service. The application performance requirements must also be considered so that the traffic does not overburden the router that must translate the IP addresses of the packets. Because NAT occurs centrally for all customers using this path, IP address pools can be shared; thus, the total number of subnets required is reduced. In the above case, in the egress PE router, first MPLS labels are removed by ingress LC and the IP packet is sent to ISM card. Customer private IP addresses are NAT'ed to public IP address by ISM card and the packets are forwarded to shared service network. In the reverse path, ISM translates Public destination IP to private IP addresses and then adds MPLS labels before sending it to egress LC. Egress LC swaps one label and sends the packet out of ASR9K towards MPLS network. NOTE: MPLS labels should be removed by LC before sending the packet to ISM as IP packet. However, ISM can add the required MPLS labels before sending the packet to egress LC. For sample MPLS VPN configuration, please refer to "Configuration Example: MPLS VPN on XR" document. Back to top 26. NAT Hairpining Host behind a NAT device (on Private side) communicating with another host behind the same NAT device (on Private side) is called "NAT Hairpinning". Let us consider the following deployment use-case. We have a server behind a NAT device which has a Public IP as well, created using Static Port Forwarding configuration. Now, this server can be accessed from hosts on public side of NAT device as well as Private side of the NAT device. When a host on Private side of the NAT device tries to communicate with the Server, it uses NAT Hairpinning. Following diagram depicts how NAT hairpinning works with NAT44 on ISM. In this case, we first add a Static Port Forwarding entry for a service offered by the Server at 10.100.1.2:2055. Let's say, the Public IP pool configured is 200.2.0.0/24. NAT44 application allocates Public IP and port as 200.2.0.210:2055 (Please note that the port is preserved here). This Public IP and port is published / made available to the hosts. Let's the packet flow now when a host with Private IP 10.10.10.2 tries to access the service. Step 1: Host sends a packet (with Source IP 10.10.10.2 and port 3000) to Server's Public IP and port and the packet reaches ASR9K on TenGigE 0/0/0/0.1 which is in VRF ivrf. Step 2: Because of Static Route configuration in VRF ivrf, the packet is forwarded to ISM card via (Inside) ServiceApp 1 interface. Step 3: Application treats this as I2O packet. A new dynamic NAT entry is created by NAT44 application and it translates source IP/Port from (Private) 10.10.10.2:3000 to (Public) 200.2.0.192:23516. Please note that the Destination IP address and port are not touched here. The packet is sent out and because of Static route definition for Public IP pool, the packet comes back to ISM card via (Outside) ServiceApp 2. Step 4: Application treats this as O2I packet. As there is already a (Static) NAT entry exists for Destination IP and port, NAT44 application translates Destination IP address/Port from (Public) 200.2.0.210:2055 to (Private) 10.100.1.2:2055. Please note that the Source IP address and port are not touched here. The packet is sent out and via regular forwarding rule, it reaches egress i/f TenGigE 0/0/0/0.2. Step 5: The packet is forwarded to the Server with Source IP/Port as 200.2.0.192:23516 and Destination IP/Port as 10.100.1.2:2055 and the server consumes it. Response from the Server will take the reverse path and it will use Dynamic NAT rule in first reverse NAT and then Static NAT rule in second NAT processing, to reach the host. In this case, Static Port Forwarding configuration is used for Server as an use-case. But, it can be another host as well for which there is no Static Port Forwarding configuration. Only important thing is sender needs to know the Public IP and port of the receiver. Please note that there is no specific NAT44 configuration is needed for this. Regular / basic NAT44 configuration will work here. Hence, no configuration is provided here. Back to top 27. Handling of Asymmetric I2O and O2I traffic with better performance ISM will provide optimal performance when I2O and O2I traffic is symmetric in nature i.e., equally balanced. However, it may not be always true. Many times, O2I traffic (download from internet) is much higher than I2O (upload to internet) traffic. To obtain optimal / maximum performance, you can divide traffic in each logical partition into 2 sub-groups (which means Public IP address pool as well needs to divided), with another pair of ServiceApps and switching the order of direction (Inside / Outside) of ServiceApps. Following table tries to summarize it with 2 pairs of ServiceApps. Logical Group 1 Logical Group 2 Inside ServiceApp Outside ServiceApp ServiceApp 1 ServiceApp 2 ServiceApp 6 ServiceApp 5 Inside ServiceApp Outside ServiceApp ServiceApp 3 ServiceApp 4 ServiceApp 8 ServiceApp 7 Please note that ServiceApp 5 and ServiceApp 7 are configured as Outside ServiceApp, instead of Inside ServiceApp. Similarly, ServiceApp 6 and ServiceApp 8 are configured as Inside ServiceApp, instead of Outside ServiceApp. You will get a better performance provided you can distribute your traffic in nearly equal subscriber groups. Following table captures sample configuration for the same. Configuration for Handling Asymmetric I2O and O2I traffic for better performance   ! Inside ServiceApp configuration interface ServiceApp 1   vrf ivrf1   ipv4 address 1.1.1.1/30 interface ServiceApp 3   vrf ivrf2   ipv4 address 1.1.3.1/30 interface ServiceApp 6   vrf ivrf3   ipv4 address 1.1.6.1/30 interface ServiceApp 8   vrf ivrf4   ipv4 address 1.1.8.1/30 ! Outside ServiceApp configuration interface ServiceApp 2   ipv4 address 1.1.2.1/30 interface ServiceApp 4   ipv4 address 1.1.4.1/30 interafce ServiceApp 5   ipv4 address 1.1.5.1/30 interface ServiceApp 7   ipv4 address 1.1.7.1/30 ! ABF Configuration to divert traffic to all Inside ServiceApp interfaces ipv4 access-list cgn_abf   ! For traffic to ServiceApp 1   10 permit 10.0.1.0/24 any nexthop1 vrf ivrf1 1.1.1.2   ! For traffic to ServiceApp 3   30 permit 10.0.2.0/24 any nexthop1 vrf ivrf2 1.1.3.2   ! For traffic to ServiceApp 6   60 permit 10.0.3.0/24 any nexthop1 vrf ivrf3 1.1.6.2   ! For traffic to ServiceApp 8   80 permit 10.0.4.0/24 any nexthop1 vrf ivrf4 1.1.8.2   100 permit any any ! Service CGN Configuration service cgn cgn1   service-type nat44 nat1   ! Address Pool for ServiceApp 1 and 2 pair   inside-vrf ivrf1     map outsideServiceApp 2 address-pool 100.0.1.0/24   ! Address Pool for ServiceApp 3 and 4 pair   inside-vrf ivrf2     map outsideServiceApp 4 address-pool 100.0.2.0/24   ! Address Pool for ServiceApp 6 and 5 pair   inside-vrf ivrf3     map outsideServiceApp 5 address-pool 100.0.3.0/24   ! Address Pool for ServiceApp 8 and 7 pair   inside-vrf ivrf4     map outsideServiceApp 7 address-pool 100.0.4.0/24 Validation of Configuration Once NAT44 on ISM is configured, we need different ways (mainly, via "show" CLIs) to validate those. In this section, important NAT44 related CLIs are listed and also explained how those can be used for validation. Back to top 28. show run As a first step, different "show run" output should be verified to ensure that configuration commands are rightly applied. Following commands can be used. 'show run' commands Usage of it show running-configuration  or show run - Displays all the configuration on ASR9K. - If you have less configuration, this is better way to get all NAT44 configuration at one shot. - Otherwise (if you have lot of configuration, this may not be the best way to get all NAT44 configuration show run service cgn * - Displays only CGN related information show run interface service* - Displays all ServiceInfra and ServiceApp interface configuration - Also includes Service-Mgmt and Service-Engine interface configuration (which can be ignored) show run router static - Displays all static route information. Back to top 29. show cgn nat44 <> statistics This is a very important CLI related to NAT44. A sample output of the CLI is shown below. Sample "show cgn nat44 <> statistics" CLI Output     RP/0/RSP0/CPU0:nat-tasix#Show cgn nat44 nat statistics Fri Aug 16 23:34:40.018 TSK Statistics summary of NAT44 instance: 'nat' Number of active translations: 91407 Number of sessions: 39716 Translations create rate: 542 Translations delete rate: 1206 Inside to outside forward rate: 147827 Outside to inside forward rate: 204847 Inside to outside drops port limit exceeded: 10077470 Inside to outside drops system limit reached: 0 Inside to outside drops resource depletion: 0 No translation entry drops: 194533431 PPTP active tunnels: 3 PPTP active channels: 2 PPTP ctrl message drops: 1246 Number of subscribers: 0 Drops due to session db limit exceeded: 0 Pool address totally free: 498 Pool address used: 1550 Pool address usage: -------------------------------------------------   External Address       Ports Used -------------------------------------------------   213.230.92.0           79   213.230.92.4           176   213.230.92.8           59   213.230.92.12          36   213.230.92.16          79   213.230.92.20          52   213.230.92.24          65   ...   213.230.103.239        2   213.230.103.247        1   213.230.103.255        1 ------------------------------------------------- Following table explains the impotant fields from the above CLI and how it can be used for validation. Please note that the values displayed by the CLI is valid at that instant for that particular ISM card. It may change at a later point of time. Field Name Description Additional Comments for Validation Number of active translations Number of NAT DB (Static + Dynamic) Entries Monitoring this over a period of time will provide the NAT44 translation scale used in the ISM card. Number of sessions Number of Session DB (Static + Dynamic) Entries Monitoring this over a period of time will provide the NAT44 session  scale used in the ISM card. Translations create rate Number of NAT44 translations created in last 1 second Monitoring this over a period of time will provide the NAT44 session  setup rate on the ISM card. Translations delete rate Number of NAT44 translations deleted  in last 1 second Monitoring this over a period of time will provide the NAT44 session  teardown rate on the ISM card. Inside to outside forward rate Number of packets gone through NAT44 translation in I2O direction in last 1 second Monitoring this over a period of time will provide the NAT44 packet processing rate on ISM card (PPS) in I2O direction Outside to inside forward rate Number of packets gone through NAT44 translation in O2I direction in last 1 second Monitoring this over a period of time will provide the NAT44 packet processing rate on ISM card (PPS) in O2I direction Inside to outside drops port limit exceeded Number of I2O packets dropped because no NAT entries cannot be created for the specific users which has reached port limit. If you have configured port limit value too less, you may hit this errors very often. You may need to consider to increase the port limit configuration in that case. Inside to outside drops system limit reached Number of I2O packets dropped because NAT DB or User DB is full. In case you are hitting this very often, you may need to check if you're hitting the scale limit and if so, you may need to go for another ISM card. Inside to outside drops resource depletion Number of I2O packets dropped because Layer 4 port could not be allocated.   No translation entry drops Number of O2I packets dropped because there was no NAT entry existing.   Number of subscribers Number of NAT Users   Drops due to session db limit exceeded Number of I2O packets dropped because Session DB is full.   Back to top 30. show cgn nat44 <> inside-translation This command is helpful in seeing the list of all Outside / Public IP address + Port mapping for a specific Inside / Private IP address (and a Port range). Following is a sample output of the command. Sample "show cgn nat44 <> inside-translation" CLI Output   RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 inside-translation protocol udp inside-vrf red                         inside-address 10.2.5.134 port start 1 end 4580 Wed May 15 14:25:09.713 UTC Inside-translation details --------------------------- NAT44 instance : nat1 Inside-VRF     : red --------------------------------------------------------------------------------------------    Outside         Protocol  Inside       Outside       Translation   Inside      Outside    Address                   Source       Source        Type          to          to                              Port         Port                        Outside     Inside                                                                       Packets     Packets --------------------------------------------------------------------------------------------   100.200.2.75       udp     4500         65024         dynamic       9236        2213   100.200.2.75       udp     4501         65153         dynamic       2236        0   100.200.2.75       udp     4502         65152         dynamic       4232        1218   100.200.2.75       udp     4503         65155         dynamic       2236        7128   100.200.2.75       udp     4504         65154         dynamic       2235        1821   100.200.2.75       udp     4505         65157         dynamic       5236        3129   100.200.2.75       udp     4506         65156         dynamic       2238        8172   100.200.2.75       udp     4507         65159         dynamic       7238        2189   100.200.2.75       udp     4508         65158         dynamic       2236        3119   ...   100.200.2.75       udp     4576         65226         dynamic       1239        2178   100.200.2.75       udp     4577         65229         dynamic       8235        4171   100.200.2.75       udp     4578         65228         dynamic       2235        0   100.200.2.75       udp     4579         65231         dynamic       4230        4127   100.200.2.75       udp     4580         65230         dynamic       2235        0 RP/0/RSP0/CPU0:# Above table shows all Outside IP (100.200.2.75) and Port combination for Inside IP (10.2.5.134) and Port Range (1 through 4580).   NOTE: - As one Inside address can only map to one Outside address, there are not multiple Outside addresses in 1st column. - The number of entries in this table should not exceed Port Limit value. Back to top 31. show cgn nat44 <> outside-translation This command is helpful in seeing the list of all Inside / Private IP address + Port mapping for a specific Outside / Public IP address (and a Port range). Following is a sample output of the command. Sample "show cgn nat44 <> outside-translation" CLI Output   RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 outside-translation protocol udp                         outside-address 100.200.2.75 port start 1 end 65535 Wed May 15 14:17:44.640 UTC Outside-translation details --------------------------- NAT44 instance : nat1 Outside-VRF    : default --------------------------------------------------------------------------------------------    Inside          Protocol  Outside      Inside         Translation  Inside      Outside    Address                   Destination  Destination    Type         to          to                              Port         Port                        Outside     Inside                                                                       Packets     Packets --------------------------------------------------------------------------------------------   10.2.5.134         udp     65024        4500          dynamic       4236        1216   10.2.5.134         udp     65025        4885          dynamic       9233        5161   10.2.5.134         udp     65026        4886          dynamic       4244        3491   10.2.5.134         udp     65027        4887          dynamic       1281        6018   10.2.5.134         udp     65028        4888          dynamic       2237        4187   ...   10.2.5.142         udp     65532        4882          dynamic       6261        3178   10.2.5.142         udp     65533        4881          dynamic       7282        0   10.2.5.142         udp     65534        4884          dynamic       1211        4178   10.2.5.143         udp     65535        4883          dynamic       1261        5189 RP/0/RSP0/CPU0:# Above table shows all Inside IPs (10.2.5.134, 10.2.5.142) and Port combination for Inside IP (100.200.2.75) and Port Range (1 through 65535). In this case, there are multiple Inside IP addresses are mapped to one Public IP address. Back to top 32. show services redundancy This CLI is used to see the current state of CGN Service redundancy - which ISM is active and which is Standby, as shown by the sample output below. Same "show services redundancy" CLI Output   RP/0/RSP0/CPU0:ios#sh services redundancy Wed May  2 07:35:26.070 UTC Service type     Name                    Pref. Active        Pref. Standby -------------------------------------------------------------------------------- ServiceInfra     ServiceInfra1           0/2/CPU0 Active ServiceInfra     ServiceInfra2           0/0/CPU0 Active ServiceCgn       cgn1                    0/2/CPU0 Standby    0/0/CPU0 Active As shown above, Preferred Active ISM card at slot 2 is now in Standby mode whereas, Preferred Standby ISM card at slot 0 is now in Active mode. It indicates, there was atleast one failure happened on ISM card at slot 2. Useful Deployment Tips Go through this Deployment Guide to clearly understand             how different feature works what parameters are configurable and need to be considered for specific deployment what are the different deployment use cases. Go through "CGv6 on ISM: Features Supported Across IOS-XR Release" page  to exactly know from which release specific features that you're  interested in are supported. This will help you in selecting the right  IOS-XR release.             Select the configuration parameter values wisely, considering            pros and cons, best-case and worst-case scenarios side effects Consider using ISM redundancy, to minimize traffic downtime during failure. Draw network diagram for deployment use-case including interface names (with VRF and IP address/subnet) and review for correctness. Try to find out what type of application traffic (VPN, Games, SIP, Video, etc.) may undergo NAT44 and whether there could be issues or not (not all applications are NAT friendly)            If possible, try it in a lab setup, before actual deployment Estimate traffic throughput and nature of it (burstiness, average, etc.), scale of sessions/users and accordingly use correct number of ISM cards and related configuration. If possible, deploy in phases. First, send selected traffic (you can use ABF) through ISM and see if there are any issues. If not, you can send more and more traffic through ISM. Related Information CGv6 on ISM: Features Supported Across IOS-XR Releases ASR9K Configuration Guides ASR9K Command Reference Guides ASR9K BNG Deployment Guide IANA Service Name and Transport Protocol Port Port Number Registry Configuration Example: MPLS VPN on IOS-XR Cisco ASR9000 S/w License Operations White Paper - 0 - Back to top ... View more