cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2056
Views
0
Helpful
25
Replies
Highlighted
Cisco Employee

ISM-100 to VSM-500 Migration and Redundancy managment guide with key performance differances

Contents

  1. ISM-100 to VSM-500 Migration guide
  2. VSM-500 Redundancy management config inputs
  3. ISM-100 to VSM-500 key performance and scalability points

Graceful insertion of VSM card on ASR9000 routers

Part-A: VSM card readiness

 

  1. Pre-Installation Information
    1. VSM-500 card supported on ASR9904,ASR9006,ASR9010,ASR9912,ASR9922 chassis
    2. This card supported from IOS-XR 5.1.1 release
    3. VSM-500 is supported with RSP440 / RP2/RSP880
    4. VSM-500 supports CGN and TMS-Arbor services

 

  1. Prerequisites files for VSM operations are :

 

VSM Services Infra Pie

asr9k-services-infra.pie

CGv6 Services pie

asr9k-services-px.pie

FPD PIE

asr9k-fpd-px.pie

VSM OVA Package

asr9k-vsm-cgv6-<version>.ova

Mandatory SMU(s) for target release

asr9k-CSCxxxxx-px.pie

 

 

Part-B: installation with service configuration

 

  1. Make sure all above mentioned PIEs are installed.

Check with Show install summary

  1. Make sure FPD is correctly upgraded

Check with sh hw-module fpd location all

  1. Install mandatory SMUs

Note: - To identify the bare minimum mandatory SMUs we need for VSM in key releases using CSM, Please refer below link.

https://supportforums.cisco.com/document/12554686/smu-list-through-cisco-software-manager-csm-recommended-smu-list-using-csm

 

  1. Install & activation of CGv6.ova for CGN services:-

 

  1. Once VSM card in IOS-XR RUN state then we need to perform upgrade activity.

 

Note: - Pre-533 IOS XR code, disabling CoPP TFTP config was required. From IOS-XR 5.3.3 no TFTP config changes required.

 

RP/0/RSP0/CPU0:ESC_CGN(config)#show run

Building configuration...

control-plane

 management-plane

  out-of-band

   interface all

    allow TFTP

end

 

RP/0/RSP0/CPU0:ESC_CGN(config)#commit

 

  1. Installation VSM Ova package with CGv6 config: -

 

Step 1: Copy cgn.ova file to RSP (E.g. to disk0:)

Step 2: Enable virtual service.

 

RP/0/RSP0/CPU0:Esc_CGN(config)#virtual-service enable

RP/0/RSP0/CPU0:Esc_CGN(config)#commit

 

Step 3: Install CGN OVA using CLI from execution mode:

 

RP/0/RSP0/CPU0:Esc_CGN# virtual-service install name cgn123 package disk0:/asr9k-vsm-cgv6-<version>.ova node 0/1/CPU0

Please note: Installation will take about 6-8 minutes.

 

Step 4: Check installation progress using show virtual-service list CLI. When the status is shown as "Installed", installation is completed.

 

RP/0/RSP0/CPU0:Esc_CGN#sh virtual-service list

 

Mon May 17 18:01:38.310 UTC

 

Virtual Service List:

 

Name                     Status             Package Name

__________________________________________________​_______

 

cgn123                   Installing         asr9k-vsm-cgv6-<version>.ova

 

RP/0/RSP0/CPU0:Esc_CGN#sh virtual-service list

Mon May 17 18:10:17.291 UTC

Virtual Service List:

Name                     Status             Package Name

__________________________________________________​_______

 

cgn123                   Installed           asr9k-vsm-cgv6-<version>.ova

 

Activate CGv6 VM

 

Step 1: Configure CGv6 VM, 10-GE interfaces and Activate

 

RP/0/RSP0/CPU0:Esc_CGN#conf t

Mon May17 18:10:25.525 UTC

 

RP/0/RSP0/CPU0:Esc_CGN(config)#virtual-service cgn123

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#vnic interface tenGigE 0/1/1/0

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#vnic interface tenGigE 0/1/1/1

………

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#vnic interface tenGigE 0/1/1/10

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#vnic interface tenGigE 0/1/1/11

 

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#commit

Mon May17 18:11:34.285 UTC

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#activate

RP/0/RSP0/CPU0:Esc_CGN(config-virt-service)#commit

 

Step 2: Un-shut and assign description to VNIC interface

 

Bring up 10-GE interfaces on VSM

Must before configuring any ServiceInfra/App interfaces

 -------------------------------------------------------

Interface TenGigE 0/1/1/0

Description VSM VNIC 1 ===>>>

 No shut

Interface TenGigE 0/1/1/1

Description VSM VNIC 2

  No shut

.......

Interface TenGigE 0/1/1/11

Description VSM VNIC 12

  No shut

 

 (Note: Adding interface Description is best practice. This will help VNIC interfaces in Admin UP state [Post Installation/Config activity] once VSM          card reloaded due to any operational reasons. )

 

Step 3: Check VM status via show virtual-service list CLI till it shows as "Activated".

 

RP/0/RSP0/CPU0: Esc_CGN#sh virtual-service list

Mon May 2 18:12:23.863 UTC

Virtual Service List:

Name                     Status             Package Name

__________________________________________________​_______

 

cgn123                   Activated           asr9k-vsm-cgv6-<version>.ova

 

Please note: After activating the VM, it will take some 5 min for CGv6 Application processes to come up.

 

Step 4: Configure ServiceInfra interface.

 

RP/0/RSP0/CPU0:Esc_CGN#conf t

RP/0/RSP0/CPU0:Esc_CGN(config)# interface ServiceInfra 1

RP/0/RSP0/CPU0:Esc_CGN(config-int)# ipv4 address <IP> <mask>

RP/0/RSP0/CPU0:Esc_CGN(config-int)# service-location 0/1/CPU0

RP/0/RSP0/CPU0:Esc_CGN(config-int)# commit

 

  • After ServiceInfra interface is configured, CGN HA agent will start sending hello packets to the CGv6 Application processes with retry after every 150 seconds.
  • Once response for hello packets is received, configurations will be sent to CGv6 Application processes.

 

Step 5: Configure Ingress interfaces

 

interface TenGigE1/0/0/0

  Add it to Inside VRF

 vrf ivrf1

  Assign IP address to it

 

interface TenGigE1/0/0/1

 vrf ivrf2

  Assign IP address to it

 

Step 6: Configure Egress interfaces

 

interface TenGigE0/2/0/0

 vrf ovrf1

Add it to Inside VRF

 

 

interface TenGigE0/2/0/1

 vrf ovrf2

 Add it to Inside VRF

 

Step 6: Configure CGN and NAT44 Service parameters

 

Define CGN Instance

 

service cgn cgn1

-- Define location

 service-location preferred-active 0/1/CPU0

 - Define NAT44 Service Type and instance (one per VSM card)

 service-type nat44 nat1

  - Define per-user Port limit for NAT44 instance

  portlimit 250

  inside-vrf ivrf1

    - Define Public IP address Pool

    map outside-vrf ovrf1 address-pool 9.0.0.0/24

   - Define inside VRF

  inside-vrf ivrf2

   - Define Public IP address Pool

    map outside-vrf ovrf2 address-pool 9.1.0.0/24

 

Step 7: Configure Inside and Outside ServiceApp interfaces

 

Define Inside Service App interfaces

 

interface ServiceApp 1

-- Define VRF where it should belong to (optional)

 vrf ivrf1

  - Assign IPv4 address and netmask

  - Define CGN instance and service type in which it belongs to

  service cgn cgn1 service-type nat44

interface ServiceApp 3

 vrf ivrf2

  - Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

-- Define Outside Service App interfaces

 

interface ServiceApp 2

 vrf ovrf1

  - Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

interface ServiceApp 4

 vrf ovrf2

  - Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

 

Step 8: Configure Inside and Outside ServiceApp interfaces

 

Define inside Service App interfaces

 

interface ServiceApp 1

-- Define VRF where it should belong to (optional)

 vrf ivrf1

  - Assign IPv4 address and netmask

  - Define CGN instance and service type in which it belongs to

  service cgn cgn1 service-type nat44

interface ServiceApp 3

 vrf ivrf2

  - Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

-- Define Outside Service App interfaces

 

interface ServiceApp 2

 vrf ovrf1

- Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

interface ServiceApp 4

 vrf ovrf2

- Assign IPv4 address and netmask

  service cgn cgn1 service-type nat44

 

Step 9: Define static route for Inside-to-outside traffic

 

router static

-- Specify the Inside VRF

 vrf ivrf1

  - Define address family

  address-family ipv4 unicast

   - Re-direct all traffic to Inside Service App

   0.0.0.0/0 ServiceApp 1

router static

 vrf ivrf2

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp 3

 

Step 10: Define static route for Outside-to-inside traffic

 

router static

 vrf ovrf1

  address-family ipv4 unicast

  -Assign IP pool ServiceApp 2

router static

 vrf ovrf2

  address-family ipv4 unicast

            -Assign IP pool ServiceApp 4

 

Refer link for service Configuration

http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r5-3/cg-nat/configuration/guide/b_cgnat_cg53xasr9k/b_cgnat_cg53xasr9k_chapter_0100.html

 

  1. Health checks / service verification of VSM by using below commands.
  2. show virtual-service list
  3. show services interfaces serviceinfra <>
  4. show interfaces serviceinfra <> accounting
  5. sh int tenGigE 0/<>/<>/* | i line protocol
  6. show services cgn vsm ha trace all location <>

 

Part-C: Traffic restoration on VSM

 

  1. Restore traffic.
  2. Verify the services once traffic is restored completely.
  3. Once traffic flow recorded fine on all LCs then diverts Test IP pools traffic on VSM card and verify the services.
    1. show interfaces serviceinfra <> accounting
    2. show services interfaces
    3. sh int tenGigE 0/<>/<>/* | i packets out
    4. show cgn nat44 nat1 statistics
    5. sh cgn nat44 <> inside-vrf <> counters

 

Redundancy options on VSM card.

 

  • HA requirement for CGv6 on VSM is same as that of CGv6 on ISM; there are some similarities as well as differences in HA architecture / implementation across these two platforms.
  • As both implementations use SVI as virtual interface architecture, both can use the service redundancy support that the infra-structure provides and can divert traffic during failover using the same. However, as CGv6 on ISM runs on a native (non-virtualized) platform whereas on VSM, it runs on a virtualized platform, there are differences in HA triggers as well as how we handle those triggers.

 

 

Note 1: - on VSM-500 card for redundancy enabling reload we don’t need to enable

  service-cgv6-ha location 0/1/CPU0 puntpath-test command. This command is only applicable    

 to ISM-100 card.

Note 2: The data/punt path failures will be detected even if the CLI is not enabled but the card won’t be

reloaded if any problem encounter between CGv6 App to CGv6 HA Agent or vice versa.

           

 

VSM on ASR9000 supports 1:1 warm standby redundancy.

 

  1. Warm-standby
  • Translation state is not synchronized between active and standby, all connections
    will be re-established on failover
  • Pros: simple to configure, a single map pool is used
  • Cons: only 1:1, one card stay unused in standby state.
  • Convergence time takes ~18 sec [Data-path HA packets take 5 x 3 =15 sec for detection and action of failure + ~3 sec for rebuilding translation DB]

 

  1. An alternative with ABF is available
  • ABF is used to divert Ingress traffic /Private side / Subscriber side IP Pools/traffic towards Public / Core / Network side.
  • Pros: offers more options like n:1 redundancy, converges very quickly
  • Cons: - CGN-App of VSM can’t be monitored. To fix this problem user can use ICMP IPSLA probes for ISM infra/Services state maintenance. Or,
  • Enable “datapath-test” knob for problem detection and action at CGv6-APP.
  • Convergence time takes:-
    1. ~1 Sec if VSM-infra [XR operations] get impacted.
    2. ~16 Sec  it takes if data-path test enable for failure detection CGV6-App
    3. ~4-6 Sec it takes if ICMP-IPSLA with NH tracking is enabled between ServiceApp and outgoing interface IP address

 

1:1 Warm-Standby Redundancy

 

  • Configuration

 

RP/0/RSP0/CPU0:Esc_CGN(config)#

service cgn HA_CGN

service-location preferred-active 0/1/CPU0 preferred-standby 0/2/CPU0 >>>>>

 

RP/0/RP0/CPU0:Esc_CGN#show services redundancy

Service type     Name                    Pref. Active        Pref. Standby     

--------------------------------------------------------------------------------

ServiceInfra     ServiceInfra1           0/1/CPU0 Active   

ServiceInfra     ServiceInfra2           0/2/CPU0 Active   

ServiceCgn       HA_CGN               0/2/CPU0 Standby    0/1/CPU0 Active   

 

For redundancy enabling reload for data path tests need to config below command on CGN router for VSM-500 card:-

RP/0/RSP1/CPU0:Esc_CGN#configure

RP/0/RSP1/CPU0:Esc_CGN(config)#service-cgv6-ha location 0/x/CPU0 datapath-test

RP/0/RSP1/CPU0:Esc_CGN(config)#commit

 

 

CGN n:1 Redundancy with ABF (NAT44)

 

  • Configuration

 

Note: - User needs to configure datapath tests knob to detect failure at CGv6 App to CGv6 HA Agent of VSM-500 card

 

service cgn ABF-cgn-HA1

 service-location preferred-active 0/0/CPU0

 service-type nat44 nat44-ABF

  inside-vrf Inside-1

   map address-pool 10.0.0.0/24

!

service cgn ABF-cgn-HA2

 service-location preferred-active 0/1/CPU0

 service-type nat44 nat44-ABF2

  inside-vrf Inside-2

   map address-pool 10.0.1.0/24

!

service cgn ABF-cgn-HA-backup

 service-location preferred-active 0/2/CPU0

 service-type nat44 nat44-ABF-backup

  inside-vrf iBackUp-1

   map address-pool 10.0.0.0/24

 inside-vrf iBackUp-2

   map address-pool 10.0.1.0/24

!

 

ipv4 access-list CGN-ABF-HA

 10 permit ipv4 9.1.0.0/24 any nexthop1 vrf Inside-1 ipv4 100.68.51.6 nexthop2 vrf iBackUp-1 ipv4 100.68.33.6

 20 permit ipv4 9.2.1.0/24 any nexthop1 vrf Inside-2 ipv4 100.68.151.6 nexthop2 vrf iBackUp-2 ipv4 100.68.33.6

 100 permit ipv4 any any

!

router static

 address-family ipv4 unicast

  110.1.0.0/16 100.1.1.2 description Ixia-i2o-Default

  151.0.0.0/24 ServiceApp2 1 description Static-Ixia-o2i-ABF

  151.0.1.0/24 ServiceApp4 1 description Static-Ixia-o2i-ABF

  151.0.0.0/24 ServiceApp6 2 description Floating-static-Ixia-o2i-ABF

  151.0.1.0/24 ServiceApp6 2 description Floating-static-Ixia-o2i-ABF

 

Performance / Scalability information between ISM and VSM card.

 

 

Per Blade Limits

ISM

VSM

NAT44 instances supported

1 per card

1 per card

End-point Dependent Filtering

Not Supported

Supported

TCP Sequence Check

Not Supported

Supported

CLI: map ip many-to-one

Supported

Not Supported

Number of service infra

1

1

Number of service app

244 (per system)

256 per card

IP pool supported

/16 to /30 (max 65535 addresses)

/16 to /27(max 65535 addresses)

Max Static Port forwarding

6K

6K

Max number of NAT users

1M

4M

FIB scale

512k

1M

Configuration CLIs

Same

Same

Uses SVI

Yes

Yes

Network Processor

No, handled by a dedicated process

Yes (Typhoon)

Egress FIB Lookup

Within CGv6 App

On Typhoon NPU

ServiceApp placement

Associated with Niantic port/VQI

Associated with NP ports / Niantic ports

# of CGv6 instances

8

48

Stateless protocols (in CGN card)

6rd, MAP-T/E

6rd, MAP-T/E

Inline support

Yes for SL protocols

With Typhoon & Tomahawk LC for    MAP-T/E & 6rd

 

 

 

25 REPLIES 25
Highlighted

Hi all

I have simulated the ISM redundancy and it functions well

When i turn down one of the modules (the primary) , the traffic is redirected to the secondary module via ABF , the ping continues but the web traffic is lost for a while and comes back again , why the ping is keep working and is there a specific timer that I can tune in order to change the TCP/UDP traffic behavior?

BR,

Mohammad

Highlighted

Hi Nitin,

I am trying to implement "cold-redundancy" with 2 VSM located on 2 different ASR9912 and using ABF with Track feature.

First of all, regardless of the Track feature, I only manage to direct traffic to the inside VRF with this ABF line:

720 permit ipv4 host 172.16.19.9 any nexthop1 vrf PAT3

RP/0/RP0/CPU0:RZ01-9912-01#sh access-lists ipv4 CGN hardware ingress location 0/1/CPU0
Mon Jul  3 11:58:56.951 mesz
ipv4 access-list CGN
 720 permit ipv4 host 172.16.19.9 any (next-hop: addr=0.0.0.0, vrf name=PAT3)

If I put the IP address of the ServiceApp interface together with the inside VRF (see below), it won't work.

720 permit ipv4 host 172.16.19.9 any nexthop1 vrf PAT3 ipv4 172.17.77.9

This is the output of the above command:

RP/0/RP0/CPU0:RZ01-9912-01#sh access-lists ipv4 CGN hardware ingress location 0/1/CPU0
Mon Jul  3 12:00:36.341 mesz
ipv4 access-list CGN
 720 permit ipv4 host 172.16.19.9 any

When I then add the track option, even the first example won't work.

720 permit ipv4 host 172.16.19.9 any nexthop1 vrf PAT3 track PAT3

Any idea about what could be wrong?

Thanks in advance for your help!

Cheers,

Stefano

Highlighted

And for scalability , I am confused regarding the max number of NAT users mentioned in the table above (which says 4M for VSM and 1M for ISM)

RP/0/RSP0/CPU0:UM-HQ1-CGN-3G-NAT1#sh cgn nat44 nat1 statistics
Sun Sep  4 13:31:04.144 AST

Statistics summary of NAT44 instance: 'nat1'
Number of active translations: 5451912
Number of sessions: 246854
Translations create rate: 26727
Translations delete rate: 25376
Inside to outside forward rate: 1047085
Outside to inside forward rate: 1685388
Inside to outside drops port limit exceeded: 5320844
Inside to outside drops system limit reached: 0
Inside to outside drops resource depletion: 0
No translation entry drops: 4902541717
PPTP active tunnels: 4
PPTP active channels: 4
PPTP ctrl message drops: 7113
Number of subscribers: 311582
Drops due to session db limit exceeded: 0
Drops due to source ip not configured: 0

Is it Number of sessions ?

BR,

Mohammad

Highlighted

Hey Mohammad,

Number of active translations: 5451912 >>> This number indicates per user translation request.

Number of subscribers: 311582 >>> This number indicates total number of active users.

Aside to the query, by seeing the stats, it looks you have either wrong load balancing of traffic among the service apps or less configured port limit. There are high number of drops.

high time for you to optimize the CGN solution over VSM. if port-limit is default then increase it either globally or per VRF basis and check the performance.

Thanks

Nitin Pabbi

Highlighted

Hi Nitin

What am thinking of is restrictions on ABF

Is there an limitation on ABF ? as I read that the ABF will not work when the nexthop is VRF?

Thanks

BR,

Mohammad

Highlighted

Just a comment. I would revise:

Note: - Pre-533 IOS XR code, disabling CoPP TFTP config was required. From IOS-XR 5.3.3 no TFTP config changes required.

OVA didn't install properly until I added tftp to control-plane config.

Regards,

c.

Highlighted

Hey Carlos,

If user don't disable management plane config or not edit MPP to allow TFTP then OVA installation will get fail.

This is because of due to default binding between MPP and application.

We have fixed this behavior in 532 code through DDTS CSCut56161.   

That way you dont need to disable COPP or enable TFTP on MGMT interface from 532  release onwards.

HTH

Thanks

Nitin Pabbi

Highlighted

What I was trying to say (maybe poorly) is that I know for a fact that it didn't work for me in 5.3.3.

Highlighted

Hey Carlos,

If you help me with error log or can share the access to your node which is in broken state then i can understand the issue.

Otherwise if you have opened SR with Cisco then please share with me.

Below is the error you get when OVA installation fails due to the bug i shared:-

RP/0/RSP0/CPU0:Node1#RP/0/RSP0/CPU0:Mar 25 14:56:55.648 : service_mgr[407]: %OS-SMGR-3-VM_INSTALL_FAIL : 'Install' of service VM 'cgn1' Failed. Error: SM SVC[cgn1]: 'cgn1' VM Installation failed, err code [1], err : VM [cgn1, 0/2/CPU0]: Install failed, server failure result: Failed to tftp
RP/0/RSP0/CPU0:Mar 25 14:56:56.026 : service_mgr[407]: %OS-LIB_VMAN-3-INSTALL_FAILURE : Virtual Service[cgn1]::Install::The installation of the virtual service failed

Thanks

Nitin Pabbi

Highlighted

Well, I opened 680802487, but the case is not related to my comments above. Thanks, Nitin.

Highlighted

Hi, Nitin:

I was wondering if you could help with this. I have 2 VSMs, one of them seems to be working fine since I see this:

RP/0/RSP0/CPU0:XXXXX# show services redundancy
Mon Aug 22 09:16:59.876 CST
Service type Name Pref. Active Pref. Standby
--------------------------------------------------------------------------------
ServiceInfra ServiceInfra1 0/7/CPU0 Active
ServiceCgn cgn1 0/7/CPU0 Active

RP/0/RSP0/CPU0:XXXXX#

On the other one, I see this (check log message):

RP/0/RSP0/CPU0:YYYYY#show services redundancy
Mon Aug 22 09:25:09.468 CST
Service type Name Pref. Active Pref. Standby
--------------------------------------------------------------------------------
ServiceInfra ServiceInfra1 0/7/CPU0 Active
ServiceCgn cgn1 0/7/CPU0 Offline

RP/0/RSP0/CPU0:YYYYY#

and

LC/0/7/CPU0:Aug 22 09:21:29.350 : secontroller[326]: %SERVICES-SESVI_EA-3-INTERNAL_ERR : ens_read_failed (ignore this error if cgv6 is not configured or sesvi_ea process is not yet spawned) with error : No such process 

Both were installed with the same procedure and (up to this point) are configured exactly alike. Is there anything that jumps out at you?

Thanks in advance!

c.

Forgot to add this:

RP/0/RSP0/CPU0:XXXXX#show cgn nat44 NAT44 statistics
Mon Aug 22 09:51:43.606 CST
Unable to obtain requested information Error:'cgn' detected the 'warning' condition 'LWM Channel not open'
RP/0/RSP0/CPU0:XXXXX#

Content for Community-Ad