Solution:
IN-LINE MAP-T / E deployment using 3rd generation ASR9k line cards and using ASR9k as border relay router.
Details:
MAP-T Introduction
- Enables the delivery of IPv4 services over an IPv6 infrastructure through the use of stateless translation (MAP-T)
- Uses Address and Port (A+P) architecture to enable the sharing of IPv4 address by distributing the port-space, i.e. Embeds IPv4 address + Port into IPv6 address.
- Use-Cases: Residential wire line operators constrained by run-out with desire to deploy an IPv6-only access/aggregation
- Limitation: Provider make sure CPEs are capable of doing NAT44 operations.
Understanding MAP- T Exception Handling
The MAP-T Exception Handling with VSM feature handles fragmented packets, TCP, MSS clamping, path MTU, UDP and ICMP packets. Depending on the header details of the fragmented packets, the CGN application directly interacts with the line cards to process or forward the packets to VSM.
Note: When MAP-T is configured for exception handling with VSM, the static route details are automatically configured with Service App details. No manual configuration is required to configure the static routes.
Reference manuals & Hardware Supported
- Internet Drafts :
- draft-ietf-softwire-map - RFC 7599
- draft-ietf-softwire-map-dhcp - RFC 7598
- draft-ietf-softwire-stateless-4v6-motivation
- Chassis : ASR90xx and 99xx chassis
- RP : RSP-880-SE/TR
- Line card : 8*100G (Tomahawk) or MoD400 (for handling inline traffic)
- Services : A9K-VSM-500 (VSM Card) (For Handling Fragmented, IPv6 Extension headers Containing Packets, TCP Max Segment Size, MTU checks, ICMP v4/v6)
MAP-T Topology Overview
MAP Components
- MAP CPE :
- Performs NAT44 (translates source IPv4 address and TCP/UDP port to assigned range)
- Performs stateless translation (MAP-T) of IPv4 to IPv6
- MAP Border Relay :
- Managed by SP deployed at edge with IPv4 and IPv6 Internet connectivity
- Performs translation based on predefined rules
- Port-set :
- Separate transport layer port space
- Port-set ID :
- Allows the identification of set of ports assigned to a CE
Note:
Provider make sure that customer CPE is capable to perform NAT44 operations. Without such capable CPEs, this solution is not possible.
Types of exception packets handled
- In 6.2.1 release, we support functionality of Map-T translations on Tomahawk line card and exception packets by VSM.
- ICMP : In Inline MAPT, without VSM echo request and reply was handled by Tomahawk. Now all ICMP traffic is handled by VSM.
- All Destination unreachable (except code 14 in v4tov6)
- Time exceeded
- All Parameter Problem( except code 1 in v4tov6 and code 2 in v6tov4)
- ICMP error message
- ICMP query messages
- V4/V6 fragmented packets.
- TCP Maximum Segment Size :
- TCP packets with SYN flag set is handled in VSM for recalculation of MSS.
- Path MTU Checks :
- IPv4 to IPv6 path MTU – packets with DF bit set and packet length > MTU
- IPv6 to IPv4 path MTU – packet length > MTU
- IPv4 options :
- Packets with LSR and SSR IPv4 options will be dropped and ICMPv4 error generated
- IPv6 extended headers
- Hop-by-Hop
- Destination Header
- Routing header
- Fragment header
- Authentication header
Note: Only default vrf is supported for MapT exception handling feature
Configuration
Interface ServiceInfra : Configure serviceInfra interface
Interface ServiceApp : Configure service App interfaces for each ipv4 and ipv6
Service cgv6 <service-name> : Configure CGv6 service name
- service-location preferred-active node-id : mention VSM location for handling exception traffic
- service-inline interface-name/s : mention tomahawk ingress and egress interfaces
- service-type map-t-cisco <instance-name> : Configure map-t instance
- Within each map-T instance, parameters to be configured are mentioned in following slide
Under each Map-T instance, configure :
- address-family :
- ipv4 and corresponding serviceApp interface
- ipv6 and corresponding serviceApp interace
- Tcp mss and path-mtu value (non-mandatory field, default MTU is 1500)
- cpe-domains for :
- ipv4 prefix length value
- ipv6 vrf default
- ipv6 prefix length value
- Sharing ratio and contiguous-ports number
- Single or multiple cpe-domain/s :
- cpe-domain-name <domain-name1> ipv4-prefix <ipv4 address/prefix> ipv6-prefix <ipv6 address/prefix>……
- Single ext-domain :
- Ext-domain-name <domain-name> ipv6-prefix <ipv6 address/prefix> ipv4-vrf default
Sample Config
Show run interface serviceapp *:
interface ServiceApp1
ipv4 address 40.40.40.1 255.255.255.0
service cgv6 cgn123 service-type map-t-cisco
!
interface ServiceApp2
ipv6 address 2345::8/64
service cgv6 cgn123 service-type map-t-cisco
!
Show run interface serviceinfra *
interface ServiceInfra1
ipv4 address 1.1.1.1 255.255.255.252
service-location 0/3/CPU0
!
Show run interface TenGigE0/2/0/5/9:
interface TenGigE0/2/0/5/9
ipv4 address 4.4.4.1 255.255.255.0
ipv6 address 2001::1/64
load-interval 30
!
Service CGv6 Configuration:
service cgv6 cgn123
service-location preferred-active 0/3/CPU0
service-inline interface TenGigE0/2/0/5/9
service-type map-t-cisco map1
address-family ipv4
interface ServiceApp1
tcp mss 335
path-mtu 1200
!
address-family ipv6
interface ServiceApp2
tcp mss 1254
path-mtu 1500
!
cpe-domain ipv4 prefix length 24
cpe-domain ipv6 vrf default
cpe-domain ipv6 prefix length 48
sharing-ratio 256
contiguous-ports 16
cpe-domain-name cpe0 ipv4-prefix 192.1.1.0 ipv6-prefix 2301:0:1122::
ext-domain-name ext1 ipv6-prefix 6301:d01:1122::/48 ipv4-vrf default
!
!
Show Commands
sh int te 0/2/0/5/9 accounting :
TenGigE0/2/0/5/9
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 119429796 141175259448 57549767 25454289540
IPV6_UNICAST 57573860 26617932084 119419021 144497009720
IPV6_MULTICAST 7 472 0 0
IPV6_ND 20 1600 24 1920
sh int serviceapp * accounting
ServiceApp1
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 8141 3485668 193852302 229133420964
ServiceApp2
Protocol Pkts In Chars In Pkts Out Chars Out
IPV6_UNICAST 193856659 234566557390 8148 3718740
show cgv6 map-t-cisco map1 statistics
Map-t-cisco IPv6 to IPv4 counters:
======================================
Translated Udp Count: 652567655
Translated Tcp Count: 0
Translated Icmp Count: 0
PSID Drop Udp Count: 0
PSID Drop Tcp Count: 0
PSID Drop Icmp Count: 0
Map-t-cisco IPv4 to IPv6 counters:
======================================
Translated Udp Count: 0
Translated Tcp Count: 0
Translated Icmp Count: 0
PSID Drop Udp Count: 0
PSID Drop Tcp Count: 0
PSID Drop Icmp Count: 0
Map-t-cisco exception IPv6 to IPv4 counters:
======================================
TCP Incoming Count: 0
TCP NonTranslatable Drop Count: 0
TCP Invalid NextHdr Drop Count: 0
TCP NoDb Drop Count: 0
TCP Translated Count: 0
TCP Psid Drop Count: 0
UDP Incoming Count: 126796
UDP NonTranslatable Drop Count: 0
UDP Invalid Next Hdr Drop Count: 0
UDP No Db Drop Count: 0
UDP Translated Count: 126796
UDP Psid Drop Count: 0
ICMP Total Incoming Count: 0
ICMP No DB Drop Count: 0
ICMP Fragment drop count: 0
ICMP Invalid NxtHdr Drop Count: 0
ICMP Nontanslatable Drop Count: 0
ICMP Nontanslatable Fwd Count: 0
ICMP UnsupportedType Drop Count: 0
ICMP Err Translated Count: 0
ICMP Query Translated Count: 0
ICMP Psid Drop Count: 0
Map-t-cisco IPv6 to IPv4 counters:
======================================
Translated Udp Count: 652567655
Translated Tcp Count: 0
Translated Icmp Count: 0
PSID Drop Udp Count: 0
PSID Drop Tcp Count: 0
PSID Drop Icmp Count: 0
Map-t-cisco IPv4 to IPv6 counters:
======================================
Translated Udp Count: 0
Translated Tcp Count: 0
Translated Icmp Count: 0
PSID Drop Udp Count: 0
PSID Drop Tcp Count: 0
PSID Drop Icmp Count: 0
Map-t-cisco exception packets IPv4 to IPv6 counters:
======================================
TCP Incoming Count: 0
TCP No Db Drop Count: 0
TCP Translated Count: 0
TCP Psid Drop Count: 0
UDP Incoming Count: 2134370
UDP No Db Drop Count: 0
UDP Translated Count: 2134370
UDP FragmentCrc Zero Drop Count: 0
UDP CrcZeroRecy Sent Count: 0
UDP CrcZeroRecy Drop Count: 0
UDP Psid Drop Count: 0
ICMP Total Incoming Count: 0
ICMP No Db Drop Count: 0
ICMP Fragment drop count: 0
ICMP UnsupportedType Drop Count: 0
ICMP Err Translated Count: 0
ICMP Query Translated Count: 0
ICMP Psid Drop Count: 0
Subsequent Fragment Incoming Count: 264661768
Subsequent Fragment No Db Drop Count: 0
Subsequent Fragment Translated Count: 264661768
Subsequent Fragment Drop Count: 0
Subsequent Fragment Throttled Count: 0
Subsequent Fragment Timeout Drop Count: 36
Subsequent Fragment TCP Input Count: 0
Subsequent Fragment UDP Input Count: 2134370
Subsequent Fragment ICMP Input Count: 0
Options Incoming Count: 0
Options Drop Count: 0
Options Forward Count: 0
Options No DB drop Count: 0
Unsupported Protocol Count: 0
ICMP generated counters :
=======================
IPv4 ICMP Messages generated count: 0
IPv6 ICMP Messages generated count: 0
Troubleshooting tips
- Checking core file at VSM:
- RP/0/RSP0/CPU0:ROUTER#virtual-service connect name Mapt console node 0/1/CPU0
Trying 192.0.131.3...
Connected to 192.0.131.3.
Escape sequence is '^^e'. - Red Hat Enterprise Linux Server release 5.3 (Tikanga)
Kernel 2.6.18-128.el5 on an x86_64 - localdomain login: root
Password:rootroot - [root@localhost ~]# cd VSM
[root@localhost VSM]# pwd
/root/VSM
[root@localhost VSM]# ls -l
total 2445520
<> - -rw------- 1 root root 425799680 Oct 5 23:16 core.7088
- RP/0/RSP0/CPU0:ROUTER#virtual-service connect name Mapt console node 0/1/CPU0
- Checking VQI/UIDB Configuration in NP Datastructures
- show uidb data location 0/4/CPU0 serviceApp 1 extension
- show cef adjacency serviceApp 1 hardware ingress detail location 0/0/CPU0 | inc sfp
- sh controllers pm vqi location 0/2/CPU0
- Checking Routes/PBR entries for CPE IPv4 and EXT IPv6
- sh pbr service-node table summary
Note :Depending on the number of MAP instances configured we will see those many 1001 and 3001 (These are for the default classes). 5001 values gets incremented accordingly when we keep adding the CPE-DOMAINS i.e say if we add one more CPE-Domain we will see in the above table additional NAME as CGN_5002 with VIdx 5002. Similarly the 7001 value gets incremented when we keep adding additional EXT-DOMAINS.
-
- show policy-map transient type pbr
- show pbr-pal ipolicy <> iclass all stats location <> -> Check PBR stats for drops pkts
- show cgv6 map-e mape1 statistics -> To check CGV6 stats
- Show controller np counter np<> location <>
Note: If the pkt is dropping in the inline interface and if the Map stats cli is not showing incremental counters than above CLI to be use to know which drop counters are increasing.If drop counters is MAPE v4 to v6 drop/MAPE v6 to v4 drop, then it can be issue with, Wrong PSID, wrong ipv6 source port, wrong ipv4 destination port, etc..
Apart than other drop counters, Counters to specifically monitor for MAP operations health are :
RSV_OPEN_NETWORK_SERVICE_TRIGGER_SVC --> It Implies the pkts have hit our service.
VIRTUAL_IF_PROTO_IPV4_UCST_INPUT_CNT --> Implies that the V6 pkts have been translated to V4
VIRTUAL_IF_PROTO_IPV6_UCST_INPUT_CNT --> Implies that the V4 pkts have been translated to V6
PARSE_OPEN_NETWORK_SERVICE_SVC_LKUP --> Pkt counts which got processed after the service lookup.
If the packets are getting dropped in the egress port, then verify whether the map-e statistics are getting incremented. If they are getting incremented, then the translated address has got dropped. Check the route is present in routing table for the translated address.
-
- show route
- show route ipv6
Note: If map-e statistics is not getting incremented, then the translation has not happened and the normal unicast packet has been forwarded to the egress. Here, in case of ipv4 address, need to check the destination address and the cpe-domain are in the same subnet.
- Checking for VQI Related Drops
- sh controllers fabric fia drops ingress location 0/0/CPU0 | inc Vqi
- Checking Ucode Datastructures related to CGN
- show controllers np struct 113 detail all-entries np0 location 0/0/CPU0 > We will have 4 entries for one instance.
- show controllers np struct 114 detail all-entries np0 location 0/0/CPU0 > We will have 14 entries for one Instance
Note: Above documented TS tips help an individual to undestand the problem. Based on this data you can log a case with Cisco TAC for further assessment if in case user unable to fix the problem by self.
- Checking P2MP Node counter in VSM and in XR
To connect to virtual host use root/rootroot .
RP/0/RSP0/CPU0:ROUTER#show virtual-service list
Virtual Service List:
Service Name Status Package Name Node Name
______________________________________________________________________________
Mapt Activated asr9k-vsm-cgv6-6.2.1.00- 0/1/CPU0
RP/0/RSP0/CPU0:ROUTER#virtual-service connect name Mapt console node 0/1/CPU0
Trying 192.0.131.3...
Connected to 192.0.131.3.
Escape sequence is '^^e'.
Red Hat Enterprise Linux Server release 5.3 (Tikanga)
Kernel 2.6.18-128.el5 on an x86_64
localhost.localdomain login: root
Password: rootroot
[root@localhost ~]# cd /var/log/cgv6/
[root@localhost cgv6]# pwd
/var/log/cgv6
[root@localhost cgv6]# p2mp_debugger
Dump options:
0 -> Policy
1 -> Main DB
2 -> User DB
3 -> Hashes DB
4 -> VRF Map
5 -> Summary DB
6 -> Dump Statistics
7 -> Clear Statistics
8 -> Dump node counter
9 -> Clear node counter
10 -> Dump CNAT counter
11 -> Dump Virtual Assembly (VA)
12 -> Show Configuration
13 -> Show Netflow V9 Configuration
14 -> Show Inside VRF Information
15 -> Show Outside VRF Information
16 -> Logging util test
17 -> Configure PD/PI/L2 Debug Level
18 -> Dump PD/P2/L2 Debug Level
19 -> Set Traffic Flags
22 -> Dump Main DB Summary
23 -> Dump User DB Summary
25 -> Dump Timeout DB Summary
27 -> Set bulk size for nat44
28 -> Dump bulk port statistics
29 -> Clear bulk port statistics
30 -> Show bulk port allocation for subscriber
Enter dump option <0-30>:8 <------- option 8 to dump cgn node counters
Enter the coremask in hex: ffffffffffff
Dump_option: 8, core_mask : FFFFFFFFFFFF
[root@localhost cgv6]#
Core1: NODE Counter dump
-----------------------------------------
Node Counters
-----------------------------------------
vsm/inject
injected and forwarded: 5965
-------------------------------
vsm_decode
Forwarded to INFRA: 3501
Forwarded to MAPT_CISCO: 2464
-------------------------------
vsm_infra_classifier
Infra to CLI: 8250
646
Infra to Data path Test: 2855
-------------------------------
vsm_ha
HA Packets(DP TX) injected: 2855
-------------------------------
vsm_infra_l3_tx
Infra L3 Tx injected and forwa: 646
Forwarded for FIB lookup: 2855
-------------------------------
vsm_xlat_classifier
xclsfr6 v4 tcp frag: 2464
-------------------------------
cnat_cli_input
CNAT config messages processed: 646
-------------------------------
cnat_db_scanner
Scan timer callback invoked: 1423132
-------------------------------
xlat_v4_to_v6_tcp
Xmit - v4 to_v6 tcp: 2464
-------------------------------
xlat_v4_frag
v4_frag_tcp: 2464
Scale
- Maximum of 32 domain definitions/MapT instances per VSM.
- Maximum of 256 CPE domains per MapT instance.
- Maximum of 1024 CPE domains across all MAP-T instances per VSM
- Single EXT domain per MapT instance
Exception Traffic Flow:
