In this article we'll explain how dhcp relay is working and how broadcasts are forwarded within IOS-XR, because the DHCP relay functionality is configured and handled differently within XR vs regular IOS.
Generally broadcasts are not forwarded by routers as they typically only live on the local subnet. However sometimes it is required to forward these broadcasts and this article will detail out how that configuration works and what the limitations are.
Considering that broadcasts are handled locally on the subnet, a router will not forward these broadcasts out and locally consume them. This presents a problem when a particular interface is servicing a subnet with clients that want to use DHCP to obtain their address information. IOS-XR at the time of writing does not have a dhcp server which means that clients cannot obtain their address from an XR router.
Even if the OS would support DHCP server, like IOS, you may want to centralize your DHCP servers in a designated area of your network which would require you to forward the client's DHCP requests to that server.
IOS employs the concept of an ip-helper-address to take the broadcasts and send out a directed broadcast or unicast to one or multiple servers.
Note that by default broadcasts are NOT forwarded by a router.
Also only UDP broadcasts can be forwarded, dhcp (which is udp based) can benefit from this only.
In order to forward udp broadcasts, IOS uses the ip helper-address configuration underneath the interface that is expected to receive broadcasts that are to be forwarded. This pertains then to ALL udp broadcasts that are received.
fp7200-6(config-if)#ip helper-address ? A.B.C.D IP destination address
In IOS-XR, DHCP broadcasts are handled and configured differently then other udp based broadcasts.
For IOS-XR to forward broadcasts (non DHCP) the following configuration is required:
RP/0/RSP1/CPU0:A9K-BOTTOM(config)#forward-protocol udp ? <1-65535> Port number
Some well known port numbers are converted into their names, such as DNS and NBNS
The interface configuration for IOS-XR looks like this:
RP/0/RSP1/CPU0:A9K-BOTTOM(config)#int g0/1/0/14 RP/0/RSP1/CPU0:A9K-BOTTOM(config-if)#ipv4 helper-address ? A.B.C.D IPv4 destination address
Note again that this configuration will not forward DHCP packets.
There are 2 key ways to handle DHCP, one is via DHCP RELAY and one is via DHCP PROXY.
Relay has been supported for a long time. XR421 adds the full capability of proxy (in combination with IP subscribers)
Configuration: in its simplist form
dhcp ipv4 profile MYGROUP relay helper-address vrf default 22.214.171.124 helper-address vrf default 126.96.36.199 ! interface GigabitEthernet0/1/0/14 relay profile MYGROUP !
There is no further interface specific configuration required as what you're used to from IOS-XR that all protocol specific configuration is done under the protocol definition and not split across different sections.
Is more dumb in the handling of dhcp broadcasts, we are effectively taking the dhcp broadcast messages and convert them into a unicast destination which are defined by the helper addresses and we send the messages back and forth, or in fact we are relaying them between the client and the helper addresses. We maintain no state pertaining to DHCP as we are simply sending them across. The return packets from the helpers are returned as broadcasts back to the subnet we are serving.
In dhcp relay mode (or broadcast forwarding for that matter) packets are sent to all helper addresses.
In XR dhcp relay, prior to XR 4.1 and as described by CSCto39520, we only forward one offer back to the client if we receive multiple offers from the various helper addresses. While this provides still redundancy, it is in certain cases not desirable to have only offer handed to the client.
The rationale in XR was that we'd be maintaining a temporary state that links the request ID to the interface, once the first offer comes in, that state is removed. This prevents the forwarding of subsequent offers.
While normally this state and interface linkage is not needed, because we could look up the giaddr to the interface and use that, in the scenario where we are unnumbered to a loopback serving multiple interfaces, this state is useful so we send it out only on the interested interface.
Post XR 4.1 we'll leave that state in place for 1 minute so all offers received in that time period are forwarded.
Memory is not an issue, we could theoretically serve 1M requests per minute.
However things are bound by the LPTS punt policer and the process prioritization.
For some actual numbers, DHCPv4-Relay over BVI can accommodate 200K sessions and qualified CPS rate is 150 RSP440 and 250 on RSP880. DHCPv4-Relay is stateless and there are no sessions created.
The 200K limitation is enforced because of Adjacency creation as the adjacency is framed with ARP learning for end-user while resolving gateway ip-addresses.
Consider 250 CPS (Calls Per second), since ASR9K acting as Relay so access-side 4 message transactions (Discover, Offer, Request, Ack) and also server-side 4 message transactions.
So, overall 8 messages transactions per session.
That means, overall 250*8*60 = 120K message transactions per second.
Currently dhcp replies being sent as broadcast to the client so after enabling the CLI "[no]broadcast-flag policy check" it will send unicast reply to the client.
In dhcp PROXY mode we do maintain state as we are handling the dhcp request on behalf of the requesting client. It looks like as we are the originator of the request to the dhcp servers. Proxy for that matter is stateful.
Proxy also stores the binding on the proxy agent locally, which relay does not.
Setup configuration, sample operation and debug verification
Note: One common forgotten thing is that both dhcp servers need to have a route back to the 188.8.131.52 address in this example.
So make sure there is a static route or when running an IGP that this interface/address is included (passively)
ASR9000 related configuration provided above.
IOS dhcp server configuration is as follows:
ip dhcp excluded-address 184.108.40.206 220.127.116.11
ip dhcp pool SERVER1 network 18.104.22.168 255.255.255.0 default-router 22.214.171.124 dns-server 126.96.36.199
ip dhcp excluded-address 188.8.131.52 184.108.40.206
ip dhcp pool SERVER2 network 220.127.116.11 255.255.255.0 default-router 18.104.22.168 dns-server 22.214.171.124 !
No specific interface configuration required on the interfaces other then ip addresses and proper routing.
dhcpd relay all flag is ON dhcpd relay errors flag is ON dhcpd relay events flag is ON dhcpd relay internals flag is ON dhcpd errors flag is ON dhcpd events flag is ON dhcpd packet flag is ON
RP/0/RSP1/CPU0:A9K-BOTTOM(config-if)#RP/0/RSP1/CPU0:Mar 29 13:23:27.980 : dhcpd: DHCPD PACKET: TP564: L3 Packet RX from addr = 0.0.0.0, port = 68, application len 576, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.980 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_conn_hlr: recv, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, RP/0/RSP1/CPU0:Mar 29 13:23:27.981 : dhcpd: DHCPD PACKET: TP608: BOOTREQUEST Rx, chaddr 0003.a0fd.28a8, interface GigabitEthernet0_1_0_14 RP/0/RSP1/CPU0:Mar 29 13:23:27.981 : dhcpd: DHCPD PACKET: TP982: DISCOVER Rx, base mode, interface GigabitEthernet0_1_0_14, chaddr 0003.a0fd.28a8
received dhcp request on interface
RP/0/RSP1/CPU0:Mar 29 13:23:27.981 : dhcpd: DHCPD PACKET: TP615: DISCOVER Rx, Relay chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.981 : dhcpd: DHCPD PACKET: TP790: Client request opt 82 not untrust, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.982 : dhcpd: DHCPD PACKET: TP791: Giaddr not present, Set giaddr 126.96.36.199, chaddr 0003.a0fd.28a8
Setting GIADDR to the interface receiving the packet, this GIADDR is used to locate the right pool on the servers. So the server sneed to have a pool for the 188.8.131.52/24 subnet.
RP/0/RSP1/CPU0:Mar 29 13:23:27.982 : dhcpd: DHCPD PACKET: TP792: Client request opt 82 nothing to add, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.982 : dhcpd: DHCPD PACKET: TP571: L3 packet TX unicast to dest 184.108.40.206, port 67, source 220.127.116.11, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384)
Unicast packet forwarded to the first helper
RP/0/RSP1/CPU0:Mar 29 13:23:27.982 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_unicast_packet: xmit, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, FROM L3 RP/0/RSP1/CPU0:Mar 29 13:23:27.982 : dhcpd: DHCPD PACKET: TP571: L3 packet TX unicast to dest 18.104.22.168, port 67, source 22.214.171.124, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384)
Unicast packet forwarded to the second helper
they are sent in the order of configuration and the configuration is maintained in order from low to high ip address.
RP/0/RSP1/CPU0:Mar 29 13:23:27.983 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_unicast_packet: xmit, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, FROM L3 RP/0/RSP1/CPU0:Mar 29 13:23:27.983 : dhcpd: DHCPD PACKET: TP835: DISCOVER Tx, chaddr 0003.a0fd.28a8
RP/0/RSP1/CPU0:Mar 29 13:23:27.984 : dhcpd: DHCPD PACKET: TP564: L3 Packet RX from addr = 126.96.36.199, port = 67, application len 300, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.984 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_conn_hlr: recv, src 188.8.131.52 dst 184.108.40.206, L3I GigabitEthernet0_1_0_1, Null output, RP/0/RSP1/CPU0:Mar 29 13:23:27.984 : dhcpd: DHCPD PACKET: TP609: BOOTREPLY Rx, chaddr 0003.a0fd.28a8, interface GigabitEthernet0_1_0_1 RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: TP616: OFFER Rx, Relay chaddr 0003.a0fd.28a8, yiaddr 220.127.116.11
Offer received and IP address offered to us
RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: TP799: Server reply opt 82 not present, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: TP892: Broadcast-flag policy Ignore, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: TP572: L3 packet TX bcast on intf GigabitEthernet0/1/0/14 to port 68, source 18.104.22.168, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_b roadcast_packet: xmit, src 22.214.171.124 dst 126.96.36.199, L3I GigabitEthernet0_1_0_1, Null output, FROM L3 RP/0/RSP1/CPU0:Mar 29 13:23:27.985 : dhcpd: DHCPD PACKET: TP838: OFFER Tx, chaddr 0003.a0fd.28a8, yiaddr 188.8.131.52 RP/0/RSP1/CPU0:Mar 29 13:23:27.991 : dhcpd: DHCPD PACKET: TP564: L3 Packet RX from addr = 0.0.0.0, port = 68, application len 576, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.991 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_conn_hlr: recv, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, RP/0/RSP1/CPU0:Mar 29 13:23:27.991 : dhcpd: DHCPD PACKET: TP608: BOOTREQUEST Rx, chaddr 0003.a0fd.28a8, interface GigabitEthernet0_1_0_14 RP/0/RSP1/CPU0:Mar 29 13:23:27.991 : dhcpd: DHCPD PACKET: TP617: REQUEST Rx, Relay chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.991 : dhcpd: DHCPD PACKET: TP790: Client request opt 82 not untrust, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.992 : dhcpd: DHCPD PACKET: TP791: Giaddr not present, Set giaddr 184.108.40.206, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.992 : dhcpd: DHCPD PACKET: TP792: Client request opt 82 nothing to add, chaddr 0003.a0fd.28a8
Request packet received from client.
RP/0/RSP1/CPU0:Mar 29 13:23:27.992 : dhcpd: DHCPD PACKET: TP571: L3 packet TX unicast to dest 220.127.116.11, port 67, source 18.104.22.168, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.992 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_unicast_packet: xmit, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, FROM L3
Forwarded to helper 1
RP/0/RSP1/CPU0:Mar 29 13:23:27.992 : dhcpd: DHCPD PACKET: TP571: L3 packet TX unicast to dest 22.214.171.124, port 67, source 126.96.36.199, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.993 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_unicast_packet: xmit, src 0.0.0.0 dst 255.255.255.255, L3I GigabitEthernet0_1_0_14, Null output, FROM L3
Forwarded to helper 2, our reply is sent to both servers
RP/0/RSP1/CPU0:Mar 29 13:23:27.993 : dhcpd: DHCPD PACKET: TP841: REQUEST Tx, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: TP564: L3 Packet RX from addr = 188.8.131.52, port = 67, application len 300, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_conn_hlr: recv, src 184.108.40.206 dst 220.127.116.11, L3I GigabitEthernet0_1_0_1, Null output, RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: TP609: BOOTREPLY Rx, chaddr 0003.a0fd.28a8, interface GigabitEthernet0_1_0_1 RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: TP619: ACK Rx, Relay chaddr 0003.a0fd.28a8, yiaddr 18.104.22.168 RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: TP799: Server reply opt 82 not present, chaddr 0003.a0fd.28a8 RP/0/RSP1/CPU0:Mar 29 13:23:27.994 : dhcpd: DHCPD PACKET: TP892: Broadcast-flag policy Ignore, chaddr 0003.a0fd.28a8
ACK received from our server
RP/0/RSP1/CPU0:Mar 29 13:23:27.995 : dhcpd: DHCPD PACKET: TP572: L3 packet TX bcast on intf GigabitEthernet0/1/0/14 to port 68, source 22.214.171.124, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:27.995 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_broadcast_packet: xmit, src 126.96.36.199 dst 188.8.131.52, L3I GigabitEthernet0_1_0_1, Null output, FROM L3 RP/0/RSP1/CPU0:Mar 29 13:23:27.995 : dhcpd: DHCPD PACKET: TP847: ACK Tx, chaddr 0003.a0fd.28a8, yiaddr 184.108.40.206 RP/0/RSP1/CPU0:Mar 29 13:23:29.985 : dhcpd: DHCPD PACKET: TP564: L3 Packet RX from addr = 220.127.116.11, port = 67, application len 300, vrf 0x60000000 (1610612736), tbl 0xe0000000 (3758096384) RP/0/RSP1/CPU0:Mar 29 13:23:29.985 : dhcpd: DHCPD PACKET: >dhcpd_iox_l3_conn_hlr: recv, src 18.104.22.168 dst 22.214.171.124, L3I GigabitEthernet0_1_0_0, Null output, RP/0/RSP1/CPU0:Mar 29 13:23:29.986 : dhcpd: DHCPD PACKET: TP609: BOOTREPLY Rx, chaddr 0003.a0fd.28a8, interface GigabitEthernet0_1_0_0 RP/0/RSP1/CPU0:Mar 29 13:23:29.986 : dhcpd: DHCPD PACKET: TP259: BOOTREPLY Drop, Client not found, interface GigabitEthernet0_1_0_0, chaddr 0003.a0fd.28a8
This is the symptom described, the 2nd offer pre XR 4.1 is dropped by the dhcp relay agent in ios-xr. Post 4.1 we will forward all offers to the client.
This picture illustrates that behavior whereby the 2nd offer is not forwarded. In this case the red line is dashed meaning it made it through. The solid green line is not forwarded and is dropped by the ASR9000 dhcp relay agent.
How RELAY, SNOOPING and PROXY scale, come together and its relation to BNG
if you have relay then we punt the dhcp packets from the client send them over to the dhcp process, convert them in a directed broadcast or unicast towards the helper and really relay only the messages back and forth. Other then CPU there is no operational overhead. No state is maintained, except for a few second “hole” between server and client for when the offer comes back.
There is no scale associated with this per-se other then cpu (ran off the LC CPU for phy/subinterfaces and RP for bundle/bvi).
if you have proxy, the transaction is controlled by the dhcp process. so client talks to a9k. a9k talks to the dhcp server and it will maintain a binding that expires on lease and absence of a renew.
The scale associated here is cpu for the message transaction. ARP interaction since dhcp proxy will install a arp forwarding adj also and memory for the binding table itself. the other scale factor is here how far the TX ADJ table can grow from ARP, we tested up to 128k, but if you have enough shared mem, this can go further than that, depending on the LC CPU mem (phy sub/if) or RP (bvi/bundle) can increase its shmem window (this is really like a ram disk and can dynamically grow as needed). the other part is the tx adj table of the npu, there is a defined limit of about 128k I thought it was (mem size in resolve)
Then there is snooping; this is really the same as proxy, but in an L2 environment AND with the notion that the binding table built is installed in the hw forwarding. this is capped at 32k per npu. Advantage of snooping is that with the binding in hw, we can do extra verification checks on ingress to make sure the mac/ip/port matches what is in the table. for egress outbound the tx adj is used installed.
Bng and Proxy
Now when you do BNG with proxy, it combines the operation of proxy and snooping: a binding created by proxy is now installed in hw also, with those checks mention in effect. this caps bng/proxy to 32k per npu.
For proxy, snoop, bng/proxy the 128k limit is really a testing limit defined by: - (lc/rp) cpu capabity - (lc/rp) cpu memory - tx adj table for snoop and bng/proxy since it is hw installed it caps it at 32k per npu, which is a hw limit. for those sessions not needing bng, but requiring dhcp services, to allow for better mem/cpu control and scale, it might make more sense to do relay here, but there is nothing against use of proxy per-se.
All broadcasts are handled on the RP, in the ASR9000 or any XR platform for that matter, we use LPTS (local packet transport services) similar as what IOS calls COPP (control plane policing)
The policing is on a per NPU basis. Look at Packet trace and troubleshooting in the ASR9000
for more information on packet troubleshooting in the ASR9000.
... View more
This document shows you how to reset a lost password or when you have locked yourself out due to a problematic AAA configuration.
Because IOS-XR is substantially different in the way config files are managed, the standard trick of conf-reg 0x2142 will not work for IOS-XR.
You can lock yourself out if you are configuring aaa authentication to tacacs with no local fall back, if the tacacs server is unavailable there
is no way for you to get in.
aaa authentication login default groupt tacacs
Also this procedure is good when you have forgotten the password to your super user in IOS-XR to manage your machine.
The following step through guide can be tried, the details of each step are listed below with more explanation:
•1) Fixing AAA configuration errors
•a. On the standby RP/RSP from the CONSOLE port hit the ESC key and type ‘ksh’ without quotes and hit ENTER
i. Login with a local username and password
ii. If this fails get the standby RP/RSP into ROMMON
iii. Bypass KSH authentication with AUX_AUTHEN_LEVEL=0 and boot
iv. Try step 1a again or use the AUX port and go to step 1b
•b. View and edit the configuration from KSH
i. Save the configuration to harddisk with ‘nvgen -c -l 1 -t 1 -o 1 > harddisk:/backupconfig.txt’
ii. Edit out the bad AAA statements with ‘nano –e /harddisk:/backupconfig.txt’
•c. Try to roll back the configuration with ‘config_rollback –n 0x1’
•d. Bypass AAA and enter exec mode with ‘/pkg/bin/exec –a’
•e. Attempt to use show commands or change the configuration
i. If this fails reload all RP/RSP ROMMON
ii. On the standby card set IOX_CONFIG_FILE=/harddisk:/backupconfig.txt or use ‘boot <image> -a <bogus_config>’ and boot
iii. Also follows step 2g if you saw issues in 1a
iv. If nothing above worked then this is the only option
•2) Fixing a lost local username/password
•a. Get the standby RP/RSP into ROMMON
i. Bypass KSH authentication with AUX_AUTHEN_LEVEL=0 and boot
•b. View the admin configuration with ‘nvgen –b /admin/cfg’
•c. Save the admin configuration to the harddisk and edit out any and all users if you need other portions of this file
•d. Bypass AAA and enter exec mode with ‘/pkg/bin/exec –a’
•e. Attempt to use show commands or change the configuration
•f. If this fails reload all RP/RSP to ROMMON
•g. Set confreg 0x142 or IOX_ADMIN_CONFIG_FILE=/harddisk:/backupconfig.txt on the standby card or ‘boot <image> -o <bogus_config>’ and boot
i. Note that this does not ignore the exec configuration and will not help if the issue is AAA related
•h. Enter a new username and password when prompted
•3) Fixing both issues
•a. If you do not know a local login or cannot use the KSH method to recover the configuration then both the IOX_CONFIG_FILE and IOX_ADMIN_CONFIG_FILE will need to be pointed towards non-existent files. Both the admin and exec configurations will be cleared by this method
•4) Make sure to remove any ROMMON variables which were change
5) XR-VM Username/Password reset procesdure using Sysadmin VM
There are 2 steps to this process.
1) Override the BASE running configuration
When you configure the problematic AAA statement sample as above.
2) Override the admin configuration that stores local usernames and passwords
When you don't remember any of the local usernames/passwords you have defined locally.
Overriding the Base configuration in XR:
In rommon set the following variable:
the file no-config is just a non existent file, you can give any name here really.
Note: This ROMMON variable will persist and needs to be removed after password recovery. Check the 'clean up' section.
And issue 'sync', this will make the change persistent in the rommon config vars.
Issue 'i' or 'reset' and when the rsp is booting up, it should ignore the config file, since there's no config file found on /harddisk: called no-config
Overriding the ADMIN configuration in XR:
In Admin configuration we store all the local usernames and passwords.
Similarly you can do the same thing for admin config:
You should get prompted for root user/pass and will have a blank config on the box.
You need to load your config and do your modification.
Note: This ROMMON variable will persist and needs to be removed after password recovery. Check the 'clean up' section.
Step 2 and 3
are the same as for the base xr config file.
Another way of recovery of the password is to enable the following again in rommon:
Which will allow the aux port to drop to ksh upon the RSP bootup with no prompt for login.
At the prompt you can either type:
Which will give you a router prompt: Or simply
Which drops you into EXEC config mode.
# uname -a
QNX node0_RSP0_CPU0 6.4.0 2009/12/10-13:43:22PST asr9k ppcbe
# /pkg/bin/exec -a
Make sure that after you're done with your changes, in case you made the rommon vars persistent, you may want to unset
the variables to get back to the normal files that are used.
rommon> unset IOX_ADMIN_CONFIG_FILE
rommon> unset IOX_CONFIG_FILE
If you forget the cleanup, you might see these lines:
RP/0/RSP1/CPU0:Oct 28 07:18:37.141 : locald_DSC: %SECURITY-LOCALD-3-LWA_ADD_FAIL : Failed to add the username admin to lightweight authentication password database: No such file or directory
Another way to clear the variable:
more nvram:/classic-rommon-var location 0/RSP1/CPU0
run iox_on 0/RSP1/CPU0 nvram_rommonvar IOX_CONFIG_FILE ""
XR-VM Username/Password reset procedure using Sysadmin VM
Note: This is not a process to hack router but user need sysadmin username / password for the accessibility of box by bypass XR credentials.
Steps to perform this activity:
1. login to router : I was having console access to the box.
bgl-xdm-009:112> telnet 10.67.30.20 2037 Trying 10.67.30.20... Connected to 10.67.30.20. Escape character is '^]'.
User Access Verification
Password: Password OK
2. Pass interrupts "ctrl + o" to toggle to sysadmin
sysadmin-vm:0_RP0# *** IDLE TIMEOUT ***
System Admin Username:
3. enter sysadmin username and password
System Admin Username: xxxxx
xxxxx connected from 127.0.0.1 using console on sysadmin-vm:0_RP0 sysadmin-vm:0_RP0#
4. with this login you can access "sysadmin- VM prompt"
5. From Sysadmin VM to access XR - VM perform following action
i. list sdr ips sysadmin-vm:0_RP0# show sdr Tue Apr 3 05:12:47.110 UTC
SDR: default-sdr Location IP Address Status Boot Count Time Started ----------------------------------------------------------------------------- 0/RP0/VM1 192.0.0.4 RUNNING 1 03/01/2019 02:10:28 0/RP0/VM2 192.0.0.6 RUNNING 1 03/01/2019 02:10:56
ii. ssh SDR VM1 address
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. <> Last login: Fri Mar 1 02:14:39 2019 from 192.0.0.4
iii. Now you are at XR-VM and to disable credential pass "ctrl + a" interrupt
6. After disabling credential you can access XR-VM
at this stage you can create/ delete / modify user credentials to access router at XR-VM directly.
P/0/RP0/CPU0:customer2#show running-config Tue Apr 3 05:13:02.445 UTC Building configuration... !! IOS XR Configuration version = 6.3.3 !! Last configuration change at Sun Feb 17 23:31:51 2019 by ZTP ! hostname customer2 >>> username asd group root-lr group cisco-support secret 5 $1$rdaY$qJt7aNcc8uFKqhP/rK11V1 !
It has been seen that sometimes a system autonomously enters password recovery mode. This is identified with:
“enter root-system username”
This is due to a ddts known as CSCth03923
You end up providing what you think is a known username and password combination and it failes to get you in.
The solution is simple, just enter a fake username/password that you know for sure has not been configured yet and you're in!
Xander Thuijs - CCIE #6775
Sr Tech Lead ASR9000
... View more
Wanted to verify with what packet size you are pinging, the asr9k doesn't do fragmentation in hardware and everything gets sent to the CPU for processing then in order to fragmentation. To find out if the 9k is part of the issue you can do the following troubleshooting guide for packet tracing within the 9k: ASR9000 packet troubleshooting and understanding NP drop counters The CRS is known to have very strict LPTS policers for locally destined Low priority traffic such as ICMP pings, hence you will see losses with a larger amount of pings. the show lpts commands provided earlier . From the further tests you've done, it looks like you might be looking at the 6k for this particular issue. xander Xander Thuijs - CCIE #6775 Sr Tech Lead ASR9000
... View more
Introduction This document provides for a couple of examples as to how untagged L2 protocols like CDP, BPDU's, VTP, LACP etc are treated in L2 or L3 scenarios on the ASR9000. Core Issue Because the ASR9000 follows the EVC model for L2 connections, the handling of L2 adjacency related traffic works a bit different then what one might be used to from the 7600 or IOS based platforms. In XR/ASR9000 some changes to this behavior have been made on a per protocol basis, which will be discussed in this document. Resolution Topology used for this example and verification: When “7200” is sending cdp to the ASR9000 and ... : Scenarios: 1) the 9k "BOT" has cdp enabled on its main interface then you’ll establish a cdp relation between “7200” and “9k-BOT”. No untagged EFP defined on the respective interfaces. 2) the 9k BOT does not have cdp enabled on its main interface AND you have an untagged EFP defined on the 9k which is put in a xconnect for instance connecting g0/1/0/14 to te 0/4/0/1 on ASR9K-BOT, THEN “7200” will establish a cdp relationship with “ASR9K-TOP”: the CDP packets are transported over the XCON. 3) the 9k BOT has cdp enabled on its main interface AND you have an untagged EFP defined but this EFP is not put in a BD or XCON, then the 7200 sees BOT as its neighbor, and: a) If your XR version is 4.2.0 or before then: 9K-BOT doesn’t see any neighbor (received cdp packets are transported or consumed by the EFP) and 9K-TOP doesn’t see anyone as there is no xconnect to transport the CDP Packets over, yet EFP is defined. b) If your XR version 4.2.1, 4.2.3, 4.3.0 or newer then: Eventhough there is an untagged EFP present, the CDP packets in the presence of a local cdp configuration are peeled out of the untagged EFP and will not be subject to the EFP service definition.. 4) and the 9k has cdp enabled on its main interface AND you have an untagged EFP defined, with has an xconnect to TOP then a) If your XR version is 4.2.0 or before then: the 7200 sees BOT AND TOP as its neighbor, 9K-BOT doesn’t see any neighbor (received cdppackets are transported or consumed by the EFP), and TOP sees 7200. b) If your XR version 4.2.1, 4.2.3, 4.3.0 or newer then: 7200 sees BOT and TOP as neighbors (xconnect present and no CDP configured on TE0/4/0/1 on BOT), BOT sees 7200 and TOP sees no one. TOP doesn't see anyone because there is no cdp configured on te 0/4/0/1 on BOT, and the cdp packets from 7200 are consumed locally. In these cases 3 and 4 are inconsistent configurations of course. The behavioral difference between releases prior to XR 4.2.1 and after is defined by CSCtg04453. This DDTS changes the handling of CDP traffic only in relation to untagged EFP. Associated Configurations ASR9K-BOT configuration: EFP to transport the untagged packets to the ASR9K-TOP interface TenGigE0/4/0/1.1 l2transport encapsulation untagged ! To capture the CDP packets on to an EFP define this: interface GigabitEthernet0/1/0/14.1 l2transport encapsulation untagged ! Main interface configuration facing the 7200. The "cdp" keyword here is present depending on the test cases above. interface GigabitEthernet0/1/0/14 cdp load-interval 30 ! Cross connect configuration l2vpn xconnect group cdp p2p cdp interface TenGigE0/4/0/1.1 interface GigabitEthernet0/1/0/14.1 ! ! ASR9K-TOP configuration: interface TenGigE0/4/0/1 cdp ! Debug and verification Seen from ASR9K-TOP LC/0/4/CPU0:Mar 22 17:42:58.746 : cdp: Packet received from fp7200-6 (0003.a0fd.28a8) on interface TenGigE0/4/0/1 LC/0/4/CPU0:Mar 22 17:42:58.746 : cdp: **Entry found in cache** LC/0/4/CPU0:Mar 22 17:42:58.746 : cdp: Received Port ID GigabitEthernet6/0 For CASE-4, whcih is the most interesting one, the output from 7200: fp7200-6#sh cdp ne Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge S - Switch, H - Host, I - IGMP, r - Repeater Device ID Local Intrfce Holdtme Capability Platform Port ID A9K-BOTTOM Gig 6/0 139 R ASR9K Seri Gig 0/1/0/14 A9K-TOP.cisco.com Gig 6/0 169 R ASR9K Seri TenGigE0/4/0/1 A9K-BOT doesn't see anyone as expected in case-4 RP/0/RSP1/CPU0:A9K-BOTTOM#sh cdp ne Tue Mar 22 12:47:05.939 EST Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge S - Switch, H - Host, I - IGMP, r - Repeater Device ID Local Intrfce Holdtme Capability Platform Port ID RP/0/RSP1/CPU0:A9K-BOTTOM# Related Information Are CDP, VTP and BPDU crossing an xconnect or bridge-domain (BD) ? Yes they did, originally. Depending on release (lovely I know). The original behavior prior to XR 4.2.1 was that untagged traffic regardless of local configuration was consumed by the untagged EFP. This is at times undesirable for especially STP and CDP also as seen above. After XR 4.2.1 we made 2 changes to the logic for notably CDP and (M)STP BPDU's. If there is an untagged EFP present but the protocl is not configured locally, transport it as part of the untagged EFP service definition. If the service is configured locally, as seen in the example above for cdp, then consume the packet locally and not transport it. The ddts that changed the behavior for STP is CSCtx28089. You can use:“ethernet filter dot1q” to drop the BPDUs from each side and keep them isolated from each other. If not configured, the BPDUs will cross over and interfere with each other. You can also use MAC based ACLs to prevent BPDUs/ CDP packets from crossing over. This example will filter PVST and MSTP packets on an L2 interface for specifically VLAN 100. (note that PVST is vlan tagged BPDU's and MSTP are untagged BPDU's, so to filter MSTP packets this ACL is to be applied to an interface with the "encapsulation untagged" keyword) ethernet-services access-list L2ACL 5 remark Access-list for blocking PVST/MSTP BPDU's 10 deny any host 0100.0ccc.cccd 20 deny any host 0180.c200.0000 30 permit any any ! interface TenGigE0/4/0/1.100 l2transport encapsulation dot1q 100 ethernet-services access-group L2ACL ingress ! Xander Thuijs, CCIE #6775 Principal Engineer ASR9000
... View more
This document provides some extra documentation and use cases on the use of port spanning or port mirroring.
You can monitor traffic passing in & out of a set of L2 or L3 Ethernet interfaces (including bundle-Ether).
ASR 9000 is the only platform implementing SPAN on XR (Only support on ethernet linecards, not on SIP-700.)
You can use SPAN/Mirror in the follow scenarios
- L2 & L3 interfaces. - Local, R-SPAN, and PW-SPAN only (no ER SPAN.) - Scale limits: 8 monitor sessions 800 total source ports 1.5 Gig bidirectional replication limit toward fabric for bundle interfaces and 10 Gig ports. Guideline: ~ 10% - 15% total bandwidth can be mirrored system-wide - Source ports: Physical, EFPs, and bundles interfaces (L2 & L3) - Destination ports: Ethernet interfaces, EFPs, and PW-SPAN. (No bundle) [ only L2 transport interfaces are supported as destination ports]
- Ability to use ACL's to define which traffic is to be captured
- Capture multicast traffic is possible
Note: some of the functionality mentioned are enhancements to the XR 4.0.1 release, this document assumes you are using this release or later.
A good reference on the terminology of SPAN/Mirror can be found here:
SPAN order of operation
SPAN mirrors what is on the wire For ingress, this means packets are mirrored before QOS, ACL, and encapsulation rewrite operations. For egress, this means packets are mirrored after QOS, ACL, and encapsulation rewrite operations.
Partial Packet Mirroring
User can configure to mirror first 64 upto 256 bytes of the packet. Note: The actual mirrored packet will be the configured size plus 4-byte trailling CRC.
interface GigabitEthernet0/6/0/20 l2transport monitor-session PW mirror first 100 <== valid range: [64, 256], inclusively ! !
Note: The mirrored packet received at sniffer will have the size of 104 (4-byte of trailing CRC added by transmit MAC layer.)
ACL based Mirroring
“permit/deny” determines the behavior of the regular traffic (forwarded or dropped) “ capture ” determines whether the packet is mirrored to the SPAN destination.
On SPAN: mirror traffic on the wire (regardless with or without ACL.)
ACL on ingress direction: SPAN will mirror traffic even regular traffic dropped by ACL: Always mirror! ACL on egress direction Will mirror if regular traffic is forwarded (Permit) Will not mirror if regular traffic is dropped (Deny.)
Inconsistent configurations: “acl” is configured on SPAN source port but ACL has no “capture” keyword: No traffic gets mirrored. “acl” is NOT configured on SPAN source port but ACL has “capture” keyword: Mirroring traffic as normal , no ACL performed.
The ACL can also be an L2 ACL :
ethernet-services access-list esacl_t2 10 deny 1234.5678.90ab 0000.0000.0000 any capture
L3 Spanning Example
monitor-session TEST destination interface GigabitEthernet0/1/0/2 (<<<< this is NP3) ! interface GigabitEthernet0/1/0/14 (<<<< this is NP2) ipv4 address 126.96.36.199 255.255.255.0 monitor-session TEST acl ! load-interval 30 ipv4 access-group span ingress ! ipv4 access-list span 10 permit ipv4 any host 188.8.131.52 capture 15 permit ipv4 any host 184.108.40.206 capture 20 permit ipv4 any host 220.127.116.11 30 permit ipv4 any any
Sample TRAFFIC GEN: (sending multicast in this example) tgn rate 1000 L2-dest-addr 0100.5E01.0101 L2-src-addr 0003.A0FD.28A8 L3-src-addr 18.104.22.168 L3-dest-addr 22.214.171.124
Checking NP2: (the port that we are spanning) Show global stats counters for NP2, revision v3
Read 12 non-zero NP counters: Offset Counter FrameValue Rate (pps) ------------------------------------------------------------------------------- 22 PARSE_ENET_RECEIVE_CNT 5478 1001 31 PARSE_INGRESS_DROP_CNT 3 1 33 RESOLVE_INGRESS_DROP_CNT 5474 1000 (there is no mcast recipient for this mcast addr, but we’re still replicating, see red line) 40 PARSE_INGRESS_PUNT_CNT 1 0 50 MODIFY_RX_SPAN_CNT 5475 1000 54 MODIFY_FRAMES_PADDED_CNT 5475 1000 68 RESOLVE_INGRESS_L3_PUNT_CNT 1 0 104 LOOP 1 0 224 PUNT_STATISTICS 9 2 480 RESOLVE_IPM4_ING_RTE_DROP_CNT 5475 1000 565 UIDB_TCAM_MISS_AGG_DROP 3 1 570 UIDB_TCAM_MISS_PORT4_DROP_FOR_HOST 3 0
NP3 is the span monitor interface: Show global stats counters for NP3, revision v3
Read 16 non-zero NP counters: Offset Counter FrameValue Rate (pps) ------------------------------------------------------------------------------- 22 PARSE_ENET_RECEIVE_CNT 36 0 23 PARSE_FABRIC_RECEIVE_CNT 79656 1000 30 MODIFY_ENET_TRANSMIT_CNT 79655 1000
Packets received from fabric and sent off to the Ethernet on the span port!
PW SPAN example
For PW span to work, you need to define a local monitor session with a destination pseudo wire. You apply that span session to the interface of interest and define an xconnect group that also leverages that span session as one of the pw ends.
On the remote side where the PW terminates, you just configure regular VPWS.
Here an example:
On the Local Side, besides my Span configuration, there is also a local cross connect between the interested session we want to span over the PW
xconnect group TEST p2p TEST interface GigabitEthernet0/1/0/39
! port 39 is the port where we apply the span on. interface GigabitEthernet0/1/0/20.100 ! this is just a random AC to have traffic flowing between the spanned port. !
interface GigabitEthernet0/1/0/20.100 l2transport encapsulation dot1q 100 rewrite ingress tag pop 1 symmetric ! the tag is popped because the other XCON end is a plain ethernet without vlan. The explanation and use cases of tag popping can be found a related
! Tech note article.
Configuration on the remote side:
Regular VPWS configuration:
RP/0/RSP0/CPU0:A9K-TOP#sh run l2vpn l2vpn xconnect group PW-SPAN p2p PW-SPAN_1 interface GigabitEthernet0/0/0/39 neighbor 126.96.36.199 pw-id 1 ! ! ! interface GigabitEthernet0/0/0/39 load-interval 30 transceiver permit pid all l2transport ! !
the neighbor in the l2vpn configuration is the LDP neighbor ID between which the PW is built.
Show on remote side: RP/0/RSP0/CPU0:A9K-TOP#show l2vpn xcon group PW-SPAN det
Group PW-SPAN, XC PW-SPAN_1, state is up; Interworking none AC: GigabitEthernet0/0/0/39, state is up Type Ethernet MTU 1500; XC ID 0x4000a; interworking none Statistics: packets: received 0, sent 16570475 bytes: received 0, sent 994228500
! packets received from the PW are sent out hte Attachment circuit's interface. The analyzer is connected to G0/0/0/39 PW: neighbor 188.8.131.52, PW ID 1000, state is up ( established ) PW class not set, XC ID 0x4000a Encapsulation MPLS, protocol LDP PW type Ethernet, control word disabled, interworking none PW backup disable delay 0 sec Sequencing not set
MPLS Local Remote ------------ ------------------------------ ----------------------------- Label 16002 16027 Group ID 0xa40 0x2 Interface GigabitEthernet0/0/0/39 PW/TM/MS MTU 1500 1500 Control word disabled disabled PW type Ethernet Ethernet VCCV CV type 0x2 0x2 (LSP ping verification) (LSP ping verification) VCCV CC type 0x6 0x6 (router alert label) (router alert label) (TTL expiry) (TTL expiry) ------------ ------------------------------ ----------------------------- MIB cpwVcIndex: 4294705162 Create time: 04/04/2011 14:36:42 (00:20:07 ago) Last time status changed: 04/04/2011 14:36:42 (00:20:07 ago) Statistics: packets: received 16570475, sent 0 bytes: received 994228500, sent 0
! Packets received on the Pseudo Wire from the SPAN port
NOTE: Pseudo Wire counters on the span side are not incrementing.That is the XCON group "cisco" in this picture config example.
This is intentional. You can review the SPANNING also with this command:
RP/0/RSP1/CPU0:A9K-BOTTOM#sh monitor-session counters
Monitor-session PW_TM_MS GigabitEthernet0/1/0/39 Rx replicated: 58488205 packets, 3743245120 octets Tx replicated: 58488206 packets, 3743245184 octets Non-replicated: 0 packets, 0 octets
R-SPAN is natively support with the capability of ASR9000 to do vlan imposition:
destination interface gig0/2/0/19.10
interface gig0/2/0/12.10 l2transport
encapsulation dot1q 10 <<< Monitoring vlan 10 traffic
interface gig0/2/0/19.10 l2transport (*)
encapsulation dot1q 100 <<< VLAN 100 will get imposed.
(*) Monitor destination could be any supported destination interface regardless of monitor source
Xander Thuijs, CCIE #6775
Sr. Tech Lead ASR9000
... View more
This document provides details on how QOS is implemented in the ASR9000 and how to interpret and troubleshoot qos related issues.
QOS is always a complex topic and with this article I'll try to describe the QOS architecture and provide some tips for troubleshooting.
Based on feedback on this document I'll keep enhancing it to document more things bsaed on that feedback.
The ASR9000 employs an end to end qos architecture throughout the whole system, what that means is that priority is propagated throughout the systems forwarding asics. This is done via backpressure between the different fowarding asics.
One very key aspect of the A9K's qos implementation is the concept of using VOQ's (virtual output queues). Each network processor, or in fact every 10G entity in the system is represented in the Fabric Interfacing ASIC (FIA) by a VOQ on each linecard.
That means in a fully loaded system with say 24 x 10G cards, each linecard having 8 NPU's and 4 FIA's, a total of 192 (24 times 8 slots) VOQ's are represented at each FIA of each linecard.
The VOQ's have 4 different priority levels: Priority 1, Priority 2, Default priority and multicast.
The different priority levels used are assigned on the packets fabric headers (internal headers) and can be set via QOS policy-maps (MQC; modular qos configuration).
When you define a policy-map and apply it to a (sub)interface, and in that policy map certain traffic is marked as priority level 1 or 2 the fabric headers will represent that also, so that this traffic is put in the higher priority queues of the forwarding asics as it traverses the FIA and fabric components.
If you dont apply any QOS configuration, all traffic is considered to be "default" in the fabric queues. In order to leverage the strength of the asr9000's asic priority levels, you will need to configure (ingress) QOS at the ports to apply the priority level desired.
In this example T0 and T1 are receiving a total of 16G of traffic destined for T0 on the egress linecard. For a 10G port that is obviously too much.
T0 will flow off some of the traffic, depending on the queue, eventually signaling it back to the ingress linecard. While T0 on the ingress linecard also has some traffic for T1 on the egress LC (green), this traffic is not affected and continues to be sent to the destination port.
The ASR9000 has the ability of 4 levels of qos, a sample configuration and implemenation detail presented in this picture:
Policer having exceed drops, not reaching configured rate
When defining policers at high(er) rates, make sure the committed burst and excess burst are set correctly.
This is the formula to follow:
Set the Bc to CIR bps * (1 byte) / (8 bits) * 1.5 seconds
Default burst values are not optimal
Say you are allowing 1 pps, and then 1 second you don’t send anything, but the next second you want to send 2. in that second you’ll see an exceed, to visualize the problem.
Alternatively, Bc and Be can be configured in time units, e.g.:
police rate percent 25 burst 250 ms peak-burst 500 ms
For viewing the Bc and Be applied in hardware, run the "show qos interface interface [input|output]".
Why do I see non-zero values for Queue(conform) and Queue(exceed) in show policy-map commands?
On the ASR9k, every HW queue has a configured CIR and PIR value. These correspond to the "guaranteed" bandwidth for the queue, and the "maximum" bandwidth (aka shape rate) for the queue.
In some cases the user-defined QoS policy does NOT explicitly use both of these. However, depending on the exact QoS config the queueing hardware may require some nonzero value for these fields. Here, the system will choose a default value for the queue CIR. The "conform" counter in show policy-map is the number of packets/bytes that were transmitted within this CIR value, and the "exceed" value is the number of packets/bytes that were transmitted within the PIR value.
Note that "exceed" in this case does NOT equate to a packet drop, but rather a packet that is above the CIR rate on that queue.
You could change this behavior by explicitly configuring a bandwidth and/or a shape rate on each queue, but in general it's just easier to recognize that these counters don't apply to your specific situation and ignore them.
What is counted in QOS policers and shapers?
When we define a shaper in a qos pmap, the shaper takes the L2 header into consideration.
The shape rate defined of say 1Mbps would mean that if I have no dot1q or qinq, I can technically send more IP traffic then having a QIQ which has more L2 overhead. When I define a bandwidth statement in a class, same applies, also L2 is taken into consideration.
When defining a policer, it looks at L2 also.
In Ingress, for both policer & shaper, we use the incoming packet size (including the L2 header).
In order to account the L2 header in ingress shaper case, we have to use a TM overhead accounting feature, that will only let us add overhead in 4 byte granularity, which can cause a little inaccuracy.
In egress, for both policer & shaper we use the outgoing packet size (including the L2 header).
ASR9K Policer implementation supports 64Kbps granularity. When a rate specified is not a multiple of 64Kbps the rate would be rounded down to the next lower 64Kbps rate.
For policing, shaping, BW command for ingress/egress direction the following fields are included in the accounting.
Port level shaping
Shaping action requires a queue on which the shaping is applied. This queue must be created by a child level policy. Typically shaper is applied at parent or grandparent level, to allow for differentiation between traffic classes within the shaper. If there is a need to apply a flat port-level shaper, a child policy should be configured with 100% bandwidth explicitly allocated to class-default.
Understanding show policy-map counters
QOS counters and show interface drops:
Policer counts are directly against the (sub)interface and will get reported on the "show interface" drops count. The drop counts you see are an aggregate of what the NP has dropped (in most cases) as well as policer drops.
Packets that get dropped before the policer is aware of them are not accounted for by the policy-map policer drops but may show under the show interface drops and can be seen via the show controllers np count command.
Policy-map queue drops are not reported on the subinterface drop counts. The reason for that is that subinterfaces may share queues with each other or the main interface and therefore we don’t have subinterface granularity for queue related drops.
Counters come from the show policy-map interface command
Class name as per configuration
Statistics for this class
Classification statistics (packets/bytes) (rate - kbps)
Packets that were matched
Matched : 31583572/2021348608 764652
packets that were sent to the wire
Transmitted : Un-determined
packets that were dropped for any reason in this class
Total Dropped : Un-determined
Policing statistics (packets/bytes) (rate - kbps)
Packets that were below the CIR rate
Policed(conform) : 31583572/2021348608 764652
Packets that fell into the 2nd bucket above CIR but < PIR
Policed(exceed) : 0/0 0
Packets that fell into the 3rd bucket above PIR
Policed(violate) : 0/0 0
Total packets that the policer dropped
Policed and dropped : 0/0
Statistics for Q'ing
Queueing statistics <<<----
Internal unique queue reference
Queue ID : 136
how many packets were q'd/held at max one time
(value not supported by HW)
High watermark (Unknown)
number of 512-byte particles which are currently
waiting in the queue
Inst-queue-len (packets) : 4096
how many packets on average we have to buffer
(value not supported by HW)
packets that could not be buffered because we held
more then the max length
Taildropped(packets/bytes) : 31581615/2021223360
see description above (queue exceed section)
Queue(conform) : 31581358/2021206912 764652
see description above (queue exceed section)
Queue(exceed) : 0/0 0
Packets subject to Randon Early detection
and were dropped.
RED random drops(packets/bytes) : 0/0
Understanding the hardware qos output
RP/0/RSP0/CPU0:A9K-TOP#show qos interface g0/0/0/0 output
With this command the actual hardware programming can be verified of the qos policy on the interface
(not related to the output from the previous example above)
Tue Mar 8 16:46:21.167 UTC Interface: GigabitEthernet0_0_0_0 output Bandwidth configured: 1000000 kbps Bandwidth programed: 1000000 ANCP user configured: 0 kbps ANCP programed in HW: 0 kbps Port Shaper programed in HW: 0 kbps Policy: Egress102 Total number of classes: 2 ---------------------------------------------------------------------- Level: 0 Policy: Egress102 Class: Qos-Group7 QueueID: 2 (Port Default) Policer Profile: 31 (Single) Conform: 100000 kbps (10 percent) Burst: 1248460 bytes (0 Default) Child Policer Conform: TX Child Policer Exceed: DROP Child Policer Violate: DROP ---------------------------------------------------------------------- Level: 0 Policy: Egress102 Class: class-default QueueID: 2 (Port Default) ----------------------------------------------------------------------
Default Marking behavior of the ASR9000
If you don't configure any service policies for QOS, the ASR9000 will set an internal cos value based on the IP Precedence, 802.1 Priority field or the mpls EXP bits.
Depending on the routing or switching scenario, this internal cos value will be used to do potential marking on newly imposed headers on egress.
If the node is L3 forwarding, then there is no L2 CoS propagation or preservation as the L2 domain stops at the incoming interface and restarts at the outgoing interface.
Default marking PHB on L3 retains no L2 CoS information even if the incoming interface happened to be an 802.1q or 802.1ad/q-in-q sub interface.
CoS may appear to be propagated, if the corresponding L3 field (prec/dscp) used for default marking matches the incoming CoS value and so, is used as is for imposed L2 headers at egress.
If the node is L2 switching, then the incoming L2 header will be preserved unless the node has ingress or egress rewrites configured on the EFPs. If an L2 rewrite results in new header imposition, then the default marking derived from the 3-bit PCP (as specified in 802.1p) on the incoming EFP is used to mark the new headers.
An exception to the above is that the DEI bit value from incoming 802.1ad / 802.1ah headers is propagated to imposed or topmost 802.1ad / 802.1ah headers for both L3 and L2 forwarding;
ASR9000 Quality of Service configuration guide
Xander Thuijs, CCIE #6775
... View more
Introduction In this article we'll be discussing the EVC (Ethernet Virtual Circuit) infrastructure that the ASR9000 and IOS-XR use to define L2 services. You'll find in this article how packets coming in on an interface are being matched against an EFP (Ethernet Flow Point), how to manipulate the vlan tags and use them in XCONNECT (VPWS) or Bridging (eg VPLS) scenarios with the optional L3 endpoint to provide IRB (Integrated Route Bridging) Core Issue The way that IOS-XR handles EVC's is a bit different then the way that IOS handles it when for instance using the 7600. When starting with IOS-XR and the ASR9000 there are a few things that you will want to be aware of when defining L2 services. A couple of the key differences are: - XR does have the concept of a trunk interface - therefore XR cannot do vlan pruning - XR does not strip vlan tags by default. - XR does not have the concept of an "interface VLAN" a.k.a. SVI's (Switch Virtual Interface), instead it uses a BVI (Bridge Virtual Interface) that can be inserted into the bridge-domain. Flexible VLAN matching How the ASR9000 matches traffic to an EFP TAG rewriting how to modify the vlan headers and how that works Defining L2 services and a comparison with IOS Providing L3 services in L2VPN using IRB/BVI interfaces Resolution In order to define an L2 service, we need to match traffic to a particular interface. In IOS-XR and the ASR9000 we use the Ethernet Flow Point (EFP) to match this traffic. An EFP is effectively a subinterface of a physical interface with the keyword "l2transport" attached to it. This l2transport defines that we are going to use this (sub) interface for L2 Services. We can match L2 and L3 interfaces on a single physical port: Flexible VLAN matching When traffic is coming in on a port, we use the TCAM to find out which particular subinterface this traffic matches on. With that in mind there are a couple of rules as to how traffic is matched. An EFP is defined as follows: RP/0/RSP0/CPU0:asr(config)# int gig 0/0/0/4.100 l2transport RP/0/RSP0/CPU0:asr(config- subif )#encapsulation ? default Packets unmatched by other service instances If a particular packet is not matching any other specific EFP on this physical port, this "Default" will capture all unmatched traffic. dot1ad IEEE 802.1ad VLAN-tagged packets dot1q IEEE 802.1Q VLAN-tagged packets untagged Packets with no explicit VLAN tag If there is no vlan tag on the packet, the "untagged" EFP will capture this traffic, this is effectively plain ethernet and useful for instance to capture BPDU's for instance. When we are going to match on say dot1q encapsulated traffic we have a variaty of how we can match vlan tagged traffic (see also foot note below in the "Related Information" section on the ethertypes used). RP/0/RSP1/CPU0:A9K-BOTTOM(config-subif)#encapsulation dot1q ? <1-4094> Start of VLAN range <1-4094> Single VLAN id any Match any VLAN id priority-tagged IEEE 802.1ad priority-tagged packets At the first level of dot1q classification we can select a vlan, vlan-range or any. These are obvious. The option "Priority tagged" allows us to capture vlan encapped traffic that is with a vlan id of 0. RP/0/RSP1/CPU0:A9K-BOTTOM(config-subif)#encapsulation dot1q 100 ? comma comma exact Do not allow further inner tags ingress Perform MAC-based matching second-dot1q IEEE 802.1Q VLAN-tagged packets Here is an important concept that is to be highlighted. You see the "word" exact" here. What that means is, in the absence of the keyword exact, if the outter vlan header is "100" in this example, this EFP is matched. so that means that also qinq frames that are of the 100 outter and 200 inner kind (if there is no specific EFP for the qiq combo 100/200 available) will match this EFP. Just a few examples: encapsulation dot1q 100: will match any number of vlan headers as long as the outter vlan id is 100 encapsulation dot1q 100 second any: will match any qiq frame where the outter vlan is 100 encapsulation dot1q 100 second 200: will match vlan tagged packets whereby the outter is 100, the inner is 200 and also a potential vlan combo of 100/200/300 encapsulation dot1q 100 second 200 exact: will match vlan tagged packets whereby the outter is 100, the inner is 200 and no other vlan tags are on the packet then these 2 specified. Normally "longest match" will win, or better put, the most specific match will win. ASR9000 doesn't support best match for VLAN ranges, but we do support best match if the "any" keyword is used. So the configuration : EFP 1 VLAN: S-VLAN=1000, C-VLAN_range=1-4095 EFP 2 VLAN= S-VLAN=1000, C-VLAN=2000 isn't allowed because the more specific C-vlan is part of the range. The parser will reject this config upon commit. The following options A and B, achieving the same, are allowed : A) EFP 1) VLAN: S-VLAN=1000, C-VLAN_range=1-1999,2001-4095 EFP 2) VLAN= S-VLAN=1000, C-VLAN=2000 B) EFP 1) VLAN: S-VLAN=1000, C-VLAN=any EFP 2) VLAN= S-VLAN=1000, C-VLAN=2000, VLAN TAG rewriting The ASR9000/XR is capable of doing many translations on vlan to a packet. The behavior is always symmetric (though this keyword is to be provided as part of that config command). The symmetry means that if we pop a tag on ingress, we push it back on egress. the following rewrites are possible: RP/0/RSP0/CPU0:A9K(config)#int gig 0/0/0/4.100 l2transport RP/0/RSP0/CPU0:A9K(config-subif)#rewrite ingress tag ? pop Remove one or more tags push Push one or more tags translate Replace tags with other tags RP/0/RSP0/CPU0:A9K(config-subif)#rewrite ingress tag pop ? 1 Remove outer tag only 2 Remove two outermost tags RP/0/RSP0/CPU0:A9K(config-subif)#rewrite ingress tag push ? dot1ad Push a Dot1ad tag (ethertype 88a8) dot1q Push a Dot1Q tag (ethertype 8100 by default, or 9100/9200 is specied on the main interface for the outter most tag) RP/0/RSP0/CPU0:A9K(config-subif)#rewrite ingress tag push dot1q 100 ? second-dot1q Push another Dot1Q tag symmetric All rewrites must be symmetric RP/0/RSP0/CPU0:A9K(config-subif)#rewrite ingress tag translate ? 1-to-1 Replace the outermost tag with another tag 1-to-2 Replace the outermost tag with two tags 2-to-1 Replace the outermost two tags with one tag 2-to-2 Replace the outermost two tags with two other tags If you want to make a cross connect or bridge between two EFP's where one EFP is vlan 100 and the other EFP is vlan 200, you need to make sure you pop the tags so that the vlan 100 is removed from the packet so it, by means of symmetry, will get vlan 200 on the egress of the other EFP. L2VPN configuration examples and comparison to 7600 The following picture highlights how to create L2 services on the ASR9000. Using IRB/BVI To provide L3 services in a bridge-domain, you can add a routed interface to the bridge domain. What is important here is that the BVI is not vlan tagged. So in order for the EFP's to talk to the BVI, we need to pop ALL Tags on ingress!! This means that frames with more then 2 tags cannot be natively using the BVI, unless we do some workarounds such as loopback cables to pop more tags. interface BVI5 ipv4 address 184.108.40.206 255.255.255.0 mac-address 0.4343.3434 ! when creating the bvi verify the show int bvi to see if the mac address is correctly programmed, these macs are coming from the backplane mac table, ! if we ran out because of so many bundle interfaces etc, you may need to provide a mac manually. interface TenGigE0/7/0/3.100 l2transport encapsulation dot1q 100 rewrite ingress tag pop 1 symmetric l2vpn bridge group testing bridge-domain testing interface TenGigE0/7/0/3.100 ! interface GigabitEthernet0/1/0/14 ! routed interface BVI5 ! Related Information XR MTU calculations Dot1q, Dot1ad and configuring "dot1q tunneling ethertype on the physical interface: As per IEEE documentation, Ethertype 8100 is for 802.1q and Ethertype 88A8 is for 802.1ad. IEEE also calls any kind of double tagging as QinQ, including dot1ad. That means that under IEEE's point of view, dot1ad = qinq. Cisco, on the other hand, calls qinq a double dot1q, where both Ethertypes (inner and outer) are 8100. Ethertype 9100 and 9200 were used as an option to differentiate the inner Ethertpe form the outer Ethertype, but they are obsolete now and the standard became dot1ad (with Ethertype 88A8). In order to configure ethertype 9100 and 9200 you have to explicitly issue the command dot1q tunneling ethertype 0x9100 or dot1q tunneling ethertype 0x9200 under the physical interface you are configuring. Summarizing: encapsulation dot1q second dot1q -> inner and outer ethertype are 8100 encapsulation dot1ad second dot1q -> inner 8100, outer 88A8 encapsulation dot1q second dot1q (with ethertype 9100) -> inner 8100, outer 9100 encapsulation dot1q second dot1q (with ethertype 9200) -> inner 8100, outer 9200 Xander Thuijs, CCIE #6775 Sr. Tech Lead ASR9000
... View more
Renne, here is a configuration snippet of how you can make it work. note however that I am providing you merely a configuration to get this going, but it would not be my preferred design. you're probably better of using a catalyst switch that can do vlan tagging and the LES facing port being the trunk. interface FastEthernet2/0 description data network no ip address duplex half no cdp enable bridge-group 10 interface GigabitEthernet6/0 description to LES service no negotiation auto interface GigabitEthernet6/0.100 descr vlan encap for data service encapsulation dot1Q 100 no snmp trap link-status no cdp enable bridge-group 10 bridge 10 protocol ieee bridge 10 route ip Optional, routed interface for this bridge domain: interface BVI10 ip address 220.127.116.11 255.255.255.0 Verification of learning and displaying bridging stats: show bridge 10 verbose Total of 300 station blocks, 296 free Codes: P - permanent, S - self BG Hash Address Action Interface VC Age RX count TX count 10 09/0 0017.0e43.a1a8 forward FastEthernet2/0 - 0 1270 1223 10 62/0 001b.53ff.8cee forward Gi6/0.100 - 3 1113 1111 Flood ports (BG 10) RX count TX count FastEthernet2/0 54 0 GigabitEthernet6/0.100 0 54
... View more
Hi Renne, Yup that detail helps. Let me explain a few concepts before we start talking details. When you have 1 subnet that is spread over multiple locations you can do bridging. When devices are in different subnets you need to be routing. What IRB gives you is say you have 2 interfaces on router, both these interfaces are serving the same subnet. you'll probably want to be bridigng between these two interfaces then. When these two bridged networks need to talk to a different network, stations would need to route. the BVI interface will help here. It provides an L3 endpoint of the subnet that you're briding from 2 networks. the default gateway of your stations would point to that BVI in that case. So to answer your question more directly, yes what you want to do you can. Ideally you want to do some vlan tagging here and mark your voice with a vlan and your data with a vlan so that that you can manage it more nicely. If the provider can't transport your vlans, then even you can use a single circuit and have both subnets over that same cable. However a phone can't talk to a workstation unless there is routing in between. Obviously you don't want to transport local traffic between say a phone and a workstation of siteA over the service, in that regard the BVI will help to be able to route locally. But then, does a PC need to talk to the Phone? (ok there are cases such as Attendant console etc), but regular desk phones generally dont need to talk to the PC directly. For security and voice quality, you probably want to keep your voice and data strictly separated with vlan tagging over your provider link. That would be my advice. thanks Xander, CCIE #6675 Sr. Tech Lead ASR9000
... View more
Renne, your question will probably give you many responses, hence options as to what you can do. Here are a few of my thoughts. When you say that you have a cisco (thanks!) router on each site on the LES, why don't you use the LES to provide connectivity on L3 between these 2 routers and route between the different LANs. If you have the ability to and you can, probably it is a good idea to separate your bcast domains. Few other things to think about: what is the LES service you obtained from the provider, in terms of speed and guarantee. Why is that important? well a LES service depending on how their core is built might start to loadbalance, and what do they then loadbalance on. When you do routing between your les circuit, the MAC pairs remain the same, so if you purchased say a 12G service and they do MAC lb, you'll never get what you pay for. I hate the answer "it depends" but in this case it might apply, you can go either way, L2 or L3. My preference is L3, eventhough I am as much familiar with L2 (so not a biased opinion). I just believe that L3 networks, while may be a bit more complex to manage (routing required) provide for more flexibility in services and security operations ... regards Xander, CCIE #6675 Sr. Tech Lead ASR9000
... View more
hey guys, Looks like my article I just wrote is right on time. It is focussed on the ASR9000, but it gives you a good idea on whatis going on with these protocol drops which seems alarming, but it is not that bad check this reference and let me know if there are any open items after reading that... Unrecognized Protocol Drops regards! xander
... View more
Introduction This section describes the equations on how the MTU is being calculated for interfaces in IOS-XR. The MTU is important since it is used as part of the signaling for Pseudo Wires, hence it is important to understand the way XR calculates MTU. Also OSPF will benefit from the right mtu tuning Core Issue This section describes the symptoms of the problem and the main issue the document resolves. Resolution The L2 MTU of the sub-interface is calculated as follows: If no L2 MTU is configured on the sub-interface, then the L2 MTU is derived from the L2 MTU inherited from the parent node. sub-l2-mtu = parent-l2-mtu + (4 * encaps-tag-count). If the sub-interface has an explicit MTU configured then the L2 MTU of the sub-interface is the minimum of the configured value and the value calculated from the parent-l2-mtu as described above. E.g. sub-l2-mtu = min (cfg-sub-l2-mtu, (parent-l2-mtu + (4 * encaps-tag-count))) The intention behind the L2 MTU definition is to try and preserve an IP payload of 1500 bytes under default configuration and hence to increase the interface L2 MTU to accommodate space for the tags that are known to be present due to the encapsulation. The L2 payload MTU is calculated as follows: The payload MTU is always derived from the L2 MTU based on the following formula: sub-l2-payload-mtu = sub-l2-mtu – (14 + (4 * (encaps-pop-tags-count – encaps-push-tags-count))) The rationale behind the payload MTU calculation is to get the correct maximum payload size of frames that may be carried over an xconnected PW relative to the L2 MTU of the interface. The IM L2 payload MTU is calculated as follows: The IM L2 payload MTU can be derived from the L2 MTU based on the following formula: im-l2-payload-mtu = sub-l2-mtu – (14 + (4 * encaps-tag-count)) The IM MTU is defined to be correct L2 payload MTU for L3 sub-interfaces, and convenient to calculate for L2 sub-interfaces subject to the restrictions placed on MTU handling by IM. Notes on the calculations above: All MTUs are calculated in bytes. All MTUs exclude the 4 byte Ethernet FCS bytes. The size of the Ethernet header is 14 bytes. The size of a 802.1Q tag is 4 bytes. sub-l2-mtu: The sub-interface L2 MTU. parent-l2-mtu: The parent-interface L2 MTU. cfg-sub-l2-mtu: The L2 MTU configured on a sub-interface (or UINT16_MAX) if no MTU has been configured. sub-l2-payload-mtu: The sub-interface L2 payload MTU. im-l2-payload-mtu: The sub-interface IM L2 payload MTU. encaps-tag-count: The number of tags matched in an EFP encapsulation configuration, or VLAN Id configuration. Ranges count as a tag match, but an inner “any” tag match does not. rewrite-pop-tags-count: The number of tags popped in the EFP rewrite configuration, or the number of tags matched in VLAN Id configuration (since the VLAN Id configuration has an implicit rewrite operation of popping all tags matched). rewrite-push-tags-count: The number of tags pushed in the EFP rewrite configuration. Notes related to specific EFP encapsulation commands: The following encapsulations count as matching 0 tags, i.e. are defined as having an encaps-tag-count of 0: encapsulation untagged encapsulation default The following encapsulation modifiers do not affect the encaps-tag-count: native payload-ethertype exact cos ingress source-mac or ingress destination-mac encapsulation dot1q|dot1ad priority tagged counts as matching a single tag, i.e. encaps-tag-count of 1. The any keyword used as the innermost tag match does not increase the encaps-tag-count. (This is to ensure consistency with the old style XR VLAN Id semantics). E.g. Encapsulation dot1q any has an encaps-tag-count of 0. Encapsulation dot1ad 10 dot1q any has an encaps-tag-count of 1 Encapsulation dot1ad any dot1q 7 has an encaps-tag-count of 2. Ranges of vlan-ids count as a encaps tag match, e.g. Encapsulation dot1q 10-100 has an encaps-tag-count of 1. The encapsulation MTU overhead of an EFP that is a disjunctive match is treated as having the MTU of its highest element. E.g. Encapsulation dot1q 10-100, untagged has an encaps-tag-count of 1. Examples: Interface GigabitEthernet0/0/0/0 mtu 2014 interface GigabitEthernet0/0/0/0.1 dot1q vlan 10 20 interface GigabitEthernet0/0/0/0.2 l2transport service instance ethernet encapsulation dot1q 11 rewrite ingress tag push dot1ad 10 dot1q 20 symmetric ! ! For GigabitEthernet0/0/0/0: l2-mtu = 2014 im-l2-payload-mtu = 2000 l2-payload-mtu = 2000 For GigabitEthernet0/0/0/0.1: encaps-tag-count = 2, rewrite-push-tags-count = 0, rewrite-pop-tags-count = 2 sub-l2-mtu = 2014 + 4 * 2 = 2022 im-l2-payload-mtu = 2000 sub-l2-payload-mtu = 2022 – (14 + (4 * (2 – 0))) = 2000 For GigabitEthernet0/0/0/0.2 encaps-tag-count = 1, rewrite-push-tags-count = 2, rewrite-pop-tags-count = 0 sub-l2-mtu = 2014 + 4 * 1 = 2018 im-l2-payload-mtu = 2000 sub-l2-payload-mtu = 2018 – (14 + (4 * (0 - 2))) = 2012 To further explain the example for GigabitEthernet0/0/0/0.2. If a 2026 byte frame was received from the PW on the disposition path then once the 2 tags have been popped off due to the rewrite policy the packets would be of size 2018, hence matching the MTU of the Egress interface.  Note the MTU negotiated is the L2 payload MTU and hence the actual maximum size of the frame that may be carried is 14 bytes larger for a type 5 PW and 18 bytes larger for a type 4 PW, although in the case of the type 4 PW the PW 802.1Q tag is effectively discarded and ignored for the purpose of the L2VPN MTU calculations (since it isn’t part of the payload). NOTE Starting in XR release 5.1.1 the ethernet interface driver configuration for MTU and MRU is determined by the XR config. Prior to these changes MTU/MRU calculation was simply based on the configured MTU + 12 bytes for the addition of 2 Ethernet Tags and the CRC field as mentioned above. However from 511 onwards, the config determines the actual MTU/MRU. If no VLAN tags are configured - then driver MTU/MRU = configured MTU on interface + 4 CRC bytes, ie 1514+4=1518 If one VLAN is configured - MTU/MRU = configured MTU + 8 bytes (2 tags + CRC) If two VLAN tags are configured - MTU/RMU = configured MTU + 12 bytes (3 tags + CRC) Related Information n/a Xander Thuijs, CCIE #6775 Principal Engineer ASR9000
... View more
with the current hardware that is 512k MAC addresses. this is system wide. All these macs can be on a single linecard or even a port and or spread over multiple linecards/ports. next generation hardware coming out the end of this year will go much further. Xander sr Tech Lead, ASR9000
... View more
Introduction In this article we'll discuss how to troubleshoot packet loss in the asr9000 and specifically understanding the NP drop counters, what they mean and what you can do to mitigate them. This document will be an ongoing effort to improve troubleshooting for the asr9000, so if after reading this article things are still not clear, please do comment on the article and I'll have details added in case my current descriptions are not explanatory enough. Core Issue Before we are going to dive into the packet troubleshooting on the asr9000 it is important to undertand a bit of the architecture of the system. Packets are received by the mac/phy, forwarded to the NPU who is processing the features, the NPU hands it over to the bridge who in turn delivers the packets to the FIA (Fabric Interfacing Asic) upon which it is sent over the fabric (residing physically on the RSP) and handed over to the egress Linecard who follows the FIA->bridge->NP-MAC path then. The NP processes packets for both ingress and egress direction. The Bridge is just a memory interface converter and is non blocking, unless you have QOS back pressure, there is generally no reason for the Bridge to drop packets. This article will not focus on the bridge and FIA drop counters as they are generally not common. Both NP, Bridge, FIA and Fabric are all priority aware and have separate queues for High and Low priority traffic as well as Unicast and Multicast. So when you look at the bridge you see UC/MC (unicast/mcast) as well as LP/HP queues. A different article will focus on that. Resolution Here a graphical overview of how the individual asics are interconnecting with the different commands that point to the actual asic of where we are viewing statistics from. In this picture Y is the slot where the linecard is inserted on and A identifies the individual interface. The first place to verify drops is the interface. The regular show interface command is well known. Drops reported on the interface are an aggregate of phyiscal layer drops as well as (certain!!) drops that were experienced in the NP. When printing the show controller np counters command, you may not know which interface identifier connects to which, therefore use the "show controller np ports all location 0/Y/CPU0" to find out which interface maps to which NP. RP/0/RSP1/CPU0:A9K-BOTTOM#sh controllers np ports all loc 0/7/CPU0 Fri Mar 4 12:09:41.132 EST Node: 0/7/CPU0: ---------------------------------------------------------------- NP Bridge Fia Ports -- ------ --- --------------------------------------------------- 0 0 0 TenGigE0/7/0/3 1 0 0 TenGigE0/7/0/2 2 1 0 TenGigE0/7/0/1 3 1 0 TenGigE0/7/0/0 In case you have an oversubscribed linecard, 2 TenGig interfaces can be connected to the same NP. For 80G linecards such as the A9K-8T, you'll see 8 NP's listed. For the 40 port 1GE, you'll see 10 1-Gig interfaces connecting to an NP, note that this is not oversubscription. With that info you can now view the controller statistics and some information as to what the controller is doing. Note that not all counters in this command constitute a problem! They are regular traffic counters also. There is a simple rate mechanism that keeps the shadow counts from the previous command invokation. Upon the next time the command is run delta's are taken and divided over the elapsed time to provide some rate counts. It is important for more accurate rate counts instantly that you run this command twice the first time with a 2/3 second interval. The output may look similar to this: RP/0/RSP1/CPU0:A9K-BOTTOM#show controller np count np1 loc 0/7/CPU0 Node: 0/7/CPU0: ---------------------------------------------------------------- Show global stats counters for NP1, revision v3 Read 30 non-zero NP counters: Offset Counter FrameValue Rate (pps) ------------------------------------------------------------------------------- 22 PARSE_ENET_RECEIVE_CNT 51047 1 23 PARSE_FABRIC_RECEIVE_CNT 35826 0 30 MODIFY_ENET_TRANSMIT_CNT 36677 0 31 PARSE_INGRESS_DROP_CNT 1 0 34 RESOLVE_EGRESS_DROP_CNT 628 0 40 PARSE_INGRESS_PUNT_CNT 3015 0 41 PARSE_EGRESS_PUNT_CNT 222 0 .... What all these mnemonics mean is listed out in this table below: 1. Common Global Counters Counter Name Counter Description IPV4_SANITY_ADDR_CHK_FAILURES Ingress IPV4 address martian check failure 127.0.0.0/8 18.104.22.168/8, not 22.214.171.124/4 (126.96.36.199/8 and 188.8.131.52/8 don't increment error count) When source address is below, ASR9K forward the packet without any error. 0.0.0.0/8 10.0.0.0/8 184.108.40.206/16 169.254.0.0/16 172.16.0.0/12 220.127.116.11/16 192.0.0.0/24 18.104.22.168/24 240.0.0.0/4 DROP_IPV4_LENGTH_ERROR Ingress IPV4 header length invalid or frame length incompatible with fragment length DROP_IPV4_CHECKSUM_ERROR Ingress IPV4 checksum invalid IPV6_SANITY_ADDR_CHK_FAILURES Ingress IPV6 frame source or destination address invalid DROP_IPV6_LENGTH_ERROR Ingress IPV6 frame length invalid MPLS_TTL_ERROR Ingress MPLS frame outer label TTL is 0 MPLS_EXCEEDS_MTU MPLS TTL exceeded – punt frame to host DROP_IPV4_NOT_ENABLED Ingress IPV4 frame and IPV4 not enabled in L3 ingress intf struct DROP_IPV4_NEXT_HOP_DOWN Drop bit set in rx adjacency (ingress) or tx adjacency (egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – drop bit in non-recursive adjacency or adjacency result IPV4_PLU_DROP_PKT Drop bit set in leaf or non-recursive adjacency lookup miss (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface. IPV4_RP_DEST_DROP IPV4 RP drop bit set in non-recursive adjacency or adjacency result and ICMP punt disabled in intf struct (ingress/egress) for the interface IPV4_NULL_RTE_DROP_PKT No IPV4 route (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – null route set in leaf or recursive adjacency DROP_IPV6_NOT_ENABLED Ingress IPV6 frame and IPV6 not enabled in L3 ingress intf struct DROP_IPV6_NEXT_HOP_DOWN Drop bit set in rx adjacency (ingress) or tx adjacency (egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – drop bit in non-recursive adjacency or adjacency result DROP_IPV6_PLU Drop bit set in leaf or non-recursive adjacency lookup miss (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface. IPV6_RP_DEST_DROP RP drop bit set in non-recursive adjacency or adjacency result and ICMP punt disabled in intf struct (ingress/egress) for the interface IPV6_NULL_RTE_DROP_PKT No IPV6 route (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – null route set in leaf or recursive adjacency MPLS_NOT_ENABLED_DROP Ingress MPLS frame and MPLs not enabled in L3 ingress intf struct MPLS_PLU_NO_MATCH Ingress MPLS frame and outer label lookup miss and not a VCCV frame associated with a pseudo-wire MPLS_PLU_DROP_PKT Ingress MPLS frame and label lookup results return a NULL route, or have no forwarding control bits set, OR one of the following: 1) Next hop down – Drop bit set in rx adjacency (ingress) or tx adjacency (egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – drop bit in non-recursive adjacency or adjacency result 2) Leaf drop - Drop bit set in leaf or non-recursive adjacency lookup miss (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface. 3) RP drop - RP drop bit set in non-recursive adjacency or adjacency result and ICMP punt disabled in intf struct (ingress/egress) for the interface 4) No route - No route (ingress/egress) and ICMP punt disabled in intf struct (ingress/egress) for the interface – null route set in leaf or recursive adjacency 5) Punt bit set in leaf results or recursive adjacency results – egress mpls punt not supported MPLS_UNSUPP_PLU_LABEL_TYPE Ingress MPLS frame outer label is 15 or less and not a router alert or null tag. DEAGG_PKT_NON_IPV4 Ingress MPLS frame outer label lookup control bits are L3VPN deagg. And the payload is not IPV4. 2. NP debug and drop counters Bulk Forwarding Counts Inject/Punt type counts Main Drop counts QOS/ACL drop counts Counter Name Counter Description PARSE_ENET_RECEIVE_CNT Total frames received from Ethernet on an ingress NP PARSE_FABRIC_RECEIVE_CNT Total frames received by an egress NP from the fabric – Note that this currently includes loopback frames MODIFY_FABRIC_TRANSMIT_CNT Total frames sent to fabric by an ingress NP MODIFY_ENET_TRANSMIT_CNT Total frames sent to Ethernet by an egress NP PARSE_LOOPBACK_RECEIVE_CNT Total loopback frames received by an NP – Note that these are egress loopback frames only and do not include ingress loopback frames such as Interflex PARSE_INTERFLEX_RECEIVE_CNT Total frames received by an NP from the interflex loopback port – these are ingress loopback frames MODIFY_FRAMES_PADDED_CNT Total number of frames sent to fabric that were undersized and required padding to 60 bytes. MODIFY_RX_SPAN_CNT Ingress frames sent to fabric on a SPAN link – these frames are replicated MODIFY_TX_SPAN_CNT Egress frames sent to Ethernet on a SPAN link – these frames are replicated MODIFY_LC_PUNT_CNT Total frames punted to LC host (all punts are handled by the modify engine) MODIFY_RP_PUNT_CNT Total frames punted to RP host (all punts are handled by the modify engine) MODIFY_PUNT_DROP_CNT Total punt packets dropped by ingress modify engine due to punt policing, OR due to punt reason lookup fail. MODIFY_LC_PUNT_EXCD_CNT Punt frames to LC dropped due to punt policing MODIFY_RP_PUNT_EXCD_CNT Punt frames to RP dropped due to punt policing PARSE_INGRESS_DROP_CNT Total packets dropped by ingress NP in the parse engine PARSE_EGRESS_DROP_CNT Total packets dropped by egress NP in the parse engine RESOLVE_INGRESS_DROP_CNT Total packets dropped by ingress NP in the resolve engine RESOLVE_EGRESS_DROP_CNT Total packets dropped by egress NP in the resolve engine MODIFY_INGRESS_DROP_CNT Total packets dropped by ingress NP in the modify engine MODIFY_EGRESS_DROP_CNT Total packets dropped by ingress NP in the modify engine PARSE_INGRESS_PUNT_CNT Total packets flagged for punting by the ingress NP in the parse engine PARSE_EGRESS_PUNT_CNT Total packets flagged for punting by the egress NP in the parse engine RESOLVE_INGRESS_L3_PUNT_CNT Total Layer3 packets flagged for punting by an ingress NP in the resolve engine due to lookup results or other cases such as ICMP unreachable RESOLVE_INGRESS_IFIB_PUNT_CNT Total Layer3 packets flagged for punting due to an ingress IFIB lookup match RESOLVE_INGRESS_L2_PUNT_CNT Total Layer2 packets flagged for punting by the egress NP in the resolve engine due to cases such as IGMP/DHCP snooping or punting of ingress L2 protocol such as LAG or MSTP or CFM RESOLVE_EGRESS_L3_PUNT_CNT Total Layer3 packets flagged for punting by an ingress NP in the resolve engine due to lookup results or other cases such as ICMP unreachable RESOLVE_EGRESS_L2_PUNT_CNT Total Layer2 packets flagged for punting by the egress NP in the resolve engine due to cases such as IGMP/DHCP snooping or punting of ingress L2 protocol such as MSTP or CFM PARSE_LC_INJECT_TO_FAB_CNT Frames injected from LC host that are intended to go directly to fabric to an egress LC without modification or protocol handling (inject header is stripped by microcode) PARSE_LC_INJECT_TO_PORT_CNT Frames injected from LC host that are intened to go directly to an Ethernet port without modification or protocol handling (inject header is stripped by microcode) PARSE_FAB_INJECT_IPV4_CNT Total number of IPV4 inject frames received by the egress NP from the fabric (sent from RP) PARSE_FAB_INJECT_PRTE_CNT Total number of layer 3 preroute inject frames received by the egress NP from the fabric (sent from RP) PARSE_FAB_INJECT_UNKN_CNT Total number of unknown inject frames received by the egress NP from the fabric (sent from RP). These are punted to the LC host. PARSE_FAB_INJECT_SNOOP_CNT Total number of IGMP/DHCP snoop inject frames received by the egress NP from the fabric (sent from RP). For DHCP, these are always re-injects from the RP. For IGMP, these can be re-injects or originated injects from the RP. PARSE_FAB_INJECT_MPLS_CNT Total number of mpls inject frames received by the egress NP from the fabric (sent from RP). PARSE_FAB_INJECT_IPV6_CNT Total number of IPV6 inject frames received by the egress NP from the fabric (sent from RP) INJECT_EGR_PARSE_PRRT_NEXT_HOP Inject preroute frames received from the fabric that are next hop type. INJECT_EGR_PARSE_PRRT_PIT Inject preroute frames received from the fabric that are PIT type . L3_EGR_RSV_PRRT_PUNT_TO_LC_CPU Inject preroute frames that have a PIT table lookup miss. These are punted to the LC host. L2_INJECT_ING_PARSE_INWARD Total number of ingress CFM injects received from the LC host that are intended to be sent to the fabric. These can either be bridged or pre-routed to a specific NP. CFM_L2_ING_PARSE_BRIDGED Total number of ingress CFM injects received from the LC host that are intended to be bridged and sent to the fabric. The bridge lookups are done for these inject frames resulting in unicast or flood traffic. L2_INJECT_ING_PARSE_PREROUTE Total number of ingress CFM injects received from the LC host that are intended to be pre-routed to a specific output port on an egress NP. bridged and sent to the fabric. The bridge lookups are not done for these inject frames. L2_INJECT_ING_PARSE_EGRESS Total number of ingress CFM injects received from the LC host that are intended to be sent to a specific output port on an egress NP. INJ_UNKNOWN_TYPE_DROP Unknown ingress inject received from the LC host – these are dropped. STATS_STATIC_PARSE_INTR_AGE_CNT Total Number of Parse Interrupt frames associated with learned MAC address aging. STATS_STATIC_PARSE_INTR_STATS_CNT Total Number of Parse Interrupt frames associated with statistics gathering. PARSE_VIDMON_FLOW_AGING_SCAN_CNT Total Number of Parse Interrupt frames associated with video monitoring flow scanning. UNSUPPORTED_INTERRUPT_FRAME Unsupported interrupt frame type received on NP. Supported types are for aging and statistics only. RESOLVE_BAD_EGR_PRS_RSV_MSG_DROP_CNT Invalid parse engine to resolve engine message received for an ingress frame – this is likely a software implementation error. RESOLVE_BAD_ING_PRS_RSV_MSG_DROP_CNT Invalid parse engine to resolve engine message received for an egress frame – this is likely a software implementation error. EGR_UNKNOWN_PAK_TYPE A frame received from the fabric by an egress NP has an unknown NPH header type – these are dropped. This is likely a software implementation error. EGR_LB_UNKNOWN_PAK_TYPE A frame received from the egress loopback queue by an egress NP has an unknown NPH header type – these are dropped. This is likely a software implementation error. XAUI_TRAINING_PKT_DISCARD These are diag packets sent to the egress NP at startup from the fabric FPGA. These are ignored and dropped by software. PRS_DEBUGTRACE_EVENT This counter increments whenever a preconfigured debug trace event matches on an ingress or egress frame. Currently the debug trace is only configurable for all drop events. IN_INTF STRUCT_TCAM_MISS Ingress frame with a pre-parse TCAM lookup miss – these are dropped. This is likely due to a TCAM setup problem. IN_INTF STRUCT_NO_ENTRY Ingress frame with a pre-parse TCAM index that is invalid – these are dropped. This is likely due to a TCAM setup problem IN_INTF STRUCT_DOWN Ingress frame with a intf struct entry with down status – these are dropped, with the exeception of ingress LACP frames on a subinterface. OUT_INTF STRUCT_INTF STRUCT_DOWN Egress Layer3 frame with an egress INTF STRUCT that has line status down, and ICMP punt is disabled in the intf struct. IN_L2_INTF STRUCT_DROP Ingress Layer2 frame on a subinterface with the L2 intf struct drop bit set. ING_VLAN_FILTER_DISCARD A vlan tagged packet has a intf struct index that does points to a default layer3 intf struct – these are dropped. Only untagged pkts can be setup on a default layer3 intf struct. IN_FAST_DISC A ingress frame was dropped due to the early discard feature. This can occur at high ingress traffic rate based on the configuration. UNKNOWN_L2_ON_L3_DISCARD Unknown Layer2 frame (non-IPV4/V6/MPLS) on a Layer3 interface. These are dropped. Known layer2 protocols are punted on a layer3 interface such as ARP/LACP/CDP/ISIS/etc. L3_NOT_MYMAC A layer3 unicast IPV4/IPV6 frame does not have the destination mac address of the router, or any mac address defined in the VRRP table. These are dropped. IPV4_UNICAST_RPF_DROP_PKT With RPF enabled, an ingress IPV4 frame has a source IP address that is not permitted, or not permitted on the ingress interface – these are dropped. RESOLVE_L3_ING_PUNT_IFIB_DROP Ingress frame has TTL of 0, or discard BFD frame if BFD not enabled MPLS_VPN_SEC_CHECK_FAIL An ingress MPLS does not have the proper VPN security code, and IPV4 is not using the global routing table. MPLS_EGR_PLU_NO_MATCH MPLS frame outer label lookup on egress NP does not match – these are dropped. MPLS_EGR_PLU_DROP_PKT MPLS frame outer label lookup on egress NP has a result with the drop bit set. RESOLVE_EGR_LAG_NOT_LOCAL_DROP_CNT An egress frame hash to a bundle interface has a bundle member that is not local to the NP – these are dropped. This can be due to a bad hash, or a bad bundle member index. It can also occur normally for floods or multicast – non-local members are dropped for each copy that is a bundle output. RESOLVE_L3_ING_ERROR_DROP_CNT L3 ingress protocol handling in the resolve engine falls through to an invalid case – these are dropped. This would occur due to a software implementation error. RESOLVE_L3_EGR_ERROR_DROP_CNT L3 egress protocol handling in the resolve engine falls through to an invalid case – these are dropped. This would occur due to a software implementation error. ING_QOS_INVALID_FORMAT An ingress packet has a QOS format value in the ingress intf struct that is not valid. Only checked if QOS is enabled in the ingress intf struct. Valid formats are 20/3/2/1/0. RESOLVE_L3_ING_ACL_DENY_DROP_CNT Total number of ingress Layer3 frames dropped due to an ACL deny policy. RESOLVE_L3_EGR_ACL_DENY_DROP_CNT Total number of egress Layer3 frames dropped due to an ACL deny policy. RESOLVE_L2_ING_ACL_DENY_DROP_CNT Total number of ingress Layer2 frames dropped due to an ACL deny policy. RESOLVE_L2_EGR_ACL_DENY_DROP_CNT Total number of egress Layer2 frames dropped due to an ACL deny policy. RESOLVE_L2L3_QOS_C_DROP_CNT Total number of ingress/egress frames dropped to a child policing policy. RESOLVE_L2L3_QOS_P_DROP_CNT Total number of ingress/egress frames dropped to a parent policing policy. Layer3 Multicast related counts These include: IP mcast forwarding counts Video Monitoring counts Multicast Fast Reroute counts Counter Name Counter Description IPV4_MCAST_NOT_ENABLED Ingress IPV4 frames dropped due to ipv4 multicast not enabled. The ipv4 multicast enable check may be ignored for non-routable IGMP frames. Also, the ipv4 multicast enable check does not apply to well known multicast frames. IPV6_MCAST_UNSUPPORTED_DROP Ingress IPV6 multicast frame and IPV6 multicast not supported in the current release RESOLVE_IPM4_ING_RTE_DROP_CNT Ingress IPV4 multicast frame route lookup fails with no match. RESOLVE_IPM4_EGR_RTE_DROP_CNT Egress IPV4 multicast frame route lookup fails with no match. RESOLVE_IPM4_NO_OLIST_DROP_CNT Egress IPV4 multicast frame has zero olist members – no copies are sent to output ports. RESOLVE_IPM4_OLIST_DROP_CNT Egress IPV4 multicast frame has olist drop bit set (no copies are sent to output ports), OR egress IPV4 multicast frame olist lookup fails. Olist lookup failure could be due to a bad copy number, or a missing entry for that copy. RESOLVE_IPM4_FILTER_DROP_CNT Egress IPV4 multicast frame copy is reflection filtered – copies to the original source port are dropped. RESOLVE_IPM4_TTL_DROP_CNT Egress IPV4 multicast frame copy fails TTL check. The frame TTL is beyond the programmed threshold. MODIFY_RPF_FAIL_DROP_CNT Ingress IPV4 multicast frames dropped due to reverse-path-forwarding check failed (to prevent looping or duplicated packets); these multicast frames came in from wrong interface. RESOLVE_VIDMON_FLOWS_DELETED_BY_AGING Inactive Video Monitoring flows deleted by hardware aging mechanism. RESOLVE_VIDMON_CLASS_FLOW_LIMIT_REACHED Indicates that the Video Monitoring Maximum flows per class limit reached. RESOLVE_VIDMON_FLOW_START_INTRVL_RESET Video Monitoring flow start interval reset to current Real Time Clock Tick (instead of updating with actual next packet arrival time) because the flow has been inactive for more than 3 interval timeouts. VIDMON_PUNT_FAILED_NO_TXBUF_CNT Punt frames dropped by Video Monitoring due to no buffer PARSE_MOFRR_WATCHDOG_INTR_RCVD L3 Multicast Fast Reroute Watchdog interrupts received by nP. The count is summation of both Activity and Loss interrupt counts. PARSE_MOFRR_SWITCH_MSG_RCVD_FROM_FAB Number of L3 Multicast Fast Reroute Switch (Flood) messages received by nP from Fabric. RESOLVE_MOFRR_HRT_LKUP_FAIL_DROP Number of watchdog interrupts are dropped because of an error in MoFRR Hash Table lookup. This counter indicates that the watchdog interrupt (Activity or Loss) is received but micro code failed to process further because of MoFRR Hash look up failed. This counter indicates an error. RESOLVE_MOFRR_WDT_INVALID_RESULTS_DROP_CNT Number of watchdog interrupts are dropped because of an error in Watch Dog counter hash table lookup. This counter indicates that the watchdog interrupt (Activity or Loss) is received but micro code failed to process further because of lookup failed. This counter indicates an error. RESOLVE_MOFRR_SWITCH_MSG_HRT_LKUP_FAIL_DROP Number of MoFRR switch messages are dropped because of an error with MoFRR Hash Table lookup failure. This counter indicates that the switch message is received from fabric but micro code failed to process further because of MoFRR Hash look up failed. This counter indicates an error and could be traffic disruptive. RESOLVE_MOFRR_SWITCH_MSG_SEQNUM_MISMATCH_DROP Number of MoFRR switch messages are dropped because of sequence number mismatch. This counter indicates that the switch message is received from fabric but micro code failed to process further the sequence number received from ingress LC does not match with sequence number for same route on egress LC. This counter indicates a programming error, race condition or hash result corruption in micro code etc. This is not a usual thing and could be traffic disruptive. RESOLVE_MOFRR_HASH_UPDATE_CNT Number of times MoFRR active flag is updated in MoFRR hash table. This counter indicates the switch message is processed successfully, i.e. RDT/Hash/Direct table lookups are successful and MoFRR Active flag is updated in MoFRR hash table. RESOLVE_MOFRR_SWITCH_MSG_INGNORED Total numbers of MoFRR switch (flood) messages are filtered without being punted to LC CPU. This counter indicates switch message is processed successfully (i.e. lookups successful and MoFRR flag is updated in hardware hash table), but the message is not punted to LC CPU either because the RPF Id is not primary or RPF Id is not local to the nP. Note that this counter is informative and does not necessarily mean an error. RESOLVE_MOFRR_SWITCH_MSG_TO_FAB Number of MoFRR switch (flood) messages generated by nP as a result of Loss Activity interrupts. This counter indicates that the Loss Activity interrupt is processed successfully by ingress LC and MoFRR switch message is flooded to all LCs towards fabric. RESOLVE_MOFRR_EGR_RPF_FAIL_DROP IP Multicast MoFRR enabled packets dropped at egress LC because of RPF check failure. Layer 2 protocol Debug Counters These include: · VPLS counts · VPWS counts · DHCP/IGMP snooping counts · CFM protocol counts · VPLS Learning/Aging related counts Counter Name Counter Description IN_L2_BLOCKED Ingress L2 frames dropped and not forwarded due to ingress interface in spanning tree blocked state. OUT_L2_BLOCKED Egress L2 frames dropped and not forwarded due to egress interface in spanning tree blocked state. IN_DOT1Q_VTP_FILTERED Ingress VTP frames dropped due to dot1q filtering enabled. IN_DOT1Q_DROP Ingress 802.1q frames dropped due to dot1q filtering enabled. IN_DOT1AD_DROP Ingress 802.1ad frames dropped due to dot1ad filtering enabled. ING_SLOW_PROTO_UNKNOWN_SUBTYPE Ingress slow protocol frame on an L3 interface that is not recognized (not LACP/EFM/SYNCE). IN_ROUTER_GUARD_DROP Ingress Layer2 IGMP or PIM dropped due to Router Guard with IGMP snooping enabled. IN_DHCP_UNTRUSTED_DROP Ingress DHCP frame dropped due to the untrusted DHCP bit set in the ingress L2 intf struct. RESOLVE_L2_DHCP_SNOOP_UNTRUSTED_DROP_CNT Egress DHCP frame dropped due to the untrusted DHCP bit set in the egress L2 intf struct. ING_DHCP_PW_DROP Ingress DHCP frame from AC that is destined for a PW and is eligible for snooping due to the DHCP snoop enabled bit set in the L2 ingress INTF STRUCT. This is dropped on ingress due to an L2 unicast to PW. DHCP snooping not supported for AC to PW frames. EGR_DHCP_PW_DROP Ingress DHCP frame from AC that is destined for a PW and is eligible for snooping due to the DHCP snoop enabled bit set in the L2 ingress INTF STRUCT. This is dropped on egress due to an L2 flood with a PW member. DHCP snooping not supported for AC to PW frames. L2VPN_DEAGG_NON_VPWS_VPLS_DROP Invalid setting for L2VPN deaggregation set in Label UFIB lookup result for ingress MPLS frame. This is most likely due to a programming error in the Label UFIB entry. XID_ZERO_DISCARD Ingress VPWS XC to AC frame has a zero destination XID stored in the L2 ingress INTF STRUCT – zero XID index is invalid. XC2AC_NOMATCH_DROP Egress VPWS XC to AC frame XID lookup fails. This is caused by an invalid XID index or missing XID entry. L2UNIC_NOMATCH_DROP Egress VPLS AC to AC frame XID lookup fails. This is caused by an invalid XID index or missing XID entry. This can also occur on an IGMP snoop originated inject to an AC/PW/Bundle with invalid XID in the inject feature header. L2FLOOD_NOMATCH_DROP Egress VPLS flood copy lookup in the flood member table fails. This could be caused by an invalid copy number or missing entry in the flood member table for that copy. L2FLOOD_SRCDST_DROP Egress VPLS flood copy is reflection filtered – copies to the original source port are dropped. L2FLOOD_INACTIVE_MEMBER_DROP Egress VPLS flood copy lookup in the flood member table returns an entry that’s not active (active bit in result is cleared). L2FLOOD_EGR_XID_NO_MATCH Egress VPLS flood copy XID lookup fails – XID stored in the OLIST result is invalid or entry is missing. This can occur if the copy is intended to go over a PW and the PW is down. EGR_AC_PW_MISMATCH Egress VPWS XC to AC XID lookup returns a PW type. For VPWS, PW entry is always on ingress. Note that for VPLS AC to AC, a PW type XID is allows in order to support IGMP injects to a PW. RESOLVE_VPWS_LAG_BIT_DROP_CNT An ingress VPWS XC to Bundle frame with a LAG table lookup fail. This is due to programming error where the LAG table pointer is invalid or no LAG table entry. RESOLVE_L2_SXID_MISS_DROP_CNT An ingress VPLS frame with a source XID lookup fail. This can be due to an invalid source XID programmed in the L2 ingress intf struct, or a missing source XID entry. RESOLVE_VPLS_SPLIT_HORIZON_DROP_CNT An ingress or egress VPLS that is dropped due to a split horizon group mismatch. Note that the split horizon group check is skipped for zero split horizon group indexes. RESOLVE_VPLS_REFLECTION_FILTER_DROP_CNT An ingress unicast VPLS frame that has a source XID that matches its destination XID. These are dropped since it cannot be sent back to the source port. RESOLVE_VPLS_MAC_FILTER_DROP_CNT An ingress VPLS frame has a source or destination MAC with the filter bit set in the L2FIB result – these are dropped. RESOLVE_VPLS_ING_BD_STR_MISS_DROP_CNT An ingress VPLS frame that is dropped due to a bridge domain lookup fail. RESOLVE_VPLS_EGR_BD_STR_MISS_DROP_CNT An egress VPLS flood frame or mac notify frame that is dropped due to a BD structure lookup miss. RESOLVE_VPLS_FLOOD_BLOCK_DROP_CNT An Ingress VPLS frame that is dropped due to its associated flood block bit being set: Unknown Unicast or Multicast. RESOLVE_VPLS_STORM_DROP_CNT An ingress VPLS frame that is dropped due to rate exceed for its associated flood storm control – Unknown Unicast or Multicast or Broadcast. RESOLVE_VPLS_EGR_FLD_NO_MEMBERS_DROP_CNT Egress L2 flood frame has zero members – no copies are sent to output ports. RESOLVE_L2_EGR_INTF STRUCT_MISS_DROP_CNT An egress VPLS unicast or flood copy frame to an AC or Bundle that is dropped to an egress L2 INTF STRUCT lookup miss. This is due to an invalid INTF STRUCT index in the XID or a missing egress L2 INTF STRUCT entry. This can also occur with L2 multicast forwarded frames or L2 multicast inject frames. RESOLVE_L2_EGR_PW_INTF STRUCT_MISS_DROP_CNT An egress VPLS unicast or flood copy frame to a PW that is dropped to an egress L2 INTF STRUCT lookup miss. This is due to an invalid INTF STRUCT index in the XID or a missing egress L2 INTF STRUCT entry. This can also occur if the PW is down. . This can also occur with L2 multicast forwarded frames or L2 multicast inject frames to a PW. RESOLVE_VPWS_L2VPN_LDI_MISS_DROP_CNT An ingress VPWS or VPLS unicast frame to a PW with an L2VPN LDI table lookup fail. This can be due to an invalid L2VPN LDI index in the XID, or a missing entry in the L2VPN LDI table. RESOLVE_VPWS_RP_DROP_CNT An ingress VPWS or VPLS unicast frame to a PW with the RP drop bit or RP dest. bit set in the NON-RECURSIVE ADJACENCY or RX-ADJ lookup results. Punts are not supported for an L2 PW adjacency. RESOLVE_VPWS_NULL_ROUTE_DROP_CNT An ingress VPWS or VPLS unicast frame to a PW with the Null Route bit or IFIB punt bit set in the Leaf lookup or RECURSIVE ADJACENCY lookup results. Punts are not supported for an L2 PW adjacency. RESOLVE_VPWS_LEAF_DROP_CNT An ingress VPWS or VPLS unicast frame to a PW with the Drop bit or Punt bit set in the Leaf lookup or RECURSIVE ADJACENCY lookup results. Punts are not supported for an L2 PW adjacency. RESOLVE_VPLS_EGR_NULL_RTE_DROP_CNT An egress VPLS flood frame copy to a PW with the Null Route bit set in the Leaf lookup or RECURSIVE ADJACENCY lookup results. Also includes IGMP snoop injects to a PW. Due to a software problem, this may also get incremented due to a BD structure lookup miss on egress flood. RESOLVE_VPLS_EGR_LEAF_IFIB_DROP_CNT An egress VPLS flood frame copy to a PW with the IFIB punt bit set in the Leaf lookup or RECURSIVE ADJACENCY lookup results. Punts are not supported for an L2 PW adjacency. Also includes IGMP snoop injects to a PW. RESOLVE_VPLS_EGR_LEAF_PUNT_DROP_CNT An egress VPLS flood frame copy to a PW with the Drop bit or Punt bit or Next Hop Down bit set in the Leaf lookup or RECURSIVE ADJACENCY or NON-RECURSIVE ADJACENCY lookup results. Punts are not supported for an L2 PW adjacency. Also includes IGMP snoop injects to a PW. RESOLVE_VPLS_EGR_TE_LABEL_DROP_CNT An egress VPLS flood frame copy to a PW with the Drop bit or Punt bit or Incomplete Adjacency bit set in the Tx adjacency or Backup Adjacency or NH Adjacency lookup results. Punts are not supported for an L2 PW adjacency. Also includes IGMP snoop injects to a PW. RESOLVE_VPLS_EGR_PW_FLOOD_MTU_DROP_CNT An egress VPLS flood frame copy to a PW that fails L3 MTU check. Punts are not supported for an L2 PW adjacency with MTU fail. Also includes IGMP snoop injects to a PW. RESOLVE_VPLS_EGR_PW_FLOOD_INTF STRUCT_DOWN_DROP_CNT An egress VPLS flood frame copy to a PW with egress INTF STRUCT in down state. Punts are not supported for an L2 PW adjacency with line status down. Also includes IGMP snoop injects to a PW. RESOLVE_VPLS_EGR_PW_FLOOD_NO_NRLDI_DROP_CNT An egress VPLS flood frame copy to a PW with and NON-RECURSIVE ADJACENCY lookup miss. Also includes IGMP snoop injects to a PW. RESOLVE_INGRESS_LEARN_CNT Total number of VPLS Source MAC learns on an ingress frame – learned locals. PARSE_FAB_MACN_W_FLD_RECEIVE_CNT Count of all frames received from fabric on egress LC with NPH L2 flood type and mac notify included (mac notify for ingress SA learn included with L2 flood) PARSE_FAB_MACN_RECEIVE_CNT Count of all frames received from fabric on egress LC with NPH mac notify flood type (mac notify for ingress SA learn). This includes: 1) forward mac notifies (from ingress LC) 2) reverse mac notifies (from egress LC in response to flood 3) mac deletes (due to a mac move or port flush) PARSE_FAB_DEST_MACN_RECEIVE_CNT Count of all mac notify frames received from fabric on egress LC that are reverse mac notify type PARSE_FAB_MAC_DELETE_RECEIVE_CNT Count of all mac notify frames received from fabric on egress LC that are mac delete type RESOLVE_LEARN_FROM_NOTIFY_CNT Total number of VPLS Source MAC learns from mac notify received. RESOLVE_VPLS_MAC_MOVE_CNT Total number of source MAC moves detected as a result of an ingress learn local or mac notify receive learn. RESOLVE_BD_FLUSH_DELETE_CNT Total number of learned entries deleted due to a Bridge Domain Flush RESOLVE_INGRESS_SMAC_MISS_CNT Total number of ingress frames with source MACs that did have a learned entry. These source MACs will be relearned if learned limits have not been exceeded, learning is enabled, and the address is unicast. RESOLVE_VPLS_MCAST_SRC_MAC_DROP_CNT Total number of ingress frames with source MACs that were not unicast. These frames are invalid and are are dropped. Non-unicast source macs cannot be learned. RESOLVE_HALF_AGE_RELEARN_CNT Total mac address from ingress frames that were relearned due to exceeding the half age period. These entries are refreshed to update the age timeout. STATS_STATIC_PARSE_PORTFLUSH_CNT Egress VPLS frame that has been detected as port flushed. Not valid if port based mac accounting is disabled. The frame will be dropped and mac delete indication sent to the bridge domain. RESOLVE_PORT_FLUSH_DROP_CNT Egress unicast frames dropped because the destination port has been flushed. For each a Mac Delete frame is sent to the bridge domain to remove the destination address. RESOLVE_PORT_FLUSH_DELETE_CNT Count of learned MACs deleted due to their source port being flushed. Does not include MACs deleted due to MAC Delete message. RESOLVE_PORT_FLUSH_AGE_OUT_CNT VPLS MAC entries that have been port flushed as detected by the MAC aging scan. These frames are dropped. Also can occur if the frame XID lookup fails and the XID is global or learned local – XID has been removed. RESOLVE_VPLS_LEARN_LIMIT_ACTION_DROP_CNT Ingress VPLS Frames dropped to learned limit exceeded – port based or bridge domain based learn limits – these frames are dropped. RESOLVE_VPLS_STATIC_MAC_MOVE_DROP_CNT A static MAC entry that are deleted due to a MAC move detected – these should never move. RESOLVE_MAC_NOTIFY_CTRL_DROP_CNT Total number of processed mac notify frames – forward or reverse mac notify – dropped after processing. RESOLVE_MAC_DELETE_CTRL_DROP_CNT Total number of processed mac delete frames – forward or reverse mac notify – dropped after processing. RESOLVE_AGE_NOMAC_DROP_CNT MAC scan aging message dropped due to an L2FIB lookup fail for the scanned mac entry. RESOLVE_AGE_MAC_STATIC_DROP_CNT MAC scan aging message dropped due to static MAC address for the scanned mac entry. EGR_VLANOPS_DROP Egress L2 frame without the proper number of vlans to do egress VLAN ops – the frame from the fabric needs at least one vlan to do a pop operation or replace operation. EGR_PREFILTER_VLAN_DROP Egress L2 frame that fails EFP pre-filter check – only valid if EFP filtering enabled – attached VLAN mismatch. RESOLVE_EFP_FILTER_TCAM_MISS_DROP Egress L2 frame that fails EFP filter TCAM – only valid if EFP filtering enabled – likely due to a TCAM missing entry due to a programming error. RESOLVE_EFP_FILTER_MISS_MATCH_DROP Egress L2 frame that fails EFP filter check – only valid if EFP filtering enabled – attached VLAN mismatch. CFM_ING_PUNT Ingress CFM frame punt count. CFM_EGR_PUNT Egress CFM frame punt count. CFM_EGR_PARSE Total egress Layer2 CFM frames processed by the NP – including egress CFM inject and egress CFM forwarded frames. INJ_NO_XID_MATCH_DROP Ingress CFM frame injects with an XID lookup fail – source XID for bridged injects, and dest. XID for pre-route injects – invalid XID index or missing entry. EGR_CFM_XID_NOMATCH Egress CFM frame inject with a dest. XID lookup fail – invalid XID index or missing entry. EGR_CFM_PW_DROP Egress CFM frame injected to a PW – not supported. PARSE_INGRESS_L3_CFM_DROP_CNT Ingress CFM frame on a layer 3 interface dropped due to CFM not enabled. RESOLVE_VPLS_MC_ING_RTE_DROP_CNT Ingress L2 multicast frame route lookup fails with no match. RESOLVE_VPLS_MC_EGR_RTE_DROP_CNT Egress L2 multicast frame route lookup fails with no match. RESOLVE_VPLS_MC_NO_OLIST_DROP_CNT Egress L2 multicast frame has zero olist members – no copies are sent to output ports. EGR_L2MC_OLIST_NOMATCH Egress L2 multicast frame has olist drop bit set (no copies are sent to output ports), OR egress L2 multicast frame olist lookup fails. Olist lookup failure could be due to a bad copy number, or a missing entry for that copy. EGR_L2MC_SRCDST Egress L2 multicast frame copy is reflection filtered – copies to the original source port are dropped. EGR_L2MC_XID_NOMATCH Egress L2 multicast frame copy XID lookup fails – XID stored in the OLIST result is invalid or entry is missing. This can occur if the copy is intended to go over a PW and the PW is down. RESOLVE_L2_PROT_BYTE_ERROR_DROP_CNT Invalid L2 protocol action set – this is an internal software programming error on an ingress or egress L2 frame if incremented. RESOLVE_VPLS_ING_ERR_DROP_CNT An ingress VPLS frame has an invalid processing type. This is likely an internal error in software. RESOLVE_VPLS_EGR_STRUCT_MISS_DROP_CNT Egress VPLS frame protocol handling falls through to an invalid case. This is likely a software implementation error. 3. Punt Reason Counters For each punt reason, there are 2 counters. The total number of frames attempted to be punted by microcode is the sum of these 2 counters. Actual punted Punt drops due to policing If the punt reason lookup fails, then software will drop the packet with MODIFY_PUNT_DROP_CNT increment. A layer3 interface is either a router gateway or a default interface for untagged frames (or both). Counter Name Counter Description PUNT_INVALID PUNT_INVALID_DROP Invalid Punt Reason – this should not occur. PUNT_ALL PUNT_ALL_EXCD Punt All Set – not implemented by software. CDP CDP_EXCD Punt Ingress CDP protocol frames – layer3 interface only. ARP ARP_EXCD Punt Ingress ARP protocol frames – layer3 interface only. RARP RARP_EXCD Punt Ingress Reverse ARP protocol frames – layer3 interface only. CGMP CGMP_EXCD Punt Ingress CGMP protocol frames – layer3 interface only. LOOP LOOP_EXCD Punt Ingress ELOOP protocol frames – layer3 interface only. CFM_OTHER CFM_OTHER_EXCD Ingress NP Punt or Egress NP Punt of CFM other packets (link trace or perf. mon.). Ingress punts can occur on a Layer2 or Layer3 interface Egress punts occur on a Layer2 interface only. CFM_CC CFM_CC_EXCD Ingress NP Punt or Egress NP Punt of CFM CC packets (continuity check). Ingress punts can occur on a Layer2 or Layer3 interface Egress punts occur on a Layer2 interface only. DHCP_SNOOP_REQ DHCP_SNOOP_REQ_EXCD Ingress Punt of DHCP snoop request frames – layer2 interface only and DHCP snooping enabled. DHCP_SNOOP_REPLY DHCP_SNOOP_REPLY_EXCD Ingress Punt of DHCP snoop reply frames – layer2 interface only and DHCP snooping enabled. MSTP MSTP_EXCD Ingress NP Punt or Egress NP Punt of MSTP frames. For Layer2 VPLS or Layer3, punt is on ingress frames only. For Layer3 VPWS, punt is on egress frames only. MSTP_PB MSTP_PB_EXCD Ingress NP Punt or Egress NP Punt of MSTP Provider Bridge frames. For Layer2 VPLS or Layer3, punt is on ingress frames only. For Layer3 VPWS, punt is on egress frames only. IGMP_SNOOP IGMP_SNOOP_EXCD Ingress punt of snooped IGMP or PIM frames – layer2 interface only and IGMP snooping enabled. EFM EFM_EXCD Ingress Punt of EFM Protocol frames – layer3 interface only. IPv4_OPTIONS IPv4_OPTIONS_EXCD Ingress Punt of IP frames with options – layer3 interface only – IPV4, IPV6. IPV4_PLU_PUNT (IPV4_FIB_PUNT) IPV4_PLU_PUNT_EXCD Ingress or Egress punt of IPV4 frames with punt flag set in Leaf results or RECURSIVE ADJACENCY results – layer3 interface only. IPV4MC_DO_ALL IPV4MC_DO_ALL_EXCD Ingress punt of IPV4 Mcast frames due to punt only set for route, OR Egress punt of IPV4 Mcast frames due to punt and forward bit set for mcast group member – layer3 interface only. IPV4 Mcast punts not support for egress route. IPV4MC_DO_ALL_BUT_FWD IPV4MC_DO_ALL_BUT_FWD_EXCD Ingress punt and forward of IPV4 Mcast frames due to punt only set for route, OR Egress punt and forward of IPV4 Mcast frames due to punt and forward bit set for mcast group member – layer3 interface only. IPV4 Mcast punts not support for egress route. This can also occur as a result of the RPF source interface check. PUNT_NO_MATCH (PUNT_FOR_ICMP) PUNT_NO_MATCH_EXCD An Ingress or Egress IPV4/IPV6 unicast frame punt due to ICMP Unreachable (i.e., next hop down). IPV4_TTL_ERROR (TTL_EXCEEDED) IPV4_TTL_ERROR_EXCD An Ingress IPV4 frame with a TTL of 1 is punted (TTL of 0 is dropped). IPV4_DF_SET_FRAG_NEEDED_PUNT IPV4_DF_SET_FRAG_NEEDED_PUNT_EXCD An egress IPV4 or MPLS frame with MTU exceeded is punted. RP_PUNT RP_PUNT_EXCD An ingress IPV4 or MPLS has the RP punt bit set in the NON-RECURSIVE ADJACENCY or RX adjacency. PUNT_IFIB PUNT_IFIB_EXCD An ingress IPV4 ucast or IPV6 frame has an IFIB lookup match resulting in an IFIB punt. PUNT_ADJ PUNT_ADJ_EXCD An ingress IPV4 ucast or MPLS frame has the adjacency punt bit set in the NON-RECURSIVE ADJACENCY or RX adjacency. An egress IPV4 ucast or MPLS frame has the adjacency punt bit set in the NON-RECURSIVE ADJACENCY or TX adjacency. PUNT_UNKNOWN_IFIB PUNT_UNKNOWN_IFIB_EXCD An ingress IPV4 ucast frame has the IFIB bit set in the LEAF or RECURSIVE ADJACENCY and the IFIB lookup does not get a match, OR an ingress MPLS frame has the IFIB bit set in the LEAF or RECURSIVE ADJACENCY. PUNT_ACL_DENY PUNT_ACL_DENY_EXCD Ingress or Egress frame has a deny action set in the ACL policy and ACL punt (instead of drop) is enabled. PUNT_ACL_LOG PUNT_ACL_LOG_EXCD Ingress or Egress frame has an ACL policy match and ACL logging is enabled. Layer3 frames only. PUNT_ACL_LOG_L2 PUNT_ACL_LOG_L2_EXCD Ingress or Egress frame has an ACL policy match and ACL logging is enabled. Layer2 frames only. IPV6_LINK_LOCAL IPV6_LINK_LOCAL_EXCD Ingress IPV6 frames received that are link local packets – these are punted – layer3 interface only. IPV6_HOP_BY_HOP IPV6_HOP_BY_HOP_EXCD Ingress IPV6 frames received that are hop by hop packets - these are punted – layer3 interface only. IPV6_TTL_ERROR IPV6_TTL_ERROR_EXCD Ingress IPV6 frames that have a TTL error – these are punted – layer3 interface only. IPV6_PLU_PUNT IPV6_PLU_PUNT_EXCD Ingress or Egress punt of IPV6 frames with punt flag set in Leaf results or RECURSIVE ADJACENCY results – layer3 interface only. IPV6_PLU_RCV IPV6_PLU_RCV_EXCD Not implemented by software. IPV6_ROUT_HEAD IPV6_ROUT_HEAD_EXCD Ingress IPV6 frames received that are router header packets – these are punted – layer3 interface only. IPV6_TOO_BIG IPV6_TOO_BIG_EXCD Egress IPV6 frames that are too bit – exceed MTU – these are punted – layer3 interface only. IPV6_MISS_SRC IPV6_MISS_SRC_EXCD Not implemented by software. IPV6MC_DC_CHECK IPV6MC_DC_CHECK_EXCD Not implemented by software. IPV6MC_DO_ALL IPV6MC_DO_ALL_EXCD Not implemented by software. IPV6MC_DO_ALL_BUT_FWD IPV6MC_DO_ALL_BUT_FWD_EXCD Not implemented by software. MPLS_PLU_PUNT (MPLS_FIB_PUNT) MPLS_PLU_PUNT_EXCD Ingress MPLS frame punted due to Punt set in the Leaf or RECURSIVE ADJACENCY, or an Ingress MPLS router alert frame – these are punted. MPLS_FOR_US MPLS_FOR_US_EXCD Ingress MPLS frame punted due to IFIB (for-us) set in the Leaf or RECURSIVE ADJACENCY. PUNT_VCCV_PKT PUNT_VCCV_PKT_EXCD Ingress MPLS frame that is a PW VCCV frame – must be a PWtoAC de-aggregation flow. PUNT_STATISTICS PUNT_STATISTICS_EXCD Statistics gathering frame punted to LC host – generated by statistics scanning machine. NETIO_RP_TO_LC_CPU_PUNT NETIO_RP_TO_LC_CPU_PUNT_EXCD Preroute egress inject frame received from the RP that is sent to the LC host. MOFRR_PUNT MOFRR_PUNT_EXCD IPV4 multicast over fast reroute punt frames. VIDMON_PUNT VIDMON_PUNT_EXCD IPV4 multicast video monitoring punt frames. VIDMON_PUNT_FLOW_ADD VIDMON_PUNT_FLOW_ADD_EXCD IPV4 multicast video monitoring new flow added punt frames. PUNT_DBGTRACE Debug Trace frames that are punted. PUNT_SYNCE Punt Ingress SYNCE protocol frames – layer3 interface only. Related Information n/a Xander Thuijs - CCIE #6775 Sr Tech Lead ASR9000
... View more
Introduction This document answers frequently asked questions about the Session State Redundancy Protocol (SSRP). SSRP is an integral part of achieving sub second failover for MLP and PPP interfaces, extremely useful in mobile backhaul deployment scenarios. While APS (Automated Protection Switching) provides for Sonet level failover, with regular MR-APS your PPP sessions would still need to re-establish. By using MR-APS your outtage time ranges around 20 seconds (protocol bound as PPP sesions have to re-establish). By leveraging SSRP, you will synchronize your ppp sessions from the working/active to the standby/protect, which means that you eliminate the need for PPP session re-establishment and that saves a lot of time. Further making use of the IP-FRR (Fast ReRoute) technology brings the failover and time to forward down to several hundreds of msecs! IP-FRR has a shadow route in the FIB (forwarding information base) pointing primarily to the mlp egress interface and secondary to the standby MR-APS peer. Whenever the primary route disappears, a sub second switch is made to the secondary route. SSRP is Session State Redundancy Protocol which is used in the ASR9000 to achieve IC-SSO or Inter Chassis Stateful Switchover. The configuration for SSRP is highlighted in different articles, but for arguments sake of this Q&A a config snippet is highlighted: Interface configuration (Required on EACH serial and mlp interface that needs to be sync’d) interface Multilink0/3/0/0/1 ipv4 address 22.214.171.124 255.255.255.252 ssrp group 1 id 10001 ppp encapsulation ppp ssrp location 0/3/CPU0 << specifies a slot that is subject to synchronisation group 1 profile ssrp1 <<< define the group (unique group per LC) and attached profile ssrp profile ssrp1 << profile definition security ttl max-hops 1 << TTL hops peer ipv4 address 192.168.1.2 << peer addr (this peer address needs to be in the same vrf as all multilink interfaces) Q&A (1) Can I reuse the SSRP profile for multiple interfaces? The SSRP profile is associated with an SSRP group - and a group can be used for multiple interfaces. Whether an SSRP profile can be used for multiple groups? If so, the answer is theoretically yes - assuming both groups want to use the same peer IPv4 address. In practice, this setup would be that common (since if the groups are using the same peer ip-addr, they could be combined into one group). The only possible use I can think of is when 2 LCs in the same (or even different) routers want to connect to the same peer, they want to use the same peer address, but a different TCP port number - in which case the same profile but different group-ids could be used. You need a group per linecard effectively, so if I have 2 linecards, then I could create 2 group ID's but referencing the same profile as they are going to the same peer, that should work fine. (2) Does the group number need to be unique per linecard or per interface, So if I have a 4 OC12 on a linecard, can I have them all use the same group number? The same group number can be used for multiple interfaces, as long as they are all on the same LC, and are all replicating to the same LC on the peer. So in general, you should be using the same group for all interfaces on a LC, unless you're replicating to multiple routers/LCs. (3) Config: interface Multilink0/2/0/0/2 ipv4 address 192.168.101.3 255.255.255.254 ssrp group 1 id 10002 ppp Considering this: Link this to the group number 1, which corresponds to the same group that I linked the CPU from “1” to the ID is here 10002, it needs to be unique, do the members of the T1’s need to have the same ID or basically the ID needs to be unique per interface that is sync’d that is each serial and each mlp if need to have a unique ID and the combination of group+id identifies a unique interface in a unique slot that gets synced to the standby? The combination of group+id needs to be sufficient to uniquely identify a PPP interface (i.e. the serial or multilink interface). So within each group, it's perfectly safe to start the ID numbering 1,2,3... We detect at config verification time whether the user attempts to configure the same ID to 2 interfaces within the same group - and fail the config. So the group+id identifies a unique interface, hence the same id cannot be used on any other interface on that uses the same group. (4) Is there is a limit of how many interfaces I can sync per group? and box wide? Not explicitly, no - config allows any number of interfaces to be configured per-group. I think officially we only support up to 2600 interfaces currently. (5) How is mastership determined in SSRP? Does it look at the APS state or something? In our implementation of SSRP, SSRP doesn't actually know whether it is Active or Standby - it also runs assuming it is both active and standby. PPP itself is the one to decide, on a per-interface basis, whether each serial/multilink interface is active or standby - and this is based (indirectly) on the APS state. Depending on the whether an interface is active or standby, PPP either replicates the state to the peer over SSRP, or requests the peer to send it the active state. (6) Wondering that if both sides are perceiving them to be master, then RTR-left might send a sync state of say UP while we are receiving state down from the right side router, what will be the end result, I mean how would a router know to override its own state by what it received? Would that then be derived by the APS state, say if I am APS master, then I don't override, if I am slave/backup/standby, I am using my received SSRP state? On each router, each interface will determine its 'redundancy state' (i.e. whether it is Active or Standby) from the APS state (i.e. whether the SONET interface is up or down). When Active, it will simply ignore any SSRP State messages it receives, while when Standby, it will just use whatever info it receives in SSRP State messages. So if both sides think they are Active, this doesn't cause any problems (both sides just ignore SSRP State messages). (7) Can I link say an interface in slot 2 to an interface on the standby in slot 4? or does it require the same phy mapping? Same physical mapping isn't required at all - so linking slot 2 to slot 4 would be fine. (8) Would a state sync packet effectively include: group/id/PPP-Phase/State/Options is that how it sort of looks what we'd be sending across? How many interfaces fit in a signle update packet? Say if a single state/interface is 100 bytes, would we pack 15 interfaces in one update? Or is it a packet per interface? The State messages contain the group, id, the CP (or authentication protocol) the state is for, whether it is Up or Down, and all the negotiated PPP Options.We batch up multiple interfaces into single packets, allowing packets of up to 3000 bytes. (9) How does the syncing work? if I have an interface doing an IPCP reneg say for instance, does that get replicated immediately? Effectively, yes. When IPCP renegotations start, a State-Down message will be sent to the standby router, indicating that IPCP is no longer up (although in fact we delay these State-Down messages by ~10s, to avoid clearing replicated state during certain MR-APS events). Then when IPCP negotiations complete again, a State-Up message is sent to the standby router, indicating that IPCP is up again, and containing the negotiated IPCP options. (10) Can I tune this timer or is it fixed? so if it goes down, we wait 10 secs for sending a stat down, if the if comes backup within 10 seconds, would I send out a state up message anyway? The timer can't be tuned - it's fixed to 10s. If it comes back up within 10s, we'd send a State-Up anyway. (11) How is pacing implemented in SSRP, I mean if there are a lot of updates sent, is there some sort of ack’ing and retransmission? There is no acking. We treat TCP as reliable - so as long as the TCP socket APIs don't return an error locally, we assume the peer has received the message. If TCP does return an error, we resync to recover. Similarly, if the TCP connection flaps, we resync to recover. Additionally, the Standby router is able at any time to request a replay of state for particular interfaces, if for some reason it hasn't yet received the state. (12) Is there any full sync operation that we sync ALL states from ALL interfaces under any circumstance? Whenever the TCP connection for a group gets established (both for the first time, and if it goes down and comes back up again), a full resync is performed for that group. Related Information Configuring PPP and SSRP on the ASR9000
... View more