07-07-2015 08:18 AM
Some one could help me, I want to know MTU value for the interface of the ASR SR 100Gb, because when traffic goes through interface a problem of performance (slow traffic) appears. I think it's a MTU problem because 100Gb inteface is a high-density interface and MTU is crucial in these cases.
Regards.
07-07-2015 11:02 PM
Hi
Max configurable MTU on the ASR9K is 9216. This will give you 9220 as max allowed frame size. If you haven't configured anything in the CLI the default MTU is 1514.
Regards Peter
07-08-2015 02:07 PM
Hi Peter,
I try MTU 9216 but traffic still have a slow performance, when I do a rollback the traffic have a normal performance, that interface is a bundle, 4X10Gb.
Regards.
07-08-2015 10:28 PM
Hi
Can you clarify "slow performance" a bit more?`Don't you get expected performance or what is wrong? Have you checked the 100gb interface in both ends for drops/errors?
/Peter
07-13-2015 06:06 AM
Hi Peter,
slow connection to internet, when a pass the traffic trought de 100Gb interface, I do not see any drop and error on the interface, but when I pass trough de 4x10Gb interface (Bundle interface) the connection to internet work perfect, That's why I think it is a problem concerning MTU
07-13-2015 11:49 AM
Hi Fredy,
your interface counters are clean, all your L1 stats are solid also, and you dont have any fragmentation issue as far as I can tell:
    Input pkts 1024-1518 bytes  = 4069777
    Input pkts 1519-Max bytes   = 0
I am suspecting that you are testing the circuit with too few flows.
As Sam hinted on these 2 cards have a FIA (fabric interface asic) that use VOQ's (virtual output queues) that are shaped at 13G and in some releases with a specific configuration setting can be set to 21G (or somewhere there).
A 100G interface is represented by 10 of those VOQ's, a single 10G interface is represented by 1 VOQ and ten 1G interfaces share a VOQ together also.
VOQ's are used to prevent head of line blocking and also provide for priority propagation (since each VOQ has 4 priority VQI's = virtual queuing instance).
yeah lots of TLA's (three letter acronyms) and if you like to read more on this check out this set of info.
To identify where the drops are coming from and propose the right mitigation check this:
show controller np counters [np] location [slot loc of LC]
run that twice to get some decent rate counters with a 1-5 second interval.
this will identify if we have to punt packets to the CPU, or whether we are losing something there.
the next thing to check is the:
show controller fabric fia stats location, this will help us identify the VOQ queue depth, and (tail) drops that happen in that asic.
I suspect you see voq depth building and tail drops which would identify the fact that your flow variety is not good enough.
you need minimally 10 10G flows (and preferably more flows) in order to take the power of this card.
Flows are loadbalanced over the available VOQ's. How the loadbalancing works is part of the deck set I ref'd above also.
The Tomahawk set of linecards can do single flow 100G.
To make sure, the 1x100 or 2x100 have no flaw or anything, but were bound by the technology available at the time to develop FIA's to support rates like this.
And also the fact that the PHY (optical piece) is connecting via multiple SGMII lanes to the MAC FPD (part of the NPU). In short you need flow variety :)
regards
xander
07-14-2015 09:12 AM
Hi Alexander,
Can help this output?
07-14-2015 06:22 PM
yeah that helps,
you seem to be ABF (access list based forwarding) a lot of packets coming in on the interface back out of it again!? is that right?
may want to remove ABF for a second and see if it is forwarded the way you expect...
xander
07-15-2015 07:18 AM
Alexander,
I can not remove the ABF because the ASR has an ISM module and doing CGN using ABF.
I use ABF to divert traffic to the ISM.
Regards.
07-15-2015 07:52 AM
Hi Fredy,
NP counters don't show any drops. There are also no drops on the fabric. The traffic rate on the HundredGigE0/7/0/0 is far less than 13Gbps, so the VQI shaper doesn't come into the picture here even if you had all the traffic calculated to the same hash value. Can you explain a more comprehensively what did you exactly mean by "slow traffic"? Why are you suspecting an issue with MTU (the controller counters don't show any frames received or sent with size above 1518 bytes)?
regards,
/Aleksandar
07-15-2015 08:53 AM
Hi Aleksandar,
Here the topology

Users who use the services of 3G, FTTH, GPON, feel slow Internet connection. do not load youtube videos, web pages open slowly.
 When the traffic passing back through the interface Bundle 2x10Gb (no shutdown) everything works normally. So I thought it was something of MTU, but I think even that is not the problem.
Regards.
09-07-2015 11:54 AM
Hi all,
I want to tell you that I had solve the problem using a bundle interface then associated with the physical interface 100Gb in this way started to run smoothly.
It is strange to work this way, but it works.
:-D
Regards.
07-09-2015 04:39 AM
Which 100GbE card are you using?
How many flows do you have?
The 1 and 2x100 cards on 9K are limited to around 13G per flow.
Thanks,
Sam
07-13-2015 06:11 AM
Hi smilstea,
The card is A9K-2x100GE-TR.
I don't understand about 13G per flow, sorry, could you explain me please?
07-13-2015 06:38 AM
Here some output, I hope can help:
RP/0/RSP1/CPU0:Router#show interface hundredGigE 0/7/0/0
Thu Jun 25 00:10:25.604 UTC
HundredGigE0/7/0/0 is up, line protocol is up 
  Interface state transitions: 15
  Hardware is HundredGigE, address is d46d.5006.88a6 (bia d46d.5006.88a6)
  Layer 1 Transport Mode is LAN
  Description:  HundredGigE0/0/0/0
  Internet address is X.X.X.X/YY
  MTU 1514 bytes, BW 100000000 Kbit (Max: 100000000 Kbit)
     reliability 255/255, txload 9/255, rxload 1/255
  Encapsulation ARPA,
  Full-duplex, 100000Mb/s, link type is force-up
  output flow control is off, input flow control is off
  loopback not set,
  ARP type ARPA, ARP timeout 04:00:00
  Last input 00:00:00, output 00:00:00
  Last clearing of "show interface" counters 00:24:32
  30 second input rate 499062000 bits/sec, 313498 packets/sec
  30 second output rate 3591623000 bits/sec, 436841 packets/sec
     16640982 packets input, 3387017120 bytes, 2 total input drops
     0 drops for unrecognized upper-level protocol
     Received 2 broadcast packets, 21 multicast packets
              0 runts, 0 giants, 0 throttles, 0 parity
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     23302163 packets output, 23497866749 bytes, 0 total output drops
     Output 77 broadcast packets, 43 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     3 carrier transitions
###################################################################################
RP/0/RSP1/CPU0:Router#  show controller hundredGigE 0/7/0/0 stat    all
Thu Jun 25 00:11:15.640 UTC
Operational data for interface HundredGigE0/7/0/0:
State:
    Administrative state: enabled
    Operational state: Up
    LED state: Green On
Phy:
    Media type: MMF fiber over 10 lane optics (short reach)
    Optics:
        Vendor: CISCO
        Part number: FTLC8281RCNM-C3
        Serial number: FNS18410BB4
MAC address information:
    Operational address: d46d.5006.88a6
    Burnt-in address: d46d.5006.88a6
    No unicast addresses in filter
    No multicast addresses in filter
Autonegotiation disabled.
Operational values:
    Speed: 100Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: None (or external)
    MTU: 1518
    MRU: 1518
    Inter-packet gap: standard (12)
    BER monitoring:
        Signal Degrade: 10e-0 (no-alarm)
        Signal Fail: 10e-0 (report-alarm)
BERT status for HundredGigE0/7/0/0:
BERT State                      :       DISABLED
Test Pattern                    :       None test pattern
Time Remaining                  :       0
Time Interval                   :       0
Statistics for interface HundredGigE0/7/0/0 (cached values):
Ingress: 
    Input total bytes           = 10283938954
    Input good bytes            = 10283938954
    Input total packets         = 48313182
    Input 802.1Q frames         = 0
    Input pause frames          = 0
    Input pkts 64 bytes         = 8784740
    Input pkts 65-127 bytes     = 30709635
    Input pkts 128-255 bytes    = 2092478
    Input pkts 256-511 bytes    = 1443001
    Input pkts 512-1023 bytes   = 1213551
    Input pkts 1024-1518 bytes  = 4069777
    Input pkts 1519-Max bytes   = 0
    Input good pkts             = 48313182
    Input unicast pkts          = 48313150
    Input multicast pkts        = 30
    Input broadcast pkts        = 2
    Input drop overrun          = 0
    Input drop abort            = 0
    Input drop invalid VLAN     = 0
    Input drop invalid DMAC     = 0
    Input drop invalid encap    = 0
    Input drop other            = 0
    Input error giant           = 0
    Input error runt            = 0
    Input error jabbers         = 0
    Input error fragments       = 0
    Input error CRC             = 0
    Input error collisions      = 0
    Input error symbol          = 0
    Input error other           = 0
    Input MIB giant             = 0
    Input MIB jabber            = 0
    Input MIB CRC               = 0
Egress:
    Output total bytes          = 69519958774
    Output good bytes           = 69519958398
    Output total packets        = 64032394
    Output 802.1Q frames        = 0
    Output pause frames         = 0
    Output pkts 64 bytes        = 2793452
    Output pkts 65-127 bytes    = 11092252
    Output pkts 128-255 bytes   = 2383515
    Output pkts 256-511 bytes   = 1354220
    Output pkts 512-1023 bytes  = 1491783
    Output pkts 1024-1518 bytes = 44917172
    Output pkts 1519-Max bytes  = 0
    Output good pkts            = 64032394
    Output unicast pkts         = 64032213
    Output multicast pkts       = 60
    Output broadcast pkts       = 122
    Output drop underrun        = 0
    Output drop abort           = 0
    Output drop other           = 0
Output error other = 0
Management information for interface HundredGigE0/7/0/0:
Bay number: 0
Port number: 0
Interface handle: 0x120000c0
Config:
    Auto-negotiation: Configuration not supported (Off)
    Carrier delay (up): Not configured
    Carrier delay (down): Not configured
    Speed: Configuration not supported (100Gbps)
    Duplex: Configuration not supported (Full Duplex)
    Flow Control: Not configured (None)
    IPG: Not configured (standard (12))
    Loopback: Not configured (None)
    MTU: Not configured
    Bandwidth: Not configured
    BER-SD Threshold: Not configured (10e-0)
    BER-SD Report: Not configured (Disabled)
    BER-SF Threshold: Not configured (10e-0)
    BER-SF Report: Not configured (Enabled)
    BER-SF Signal Remote Failure: Not configured (Disabled)
Driver constraints:
    Min MTU: 64 bytes
    Max MTU: 9216 bytes
    Max speed: 100Gbps
    Interface type: HundredGigE
    Management interface: No
    Promiscuous mode: Yes
    Default carrier delay up (auto-neg on): 0 ms
    Default carrier delay down (auto-neg on): 0 ms
    Default carrier delay up (auto-neg off): 0 ms
    Default carrier delay down (auto-neg off): 0 ms
    Allowed config mask: 0xe7b
    BER:
        SD (min - max): (10e-6 - 10e-9)
        SD default: 10e-0
        SF (min - max): (10e-6 - 10e-9)
        SF default: 10e-0
Cached driver state:
    MTU: 1514 bytes
    Burnt-in MAC address: d46d.5006.88a6
Operational carrier delay:
    Carrier delay (up): 0 ms
    Carrier delay (down): 0 ms
Not a member of a bundle interface.
Satellite uplink settings:
    Not in satellite uplink (ICL) mode.
Port FSM state:
    Port is enabled, link is up
Complete FSM state:
    Admin up
    Client admin up
    Client admin tx not disabled
    Port enabled
    Port tx enabled
    Hardware link up
IDB interface state information:
    IDB client admin up
    IDB client tx admin up
    IDB error disable not set
0 Unicast MAC Addresses:
0 Multicast MAC Addresses:           
 none
Port Number         : 0 
Bay Number          : 0 
Ifinst              : 0 
Ifinst Subport      : 0 
Board Type          : 0x00280251 
Port Type           : 100GE 
Transport mode      : LAN
BIA MAC addr        : d46d.5006.88a6
Oper. MAC addr      : d46d.5006.88a6
Egress MAC addr     : d46d.5006.88a6
Port Available      : true
Status polling is   : enabled
Status events are   : enabled
I/F Handle          : 0x120000c0
Cfg Link Enabled    : tx/rx enabled
H/W Tx Enable       : yes
MTU                 : 1518
H/W Speed           : 100 Gbps 
H/W Duplex          : Full 
H/W Loopback Type   : None 
H/W FlowCtrl Type   : None 
H/W AutoNeg Enable  : Off 
H/W Link Defects    : (0x00000000)  none
H/W Raw Link Defects : (0x00000000)  none
Link Up             : yes
Link Led Status     : Link up   -- Green/Amber
Pluggable Present   : yes
Pluggable Type      : CFP-100G-SR10
Pluggable PID       : CFP-100G-SR10
Pluggable Compl.    : Compliant
Pluggable Type Supp.: Supported
Pluggable PID Supp. : Supported
NP->Mac Lane Sync   : 0x001fffff
PHY->Mac Lane Sync  : 0x000fffff
Mac Version         : 0x00000027
 --More--           
Rx        64B66B      Lane        Sync        PCS         Virt  PCS
Service   Block       Marker      Header      Lane        Lane  Lane
Lane      Lock        Sync        Err Cnt     BIP Errors  Error Mapping
--        ---------   ---------   ----------  ----------  ----- -------
0         Locked      Locked      0           0           Clean 1      
1         Locked      Locked      0           0           Clean 0      
2         Locked      Locked      0           0           Clean 3      
3         Locked      Locked      0           0           Clean 2      
4         Locked      Locked      0           0           Clean 5      
5         Locked      Locked      0           0           Clean 4      
6         Locked      Locked      0           0           Clean 7      
7         Locked      Locked      0           0           Clean 6      
8         Locked      Locked      0           0           Clean 9      
9         Locked      Locked      0           0           Clean 8      
10        Locked      Locked      0           0           Clean 10     
11        Locked      Locked      0           0           Clean 11     
12        Locked      Locked      0           0           Clean 13     
13        Locked      Locked      0           0           Clean 12     
14        Locked      Locked      0           0           Clean 15     
15        Locked      Locked      0           0           Clean 14     
16        Locked      Locked      0           0           Clean 16     
17        Locked      Locked      0           0           Clean 17     
18        Locked      Locked      0           0           Clean 18     
19        Locked      Locked      0           0           Clean 19     
CFP EEPROM port: 0
  Xcvr Type: CFP
  Ext Type: 16W, Host Lane n:n, CLEI, 
  Connector Type: MPO
  Ethernet Application Codes: 100GE-SR10, 
  Number of Lanes: Network 10, Host 10
  Max Bit Rate: Network Lane 10.4Gbit/s, Host Lane 10.4Gbit/s
 --More--             Link Reaches: SM Fiber 0KM, MM Fiber: 100M, Copper: 0M
  Device Tech1: VCSEL, DML, 
  Device Tech2: No WL, Uncool Xmtr, Xmtr not tunable, No VOA, PIN detector, No EDC, 
  Encoding: NRZ, Non-PSK, 
  Vendor Name: CISCO           
  Vendor OUI: 00.90.65
  Vendor Part Number: FTLC8281RCNM-C3 
  Vendor Serial Number: FNS18410BB4     
  Date Code (yyyymmdd): 20141006, Lot Code 00
  DDM Type: RX Avg Power, TX OMA, 
  Module DDM: Power Supply Voltage, Temperature, 
  Per Lane DDM: Laser Bias, 
  Enhanced Options: Host Lpbk, Host Emp Ctrl, 
 MSA Data (CFP NVR 1 Table - addr 0x8000-0x807F)
0x0000: 0e 61 09 03 00 00 00 00 : 00 aa 4a 34 34 00 0a 00 
0x0010: 0a 01 83 40 86 60 02 8a : 00 04 40 4f 15 3c 32 46 
0x0020: 00 43 49 53 43 4f 20 20 : 20 20 20 20 20 20 20 20 
0x0030: 20 00 90 65 46 54 4c 43 : 38 32 38 31 52 43 4e 4d 
0x0040: 2d 43 33 20 46 4e 53 31 : 38 34 31 30 42 42 34 20 
0x0050: 20 20 20 20 32 30 31 34 : 31 30 30 36 30 30 49 50 
0x0060: 55 49 42 52 34 52 41 41 : 0e 0e 02 00 01 07 0c 03 
0x0070: 02 a0 04 01 01 00 64 01 : 00 00 00 00 00 00 00 72 
        CLEI Code: IPUIBR4RAA
        Part Number: 10-2787-01    (ver.: V01 )
        Product ID: CFP-100G-SR10   
 Vendor Specific Data (Vendor Cisco NVR 1 Table - address 0x8400-0x847F) 
0x0100: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
0x0110: 43 49 53 43 4f 20 20 20 : 20 20 20 20 20 20 20 20 
0x0120: 43 46 50 2d 31 30 30 47 : 2d 53 52 31 30 20 20 20 
0x0130: 56 30 31 20 07 46 4e 53 : 31 38 34 31 30 42 42 34 
0x0140: 31 30 2d 32 37 38 37 2d : 30 31 20 20 41 30 20 20 
0x0150: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
0x0160: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
0x0170: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 a2 
 --More--           
        Module
        Thresholds:                    Alarm High         Warning High          Warning Low            Alarm Low
              Temperature:            +74.000 C             +70.000 C              +0.000 C              -4.000 C
                  Voltage:           3.665 Volt            3.465 Volt            3.135 Volt            2.935 Volt
        Temperature: +39.519 C
        Voltage: 3.319 Volt
        Lanes
        Thresholds:                    Alarm High         Warning High          Warning Low            Alarm Low
              Temperature:            +74.000 C             +70.000 C              +0.000 C              -4.000 C
                     Bias:         15.000 mAmps          14.000 mAmps           5.000 mAmps           4.000 mAmps
           Transmit Power:  1.99520 mW (2.99986 dBm)   0.79430 mW (-1.00015 dBm)   0.17370 mW (-7.60200 dBm)   0.06910 mW (-11.60522 dBm)
            Receive Power:  4.36510 mW (6.39994 dBm)   1.73780 mW (2.40000 dBm)   0.11220 mW (-9.50007 dBm)   0.04460 mW (-13.50665 dBm)
                Lane            Temp             Bias                   Tx Power                   Rx Power
                   0             N/A     10.184 mAmps                        N/A                        N/A
                   1             N/A      9.622 mAmps                        N/A                        N/A
                   2             N/A      9.944 mAmps                        N/A                        N/A
                   3             N/A     10.184 mAmps                        N/A                        N/A
                   4             N/A     10.184 mAmps                        N/A                        N/A
                   5             N/A     10.184 mAmps                        N/A                        N/A
                   6             N/A     10.184 mAmps                        N/A                        N/A
                   7             N/A      9.944 mAmps                        N/A                        N/A
                   8             N/A      9.944 mAmps                        N/A                        N/A
                   9             N/A     10.104 mAmps                        N/A                        N/A
 Threshold Data (CFP NVR 2 Table - address 0x8080-0x80ff) 
0x0080: 4a 00 46 00 00 00 fc 00 : 8f 2a 87 5a 7a 76 72 a6 
0x0090: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
0x00a0: 00 00 00 00 00 00 00 00 : 1d 4c 1b 58 09 c4 07 d0 
0x00b0: 4d f0 1f 07 06 c9 02 b3 : 4a 00 46 00 00 00 fc 00 
0x00c0: aa 83 43 e2 04 62 01 be : 00 00 00 00 00 00 00 00 
0x00d0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
0x00e0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00 
 --More--           0x00f0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 98 
  OPTICS:
   Rx LOS Fault       : No
   Tx Fault           : No
  MAC:
   Rx Local Fault     : No
   Rx Remote Fault    : No
MAC registers: 
0x0010 registers.clock_status                  : 0x000003ff 
0x0014 registers.soft_reset                    : 0x00000000 
0x0030 registers.revision_register             : 0x00000027 
0x0034 registers.software_status               : 0x00000000 
0x0100 registers.hseib_buffer_configuration    : 0x00000010 
0x0104 registers.rx_hsec_configuration0        : 0x00001c09 
0x0158 registers.oobfc_configuration           : 0x00000007 
0x0200 registers.tx_hsec_configuration0        : 0x00000003 
0x0208 registers.tx_hsec_max_packet_len        : 0x000005ee 
0x0230 registers.tx_buffer_pause_src_address1  : 0x00000000 
0x0234 registers.tx_buffer_pause_src_address0  : 0xd46d5006 
0x0400 registers.iipc_serdes0_configuration0   : 0x00000db3 
0x0404 registers.iipc_serdes0_configuration1   : 0x00000000 
0x04a8 registers.hsec_serdes0_configuration0   : 0x000004a8 
0x04ac registers.hsec_serdes0_configuration1   : 0x00000000 
0x1010 interupts.rx_hsec_err_intr              : 0x00000000 
0x1018 interupts.rx_hsec_err_mask              : 0x0001fffc 
0x1020 interupts.rx_buffer_err_intr            : 0x00000000 
0x1040 interupts.tx_hsec_err_intr              : 0x00000000 
0x1050 interupts.tx_buffer_err_intr            : 0x00000000 
0x1080 interupts.system_monitor_intr           : 0x00000000 
0x1090 interupts.seu_intr                      : 0x00000000 
0x1110 interupts.rx_hsec_bip7_0_status_intr    : 0x00000001 
0x1120 interupts.device_intr                   : 0x00000000 
0x1124 interupts.device_enable                 : 0x00000001 
0x2000 stats.rx_hsec_status                    : 0x00000004 
0x2004 stats.rx_hsec_framing_err               : 0x000fffff 
0x200c stats.rx_hsec_synced_err                : 0x000fffff 
0x2410 stats.rx_interlaken_status              : 0x00000201 
0x2418 stats.rx_interlaken_synced_err          : 0x00000000 
0x242c stats.rx_interlaken_framing_err         : 0x00000000 
0x2434 stats.rx_interlaken_crc32_err           : 0x00000000 
AFP registers: 
0x0000 cfp_reg_info_                           : 0x002c0f41 
0x0004 reg_init_stat_                          : 0x00000000 
 --More--           0x0008 reset_                                  : 0x00000000 
0x000c int_stat_                               : 0x00000000 
0x0010 int_en_                                 : 0x00000050 
0x0014 int_frc_                                : 0x00000000 
0x0018 stat_                                   : 0x00000007 
0x001c ctrl_                                   : 0x000040c5 
PHY registers: 
0x0000 nlp-reg1                                : 0x00000000 
0x0001 nlp-reg2                                : 0x00000001 
0x0002 nlp-reg3                                : 0x00000002 
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide