07-07-2015 08:18 AM
Some one could help me, I want to know MTU value for the interface of the ASR SR 100Gb, because when traffic goes through interface a problem of performance (slow traffic) appears. I think it's a MTU problem because 100Gb inteface is a high-density interface and MTU is crucial in these cases.
Regards.
07-07-2015 11:02 PM
Hi
Max configurable MTU on the ASR9K is 9216. This will give you 9220 as max allowed frame size. If you haven't configured anything in the CLI the default MTU is 1514.
Regards Peter
07-08-2015 02:07 PM
Hi Peter,
I try MTU 9216 but traffic still have a slow performance, when I do a rollback the traffic have a normal performance, that interface is a bundle, 4X10Gb.
Regards.
07-08-2015 10:28 PM
Hi
Can you clarify "slow performance" a bit more?`Don't you get expected performance or what is wrong? Have you checked the 100gb interface in both ends for drops/errors?
/Peter
07-13-2015 06:06 AM
Hi Peter,
slow connection to internet, when a pass the traffic trought de 100Gb interface, I do not see any drop and error on the interface, but when I pass trough de 4x10Gb interface (Bundle interface) the connection to internet work perfect, That's why I think it is a problem concerning MTU
07-13-2015 11:49 AM
Hi Fredy,
your interface counters are clean, all your L1 stats are solid also, and you dont have any fragmentation issue as far as I can tell:
Input pkts 1024-1518 bytes = 4069777
Input pkts 1519-Max bytes = 0
I am suspecting that you are testing the circuit with too few flows.
As Sam hinted on these 2 cards have a FIA (fabric interface asic) that use VOQ's (virtual output queues) that are shaped at 13G and in some releases with a specific configuration setting can be set to 21G (or somewhere there).
A 100G interface is represented by 10 of those VOQ's, a single 10G interface is represented by 1 VOQ and ten 1G interfaces share a VOQ together also.
VOQ's are used to prevent head of line blocking and also provide for priority propagation (since each VOQ has 4 priority VQI's = virtual queuing instance).
yeah lots of TLA's (three letter acronyms) and if you like to read more on this check out this set of info.
To identify where the drops are coming from and propose the right mitigation check this:
show controller np counters [np] location [slot loc of LC]
run that twice to get some decent rate counters with a 1-5 second interval.
this will identify if we have to punt packets to the CPU, or whether we are losing something there.
the next thing to check is the:
show controller fabric fia stats location, this will help us identify the VOQ queue depth, and (tail) drops that happen in that asic.
I suspect you see voq depth building and tail drops which would identify the fact that your flow variety is not good enough.
you need minimally 10 10G flows (and preferably more flows) in order to take the power of this card.
Flows are loadbalanced over the available VOQ's. How the loadbalancing works is part of the deck set I ref'd above also.
The Tomahawk set of linecards can do single flow 100G.
To make sure, the 1x100 or 2x100 have no flaw or anything, but were bound by the technology available at the time to develop FIA's to support rates like this.
And also the fact that the PHY (optical piece) is connecting via multiple SGMII lanes to the MAC FPD (part of the NPU). In short you need flow variety :)
regards
xander
07-14-2015 09:12 AM
Hi Alexander,
Can help this output?
07-14-2015 06:22 PM
yeah that helps,
you seem to be ABF (access list based forwarding) a lot of packets coming in on the interface back out of it again!? is that right?
may want to remove ABF for a second and see if it is forwarded the way you expect...
xander
07-15-2015 07:18 AM
Alexander,
I can not remove the ABF because the ASR has an ISM module and doing CGN using ABF.
I use ABF to divert traffic to the ISM.
Regards.
07-15-2015 07:52 AM
Hi Fredy,
NP counters don't show any drops. There are also no drops on the fabric. The traffic rate on the HundredGigE0/7/0/0 is far less than 13Gbps, so the VQI shaper doesn't come into the picture here even if you had all the traffic calculated to the same hash value. Can you explain a more comprehensively what did you exactly mean by "slow traffic"? Why are you suspecting an issue with MTU (the controller counters don't show any frames received or sent with size above 1518 bytes)?
regards,
/Aleksandar
07-15-2015 08:53 AM
Hi Aleksandar,
Here the topology
Users who use the services of 3G, FTTH, GPON, feel slow Internet connection. do not load youtube videos, web pages open slowly.
When the traffic passing back through the interface Bundle 2x10Gb (no shutdown) everything works normally. So I thought it was something of MTU, but I think even that is not the problem.
Regards.
09-07-2015 11:54 AM
Hi all,
I want to tell you that I had solve the problem using a bundle interface then associated with the physical interface 100Gb in this way started to run smoothly.
It is strange to work this way, but it works.
:-D
Regards.
07-09-2015 04:39 AM
Which 100GbE card are you using?
How many flows do you have?
The 1 and 2x100 cards on 9K are limited to around 13G per flow.
Thanks,
Sam
07-13-2015 06:11 AM
Hi smilstea,
The card is A9K-2x100GE-TR.
I don't understand about 13G per flow, sorry, could you explain me please?
07-13-2015 06:38 AM
Here some output, I hope can help:
RP/0/RSP1/CPU0:Router#show interface hundredGigE 0/7/0/0
Thu Jun 25 00:10:25.604 UTC
HundredGigE0/7/0/0 is up, line protocol is up
Interface state transitions: 15
Hardware is HundredGigE, address is d46d.5006.88a6 (bia d46d.5006.88a6)
Layer 1 Transport Mode is LAN
Description: HundredGigE0/0/0/0
Internet address is X.X.X.X/YY
MTU 1514 bytes, BW 100000000 Kbit (Max: 100000000 Kbit)
reliability 255/255, txload 9/255, rxload 1/255
Encapsulation ARPA,
Full-duplex, 100000Mb/s, link type is force-up
output flow control is off, input flow control is off
loopback not set,
ARP type ARPA, ARP timeout 04:00:00
Last input 00:00:00, output 00:00:00
Last clearing of "show interface" counters 00:24:32
30 second input rate 499062000 bits/sec, 313498 packets/sec
30 second output rate 3591623000 bits/sec, 436841 packets/sec
16640982 packets input, 3387017120 bytes, 2 total input drops
0 drops for unrecognized upper-level protocol
Received 2 broadcast packets, 21 multicast packets
0 runts, 0 giants, 0 throttles, 0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
23302163 packets output, 23497866749 bytes, 0 total output drops
Output 77 broadcast packets, 43 multicast packets
0 output errors, 0 underruns, 0 applique, 0 resets
0 output buffer failures, 0 output buffers swapped out
3 carrier transitions
###################################################################################
RP/0/RSP1/CPU0:Router# show controller hundredGigE 0/7/0/0 stat all
Thu Jun 25 00:11:15.640 UTC
Operational data for interface HundredGigE0/7/0/0:
State:
Administrative state: enabled
Operational state: Up
LED state: Green On
Phy:
Media type: MMF fiber over 10 lane optics (short reach)
Optics:
Vendor: CISCO
Part number: FTLC8281RCNM-C3
Serial number: FNS18410BB4
MAC address information:
Operational address: d46d.5006.88a6
Burnt-in address: d46d.5006.88a6
No unicast addresses in filter
No multicast addresses in filter
Autonegotiation disabled.
Operational values:
Speed: 100Gbps
Duplex: Full Duplex
Flowcontrol: None
Loopback: None (or external)
MTU: 1518
MRU: 1518
Inter-packet gap: standard (12)
BER monitoring:
Signal Degrade: 10e-0 (no-alarm)
Signal Fail: 10e-0 (report-alarm)
BERT status for HundredGigE0/7/0/0:
BERT State : DISABLED
Test Pattern : None test pattern
Time Remaining : 0
Time Interval : 0
Statistics for interface HundredGigE0/7/0/0 (cached values):
Ingress:
Input total bytes = 10283938954
Input good bytes = 10283938954
Input total packets = 48313182
Input 802.1Q frames = 0
Input pause frames = 0
Input pkts 64 bytes = 8784740
Input pkts 65-127 bytes = 30709635
Input pkts 128-255 bytes = 2092478
Input pkts 256-511 bytes = 1443001
Input pkts 512-1023 bytes = 1213551
Input pkts 1024-1518 bytes = 4069777
Input pkts 1519-Max bytes = 0
Input good pkts = 48313182
Input unicast pkts = 48313150
Input multicast pkts = 30
Input broadcast pkts = 2
Input drop overrun = 0
Input drop abort = 0
Input drop invalid VLAN = 0
Input drop invalid DMAC = 0
Input drop invalid encap = 0
Input drop other = 0
Input error giant = 0
Input error runt = 0
Input error jabbers = 0
Input error fragments = 0
Input error CRC = 0
Input error collisions = 0
Input error symbol = 0
Input error other = 0
Input MIB giant = 0
Input MIB jabber = 0
Input MIB CRC = 0
Egress:
Output total bytes = 69519958774
Output good bytes = 69519958398
Output total packets = 64032394
Output 802.1Q frames = 0
Output pause frames = 0
Output pkts 64 bytes = 2793452
Output pkts 65-127 bytes = 11092252
Output pkts 128-255 bytes = 2383515
Output pkts 256-511 bytes = 1354220
Output pkts 512-1023 bytes = 1491783
Output pkts 1024-1518 bytes = 44917172
Output pkts 1519-Max bytes = 0
Output good pkts = 64032394
Output unicast pkts = 64032213
Output multicast pkts = 60
Output broadcast pkts = 122
Output drop underrun = 0
Output drop abort = 0
Output drop other = 0
Output error other = 0
Management information for interface HundredGigE0/7/0/0:
Bay number: 0
Port number: 0
Interface handle: 0x120000c0
Config:
Auto-negotiation: Configuration not supported (Off)
Carrier delay (up): Not configured
Carrier delay (down): Not configured
Speed: Configuration not supported (100Gbps)
Duplex: Configuration not supported (Full Duplex)
Flow Control: Not configured (None)
IPG: Not configured (standard (12))
Loopback: Not configured (None)
MTU: Not configured
Bandwidth: Not configured
BER-SD Threshold: Not configured (10e-0)
BER-SD Report: Not configured (Disabled)
BER-SF Threshold: Not configured (10e-0)
BER-SF Report: Not configured (Enabled)
BER-SF Signal Remote Failure: Not configured (Disabled)
Driver constraints:
Min MTU: 64 bytes
Max MTU: 9216 bytes
Max speed: 100Gbps
Interface type: HundredGigE
Management interface: No
Promiscuous mode: Yes
Default carrier delay up (auto-neg on): 0 ms
Default carrier delay down (auto-neg on): 0 ms
Default carrier delay up (auto-neg off): 0 ms
Default carrier delay down (auto-neg off): 0 ms
Allowed config mask: 0xe7b
BER:
SD (min - max): (10e-6 - 10e-9)
SD default: 10e-0
SF (min - max): (10e-6 - 10e-9)
SF default: 10e-0
Cached driver state:
MTU: 1514 bytes
Burnt-in MAC address: d46d.5006.88a6
Operational carrier delay:
Carrier delay (up): 0 ms
Carrier delay (down): 0 ms
Not a member of a bundle interface.
Satellite uplink settings:
Not in satellite uplink (ICL) mode.
Port FSM state:
Port is enabled, link is up
Complete FSM state:
Admin up
Client admin up
Client admin tx not disabled
Port enabled
Port tx enabled
Hardware link up
IDB interface state information:
IDB client admin up
IDB client tx admin up
IDB error disable not set
0 Unicast MAC Addresses:
0 Multicast MAC Addresses:
none
Port Number : 0
Bay Number : 0
Ifinst : 0
Ifinst Subport : 0
Board Type : 0x00280251
Port Type : 100GE
Transport mode : LAN
BIA MAC addr : d46d.5006.88a6
Oper. MAC addr : d46d.5006.88a6
Egress MAC addr : d46d.5006.88a6
Port Available : true
Status polling is : enabled
Status events are : enabled
I/F Handle : 0x120000c0
Cfg Link Enabled : tx/rx enabled
H/W Tx Enable : yes
MTU : 1518
H/W Speed : 100 Gbps
H/W Duplex : Full
H/W Loopback Type : None
H/W FlowCtrl Type : None
H/W AutoNeg Enable : Off
H/W Link Defects : (0x00000000) none
H/W Raw Link Defects : (0x00000000) none
Link Up : yes
Link Led Status : Link up -- Green/Amber
Pluggable Present : yes
Pluggable Type : CFP-100G-SR10
Pluggable PID : CFP-100G-SR10
Pluggable Compl. : Compliant
Pluggable Type Supp.: Supported
Pluggable PID Supp. : Supported
NP->Mac Lane Sync : 0x001fffff
PHY->Mac Lane Sync : 0x000fffff
Mac Version : 0x00000027
--More--
Rx 64B66B Lane Sync PCS Virt PCS
Service Block Marker Header Lane Lane Lane
Lane Lock Sync Err Cnt BIP Errors Error Mapping
-- --------- --------- ---------- ---------- ----- -------
0 Locked Locked 0 0 Clean 1
1 Locked Locked 0 0 Clean 0
2 Locked Locked 0 0 Clean 3
3 Locked Locked 0 0 Clean 2
4 Locked Locked 0 0 Clean 5
5 Locked Locked 0 0 Clean 4
6 Locked Locked 0 0 Clean 7
7 Locked Locked 0 0 Clean 6
8 Locked Locked 0 0 Clean 9
9 Locked Locked 0 0 Clean 8
10 Locked Locked 0 0 Clean 10
11 Locked Locked 0 0 Clean 11
12 Locked Locked 0 0 Clean 13
13 Locked Locked 0 0 Clean 12
14 Locked Locked 0 0 Clean 15
15 Locked Locked 0 0 Clean 14
16 Locked Locked 0 0 Clean 16
17 Locked Locked 0 0 Clean 17
18 Locked Locked 0 0 Clean 18
19 Locked Locked 0 0 Clean 19
CFP EEPROM port: 0
Xcvr Type: CFP
Ext Type: 16W, Host Lane n:n, CLEI,
Connector Type: MPO
Ethernet Application Codes: 100GE-SR10,
Number of Lanes: Network 10, Host 10
Max Bit Rate: Network Lane 10.4Gbit/s, Host Lane 10.4Gbit/s
--More-- Link Reaches: SM Fiber 0KM, MM Fiber: 100M, Copper: 0M
Device Tech1: VCSEL, DML,
Device Tech2: No WL, Uncool Xmtr, Xmtr not tunable, No VOA, PIN detector, No EDC,
Encoding: NRZ, Non-PSK,
Vendor Name: CISCO
Vendor OUI: 00.90.65
Vendor Part Number: FTLC8281RCNM-C3
Vendor Serial Number: FNS18410BB4
Date Code (yyyymmdd): 20141006, Lot Code 00
DDM Type: RX Avg Power, TX OMA,
Module DDM: Power Supply Voltage, Temperature,
Per Lane DDM: Laser Bias,
Enhanced Options: Host Lpbk, Host Emp Ctrl,
MSA Data (CFP NVR 1 Table - addr 0x8000-0x807F)
0x0000: 0e 61 09 03 00 00 00 00 : 00 aa 4a 34 34 00 0a 00
0x0010: 0a 01 83 40 86 60 02 8a : 00 04 40 4f 15 3c 32 46
0x0020: 00 43 49 53 43 4f 20 20 : 20 20 20 20 20 20 20 20
0x0030: 20 00 90 65 46 54 4c 43 : 38 32 38 31 52 43 4e 4d
0x0040: 2d 43 33 20 46 4e 53 31 : 38 34 31 30 42 42 34 20
0x0050: 20 20 20 20 32 30 31 34 : 31 30 30 36 30 30 49 50
0x0060: 55 49 42 52 34 52 41 41 : 0e 0e 02 00 01 07 0c 03
0x0070: 02 a0 04 01 01 00 64 01 : 00 00 00 00 00 00 00 72
CLEI Code: IPUIBR4RAA
Part Number: 10-2787-01 (ver.: V01 )
Product ID: CFP-100G-SR10
Vendor Specific Data (Vendor Cisco NVR 1 Table - address 0x8400-0x847F)
0x0100: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
0x0110: 43 49 53 43 4f 20 20 20 : 20 20 20 20 20 20 20 20
0x0120: 43 46 50 2d 31 30 30 47 : 2d 53 52 31 30 20 20 20
0x0130: 56 30 31 20 07 46 4e 53 : 31 38 34 31 30 42 42 34
0x0140: 31 30 2d 32 37 38 37 2d : 30 31 20 20 41 30 20 20
0x0150: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
0x0160: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
0x0170: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 a2
--More--
Module
Thresholds: Alarm High Warning High Warning Low Alarm Low
Temperature: +74.000 C +70.000 C +0.000 C -4.000 C
Voltage: 3.665 Volt 3.465 Volt 3.135 Volt 2.935 Volt
Temperature: +39.519 C
Voltage: 3.319 Volt
Lanes
Thresholds: Alarm High Warning High Warning Low Alarm Low
Temperature: +74.000 C +70.000 C +0.000 C -4.000 C
Bias: 15.000 mAmps 14.000 mAmps 5.000 mAmps 4.000 mAmps
Transmit Power: 1.99520 mW (2.99986 dBm) 0.79430 mW (-1.00015 dBm) 0.17370 mW (-7.60200 dBm) 0.06910 mW (-11.60522 dBm)
Receive Power: 4.36510 mW (6.39994 dBm) 1.73780 mW (2.40000 dBm) 0.11220 mW (-9.50007 dBm) 0.04460 mW (-13.50665 dBm)
Lane Temp Bias Tx Power Rx Power
0 N/A 10.184 mAmps N/A N/A
1 N/A 9.622 mAmps N/A N/A
2 N/A 9.944 mAmps N/A N/A
3 N/A 10.184 mAmps N/A N/A
4 N/A 10.184 mAmps N/A N/A
5 N/A 10.184 mAmps N/A N/A
6 N/A 10.184 mAmps N/A N/A
7 N/A 9.944 mAmps N/A N/A
8 N/A 9.944 mAmps N/A N/A
9 N/A 10.104 mAmps N/A N/A
Threshold Data (CFP NVR 2 Table - address 0x8080-0x80ff)
0x0080: 4a 00 46 00 00 00 fc 00 : 8f 2a 87 5a 7a 76 72 a6
0x0090: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
0x00a0: 00 00 00 00 00 00 00 00 : 1d 4c 1b 58 09 c4 07 d0
0x00b0: 4d f0 1f 07 06 c9 02 b3 : 4a 00 46 00 00 00 fc 00
0x00c0: aa 83 43 e2 04 62 01 be : 00 00 00 00 00 00 00 00
0x00d0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
0x00e0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00
--More-- 0x00f0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 98
OPTICS:
Rx LOS Fault : No
Tx Fault : No
MAC:
Rx Local Fault : No
Rx Remote Fault : No
MAC registers:
0x0010 registers.clock_status : 0x000003ff
0x0014 registers.soft_reset : 0x00000000
0x0030 registers.revision_register : 0x00000027
0x0034 registers.software_status : 0x00000000
0x0100 registers.hseib_buffer_configuration : 0x00000010
0x0104 registers.rx_hsec_configuration0 : 0x00001c09
0x0158 registers.oobfc_configuration : 0x00000007
0x0200 registers.tx_hsec_configuration0 : 0x00000003
0x0208 registers.tx_hsec_max_packet_len : 0x000005ee
0x0230 registers.tx_buffer_pause_src_address1 : 0x00000000
0x0234 registers.tx_buffer_pause_src_address0 : 0xd46d5006
0x0400 registers.iipc_serdes0_configuration0 : 0x00000db3
0x0404 registers.iipc_serdes0_configuration1 : 0x00000000
0x04a8 registers.hsec_serdes0_configuration0 : 0x000004a8
0x04ac registers.hsec_serdes0_configuration1 : 0x00000000
0x1010 interupts.rx_hsec_err_intr : 0x00000000
0x1018 interupts.rx_hsec_err_mask : 0x0001fffc
0x1020 interupts.rx_buffer_err_intr : 0x00000000
0x1040 interupts.tx_hsec_err_intr : 0x00000000
0x1050 interupts.tx_buffer_err_intr : 0x00000000
0x1080 interupts.system_monitor_intr : 0x00000000
0x1090 interupts.seu_intr : 0x00000000
0x1110 interupts.rx_hsec_bip7_0_status_intr : 0x00000001
0x1120 interupts.device_intr : 0x00000000
0x1124 interupts.device_enable : 0x00000001
0x2000 stats.rx_hsec_status : 0x00000004
0x2004 stats.rx_hsec_framing_err : 0x000fffff
0x200c stats.rx_hsec_synced_err : 0x000fffff
0x2410 stats.rx_interlaken_status : 0x00000201
0x2418 stats.rx_interlaken_synced_err : 0x00000000
0x242c stats.rx_interlaken_framing_err : 0x00000000
0x2434 stats.rx_interlaken_crc32_err : 0x00000000
AFP registers:
0x0000 cfp_reg_info_ : 0x002c0f41
0x0004 reg_init_stat_ : 0x00000000
--More-- 0x0008 reset_ : 0x00000000
0x000c int_stat_ : 0x00000000
0x0010 int_en_ : 0x00000050
0x0014 int_frc_ : 0x00000000
0x0018 stat_ : 0x00000007
0x001c ctrl_ : 0x000040c5
PHY registers:
0x0000 nlp-reg1 : 0x00000000
0x0001 nlp-reg2 : 0x00000001
0x0002 nlp-reg3 : 0x00000002
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide