- Introduction
- Disclaimer
- General
- Ethernet Line Cards
- Optics OIR
- Third Party Optics
- Transceiver permit pid all
- Statistics
- MTU
- Link Defects
- packet-gap non-standard
- small-frame-padding
- loopback
- Loopback external (Applicable only to RJ45 SFP)
- carrier-delay
- Optic Pie
- Optics DOM
- 1GE Interface
- LAN Mode
- Uni-directional link Fault
- Optics
- 1GE Optical SFP
- Copper/RJ45 SFP
- 100FX Optical SFP
- CWDM/DWDM SFP
- GLC-BX10-U/D SFP
- 10GE Interface
- LAN Mode
- WAN PHY Mode
- OTN Mode
- Proactive Protection
- Unidirectional Link Routing
- Optics
- 40GE Interface
- LAN Mode
- OTN Mode
- Optics
- 100GE Interface
- LAN Mode
- LAN PP Mode
- Optics
- FAQ
- Glossary
- Related Information
Introduction
This white paper is intended to help the readers obtain a quick understanding on 1/10/40/100GE Ethernet port on a Line Card in general and how it has been implemented in ASR9000 including the configuration detail. It also includes a ‘frequently asked questions’ (FAQ) section, which cover some of the potential field questions that might pop up while deploying this feature using ASR9k. Overall, this white paper can be a quick reference guide for anyone working on these Line Cards.
Disclaimer
This white paper is based on ASR9k Release 4.3.1.
General
The Cisco ASR 9000 Series Aggregation Services Routers is the baseline for next-‐generation Carrier Ethernet networks, providing up to 96 terabits (Tbps) per system. Its architecture design enables capabilities for next generation Internet that includes mobile aggregation & virtualization and also reduces operational cost & complexity.
Ethernet Line Cards
Ethernet Line Cards in ASR9000 can be called as First and Second generation based on the Network processor (NP) used. The Second generation Line Card is also referred to as Enhanced Ethernet Line Card.
| Line Card/Modular Port Adapter (MPA) | Generation | Supported Transport modes | Supported Optics Type |
| A9K‐40G | First Generation | 1GE LAN | SFP |
| A9K‐4T | First Generation | 10GE LAN | XFP |
| A9K‐8T/4 | First Generation | 10GE LAN | XFP |
| A9K‐2T20G | First Generation | 1GE LAN 10GE LAN/WAN PHY/OTN |
SFP for 1GE Ports; XFP for 10GE Ports |
| A9K‐8T | First Generation | 10GE LAN/WAN PHY/OTN | XFP |
| A9K‐16T/8 | First Generation | 10GE LAN/WAN PHY/OTN | SFP+ |
| A9K‐24-10GE | Second Generation | 10GE LAN/WAN PHY/OTN | SFP+ |
| A9K‐36-10GE | Second Generation | 10GE LAN/WAN PHY/OTN | SFP+ |
| A9K‐MPA‐20x1GE | Second Generation | 10GE LAN/WAN PHY/OTN | SFP |
| A9K‐MPA‐2x10GE | Second Generation | 10GE LAN/WAN PHY/OTN | XFP |
| A9K‐MPA‐4x10GE | Second Generation | 10GE LAN/WAN PHY/OTN | XFP |
| A9K‐MPA‐8x10GE | Second Generation | 10GE LAN/WAN PHY/OTN | SFP+ |
| A9K‐MPA‐1x40GE | Second Generation | 10GE LAN/OTN | QSFP+ |
| A9K‐MPA‐2x40GE | Second Generation | 10GE LAN/OTN | QSFP+ |
| A9K‐2x100GE | Second Generation | 100GE LAN, LAN PP* | CFP |
| A9K‐1x100GE | Second Generation | 100GE LAN, LAN PP* | CFP |
* LAN PP – LAN Proactive Protection – “Cisco Proprietary”
Optics OIR
After removing Optics, it is recommended to wait for 10 seconds for Software cleanup to complete before inserting another Optics. Also, after Optics insertion, it is recommended to wait for 10 seconds before removing it. After Optics insertion, software waits for power to stabilize before applying configuration. If the above recommendation is not followed, then it may affect the functionality of Optics.
Third Party Optics
ASR9000 supports only Cisco Optics listed in the following data-sheet.
https://tmgmatrix.cisco.com/?npf=83
Third Party Optics support policy is discussed here:
https://community.cisco.com/t5/service-providers-blogs/asr9000-policy-3rd-party-optics/ba-p/3106940
Product Warranty:
http://www.cisco.com/en/US/prod/prod_warranty09186a00800b5594.html
Transceiver permit pid all
ASR 9000 supported Optics has undergone extensive thermal, electrical and optical testing during qualification. Cisco policy is to support only optics mentioned in the datasheet. When a new Optics is inserted, Software reads Optics EEPROM and performs Security, Type and Product ID (PID) verification. If the inserted Optics is not a supported one, then Optics Transmit Laser will be turned off. This will cause Loss‐;of‐;signal (LOS) and Local Fault (LF) at far‐;end port. As an override for PID check for a given port, the following CLI can be used. But, it is to be noted that there is no guarantee that an unsupported Optics will work even after configuring ‘transceiver permit pid all’ CLI. An unsupported optics may also damage the Hardware since it may violate thermal and electrical specification of the Line Card. Cisco recommends only to use ASR 9000 supported Optics.
PID Override Command
PID Override command: RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-‐if)#transceiver permit pid all RP/0/RSP0/CPU0:ios(config-‐if)#commit
To verify if you are running into this scenario use the command
RP/0/RSP0/CPU0:A9K-BNG#show controller g 0/0/0/0 internal Port Number : 0 Board Type : 0x00000235 Port Type : 1GE Transport mode : LAN BIA MAC addr : 6666.4444.2222 Oper. MAC addr : 6666.4444.2222 Egress MAC addr : 6666.4444.2222 Port Available : true Status polling is : enabled Status events are : enabled I/F Handle : 0x040000c0 Cfg Link Enabled : tx/rx enabled H/W Tx Enable : yes MTU : 1526 H/W Speed : 1 Gbps H/W Duplex : Full H/W Loopback Type : None H/W FlowCtrl Type : None H/W AutoNeg Enable : Off Link Faults : (0x00000000) none Raw Link Faults : (0x00200000) XCVR_PID_UNSUP Link Up : yes Link Led Status : Link up -- Green/Amber UDLF enabled : no UDLF force event : no UDLF uni_dir enabled: no Pluggable Present : yes Pluggable Type : 1000BASE-SX Pluggable PID : Pluggable Compl. : (Service Un) - Compliant Pluggable Type Supp.: (Service Un) - Supported Pluggable PID Supp. : (Permit All) - Not supported
In this example we still do have an unsupported optic, but we override the check and the optic is brought up.
The PFM (platform Fault Manager) will still report the error however:
RP/0/RSP0/CPU0:A9K-BNG#show pfm location 0/0/cPU0 CURRENT TIME: Jun 13 10:37:30 2013 PFM TOTAL: 6 EMERGENCY/ALERT(E/A): 0 CRITICAL(CR): 6 ERROR(ER): 0 ------------------------------------------------------------------------------- Raised Time |S#|Fault Name |Sev|Proc_ID|Dev/Path Name |Handle --------------------+--+-----------------+---+-------+--------------+---------- Jun 13 02:08:47 2013|2 |DEV_SFP_PID_NOT_S|CR |180304 |SFP |0x1029000
Statistics
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/0 stats Statistics for interface GigabitEthernet0/5/0/0 (cached values): <output omitted>
Notes
- Packet input and output count doesn’t include counts for Preamble (7 bytes),SFD (1 byte), FCS (4 bytes).
- Input error giant: The total number of well-formed and valid packets that are dropped because they are larger than the configured MRU. This counter is similar to the RFC2819 definition of etherStatsOversizedPkts, but is based on MRU rather than a fixed size of 1518 octets.
- Runt: Any frame received lesser than 64 bytes is counted as ‘runt’ and dropped.
- Input MIB giant: The total number of well-formed and valid packets that are larger than 1518 bytes. (Regardless of whether the packets are dropped due to MRU consideration). This counter strictly matches the RFC2819 definition of etherStatsOversizePkts.
- Input error symbol: A Symbol error means the interface detects an undefined (invalid) Symbol received. Small amounts of symbol errors can be ignored. Large amounts of symbol errors can indicate a bad fiber or Optics.
- Input error jabbers: A jabber is a frame longer than 1518 octets (which exclude framing bits, but include FCS octets), which does not end with an even number of octets (alignment error) or has a bad FCS error.
MTU
In ASR 9000, the maximum allowed receive Ethernet frame size can’t exceed 9220 bytes which includes 4 bytes of CRC. If ‘mtu’ CLI is not configured, then default MTU of 1514 bytes is selected by software and internally 12 extra bytes are added to take care of QinQ header and CRC bytes. So, with default MTU configuration, any frame above 1526 (including CRC), will be dropped as ‘giants’. Maximum allowed user configurable MTU is 9216 bytes but in MAC hardware the maximum allowed is 9220 bytes. So, with ‘mtu 9216’ configuration, the maximum allowed receive frame size can’t exceed 9220 bytes (including 4 bytes of CRC).
NOTE
Starting in XR release 5.1.1 the ethernet interface driver configuration for MTU and MRU is determined by the XR config.
Prior to these changes MTU/MRU calculation was simply based on the configured MTU + 12 bytes for the addition of 2 Ethernet Tags and the CRC field as mentioned above. However from 511 onwards, the config determines the actual MTU/MRU.
- If no VLAN tags are configured:
MTU/MRU = configured MTU on interface + 4 CRC bytes, i.e. 1514+4=1518 - If one VLAN is configured:
MTU/MRU = configured MTU + 8 bytes (1 tag + CRC) - If two VLAN tags are configured:
MTU/RMU = configured MTU + 12 bytes (2 tags + CRC)
MTU configuration command: RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#mtu 9216 RP/0/RSP0/CPU0:ios(config-if)#commit
Link Defects
A Port link can go down for several reasons. The following CLI may be used to find out the reason for it.
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/0 internal
Port Number : 0
Board Type : 0x00000235
Port Type : 1GE
Transport mode : LAN
BIA MAC addr : 0022.bde2.b178
Oper. MAC addr : 0022.bde2.b178
Egress MAC addr : 0022.bde2.b178
Port Available : true
Status polling is : enabled
Status events are : enabled
I/F Handle : 0x0e000240
Cfg Link Enabled : tx/rx enabled
H/W Tx Enable : no
MTU : 1526
H/W Speed : 1 Gbps
H/W Duplex : Full
H/W Loopback Type : None
H/W FlowCtrl Type : None
H/W AutoNeg Enable : Off
H/W Link Defects : (0x0000026a) RX_LOS HW_LINK LASI NO_OPTICS TX_FAULT
| Defect | Cause/Meaning | Defect Source | Effect | Recovery |
| NO_OPTICS | Optics is not present | Optics | Port is receiving LOS. Link Down. | Insert an Optics |
| TX_FAULT | Optics is reporting TX Fault. Optics TX Laser is not healthy. | Optics | Port Link is down. Far-‐end may be down due to LOS. | Try port ‘shut’ followed by ‘no shut’. Try Optics OIR. If it doesn’t solve the issue, replace the Optics. |
| MOD_NOT_RDY | Optics is reporting Module Not Ready. Optics TX laser or PHY clocking issue. | Optics | Port Link is down. Far‐end may go down due to RFI. | Try port ‘shut’ followed by ‘no shut’. Try Optics OIR. If it doesn’t solve the issue, replace the Optics. |
| RX_LOS | Optics is receiving Loss-‐of-‐ Signal. This will cause MAC Local Fault (LF) and PHY LASI. | Optics | Port Link is down. Far-end may go down due to RFI. | Check the fiber; check whether far-end port is ‘no shut’. |
| XCVR_SECURITY | Not a Cisco supported Optics. Security check failed. | Optics | Port Link is down. Optics Transmit Laser is disabled. Far-end will see LOS. | Try Optics OIR. Use ASR9k supported Optics. If it is 3rd party Optics, you may try ‘service unsupported’. |
| SFP_HW_LINK | Copper/100FX Transceiver PHY is reporting Link Down | Optics | Port Link is down. Far-end may be down too. | Try Optics OIR. Check cable/fiber. Ensure both ends has matching autoneg, speed, flowcontrol commands. For 100FX, configure ‘no autoneg’ and ‘speed 100’ on both sides. |
| XCVR_PID_UNSUP | Optics Product ID is not supported for ASR9k. | Optics | Port Link is down. Optics TX laser is disabled. Far‐end will see LOS. | Try Optics OIR. If it is a 3rd party Optics, try port ‘transceiver permit pid all’ command. |
| XCVR_TYPE_UNSUP | Not a correct Optics type for this port. Instead of SFP+, a SFP is inserted in a 10GE port. | Optics | Port Link is down. Optics TX laser may be disabled. Far‐end may see LOS. | Check and replace the Optics. If it is a 3rd party Optics, try ‘service unsupported’ |
| UNSUP_CFG | User has configured an unsupported command. For example, ‘autoneg’ is configured for 100FX or ‘speed’ is configured for 1GE Optical SFP | Optics | Port Link is down. Optics TX laser is disabled | Remove the unsupported port command. |
| LASI |
PHY TX/RX link is DOWN (because of RX_LOS/PCS/PMA layers DOWN). If it is a 10GE/40GE port, then 'sh controllers tenGigE <> xgxs' provides additional information
|
PHY | Port Link is disabled. If MAC is seeing HW_LINK defect, then far-end may get RFI. | Try Optics OIR, shut/no shut. Check health of fiber and Optics. If the Optics is DOM supported, ‘sh controller <> phy’ will provide TX/RX Power information. Also, check port configuration at both ends. |
| HW_LINK | MAC is seeing local fault. MAC RX link is DOWN (because of RX_LOS/PCS/PMA layers DOWN) | MAC | Port Link is down. Far-end may be get Remote Fault. | Try Optics OIR and shut/no shut. Check health of fiber and Optics. If the Optics is DOM supported, ‘sh controller <> phy’ will provide TX/RX Power information. Also, check port configuration at both ends. |
| RFI | MAC is seeing Remote fault. Far-‐end is seeing Local Fault. | MAC | Port Link is down. Far-end may be getting Local Fault. | Try shut/no shut and Optics OIR. Problem may be with Optics Transmit or TX fiber. Check whether near-‐ end has enabled TX laser. Use ‘sh controller <> {internal | phy}. |
| AUTO_NEG | Specific to 1GE port. Auto- negotiation is not complete | MAC/PHY | Port Link is down. | Try shut/no shut and Optics OIR. Check the health of Optics, Fiber. Check port configuration both sides. |
| FIA_INIT | Fabric is not ready to receive (some initialization issue). | Fabric | All ports are down. | Line Card reload may recover. |
| FIA_SHUTDOWN | Fabric is shutdown | Fabric | All ports are down. | Line Card reload may recover |
| G709_LICENSE | No G.709 license is available | G.709 Software License | Warning message | License has expired or not available. Buy, new license. |
| ADV_OPT_PIE | Advanced Optics Pie is not installed. | G.709 Software License | Port transmit is disabled. Port Link is down. | Install Advanced Optic Pie |
| DWDM_LASER_SHUT | Port is in OTN mode but TX laser is disabled by Software | OTN | Port transmit is disabled. Port Link is down. | Configure ‘admin-‐state in-‐service’ under ‘controller dwdm <>’. |
| WANPHY_SD_SF_BER | Link is down because of Signal Fail (SF) alarm from WAN PHY. If the rate of B2 BIP error is above the SF configured threshold, the above defect is raised | WAN PHY | Port transmit is disabled. Port Link is down. | Use ‘sh controller wanphy <>’ to check B2 BIP error counts. If it is increasing, then check whether far-end is configured for WAN PHY mode. Check the health of Optics and Fiber. |
| MODE_CHANGE | Port transmit is disabled momentarily during mode change. (lan/wanphy/otn changed using 'transport-mode <>'under 'interface tengig<>') | PHY | Port Link is down. Far-end may flap due to clock configuration change. | Internal software change and reconfig of PHY and clocking logic. Will clear after few minutes. |
packet-gap non-standard
By default, Ethernet port operates at default Inter-frame-gap (IFG) of 12 bytes.
If the Far-end system needs a non-standard IFG, then the following CLI can be used to change IFG to 16 bytes. This CLI is supported only for10GE port.
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#packet-gap non-standard RP/0/RSP0/CPU0:ios(config-if)#commit
small-frame-padding
The following CLI allows ASR 9000 to pad Transmit/Egress Frames smaller than 68-bytes to 68 bytes (including CRC). This CLI is supported only for Second Generation Line Cards starting from 4.3.1 IOS XR Release.
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#small-frame-padding RP/0/RSP0/CPU0:ios(config-if)#commit
loopback
ASR 9000 supports the following loopback types for debugging purpose.
For 1GE/100GE, the loopback takes place at MAC whereas for 10GE/40GE, it is done at PHY.
RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#loopback ? external Enable external loopback (requires loopback connector) internal Enable internal loopback line Enable line loopback
Loopback external (Applicable only to RJ45 SFP)
Configure ‘loopback external’, when a port is operating at 1000 Mbps Speed using an external loopback plug. With speed 10 Mbps or 100 Mbps, external loopback plug will work fine without the below configuration.
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#loopback external RP/0/RSP0/CPU0:ios(config-if)#commit
carrier-delay
The interface configuration command ‘carrier-delay’ is used to delay the processing of hardware link down/up notifications. By default, without any ‘carrier-delay’ configuration, the hardware allows faster link-down notification for signal failure condition such as LOS. To allow soaking time against noise causing link flap, ‘carrier-delay {up | down) <msec>’ may be used. For some Line Cards, the Link UP may take several seconds since clock stabilization occurs when the receive signal state changes from LOS to normal.
RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#carrier-delay ? down Set the carrier delay down value up Set the carrier delay up value RP/0/RSP0/CPU0:ios(config-if)#carrier-delay down ? <0-2147483647> Delay in milliseconds
Optic Pie
To use the following features, ‘asr9k-optic-px.pie’ needs to be installed. Without Optic Pie, the link is brought down and also Transmit Laser is disabled.
- OTN mode in 10G port
- OTN mode in 40G port
- LAN Proactive Protection in 100G port
Optics DOM
DOM feature is supported only on Optics that provides this functionality (Ref. 5). Software periodically collects the following sensor data from supported Optics.
- Transceiver Temperature
- Transceiver power supply voltage
- Transceiver Bias Current
- Transceiver Transmit Power
- Transceiver Receive Power
The alarms and warnings available for each of the above sensors are read from Optics EEPROM.
- High Alarm
- Low Alarm
- High Warnin
- Low Warning
Line Card sends a message to RSP which in turn will raise a CISCO-ENTITY-SENSOR-MIB SNMP trap for the corresponding alarm or warning. Alarms are of major severity while warnings are of minor severity for MIB. The following CLI can be used to show DOM data.
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/1 phy
< snip>
Temperature: 40.055
Voltage: 3.265 Volt
Tx Bias: 9.464 mAmps
Tx Power: 0.344 mW (-4.64 dBm)
Rx Power: 0.260 mW (-5.85 dBm)
Oper. Status/Control: Rx Rate Select,
1GE Interface
LAN Mode
This is the default mode for a 1GE port. The following CLIs can be used to check the status of a given port.
RP/0/RSP0/CPU0:ios#sh int gigabitEthernet 0/5/0/39 GigabitEthernet0/5/0/39 is up, line protocol is up RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/39 internal Port Number : 39 Port Type : GE Transport mode : LAN BIA MAC addr : 001d.e5eb.8d17 Oper. MAC addr : 001d.e5eb.8d17 Egress MAC addr : 001d.e5eb.8d17 Port Available : true Status polling is : enabled Status events are : enabled I/F Handle : 0x0e000a80 Cfg Link Enabled : tx/rx enabled H/W Tx Enable : yes <<<< Optics Laser is Enabled UDLF enabled : no SFP PWR DN Reason : 0x00000000 SFP Capability : 0x000000e7 MTU : 1526 H/W Speed : 100 Mbps H/W Duplex : Full H/W Loopback Type : None H/W FlowCtrl type : None H/W AutoNeg Enable: Off H/W Link Defects : (0x00000000) None <<<< No Defects Link Up : yes <<<< Port Link is Up Link Led Status : Link up -- Green/Amber Input good underflow : 0 Input ucast underflow : 0 Output ucast underflow : 0 Input unknown opcode underflow: 0 Pluggable Present : yes Pluggable Type : 1000BASE-T <<<< RJ45 SFP is present Pluggable Compl. : Compliant <<<< Passed Security Check Pluggable Type Supp.: Supported <<<< Passed Optics Type Check Pluggable PID Supp. : Supported <<<< Passed PID check Pluggable Scan Flg: false
Uni-directional link Fault
Uni-directional Link Fault detection (UDLF) is a feature of 802.3 OAM that allows a host that detects a fault on the RX channel of a link to notify the peer via an 802.3 OAM message, so that the peer can disable the interface. In 10/40/100GE, when a Near-end port receives LOS, MAC sends Remote Fault (RF) to Far-end so that port is brought down. But, in case of 1GE, RF is not defined as a standard. So, this will cause Far-end port to stay UP even when a Near-end port is DOWN. By enabling UDLF feature at both ends, the Near-end notifies Far-end with OAM message whenever the port link is down due to LOS.
Configuration to Enable UDLR:
interface GigabitEthernet0/6/0/2 ethernet oam uni-directional link-fault detection action uni-directional link-fault efd
Optics
Only ‘Full-duplex’ operation is supported since ‘Half-duplex’ capability is not available in MAC. It is to be noted that the default configuration is ‘no negotiation’ which means Auto-negotiation is turned off. Please enable ‘negotiation auto’ in 1GE port if auto-negotiation is enabled at Link Partner (Far-end) port.
Enable Auto-negotiation:
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#negotiation auto RP/0/RSP0/CPU0:ios(config-if)#commit
Disable Auto-negotiation:
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#no negotiation auto RP/0/RSP0/CPU0:ios(config-if)#commit
1GE Optical SFP
With 1GE fixed Speed Optical SFP, ‘negotiation auto’ and ‘no negotiation auto’ are supported, it’s mandatory that both Near and Far-ends have matching configuration.
But, it is recommended to use Auto-negotiation (AN) at both ends to ensure Speed, Flow-control (FC) and Master/Slave are negotiated and resolved.
| Auto‐negotiation | Near‐end (Config) | Far‐end (Config) | Near‐end (Result) |
| ON | None (Advertise Speed 1000, Full‐duplex) | None | 1000 Mbps (Resolved) |
| OFF | Speed 1000 | Speed 1000 | 1000 Mbps |
| Auto‐negotiation | Near‐end (Config) | Far‐end (Config) | Near‐end (Result) |
| ON | Ingress | None | None |
| ON | Bidirectional | None | None |
| ON | Bidirectional | Ingress | Bidirectional |
| ON | Ingress | Ingress | None |
| ON | Ingress | Bidirectional | Ingress |
Resolved Speed and Flow-control with Autoneg:
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/8 internal
Port Number : 8
Port Type : GE
Transport mode : LAN <<<< LAN Mode
BIA MAC addr : 001d.e5eb.8cf8
Oper. MAC addr : 001d.e5eb.8cf8
Egress MAC addr : 001d.e5eb.8cf8
Port Available : true
Status polling is : enabled
Status events are : enabled
I/F Handle : 0x0e0002c0
Cfg Link Enabled : tx/rx enabled
H/W Tx Enable : yes <<<< Optics TX Laser Enabled
UDLF enabled : no
SFP PWR DN Reason : 0x00000000
SFP Capability : 0x00000024
MTU : 1526
H/W Speed : 1 Gbps
H/W Duplex : Full
H/W Loopback Type : None
H/W FlowCtrl type : None
H/W AutoNeg Enable: On <<<< Autoneg Enabled
H/W AutoNeg Mask : None (0x0000)
H/W AutoNeg Speed : Unknown (0)
H/W AutoNeg Duplex: Invalid (0)
H/W AutoNeg FlowC : None (0)
Resolved AutoNeg Mask : None (0x0000) <<<< Autoneg Resolved State
Resolved AutoNeg Speed : 1 Gbps (3)
Resolved AutoNeg Duplex: Full (2)
Resolved AutoNeg FlowC : None (0)
Local Autoneg RFI : None <<<< Autoneg Local end configuration
Local Autoneg Pause : Asym (Ingress)
Local Autoneg Speed : 1Ge Full
Local Autoneg Speed Mask : 1Gb
Local Autoneg Duplex Mask : full
Link Partner Autoneg RFI : None <<<< Autoneg Far-end configuration
Link Partner Autoneg Pause : None
Link Partner Autoneg Speed : 1Ge Full
Link Partner Autoneg Speed Mask : 1Gb
Link Partner Autoneg Duplex Mask : full
H/W Link Defects : (0x00000000) None <<<< No Defects
Link Up : yes <<<< Link is UP
Link Led Status : Link up -- Green/Amber <<<< LED is Green
Input good underflow : 0
Input ucast underflow : 0
Output ucast underflow : 0
Input unknown opcode underflow: 0
Pluggable Present : yes
Pluggable Type : 1000BX-10-D
Pluggable Compl. : Compliant
Pluggable Type Supp.: Supported
Pluggable PID Supp. : Supported
RP/0/RSP0/CPU0:ios#sh int gigabitEthernet 0/5/0/8
GigabitEthernet0/5/0/8 is up, line protocol is up
Interface state transitions: 15
Hardware is GigabitEthernet, address is 001d.e5eb.8cf8 (bia 001d.e5eb.8cf8)
Internet address is Unknown
MTU 1514 bytes, BW 1000000 Kbit (Max: 1000000 Kbit)
reliability 255/255, txload 0/255, rxload 0/255
Encapsulation ARPA,
Full-duplex, 1000Mb/s, link type is autonegotiation <<<< Autoneg ON, Speed 1000 Mbps
output flow control is off, input flow control is off <<<< Flow-control OFF
loopback not set,
Last input 1d07h, output never
Last clearing of "show interface" counters never
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
6 packets input, 360 bytes, 2 total input drops
0 drops for unrecognized upper-level protocol
Received 6 broadcast packets, 0 multicast packets
0 runts, 0 giants, 0 throttles, 0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 packets output, 0 bytes, 0 total output drops
Output 0 broadcast packets, 0 multicast packets
0 output errors, 0 underruns, 0 applique, 0 resets
0 output buffer failures, 0 output buffers swapped out
15 carrier transitions
Copper/RJ45 SFP
Copper SFP is a PHY based SFP that allows 10/100/1000 Mbps of Speed and Flow-control configuration whether Auto-negotiation is enabled are not. But, it is recommended to use Auto-negotiation (AN) at both ends to ensure Speed, Flow-control (FC) and Master/Slave are negotiated and resolved.
| Auto-negotiation (Both Sides) | Near-end (Config) | Far-end (Config) | Near-end (Result) |
| ON | None (Advertise Speed 1000/100/10, Full-duplex) | None | 1000 |
| ON | None (Advertise Speed 1000/100/10, Full-duplex) | Speed 1000 | 1000 |
| ON | None (Advertise Speed 1000/100/10, Full-duplex) | Speed 100 | 100 |
| ON | None (Advertise Speed 1000/100/10, Full-duplex) | Speed 10 | 10 |
| ON | Speed 1000 | None | 1000 |
| ON | Speed 100 | None | 100 |
| ON | Speed 10 | None | 10 |
| OFF | Speed 1000 | Speed 1000 | 1000 |
| OFF | Speed 100 | Speed 100 | 100 |
| OFF | Speed 10 | Speed 10 | 10 |
Resolved Speed and Flow-control with Autonegotiation:
RP/0/RSP0/CPU0:IOS#sh int gigabitEthernet 0/5/0/34
GigabitEthernet0/5/0/34 is up, line protocol is up
Interface state transitions: 5
Hardware is GigabitEthernet, address is 001d.e5eb.8d12 (bia 001d.e5eb.8d12)
Internet address is Unknown
MTU 1514 bytes, BW 100000 Kbit (Max: 100000 Kbit)
reliability 255/255, txload 0/255, rxload 0/255
Encapsulation ARPA,
Full-duplex, 100Mb/s, THD, link type is autonegotiation <<<< Resolved Speed
output flow control is off, input flow control is on <<<< Resolved Flow-control
loopback not set,
Last input 00:00:00, output 00:00:00
Last clearing of "show interface" counters never
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
1875 packets input, 395625 bytes, 0 total input drops
0 drops for unrecognized upper-level protocol
Received 0 broadcast packets, 1875 multicast packets
0 runts, 0 giants, 0 throttles, 0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
1875 packets output, 395625 bytes, 0 total output drops
Output 0 broadcast packets, 1875 multicast packets
0 output errors, 0 underruns, 0 applique, 0 resets
0 output buffer failures, 0 output buffers swapped out
5 carrier transitions
100FX Optical SFP
GLC-GE-100FX is a PHY based Optical SFP. Auto-negotiation is not supported but ‘speed 100’ needs to be configured.
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int gigabitEthernet 0/5/0/0 RP/0/RSP0/CPU0:ios(config-if)#no negotiation auto RP/0/RSP0/CPU0:ios(config-if)#commit RP/0/RSP0/CPU0:ios(config-if)#speed 100 RP/0/RSP0/CPU0:ios(config-if)#commit
CWDM/DWDM SFP
1GE port supports several CWDM and DWDM Optics (Ref. 3).
GLC-BX10-U/D SFP
GLC-BX-D and GLC-BX-U Optics (Ref. 10) operate with single strand using Single Mode Fiber (SMF). GLC-BX-D transmits 1490-nm and receives 1310-nm wavelength, whereas GLC-BX-U receives 1490-nm and transmits 1310-nm wavelength signal. Bi-directional communication is achieved with integrated Wavelength-division multiplexing (WDM) splitter inside the optics. This optics also provides Digital optical monitoring (DOM) support. No special configuration is needed to operate this SFP.
RP/0/RSP0/CPU0:ios#sh running-config interface gigabitEthernet 0/5/0/8
interface GigabitEthernet0/5/0/8
negotiation auto
RP/0/RSP0/CPU0:ios#sh running-config interface gigabitEthernet 0/5/0/9
interface GigabitEthernet0/5/0/9
negotiation auto
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/8 internal
<snip>
Pluggable Type : 1000BX-10-D
Pluggable Compl. : Compliant
Pluggable Type Supp.: Supported
Pluggable PID Supp. : Supported
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/9 internal
<snip>
Pluggable Type : 1000BX-10-U
Pluggable Compl. : Compliant
Pluggable Type Supp.: Supported
Pluggable PID Supp. : Supported
RP/0/RSP0/CPU0:ios#sh controllers gigabitEthernet 0/5/0/8 phy
SFP EEPROM port:8
Xcvr Type: SFP
Xcvr Code: 1000BX-10-D
Encoding: 8B10B
Bit Rate: 1300 Mbps
Link Reach 9u fiber (Km): 10000 meter
Link Reach 9u fiber (100m): 10000 meter
Vendor Name: CISCO-FIBERXON
Vendor OUI: 00.1f.22
Vendor Part Number: 9912C-SL10G-CIC (rev.: 10 )
Laser wavelength: 1490 nm (fraction: 0.00 nm)
<snip> ** DOM Data **
Temperature: 37.719
Voltage: 3.265 Volt
Tx Bias: 17.104 mAmps
Tx Power: 0.311 mW (-5.07 dBm)
Rx Power: 0.352 mW (-4.54 dBm)
Oper. Status/Control:
10GE Interface
LAN Mode
This is default mode of operation.
WAN PHY Mode
PHY device maps 10GE LAN streams to STS-192c SPE and this limits the data-rate to 9.584 Gbps. WAN PHY provides rich SONET OAM capabilities for link management. It is to be noted that WAN PHY line-rate is lower than 10GE LAN rate (10.3125 Gbps). When WAN PHY mode is configured, transmit Inter-frame-gap (IFG) is set to 16 bytes instead of default 12 bytes in MAC.
LAN to WAN PHY Configuration
RP/0/RSP0/CPU0:ios#sh running-config interface tenGigE 0/6/0/0 interface TenGigE0/6/0/0 transport-mode wan RP/0/RSP0/CPU0:IOS#sh running-config controller wanphy 0/6/0/0 controller wanphy0/6/0/0 wanmode on
WAN PHY to LAN Mode
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#no transport-mode RP/0/RSP0/CPU0:ios(config-if)#commit
WAN PHY Alarm Status
RP/0/RSP0/CPU0:IOS#sh controllers wanphy 0/6/0/0 alarms Interface: wanphy0/6/0/0 Configuration Mode: WAN Mode SECTION LOF = 2, LOS = 2, BIP(B1) = 0 LINE AIS = 0, RDI = 0, FEBE = 0, BIP(B2) = 0 PATH AIS = 0, RDI = 0, FEBE = 0, BIP(B3) = 0 LOP = 0, NEWPTR = 0, PSE = 0, NSE = 0 WIS ALARMS SER = 263414, FELCDP = 0, FEAISP = 0 WLOS = 2, PLCD = 2 LFEBIP = 0, PBEC = 0, PLMP = 0 Active Alarms[All defects]: los, lof, sef, Active Alarms[Highest Alarms]: los Rx(K1/K2): N/A, Tx(K1/K2): N/A S1S0 = N/A, C2 = N/A PATH TRACE BUFFER Remote IP addr: BER thresholds: SF = 10e-3 SD = 10e-6 TCA thresholds: N/A Alarm reporting enabled for:los, lof, lop
Layered Alarm Status (Optics, PHY, MAC)
(Note: This CLI is applicable only for 10GE port in Second Generation Line Card)
RP/0/RSP0/CPU0:ios#sh controllers tenGigE 0/6/0/0 xgxs
OPTICS:
Rx LOS Fault : Yes
Tx Fault : No
PHY:
Tx Align Fault : No
PMA/PMD:
Rx LOS Fault : Yes
Rx PMA link Fault : Yes
WANPHY Mode:
SECTION:
Los: Yes, Lof: Yes
LINE:
Ais: No, Rdi: No
PATH:
Ais: No, Lop: No
Plm: No, Lcd: No
PCS:
Rx PCS link Fault : Yes
Rx PCS lock Fault : Yes
MAC:
Rx Local Fault : Yes
Rx Remote Fault : No
SD/SF BER
Software sets Signal Degrade (SD) and Signal Fail (SF) thresholds to 10e-3 and 10e-6 respectively.
RP/0/RSP0/CPU0:ios#sh controllers wanphy 0/6/0/0 alarms | inc SD BER thresholds: SF = 10e-3 SD = 10e-6
SD/SF BER threshold is configurable using ‘threshold’ command.
Software monitors B2 BIP errors and asserts SD/SF BER alarms whenever the error-rate exceeds the threshold. Port link will go down once BIP error-rate exceeds SF BER threshold.
Threshold configuration:
RP/0/RSP0/CPU0:ios(config)#controller wanphy 0/6/0/0 RP/0/RSP0/CPU0:ios(config-wanphy)#threshold ? sd-ber Set Signal Degrade BER threshold sf-ber Set Signal Fail BER threshold
Alarm reporting configuration
RP/0/RSP0/CPU0:ios(config)#controller wanphy 0/6/0/0 RP/0/RSP0/CPU0:ios(config-wanphy)#report ? line-ais Set Line Alarm Indication Signal reporting status lof Set Loss of Frame reporting status lop Set Loss Of Pointer reporting status los Set Loss of Signal reporting status path-ais Set Path Alarm Indication Signal reporting status path-fe-ais Set Path Far End Alarm Indication Signal reporting status path-fe-plm Set Path Far End Label Mismatch reporting status path-lcd Set Loss Of Code-Group Delineation reporting status path-plm Set Path Payload Label Mismatch reporting status path-rdi Set Path Remote Defect Indication Signal reporting status rdi Set Remote Defect Indicator reporting status sd-ber Set LBIP BER in excess of SD threshold reporting status sf-ber Set LBIP BER in excess of SF threshold reporting status
Also, link will go down if any of the following alarm is asserted.
LOS, LOF, AIS-L, AIS-P, LOP-P, UNEQ-P.
OTN Mode
10GE LAN traffic is mapped to OPU2e or OPU1e containers. OTN provides Forward Error Correction (FEC) feature in addition to SONET like OAM capabilities.
asr9k-optic-px.pie is needed to operate in OTN mode.
Optics Transmit Laser will be disabled if Pie is not installed and activated.
LAN to OPU2e Configuration:
RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#transport-mode otn bit-transparent ? opu1e 10GBASE-R over OPU1e without fixed stuffing (11.0491Gb/s) opu2e 10GBASE-R over OPU2e with fixed stuffing (11.0957Gb/s) RP/0/RSP0/CPU0:ios#sh running-config interface tenGigE 0/6/0/0 interface TenGigE0/6/0/0 transport-mode otn bit-transparent opu2e RP/0/RSP0/CPU0:ios#sh running-config controller dwdm 0/6/0/0 controller dwdm0/6/0/0 admin-state in-service
OPU2e to LAN mode
RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#no transport-mode RP/0/RSP0/CPU0:ios(config-if)#commit
OTN Alarm Status
RP/0/RSP0/CPU0:ios#sh controllers dwdm 0/6/0/0 g709
OTU
LOS = 1 LOF = 0 LOM = 0
BDI = 0 IAE = 0 BIP = 0
BEI = 0 TIM = 0
ODU
AIS = 0 BDI = 0 OCI = 0
LCK = 0 BIP = 0 BEI = 0
PTIM = 0 TIM = 0
FEC Mode: Standard (Default)
EC(current second) = 0 EC = 0 UC = 0
pre-FEC BER < 9.34E-11 Q > 6.45 Q Margin > 6.09
Remote FEC Mode: Unknown
FECMISMATCH = 0
Detected Alarms: LOS
Asserted Alarms: LOS
Alarm Reporting Enabled for: LOS LOF LOM IAE OTU-BDI OTU-TIM OTU_SF_BER OTU_SD_BER ODU-AIS ODU-BDI OCI LCK PTIM ODU-TIM ODU_SF_BER ODU_SD_BER FECMISMATCH
BER Thresholds: OTU-SF = E-3 OTU-SD = E-6
ODU-SF = E-3 ODU-SD = E-6
Layered Alarm Status (Optics, PHY, MAC)
(Note: This CLI is applicable only to 10 port in Second Generation Line Card)
RP/0/RSP0/CPU0:ios#sh controllers tenGigE 0/6/0/0 xgxs
OPTICS:
Rx LOS Fault : Yes
Tx Fault : No
PHY:
Tx Align Fault : No
PMA/PMD:
Rx LOS Fault : Yes
Rx PMA link Fault : Yes
OTU2 Mode:
OTU:
Los: Yes, Oof: Yes, Lof: Yes
Lom: Yes, Oom: Yes
Ais: No, Iae: Yes
Bdi: Yes, Tim: No
ODU:
Bdi: Yes, Ais: No, Lck: No
Oci: No, Tim: No, Ptim: Yes
PCS:
Rx PCS link Fault : Yes
Rx PCS lock Fault : Yes
MAC:
Rx Local Fault : Yes
Rx Remote Fault : No
Supported FEC Modes
RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/6/0/0 RP/0/RSP0/CPU0:ios(config-dwdm)#g709 fec ? disable Disable FEC enhanced Enhanced FEC mode high-gain High Gain FEC mode long-haul Long Haul FEC mode standard Standard FEC mode RP/0/RSP0/CPU0:ios(config-dwdm)#g709 fec enhanced ? i.4 efec i.4 i.7 efec i.7
| Configuration | FEC Modes (Second Generation Line Card) | FEC Mode (First Generation Line Card) |
| g709 fec standard | ITU G.975 (GFEC) default | ITU G.975 (GFEC) default |
| g709 fec enhanced i.4 | ITU G.975, I.4 (EFEC) | ITU G.975, I.4 (EFEC) |
| g709 fec enhanced i.7 | ITU G.975, I.7 (UFEC) | Not Supported |
| g709 disable | No FEC | No FEC |
| g709 fec high-‐gain | Not Supported | Not Supported |
| g709 fec long-‐haul | Not Supported | Not Supported |
Loopback
When a port is in OTN mode, loopback is configured under ‘controller dwdm’.
RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/6/0/0 RP/0/RSP0/CPU0:ios(config-‐dwdm)#loopback ? internal Select internal loopback mode line Select line loopback mode
Performance Monitoring (PM) configuration
RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/6/0/0 RP/0/RSP0/CPU0:ios(config-‐dwdm)#pm ? 15-‐min Configure pm parameters of 15 minute interval 24-‐hour Configure pm parameters of 24 hour interval RP/0/RSP0/CPU0:ios(config-‐dwdm)#pm 15‐min ?
fec Configure fec g709 performance monitoring optics Configure optics performance monitoring otn Configure otn g709 performance monitoring RP/0/RSP0/CPU0:ios (config-‐dwdm)#pm 15‐min fec ? report set fec TCA reporting status
threshold Configure fec threshold
Show Performance Monitoring Information
RP/0/RSP0/CPU0:ios#sh controllers dwdm 0/6/0/0 pm interval ? 15-min Show pm data in 15 minute interval 24-hour show pm data in 24 hour interval RP/0/RSP0/CPU0:ios#sh controllers dwdm 0/6/0/0 pm history ? 15-min show all pm data in 15 minute interval 24-hour show all pm data in 24 hour interval fec show all fec data in 15 minute and 24 hour interval optics show all optics data in 15 minute and 24 hour interval otn show all otn data in 15 minute and 24 hour interval | Output Modifiers <cr>
Proactive Protection
With ‘Proactive Protection’ feature enabled, software monitors FEC errors for the configured threshold and notifies upper-layer protocols once the error-rate crosses the threshold.
RP/0/RSP0/CPU0:ios#conf t RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/6/0/0 RP/0/RSP0/CPU0:ios(config-dwdm)#admin-state out-of-service RP/0/RSP0/CPU0:ios(config-dwdm)#commit RP/0/RSP0/CPU0:ios(config-dwdm)#proactive RP/0/RSP0/CPU0:ios(config-dwdm)#commit RP/0/RSP0/CPU0:ios(config-dwdm)#admin-state in-service RP/0/RSP0/CPU0:ios(config-dwdm)#commit
BER threshold
RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/6/0/0
RP/0/RSP0/CPU0:ios(config-dwdm)#proactive ?
revert Configure Revert thresholds
trigger Configure trigger thresholds
<cr>
RP/0/RSP0/CPU0:ios#sh controllers dwdm 0/6/0/0 proactive
Proactive Protection Status: ON
Proactive Protection State: Normal -Interface is Up
Inputs affecting proactive protection state:
Transport admin state : In Service
Trigger threshold : 6E-5 (Default 6E-5)
Revert threshold : 1E-5 (Default 1E-5)
Trigger integration window : 30 ms
Revert integration window : 20000 ms
Received APS : 0x0f (No Request)
Transmitted APS : 0x0f (No Request)
SD/SF BER
Signal Degrade (SD) and Signal Failure (SF) threshold can be configured in OTU and ODU layers. Software monitors BIP errors from OTU and ODU layers for raising Alarms. Port link will go down for if BIP error rate exceeds SF threshold.
The default BER thresholds are given below.
BER Thresholds: OTU-SF = E-3 OTU-SD = E-6
ODU-SF = E-3 ODU-SD = E-6
SD/SF BER threshold configuration
RP/0/RSP0/CPU0:ios(config-dwdm)#g709 otu ? overhead Configure OTU overhead report Configure OTU alarm reporting threshold Configure OTU threshold value RP/0/RSP0/CPU0:ios(config-dwdm)#g709 otu threshold ? sd-ber Set Signal Degrade BER threshold sf-ber Set Signal Failure BER threshold RP/0/RSP0/CPU0:ios(config-dwdm)#g709 odu ? overhead Configure ODU overhead report Configure ODU alarm reporting threshold Configure ODU threshold value RP/0/RSP0/CPU0:ios(config-dwdm)#g709 odu threshold ? sd-ber Set Signal Degrade BER threshold sf-ber Set Signal Failure BER threshold
Unidirectional Link Routing
The Unidirectional Link Routing (UDLR) feature is currently supported only in 10GE LAN mode only. It allows a physical port to receive or transmit data with a single fiber.‘transport-mode {rx-only | tx-only}’ CLI is used for this purpose.
RP/0/RSP0/CPU0:ios(config)#int tenGigE 0/6/0/0 RP/0/RSP0/CPU0:ios(config-if)#transport-mode ? rx-only 10GE UDLR Mode, Receive Only tx-only 10GE UDLR Mode, Transmit Only
Optics
SR/LR/ER/ZR/DWDM Optics
Ref. 3 for all ASR 9000 supported XFP and SFP+ Optics.
Tunable Optics
Ref. 9 provides the specification for Tunable DWDM XFP. This Optics allows 80 DWDM ITU-50GHz channels for tuning.
ASR 9000 follows CRS-1 for ‘wavelength’ to Channel mapping. Please use ‘sh controller dwdm <> wavelength-map’ CLI to see the correct channel to wavelength mapping.
RP/0/RSP0/CPU0:ios#show controllers dwdm 0/6/0/7 wavelength-map
Wavelength band: C-Band
MSA ITU channel range supported: 3~84
Wavelength map table
----------------------------------------------------
ITU Ch G.694.1 Frequency Wavelength
Num Ch Num (THz) (nm)
----------------------------------------------------
3 58 196.00 1529.553
----------------------------------------------------
4 57 195.95 1529.944
----------------------------------------------------
5 56 195.90 1530.334
----------------------------------------------------
6 55 195.85 1530.725
----------------------------------------------------
7 54 195.80 1531.116
<snip>
----------------------------------------------------
80 -19 192.15 1560.200
----------------------------------------------------
81 -20 192.10 1560.606
----------------------------------------------------
82 -21 192.05 1561.013
----------------------------------------------------
83 -22 192.00 1561.419
----------------------------------------------------
84 -23 191.95 1561.826
----------------------------------------------------
To get 1531.116 nm, ‘Wavelength 7’ will be configured.
RP/0/RSP0/CPU0:IOS(config)#controller dwdm 0/6/0/7 RP/0/RSP0/CPU0:IOS(config-dwdm)#wavelength 7 RP/0/RSP0/CPU0:IOS(config-dwdm)#commit
Instead of Channel Number, desired frequency can be configured using ‘wavelength frequency <>’ command.
RP/0/RSP0/CPU0:IOS(config-dwdm)#wavelength frequency ? <19115-19610> Enter 5-digit Frequency data; e.g. 19580 for 195.8 THz
40GE Interface
40GE Modular Port Adapter (MPA) implements 40GE LAN and 40GE mapped onto OTU3 (43.0184 Gbps) modes. It also supports G.709 OTN OTU3 framing with triple FEC (GFEC (Reed-Solomon), EFEC (G.975.1, I.4) and UFEC (G.975.1, 1.7)).
LAN Mode
This is the default mode for the MPA.
RP/0/RSP0/CPU0:IOS#sh int fortyGigE 0/0/0/0 FortyGigE0/0/0/0 is up, line protocol is up Interface state transitions: 1 Hardware is FortyGigE, address is 0022.bde5.aaec (bia 0022.bde5.aaec) Layer 1 Transport Mode is LAN
OTN Mode
To change Transport mode from LAN to OTU3 (43.0184 Gbps), use ‘g709 enable’ CLI.
To revert back to LAN mode, ‘no g709 enable’ needs to be configured.
asr9k-optic-px.pie is needed to operate in OTN mode.
Optics Transmit Laser will be disabled if PIE is not installed and activated.
If a port is in OTU3 mode, then ‘admin-state out-of-service’ configuration is needed before moving to any other command. G.709 Optics pie and License are needed to support OTU3 mode.
Change from LAN to OTU3
RP/0/RSP0/CPU0:IOS(config)#controller dwdm 0/0/0/0 RP/0/RSP0/CPU0:IOS(config-dwdm)#g709 enable RP/0/RSP0/CPU0:IOS(config-dwdm)#commit RP/0/RSP0/CPU0:IOS(config-dwdm)#admin-state in-service RP/0/RSP0/CPU0:IOS(config-dwdm)#commit
Change from OTU3 to LAN
RP/0/RSP0/CPU0:IOS(config)#controller dwdm 0/0/0/0 RP/0/RSP0/CPU0:IOS(config-dwdm)#admin-state out-of-service RP/0/RSP0/CPU0:IOS(config-dwdm)#commit RP/0/RSP0/CPU0:IOS(config-dwdm)#no g709 enable RP/0/RSP0/CPU0:IOS(config-dwdm)#commit
RP/0/RSP0/CPU0:IOS#sh controllers fortyGigE 0/0/0/0 internal Tue Apr 30 00:09:44.419 UTC Port Number : 0 Board Type : 0x0000023a Port Type : 40GE Transport mode : OTN(opu3) BIA MAC addr : 0022.bde5.aaec Oper. MAC addr : 0022.bde5.aaec Egress MAC addr : 0022.bde5.aaec Port Available : true Status polling is : enabled Status events are : enabled I/F Handle : 0x000001c0 Cfg Link Enabled : tx/rx enabled H/W Tx Enable : no MTU : 1526 H/W Speed : 40 Gbps H/W Duplex : Full H/W Loopback Type : None H/W FlowCtrl Type : None H/W AutoNeg Enable : Off H/W Link Defects : (0x0000202a) RX_LOS HW_LINK LASI G709_LICENSE H/W Raw Link Defects : (0x0000202a) RX_LOS HW_LINK LASI G709_LICENSE <snip>
Alarm Status
RP/0/RSP0/CPU0:IOS#sh controllers dwdm 0/0/0/0
Tue Apr 30 09:29:16.745 UTC
Port dwdm0/0/0/0
Controller State: Up
Transport Admin State: In Service
Loopback: None
G709 Status
OTU
LOS = 1 LOF = 0 LOM = 0
BDI = 0 IAE = 0 BIP = 0
BEI = 0 TIM = 0
ODU
AIS = 0 BDI = 0 OCI = 0
LCK = 0 BIP = 0 BEI = 0
PTIM = 0 TIM = 0
FEC Mode: Standard (Default)
EC(current second) = 0 EC = 0 UC = 0
pre-FEC BER < 2.32E-11 Q > 6.66 Q Margin > 6.37
Remote FEC Mode: Standard
FECMISMATCH = 0
Detected Alarms: None
Asserted Alarms: None
Alarm Reporting Enabled for: LOS LOF LOM IAE OTU-BDI OTU-TIM OTU_SF_BER OTU_SD_BER ODU-AIS ODU-BDI OCI LCK PTIM OD
U-TIM ODU_SF_BER ODU_SD_BER FECMISMATCH
BER Thresholds: OTU-SF = E-3 OTU-SD = E-6
ODU-SF = E-3 ODU-SD = E-6
Connectivity Info
Network Port ID: Not Configured
Network Connection ID: Not Configured
OTU TTI Sent String ASCII : Tx TTI Not Configured
OTU TTI Received String ASCII : Rx TTI Not Received
OTU TTI Expected String ASCII : Exp TTI Not Configured
ODU TTI Sent String ASCII : Tx TTI Not Configured
Layered Alarm Status (Optics, PHY, MAC)
RP/0/RSP0/CPU0:ios#sh controllers fortyGigE 0/0/0/0 xgxs
OPTICS:
Rx LOS Fault : No
Tx Fault : No
PHY:
Tx Align Fault : No
PMA/PMD:
Rx LOS Fault : No
Rx PMA link Fault : No
OTU3 Mode:
OTL:
Otl_lol: No, Otl_ola: No, Otl_ola_d: No
Otl_skew_oor: No, Otl_skew_oor_d: No
OTU:
Los: No, Oof: No, Lof: No
Lom: No, Oom: No
Ais: No, Iae: No
Bdi: No, Tim: No
ODU:
Bdi: No, Ais: No, Lck: No
Oci: No, Tim: No, Ptim: No
GMP:
Oof_1027b: No, Oof_1027_d: No
Dec_513b_e: No, Sync_fail: No
PCS:
Rx PCS link Fault : No
Rx PCS lock Fault : No
MAC:
Rx Local Fault : No
Rx Remote Fault : No
Supported FEC Modes
RP/0/RSP0/CPU0:ios(config)#controller dwdm 0/0/0/0 RP/0/RSP0/CPU0:ios(config-dwdm)#g709 fec ? disable Disable FEC enhanced Enhanced FEC mode high-gain High Gain FEC mode long-haul Long Haul FEC mode standard Standard FEC mode RP/0/RSP0/CPU0:ios(config-dwdm)#g709 fec enhanced ? i.4 efec i.4 i.7 efec i.7
| Configuration | FEC Mode |
| g709 fec standard | G.975.1 (GFEC) default |
| g709 fec enhanced i.4 | G.975.1, I.4 (EFEC) |
| g709 fec enhanced i.7 | G.975.1, I.7 (UFEC) |
| g709 fec disable | No FEC |
| g709 fec high-gain | Not Supported |
| g709 fec long‐haul | Not Supported |
Loopback
When a port is in OTN mode, loopback is configured under ‘controller dwdm’. as in 10GE ports
Performance Monitoring (PM) configuration
Same as in 10GE ports
Show Performance Monitoring Information
Same as in 10GE ports
Proactive Protection
Same as in 10GE ports
BER threshold
Same as in 10GE ports
SD/SF BER
Same as in 10GE ports
Optics
40GE MPA supports the following Cisco QSFPs.
- QSFP-40G-SR4 - LAN mode only
- QSFP-40GE-LR4 - LAN mode only
- QSFP-40G-LR4 - LAN/OTU3 modes
100GE Interface
LAN Mode
LAN is the default mode for a 100GE port.
LAN PP Mode
In LAN Proactive Protection (PP) mode, ONS M6 terminates OTU4 signal and monitors Forward-error-correction (FEC) errors. Whenever the error-rate exceeds the user-configured threshold, M6 notifies ASR 9000 through a Cisco proprietary signal. This allows fast reroute (FRR) in case of signal deterioration in OTU4 link. asr9k-optic-px.pie is needed to support this feature. Optics Transmit Laser will be disabled if Pie is not installed and activated.
Proactive protection configuration
RP/0/RSP0/CPU0:ios#show run controller dwdm 0/3/0/0 controller dwdm0/3/0/0 proactive admin-state in-service
Proactive Status:
RP/0/RSP0/CPU0:ios#show controllers dwdm 0/3/0/0 proactive
Proactive Protection Status: ON
Proactive Protection State: Normal -Interface is Up <<<< proactive intf is up.
Inputs affecting proactive protection state:
Transport admin state : In Service
Trigger threshold : 6E-5
Revert threshold : 1E-5
Trigger integration window : 30 ms
Revert integration window : 20000 ms
Received APS : 0x0f (No Request)
Transmitted APS : 0x0f (No Request)
G.709 Optics Pie is needed for PP:
RP/0/RSP0/CPU0:RR02#show controllers hundredGigE 0/3/0/0 internal Port Number : 0 Board Type : 0x003b0262 Port Type : 100GE Transport mode : LAN_PP BIA MAC addr : 0022.bde5.aaa0 Oper. MAC addr : 0022.bde5.aaa0 Egress MAC addr : 0022.bde5.aaa0 Port Available : true Status polling is : enabled Status events are : enabled I/F Handle : 0x060000c0 Cfg Link Enabled : tx/rx enabled H/W Tx Enable : no MTU : 1526 H/W Speed : 100 Gbps H/W Duplex : Full H/W Loopback Type : None H/W FlowCtrl Type : None H/W AutoNeg Enable : Off Link Faults : (0x00008000) G709_LICENSE <<<< Active alarm when optic pie is absent Raw Link Faults : (0x00008000) G709_LICENSE Link Up : no Link Led Status : Link down -- Red Pluggable Present : yes Pluggable Type : Pluggable PID : CFP-100G-SR10 Pluggable Compl. : Compliant Pluggable Type Supp.: Supported Pluggable PID Supp. : Supported NP->Mac Lane Sync : 0x001fffff PHY->Mac Lane Sync : 0x000fffff Mac Version : 0x00000022
Optics
100GE ports support the following Cisco Supported Optics (Ref. 3).
- CFP-100G-LR4
- CFP-100G-SR10
FAQ
Q: 1GE port: Link is not coming up. How do I debug?
- Check configuration on both sides (autoneg, speed, flow control) using ‘sh running interface giga <>’
- If the Optics are DOM supported, check Optics Power levels using ‘sh controller giga <> phy’
- Check the reason for the link failure using ‘sh controller giga <> internal’. Meaning of Link defect is discussed in Table 2.
- Also, ensure FPGA images are up-to-date using ‘sh hw-mod location all <>’
Q: 10/100 Mbps with Copper SFP: ARP and ping packets are not received by ASR9k port
SFP-GE-T PHY doesn’t handle odd number of preamble and this will cause silent drop of Ethernet Frames. The issue is not seen with 1000 Mbps. This may happen if the far-end switch is not following IEEE standard for transmission of Preamble.
Q: 10/40/100GE port: Link is not coming up. What do I check?
- Check configuration on both sides using ‘sh running interface … <>’
- If the Optics are DOM supported, check Optics Power levels using ‘sh controller … <> phy’
- Check the reason for the link failure using ‘sh controller … <> internal’
Q: 10/40G OTN mode: Link is not coming up. How do I debug?
- Check ‘admin-state in-service’ is configured under ‘controller dwdm’
- Check both ends use correct OTN modes
- Check both ends use same FEC modes
- Check the reason for the link failure using ‘sh controller <> internal’
- Check the reason for the defect in Table 2
- Ensure Optics pie is installed
- Check Optics type used
Q: After LOS cleared, the port link takes several seconds to come up. Is it normal?
Yes, this is normal. When Receive Loss-of-Signal (LOS) state changes to Normal, PHY and clock Logic is reconfigured. Software waits for Clock lock before bringing link Up.
Q: During Line Card reload, link flap occurs. Is it normal?
Yes, this is normal. During Line Card reload, various hardware logics including MAC, PHY and Clocking undergo device configuration. This will cause link flap at Near and Far-ends.
Q: Link Flap occurred during traffic condition. How do I debug?
- Check Optics power status using ‘sh controller <> internal’
- Check ‘sh interface’ for input errors
- Check ‘sh controller <> stat’ for various input errors
- If the optical receive power is low, clean or replace the optics and fiber at both ends
- Monitor ‘symbol’ and ‘crc’ error count using ‘sh controller <> stat’
- If ‘OTN’ mode is used, check ‘sh controller dwdm <>’ for various BIP and other error counts
- If ‘WAN’ mode is used, check ‘sh controller wanphy all’ for various BIP and other error counts
- Get ‘sh log location <>’ output
If it is a first generation Line Card, then get the trace log
sh ether-ctrl <interface> trace location <location>
If it is a second generation Line Card, then get the following trace log
sh controllers vic ltrace all location <location> sh controller <interface> reg (need few snap-shots of this command with 3 minute interval)
Q: OTN mode: when a link can be considered “good enough”?
- UC (uncorrectable cells) is zero on ‘show controller dwdm’ or steady to a value
- Reported BIP errors are 0
- reported preFEC-BER smaller than the preFEC-SD (signal degradation) threshold
- H/W Link Defectsreport “none” in ‘show controller internal’
Q: Tunable Optics: How do I check what ‘wavelength’ is configured in Optics?
RP/0/RSP0/CPU0:ios#sh controllers tenGigE 0/6/0/7 phy | inc Wavelength Control Wavelength Control: setpoint=1556.50 nm, error=0.00 nm
Also, Configured Optics ‘Channel ID’(Ref. 9) can be obtained from Optics.
RP/0/RSP0/CPU0:SH6#sh controllers tenGigE 0/6/0/7 phy | inc Wavelength Chann
Wavelength Channel: 14
Glossary
- CLI
- Command Line Interface
- EFD
- Ethernet Fault Detection
- FRR
- Fast-reroute to back-up link in case the main data-link fails
- OTN
- Optical Transport Network
- MAC
- Media Access Control
- PHY
- Physical Layer device connects MAC to Optics
- Optics
- SFP, SFP+, XFP, QSFP+, CFP
- SPE
- Synchronous Payload Envelope
Related Information
Provided by our Ethernet development Tech Lead Ramki Ramachandran
