Friends, Problem : When there is a high congestion on the Remote site router wan link, our monitoring server reports it as down, and the reason being not getting reply to icmp packets or high responce time to get a reply back. I do not want this to happen during congestion period. Please note that Inbound making is done on L3 switch and Outbound service policy is applied to Remote site router WAN interface. - So when the Remote router replies to ICMP packts sent by monitoring servers , echo reply packets will be matched by default class, and due to congestions at  some point of time will be randomly droped. I belive router self generated packets like ssh, icmp, will be given default treatment. Monitoring server -------- Head end Router (ASR1000) QOS applied here too ------- (Outbound policy applied on WAN interface)Remote site Router (2951) -------- L3 Swith (3750) (Inboud marking done here). One way I think to address this is to create new class maps and policy-map on remote site router to mark the packets sent destination (Monitoring server ip ) marked with higher DSCP value or bandwith percentage. but would need to be done head end router too. Example: access-list 101 permit icmp host x.x.x.x host 10.x.x.x echo access-list 101 permit icmp host x.x.x.x host 10.x.x.x echo-reply access-list 101 permit udp host x.x.x.x host 10.x.x.x eq snmp class-map match-all QPM_ICMP-Traffic description Outbound ICMP traffic queue match access-group 101 ! policy-map QPM_Lab-Policy-Group class QPM_ICMP-Traffic bandwidth percent <X> ! interface Se0/0 ip address 192.168.251.6 255.255.255.224 service-policy output QPM_Lab-Policy-Group Note we have below configs on L3 switch: From L3 Switch at remote sites : (ACL is same at Head end) class MARK-OPS-ADMIN-MGMT set dscp af31 ip access-list extended OPS-ADMIN-MGMT-ACL permit tcp any any eq tacacs permit tcp any eq tacacs any permit tcp any any eq 161 permit tcp any eq 161 any permit udp any any eq snmp permit udp any eq snmp any permit udp any any eq snmptrap permit udp any eq snmptrap any permit udp any any eq syslog permit tcp any any eq telnet permit tcp any eq telnet any permit tcp any any eq 22 permit tcp any eq 22 any What do you suggest the best way to modify QOS ? and resolve the issue.? Thanks Sandep Rawat ... View more

Hi guys, I am having issues with a port channnel connected to the UCS server.Whenever I am putting physical port into portchannel it is going down, the physical port is going down with interface staus saying first - (Link not connected) and then finally (Hot standby in bundle). Portchannel remail down with status (No operational members) What does this status message means :  Ethernet1/11 is down (Hot standby in bundle) Below is the configuration: interface Ethernet1/11   no cdp enable   switchport mode trunk   switchport trunk allowed vlan 317,2511-2514   spanning-tree port type edge trunk   no snmp trap link-status   logging event port link-status   logging event port trunk-status   channel-group 101 mode active interface port-channel101 switchport mode trunk   vpc 101   switchport trunk allowed vlan 317,2511-2514   spanning-tree port type edge trunk   logging event port link-status   logging event port trunk-status ACC14# sh port-channel summ Flags:  D - Down        P - Up in port-channel (members)         I - Individual  H - Hot-standby (LACP only)         s - Suspended   r - Module-removed         S - Switched    R - Routed         U - Up (port-channel) -------------------------------------------------------------------------------- Group Port-       Type     Protocol  Member Ports       Channel -------------------------------------------------------------------------------- 101   Po101(SD)   Eth      LACP      Eth1/11(H) ------------------------------- Below is what I have obeserved : ACC14(config)# int eth1/11 ACC14(config-if)# no channel-group 101 mode active ACC14(config-if)# sh int eth1/11 Ethernet1/11 is up   Hardware: 1000/10000 Ethernet, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   full-duplex, 10 Gb/s, media type is 10G   Beacon is turned off   Input flow-control is off, output flow-control is off   Rate mode is dedicated   Switchport monitor is off   EtherType is 0x8100   Last link flapped 00:00:02   Last clearing of "show interface" counters never   30 seconds input rate 112 bits/sec, 14 bytes/sec, 0 packets/sec   30 seconds output rate 232 bits/sec, 29 bytes/sec, 0 packets/sec   Load-Interval #2: 5 minute (300 seconds)     input rate 152 bps, 0 pps; output rate 248 bps, 0 pps   RX     0 unicast packets  1701 multicast packets  0 broadcast packets     1701 input packets  368680 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  59044 multicast packets  5630 broadcast packets     64674 output packets  5169139 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   15 interface resets ACC14(config-if)# sh int po101 port-channel101 is down (No operational members) vPC Status: Down, vPC number: 101 [packets forwarded via vPC peer-link]   Hardware: Port-Channel, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   auto-duplex, 10 Gb/s   Beacon is turned off   Input flow-control is off, output flow-control is off   Switchport monitor is off   EtherType is 0x8100   No members   Last clearing of "show interface" counters never   0 seconds input rate 0 bits/sec, 0 bytes/sec, 0 packets/sec   0 seconds output rate 0 bits/sec, 0 bytes/sec, 0 packets/sec   Load-Interval #2: 0 seconds     input rate 0 bps, 0 pps; output rate 0 bps, 0 pps   RX     0 unicast packets  0 multicast packets  0 broadcast packets     0 input packets  0 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  0 multicast packets  0 broadcast packets     0 output packets  0 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   0 interface resets ACC14(config-if)# sh int eth1/11 Ethernet1/11 is down (Link not connected)   Hardware: 1000/10000 Ethernet, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   auto-duplex, 10 Gb/s, media type is 10G   Beacon is turned off   Input flow-control is off, output flow-control is off   Rate mode is dedicated   Switchport monitor is off   EtherType is 0x8100   Last link flapped 00:00:10   Last clearing of "show interface" counters never   30 seconds input rate 48 bits/sec, 6 bytes/sec, 0 packets/sec   30 seconds output rate 8056 bits/sec, 1007 bytes/sec, 8 packets/sec   Load-Interval #2: 5 minute (300 seconds)     input rate 288 bps, 0 pps; output rate 2.30 Kbps, 2 pps   RX     0 unicast packets  1728 multicast packets  0 broadcast packets     1728 input packets  374762 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  59637 multicast packets  5685 broadcast packets     65322 output packets  5222148 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   15 interface resets ACC14(config-if)# sh int po101 port-channel101 is down (No operational members) vPC Status: Down, vPC number: 101 [packets forwarded via vPC peer-link]   Hardware: Port-Channel, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   auto-duplex, 10 Gb/s   Beacon is turned off   Input flow-control is off, output flow-control is off   Switchport monitor is off   EtherType is 0x8100   Members in this channel: Eth1/11   Last clearing of "show interface" counters never   30 seconds input rate 2192 bits/sec, 274 bytes/sec, 0 packets/sec   30 seconds output rate 7296 bits/sec, 912 bytes/sec, 3 packets/sec   Load-Interval #2: 5 minute (300 seconds)     input rate 216 bps, 0 pps; output rate 2.38 Kbps, 2 pps   RX     0 unicast packets  450 multicast packets  0 broadcast packets     450 input packets  98735 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  980 multicast packets  12 broadcast packets     992 output packets  143742 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   0 interface resets ACC14(config-if)# sh lacp counters interface po101 ACC14(config-if)# int eth1/11 ACC14(config-if)# channel-group 101 mode active ACC14(config-if)# sh lacp counters interface po101                     LACPDUs         Marker      Marker Response    LACPDUs Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err --------------------------------------------------------------------- port-channel101 Ethernet1/11       261    599      0      0        0      0        0      ACC14(config-if)# sh lacp counters interface po101                     LACPDUs         Marker      Marker Response    LACPDUs Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err --------------------------------------------------------------------- port-channel101 Ethernet1/11       261    600      0      0        0      0        0      ACC14(config-if)# sh lacp counters interface po101                     LACPDUs         Marker      Marker Response    LACPDUs Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err --------------------------------------------------------------------- port-channel101 Ethernet1/11       261    603      0      0        0      0        0      ACC14(config-if)# ACC14(config-if)# ACC14(config-if)# sh lacp counters interface po101                     LACPDUs         Marker      Marker Response    LACPDUs Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err --------------------------------------------------------------------- port-channel101 Ethernet1/11       261    605      0      0        0      0        0      ACC14(config-if)# ACC14(config-if)# sh lacp counters interface po101                     LACPDUs         Marker      Marker Response    LACPDUs Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err --------------------------------------------------------------------- port-channel101 Ethernet1/11       262    610      0      0        0      0        0      ACC14(config-if)# ACC14(config-if)# Now no more LACP packets are sent or received ACC14(config-if)# ACC14(config-if)# sh int po101 port-channel101 is down (No operational members) vPC Status: Down, vPC number: 101 [packets forwarded via vPC peer-link]   Hardware: Port-Channel, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   auto-duplex, 10 Gb/s   Beacon is turned off   Input flow-control is off, output flow-control is off   Switchport monitor is off   EtherType is 0x8100   Members in this channel: Eth1/11   Last clearing of "show interface" counters never   30 seconds input rate 248 bits/sec, 31 bytes/sec, 0 packets/sec   30 seconds output rate 128 bits/sec, 16 bytes/sec, 0 packets/sec   Load-Interval #2: 5 minute (300 seconds)     input rate 96 bps, 0 pps; output rate 1.66 Kbps, 2 pps   RX     0 unicast packets  471 multicast packets  0 broadcast packets     471 input packets  101618 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  988 multicast packets  12 broadcast packets     1000 output packets  144925 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   0 interface resets ACC14(config-if)# sh int eth1/11 Ethernet1/11 is down (Hot standby in bundle)   Hardware: 1000/10000 Ethernet, address: 547f.ee24.4e92 (bia 547f.ee24.4e92)   MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,      reliability 255/255, txload 1/255, rxload 1/255   Encapsulation ARPA   Port mode is trunk   auto-duplex, 10 Gb/s, media type is 10G   Beacon is turned off   Input flow-control is off, output flow-control is off   Rate mode is dedicated   Switchport monitor is off   EtherType is 0x8100   Last link flapped 00:01:03   Last clearing of "show interface" counters never   30 seconds input rate 160 bits/sec, 20 bytes/sec, 0 packets/sec   30 seconds output rate 176 bits/sec, 22 bytes/sec, 0 packets/sec   Load-Interval #2: 5 minute (300 seconds)     input rate 56 bps, 0 pps; output rate 1.58 Kbps, 2 pps   RX     0 unicast packets  1755 multicast packets  0 broadcast packets     1755 input packets  378413 bytes     0 jumbo packets  0 storm suppression bytes     0 giants  0 input error  0 short frame  0 overrun   0 underrun     0 watchdog  0 if down drop     0 input with dribble  0 input discard     0 Rx pause   TX     0 unicast packets  59651 multicast packets  5685 broadcast packets     65336 output packets  5224084 bytes     0 jumbo packets     0 output errors  0 collision  0 deferred  0 late collision     0 lost carrier  0 no carrier  0 babble     0 Tx pause   15 interface resets And just to let u know This po101 is a part of vPC, between 2 N5k. Other Switch's N5k port channel is up. Thanks in advance... HTH Sandeep Rawat Message was edited by: Sandeep Rawat ... View more

First question is why i am seeing below multicast entries in mac address table for gi1/39, gernally on all interfaces in vlan 172. cisco4k#sh mac address-table interface gi1/39     -----> connects to a VM Unicast Entries vlan   mac address     type        protocols               port -------+---------------+--------+---------------------+-------------------- 172    12f3.f000.3361   dynamic ip                    GigabitEthernet1/39   Multicast Entries vlan    mac address     type    ports -------+---------------+-------+-------------------------------------------- 172    ffff.ffff.ffff   system Switch,Gi1/35,Gi1/36,Gi1/37,Gi1/38,Gi1/39                                 Gi1/40,Gi1/41,Gi1/46,Po3,Po4 cisco4k#sh ip multicast                     Multicast Routing: disabled   Multicast Multipath: disabled   Multicast Route limit: No limit   Multicast Triggered RPF check: enabled   Multicast Fallback group mode: Sparse cisco4k#sh ip multicast interface vlan 172 Vlan172 is up, line protocol is up   Internet address is x.x.x.x/24   Multicast routing: disabled   Multicast packets in/out: 59324871/0   Multicast boundary: not set   Multicast TTL threshold: 0   Multicast Tagswitching: disabled cisco4k#sh ip multicast interface vlan 172 Vlan172 is up, line protocol is up   Internet address is x.x.x.x/24   Multicast routing: disabled   Multicast packets in/out: 59324877/0   Multicast boundary: not set   Multicast TTL threshold: 0   Multicast Tagswitching: disabled ddcxs01an# -------------------------------------------- Second Q - I am seeing MAC address flaps and also CPU high on switch , could this be related to multicast entries, i belive no, what you think cisco4k#sh ver Cisco IOS Software, Catalyst 4000 L3 Switch Software (cat4000-I5S-M), Version 12.2(25)EWA10, RELEASE SOFTWARE (fc1) cisco4k#sh log | in 12:F3:F0:00:33:61 Aug 17 20:33:02 starts: %C4K_EBM-4-HOSTFLAPPING: Host 12:F3:F0:00:33:61 in vlan 172 is flapping between port Gi1/38 and port Gi1/39 cisco4k#sh mac address-table interface gi1/39 Unicast Entries vlan   mac address     type        protocols               port -------+---------------+--------+---------------------+-------------------- 172    12f3.f000.3361   dynamic ip                    GigabitEthernet1/39   Multicast Entries vlan    mac address     type    ports -------+---------------+-------+-------------------------------------------- 172    ffff.ffff.ffff   system Switch,Gi1/35,Gi1/36,Gi1/37,Gi1/38,Gi1/39                                 Gi1/40,Gi1/41,Gi1/46,Po3,Po4 cisco4k#sh mac address-table interface gi1/38 Multicast Entries vlan    mac address     type    ports -------+---------------+-------+-------------------------------------------- 172    ffff.ffff.ffff   system Switch,Gi1/35,Gi1/36,Gi1/37,Gi1/38,Gi1/39                                 Gi1/40,Gi1/41,Gi1/46,Po3,Po4 cisco4k#sh run int gi1/38 Building configuration... Current configuration : 140 bytes ! interface GigabitEthernet1/38 switchport access vlan 172 switchport mode access logging event link-status spanning-tree portfast end Current CPU is normal , as only one mac address is flapping ,I noticed a 99% cpu when there were multiple mac flaps for host in vlan 172, and PID 30 was using more than 80% cpu. cisco4k#sh proc cpu sorted | ex 0.00 CPU utilization for five seconds: 24%/0%; one minute: 26%; five minutes: 27% PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process   30  39654906482251773658          0 16.79% 18.86% 19.14%   0 Cat4k Mgmt LoPri   29  20350339682857418569          0  5.43%  5.57%  5.58%   0 Cat4k Mgmt HiPri ... View more