123 ... View more

Hello Friends, I have a cisco 2821 router (Cisco 2821 (revision 53.51) with 247808K/14336K bytes of memory), The CPU utilization is going high, When interface traffic on Gi0/1 increses, CPU Utilization of device increases, RouterL#show ip int brief Interface                  IP-Address      OK? Method Status                Protocol GigabitEthernet0/0         172.23.133.12   YES NVRAM  up                    up GigabitEthernet0/1         Public IP   YES NVRAM  up                    up FastEthernet0/0/0          10.216.111.106  YES NVRAM  up                    down FastEthernet0/0/1          10.216.111.110  YES NVRAM  up                    up NVI0                             172.23.133.12   YES unset  up                    up Tunnel190                  10.216.107.98   YES NVRAM  up                    up Tunnel191                  10.216.107.101  YES NVRAM  up                    down RouterL# ROUTER#show process cpu CPU utilization for five seconds: 87%/79%; one minute: 86%; five minutes: 95% I have checked the logs, Following is the logs observed, ROUTER#show logging Syslog logging: enabled (556138 messages dropped, 146 messages rate-limited,                 0 flushes, 0 overruns, xml disabled, filtering disabled) No Active Message Discriminator. No Inactive Message Discriminator.     Console logging: disabled     Monitor logging: level debugging, 120976 messages logged, xml disabled,                      filtering disabled     Buffer logging:  level debugging, 159547 messages logged, xml disabled,                      filtering disabled     Logging Exception size (4096 bytes)     Count and timestamp logging messages: disabled     Persistent logging: disabled     Trap logging: level debugging, 58779 message lines logged         Logging to 10.216.16.70  (udp port 514,  audit disabled,               authentication disabled, encryption disabled, link up),               58511 message lines logged,               0 message lines rate-limited,               0 message lines dropped-by-MD,               xml disabled, sequence number disabled               filtering disabled Log Buffer (4096 bytes): bitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:05:40: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 173.236.91.148(0), 1 packet .Jan 12 05:05:41: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 125.17.126.10 -> 173.192.153.141 (0/0), 2 packets .Jan 12 05:05:41: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.168 -> 173.192.153.141 (0/0), 1 packet .Jan 12 05:05:44: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.174 -> 196.202.246.4 (0/0), 1 packet .Jan 12 05:05:47: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.165 -> 196.202.246.4 (0/0), 1 packet .Jan 12 05:05:57: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.128 -> 190.232.139.252 (0/0), 1 packet .Jan 12 05:05:59: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.147 -> 190.232.139.252 (0/0), 1 packet .Jan 12 05:06:00: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.179 -> 190.232.139.252 (0/0), 1 packet .Jan 12 05:06:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). .Jan 12 05:06:12: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:06:41: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 295 packets .Jan 12 05:06:44: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:06:56: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 174.122.47.26(0), 1 packet .Jan 12 05:07:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). .Jan 12 05:07:06: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 216.36.248.248(0), 1 packet .Jan 12 05:07:13: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 114.27.8.196(0), 1 packet .Jan 12 05:07:14: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:07:39: %SEC-6-IPACCESSLOGDP: list 110 denied icmp 203.101.52.183 -> 95.83.104.193 (0/0), 1 packet .Jan 12 05:07:41: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 9 packets .Jan 12 05:07:45: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:08:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). .Jan 12 05:08:15: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:08:34: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 212.59.148.87(0), 1 packet .Jan 12 05:08:41: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 2 packets .Jan 12 05:08:46: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:08:58: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 114.42.110.221(0), 1 packet .Jan 12 05:09:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). .Jan 12 05:09:16: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/0: the fragment table has reached its maximum threshold 16 .Jan 12 05:09:40: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 2 packets .Jan 12 05:10:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). .Jan 12 05:10:35: %SEC-6-IPACCESSLOGP: list 110 denied tcp 125.17.126.10(0) -> 122.116.206.105(0), 1 packet .Jan 12 05:10:40: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 2 packets .Jan 12 05:11:05: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on FastEthernet0/0/1 (not half duplex), with NOIDA-WAN-CORE-RTR-1 GigabitEthernet1/18 (half duplex). ROUTER# ROUTER# ROUTER#show ip virtual-reassembly GigabitEthernet0/0 GigabitEthernet0/0:    Virtual Fragment Reassembly (VFR) is ENABLED...    Concurrent reassemblies (max-reassemblies): 16    Fragments per reassembly (max-fragments): 32    Reassembly timeout (timeout): 3 seconds    Drop fragments: OFF    Current reassembly count:3    Current fragment count:6    Total reassembly count:219967    Total reassembly timeout count:8265 ROUTER# Following is the Show ip traffic details, ROUTER#show ip traffic IP statistics:   Rcvd:  5595185 total, 3577131 local destination          0 format errors, 0 checksum errors, 6358 bad hop count          0 unknown protocol, 5 not a gateway          0 security failures, 0 bad options, 0 with options   Opts:  0 end, 0 nop, 0 basic security, 0 loose source route          0 timestamp, 0 extended security, 0 record route          0 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump          0 other   Frags: 514890 reassembled, 55367 timeouts, 0 couldn't reassemble          0 fragmented, 0 fragments, 0 couldn't fragment   Bcast: 16 received, 0 sent   Mcast: 796629 received, 87672 sent   Sent:  317343 generated, 511135999 forwarded   Drop:  8 encapsulation failed, 0 unresolved, 0 no adjacency          17 no route, 17330 unicast RPF, 0 forced drop          0 options denied   Drop:  0 packets with source IP address zero   Drop:  0 packets with internal loop back IP address          0 physical broadcast ICMP statistics:   Rcvd: 3 format errors, 0 checksum errors, 0 redirects, 6 unreachable         3120 echo, 6 echo reply, 0 mask requests, 0 mask replies, 0 quench         0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other         0 irdp solicitations, 0 irdp advertisements         0 time exceeded, 0 info replies   Sent: 0 redirects, 0 unreachable, 20 echo, 3120 echo reply         0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp replies         0 info reply, 6358 time exceeded, 0 parameter problem         0 irdp solicitations, 0 irdp advertisements TCP statistics:   Rcvd: 20993 total, 1 checksum errors, 499 no port   Sent: 20911 total BGP statistics:   Rcvd: 0 total, 0 opens, 0 notifications, 0 updates         0 keepalives, 0 route-refresh, 0 unrecognized   Sent: 0 total, 0 opens, 0 notifications, 0 updates         0 keepalives, 0 route-refresh IP-EIGRP statistics:   Rcvd: 0 total   Sent: 0 total PIMv2 statistics: Sent/Received   Total: 2758/2532, 0 checksum errors, 0 format errors   Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,  Hellos: 2582/2523   Join/Prunes: 0/0, Asserts: 0/0, grafts: 176/0   Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0   Queue drops: 0   State-Refresh: 0/0 IGMP statistics: Sent/Received   Total: 1278/1265, Format errors: 0/0, Checksum errors: 0/0   Host Queries: 13/1265, Host Reports: 1265/0, Host Leaves: 0/0   DVMRP: 0/0, PIM: 0/0   Queue drops: 0 UDP statistics:   Rcvd: 1634731 total, 25 checksum errors, 938 no port   Sent: 143697 total, 0 forwarded broadcasts OSPF statistics:   Rcvd: 0 total, 0 checksum errors         0 hello, 0 database desc, 0 link state req         0 link state updates, 0 link state acks   Sent: 0 total         0 hello, 0 database desc, 0 link state req         0 link state updates, 0 link state acks ARP statistics:   Rcvd: 6596 requests, 25 replies, 0 reverse, 0 other   Sent: 73 requests, 3239 replies (5 proxy), 0 reverse   Drop due to input queue full: 0 ROUTER# ROUTER# The Memory utilization is , ROUTER#show processes memory Processor Pool Total:  168481024 Used:   36920964 Free:  131560060       I/O Pool Total:   14680064 Used:    7101744 Free:    7578320 PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process RouterL#show ip int brief Interface                  IP-Address      OK? Method Status                Protocol GigabitEthernet0/0         172.23.133.12   YES NVRAM  up                    up GigabitEthernet0/1         Public IP   YES NVRAM  up                    up FastEthernet0/0/0          10.216.111.106  YES NVRAM  up                    down FastEthernet0/0/1          10.216.111.110  YES NVRAM  up                    up NVI0                       172.23.133.12   YES unset  up                    up Tunnel190                  10.216.107.98   YES NVRAM  up                    up Tunnel191                  10.216.107.101  YES NVRAM  up                    down RouterL# Also Can anybody tell What is the NVI0 interface? RouterL#show int GigabitEthernet0/0 | in rate   Queueing strategy: fifo   5 minute input rate 44957000 bits/sec, 17010 packets/sec   5 minute output rate 6671000 bits/sec, 10597 packets/sec RouterL# RouterL#show int GigabitEthernet0/1 | in rate   Queueing strategy: fifo   30 second input rate 6174000 bits/sec, 11091 packets/sec   30 second output rate 46785000 bits/sec, 17758 packets/sec RouterL# RouterL#show int FastEthernet0/0/1 | in rate   Queueing strategy: fifo   5 minute input rate 906000 bits/sec, 125 packets/sec   5 minute output rate 1000 bits/sec, 1 packets/sec RouterL# ... View more

In general there are two types of incidence for network issues. These fall in the category of outage incidents and performance incidents. Troubleshooting outage incidents in datacenter is quite straightforward. When it comes to analyze performance incidents, things become quite tricky. Generally network team’s opinion on most incidents is based on ping and traceroute results. However these two tools are not of much use in figuring out the performance related issues. Imagine a situation when you are called by application team. One major application behavior has just got aberrant in last 24 hours. Transactions are getting failed and they have given a bulky application layer log with errors. This issue calls for a detailed analysis of TCP transactions. Ping and trace can only reflect the issues with latency or any node’s outage. What is the best way to start? ... View more

VPN concept and configuration is very interesting. Attached is the Fact Sheet for configuration guidelines. ... View more

Firewalls in general use access-list tables for packet flow control. Managing firewall rules and optimizing is a critical part of firewall operation. Once in production system while we do policy fine tuning we want to segregate host based and subnet based policies. This segregation is required in order to see whether any duplicity of policy exist or the traffic flow permitted/denied is as per designed data flow or not. Most commonly we export all the rules in an excel spreadsheet and do some data analysis on the values like clubbing, filtering or cut-copy-paste of rules. One of the tasks involves segregating host to host, host to subnet, subnet to host and subnet to subnet policies. This paper gives a mathematical method to calculate the desired result. ... View more

Firewalls in general use access-list tables for packet flow control. Managing firewall rules and optimizing is a critical part of firewall operation. Once in production system while we do policy fine tuning we want to segregate host based and subnet based policies. This segregation is required in order to see whether any duplicity of policy exist or the traffic flow permitted/denied is as per designed data flow or not. Most commonly we export all the rules in an excel spreadsheet and do some data analysis on the values like clubbing, filtering or cut-copy-paste of rules. One of the tasks involves segregating host to host, host to subnet, subnet to host and subnet to subnet policies. This paper gives a mathematical method to calculate the desired result. ... View more