I'm running an ASA with 9.8(2) and I have a IPSEC tunnel with another device. The other device (pfsense) fragments ESP packets in order to fit the MTU, but the ASA does not seem to allow ESP fragments to go in, does not reassemble them and of course I can't see the decapsulated ESP payload to reach the endhost. I have opened the firewall to allow everything.
The question is how can i configure the ASA to do reassembly, as it should be and forward the payload to the endhost.
PS: I know all about PMTU and MSS, but it does not apply in my case, so I would like to reassemble the packets.
... View more
I opened a case to TAC and the conclusion was that there is not particular problem with the setup. The CPU is supposed to be high for our setup.
We captured packets that go to CPU (there is a mechanism for that in the 4500X) and we examined the packets. Nothing suspicious was found. It is just that we have about 6-7K active users and many vlans and the 4500X just can't handle it.
... View more
Yia soy Lefteri,
Well, my case is a stack of:
Switch Ports Model SW Version SW Image Mode ------ ----- ----- ---------- ---------- ---- 3 52 WS-C3650-48TD 03.03.04SE cat3k_caa-universalk9 INSTALL * 4 52 WS-C3650-48TD 03.03.04SE cat3k_caa-universalk9 INSTALL
uptime : 1 year, 42 weeks, 3 days, 4 hours, 54 minutes
- I have about 30 SVIs (vlan l3 ifs)
- about 50 lines of ACL statements on applied on SVI interfaces
- OSPF, static
- about 240 entries in the arp table, few of them in unknown state
- I have in all interfaces "ip device tracking maximum 0" in order to disable device tracking (in a later IOS you can disable it globally).
- various policing/shaping on interfaces
- my CPU right now is:
#sh proc cpu Core 0: CPU utilization for five seconds: 95%; one minute: 95%; five minutes: 95% Core 1: CPU utilization for five seconds: 92%; one minute: 94%; five minutes: 95% Core 2: CPU utilization for five seconds: 93%; one minute: 96%; five minutes: 95% Core 3: CPU utilization for five seconds: 97%; one minute: 94%; five minutes: 94%
- The CPU load has been increased in big steps of about 20% in about two months. I have also seen once, CPU load falling a 20%. When there is an increase/decrease of CPU load (a step), it stays there for weeks, even months. So I want to say that it is not going up and down as a normal CPU should do.
- FED is the problem:
#show processes cpu detailed process fed sorted | ex 0.0 PID T C TID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process (%) (%) (%) 5665 L 229196 1441256 466 72.75 73.16 73.61 1088 fed 5665 L 1 6100 1810519 1257976 0 24.12 24.19 24.30 0 fed-ots-main 5665 L 2 11007 1280626 2181769 0 14.24 12.63 12.92 0 XcvrPoll
- The CPU load is supposed to be punted packets to CPU.
#show platform punt client tag buffer jumbo fallback packets received failures alloc free bytes conv buf 27 0/1024/2048 0/5 0/5 0 0 0 0 0 65536 0/1024/1600 0/0 0/512 1012528394 1225732032 3754655604 0 0 65537 0/ 512/1600 0/0 0/512 28843424 28843424 3106482325 0 0 65538 0/ 5/5 0/0 0/5 0 0 0 0 0 65539 1/2048/1600 0/16 0/512 351496405 351662096 1015006362 0 0 65540 0/ 128/1600 0/8 0/0 11983417 11983417 2407384532 0 0 65541 0/ 128/1600 0/16 0/32 120981077 120981077 1710593502 0 0 65542 0/ 768/1600 0/4 0/0 19806318 162558624 1585792442 0 0 65544 0/ 96/1600 0/4 0/0 0 0 0 0 0 65545 0/ 96/1600 0/8 0/32 0 0 0 0 0 65546 0/ 512/1600 0/32 0/512 1516512715 1519497732 1064396594 0 0 65547 0/ 96/1600 0/8 0/32 0 0 0 0 0 65548 0/ 512/1600 0/32 0/256 2249570784 2249568845 1396835480 0 2 65551 0/ 512/1600 0/0 0/256 7 7 420 0 0 65556 0/ 16/1600 0/4 0/0 0 0 0 0 0 65557 0/ 16/1600 0/4 0/0 0 0 0 0 0 65558 0/ 16/1600 0/4 0/0 2610521 2610521 180203510 0 282 65559 0/ 16/1600 0/4 0/0 45229293 45229293 3911166554 0 3 65560 0/ 16/1600 0/4 0/0 1554691 1554691 124004004 0 4711 65561 0/ 512/1600 0/0 0/128 407124369 439136899 2988194120 0 7 65562 0/ 512/1600 0/0 0/256 0 0 0 0 0 65563 0/ 512/1600 0/0 0/256 0 0 0 0 0 65565 0/ 512/1600 0/16 0/256 0 0 0 0 0 65566 0/ 512/1600 0/16 0/256 0 0 0 0 0 65567 0/ 512/1600 0/16 0/256 0 0 0 0 0 65568 0/ 512/1600 0/16 0/256 0 0 0 0 0 65583 0/ 1/1 0/0 0/0 0 0 0 0 0 131071 0/ 96/1600 0/4 0/0 0 0 0 0 0 fallback pool: 0/1500/1600 jumbo pool: 0/128/9300
- My tcam usage is well under limits.
- Since I have CPU load big steps upwards in specific moment in time, I queried my logs (from all the devices, not only the 3650) in order to find if some event happened at that time and triggered the CPU load. I found nothing.
What I believe:
It is a BUUUGGG... It can't be the arp requests. Even your 8*/24 is not that much to have the switch in constant load for days. As you can see on the output for CPU punted packets, I have many packets that go to CPU. The switch has constant load but it is very responsive. It can even be a cosmetic bug that falsely shows high CPU load.
We should diff the 'punted to CPU packets' counters on order to derive rates and capture some packets to see if we can come to a conclusion.
Have you opened a cisco tac case?
... View more
I have the same issue on my 3650. It is exactly the same as you describe it. The funny thing is that my 3650 is actually a stack of two switches and only one switch has the problem (Its the master of the stack).
Have you opened a case? Have you made in other conclusions about the issue? Did you have the chance to try another IOS?
Thanx in advance,
... View more
I'm trying to upgrade an ASR9001 from 5.1.3 to 5.3.3 but the command:
admin install add tar ftp://xx.xxx.xx/ASR9K-iosxr-px-K9-5.3.3.tar fails.
What it says about NTP does not apply (time is right) and about code signing stuff, I really don't know what that means.
The md5 of the tar file is correct.
The output is:
RP/0/RSP0/CPU0:R4(admin)#install add tar ftp://ciscoftp:******@xx.yy.zz.40/ASR9K-iosxr-px-K9-5.3.3.tar synchronous Fri May 6 11:11:18.025 EEST Install operation 80 '(admin) install add tar /ftp://ciscoftp:*************************@xx.yy.zz.40/ASR9K-iosxr-px-K9-5.3.3.tar synchronous' started by user 'spiros' via CLI at 11:11:18 EEST Fri May 06 2016. Info: The following files were extracted from the tar file Info: '/ftp://ciscoftp:*************************@xx.yy.zz.40/ASR9K-iosxr-px-K9-5.3.3.tar' and will be added to the entire router: Info: Info: README-ASR9K-iosxr-px-k9-5.3.3 (skipped - not a pie) Info: asr9k-video-px.pie-5.3.3 Info: asr9k-services-px.pie-5.3.3 Info: asr9k-services-infra-px.pie-5.3.3 Info: asr9k-optic-px.pie-5.3.3 Info: asr9k-mpls-px.pie-5.3.3 Info: asr9k-mini-px.pie-5.3.3 Info: asr9k-mgbl-px.pie-5.3.3 Info: asr9k-mcast-px.pie-5.3.3 Info: asr9k-li-px.pie-5.3.3 Info: asr9k-fpd-px.pie-5.3.3 Info: asr9k-doc-px.pie-5.3.3 Info: asr9k-bng-px.pie-5.3.3 Info: asr9k-asr901-nV-px.pie-5.3.3 Info: asr9k-asr9000v-nV-px.pie-5.3.3 Info: asr9k-k9sec-px.pie-5.3.3 Info: Error: Cannot proceed with the add operation because the code signing certificate has expired. Error: Suggested steps to resolve this: Error: - check the system clock using 'show clock' (correct with 'clock set' if necessary). Error: - check the pie file was built within the last 5 years using '(admin) show install pie-info Error: /tmp/install/tar/instdir/34139816_349000000/asr9k-video-px.pie-5.3.3'. Error: Error: The following pies were not added due to an error: Error: asr9k-video-px.pie-5.3.3 Error: asr9k-services-px.pie-5.3.3 Error: asr9k-services-infra-px.pie-5.3.3 Error: asr9k-optic-px.pie-5.3.3 Error: asr9k-mpls-px.pie-5.3.3 Error: asr9k-mini-px.pie-5.3.3 Error: asr9k-mgbl-px.pie-5.3.3 Error: asr9k-mcast-px.pie-5.3.3 Error: asr9k-li-px.pie-5.3.3 Error: asr9k-fpd-px.pie-5.3.3 Error: asr9k-doc-px.pie-5.3.3 Error: asr9k-bng-px.pie-5.3.3 Error: asr9k-asr901-nV-px.pie-5.3.3 Error: asr9k-asr9000v-nV-px.pie-5.3.3 Error: asr9k-k9sec-px.pie-5.3.3 Install operation 80 failed at 11:30:15 EEST Fri May 06 2016.
... View more
Anyone has bought or has some experience with SG350XG and SG550XG Cisco Stackable Managed Switches?
I was wondering if they are good value for money. I understand that they are non-IOS and the low end of the switching portfolio.
... View more
We have an ASR9001. I would like some info about the sizing of BGP-FS. For example, how many /32 routes can i send over it, without reaching tcam limits?
DDOS attacks can easily reach hundred of thousands attacking hosts...
... View more
We are running into problems with a 4500X. The switch has high CPU and if we add more services to it, it collapses. Let me clarify that:
- We have 189 VLANs and 185 RPVST instances
- About 120 SVIs (VLAN interfaces) with ipv6 *only*. Typical config of the interface follows:
interface Vlanxxx description servers no ip address ipv6 address 2001:xxxx:xxxx:xx::yy/64 ipv6 enable ipv6 nd other-config-flag ipv6 nd ra interval 15 no ipv6 redirects ipv6 mld explicit-tracking ipv6 mld access-group mld_filter ipv6 verify unicast source reachable-via rx allow-default ipv6 dhcp server v6_dhcp ipv6 traffic-filter SERVERS-OUTV6 out ipv6 ospf 3323 area 0 counter ipv6
- OSPF, OSPFv3, iBGP.
- ipv6 unicast and multicast
- IP cef contains about 1200pfxs and ipv6 cef contains about 2000 pfxs.
- CPU varies from 30% to 45% depending on the traffic passing by (I think it should be uncorrelated, but CPU correlates strongly with traffic).
When we move our primary ipv6 BGP peering to the 4500X, the load of the CPU goes high. If we add ipv4 on the SVIs the load also goes up and it collapses (100% CPU).
We have about 1.5G of traffic. When we did a local SPAN on one port (mirroring)with 1.5G traffic, the monitor port transmitted only 350Mbps of traffic.... The rest was dropped. Weird and disappointing behaviour for a rather simple HW feature.
Do you have an idea what is the feature that raises up the CPU and makes it correlate with traffic?
PS: we are running the latest stable IOSXE
... View more
thanx for your response.
What kind of options/configurations are you referring at?
I'm not actually using any features. It is just a VLAN interface without any other features enabled.
interface Vlan1 ip address x.x.x.23 255.255.255.248 no ip proxy-arp ip ospf cost 100 arp timeout 300 end
One thing I can see for the some of the routing vlans:
sw5#sh interfaces stats Vlan1 Switching path Pkts In Chars In Pkts Out Chars Out Processor 23256751 2093578079 0 0 Route cache 0 0 88173408 19860239378 Total 23256751 2093578079 88173408 19860239378 Vlan10 Switching path Pkts In Chars In Pkts Out Chars Out Processor 101198618 9530828009 0 0 Route cache 611697 157207480 1217409 156343754 Total 101810315 9688035489 1217409 156343754
while the SVIs report:
Vlan1 is up, line protocol is up Internet address is x.x.x.x/29 Broadcast address is 255.255.255.255 Address determined by non-volatile memory MTU is 9000 bytes Helper address is not set Directed broadcast forwarding is disabled Multicast reserved groups joined: 184.108.40.206 Outgoing access list is not set Inbound access list is not set Proxy ARP is disabled Local Proxy ARP is disabled Security level is default Split horizon is enabled ICMP redirects are always sent ICMP unreachables are always sent ICMP mask replies are never sent IP fast switching is enabled IP Flow switching is disabled IP CEF switching is enabled IP CEF switching turbo vector IP Null turbo vector IP multicast fast switching is enabled IP multicast distributed fast switching is disabled IP route-cache flags are Fast, CEF Router Discovery is disabled IP output packet accounting is disabled IP access violation accounting is disabled TCP/IP header compression is disabled RTP/IP header compression is disabled Probe proxy name replies are disabled Policy routing is disabled Network address translation is disabled BGP Policy Mapping is disabled Input features: MCI Check Output features: Check hwidb
Vlan10 is up, line protocol is up Internet address is x.x.x.x/29 Broadcast address is 255.255.255.255 Address determined by non-volatile memory MTU is 9000 bytes Helper address is not set Directed broadcast forwarding is disabled Multicast reserved groups joined: 220.127.116.11 18.104.22.168 Outgoing access list is not set Inbound access list is not set Proxy ARP is disabled Local Proxy ARP is disabled Security level is default Split horizon is enabled ICMP redirects are always sent ICMP unreachables are always sent ICMP mask replies are never sent IP fast switching is enabled IP Flow switching is disabled IP CEF switching is enabled IP CEF switching turbo vector IP Null turbo vector IP multicast fast switching is enabled IP multicast distributed fast switching is disabled IP route-cache flags are Fast, CEF Router Discovery is disabled IP output packet accounting is disabled IP access violation accounting is disabled TCP/IP header compression is disabled RTP/IP header compression is disabled Probe proxy name replies are disabled Policy routing is disabled Network address translation is disabled BGP Policy Mapping is disabled Input features: MCI Check Output features: Check hwidb
... View more
I have a stack with two WS-C3650-48TD and IPBase license. I have enabled ipv4 routing with a few SVIs (VLAN interfaces for a few low traffic customers) and I'm running OSPF.
Yesterday, a DDOS attack was *routed* through the WS-C3650. At the input interface a had 1G of traffic with about 300Kpps and at the output interface I had 70Mbps of traffic and 17Kpps....
This means to me that the 3650 managed to route only 17Kpps of ipv4 traffic that passed by the switch and that the rest of traffic was dropped.
The million dollar question is:
Does the WS-C3650-48TD do, ipv4 routing in hardware or software? Because if it does it in software, then whats the point of supporting 1000 SVIs and 24000 routes and routing protocols as the PDF says.
... View more
I can't make IOS XRv 6.0.0 (and older ones like 5.3) to boot in vmware.
I have tried to import the ova to:
- VMware workstation 10 (on two different PCs vtx/vtd enabled)
- VMware ESXi5.5 (on server hardware)
- Oracle VirtualPC
All the above failed. When I press start, I can see the boot prompt, I'm pressing enter to boot IOS-XRv and then it says "booting IOS-XRv.." and stays there, without doing anything else. This happens, with both 6.0 and 5.3, on all the mentioned virtualization enviroments.
I can't be that wrong. There is a problem with the ova's? There is a setting, I'm missing?
... View more
I would like to configure mLACP between two ME3600X, in order to provide port/circuit redundacy to a customer.
The configuration guides always mention MPLS config, for ICCP/MLACP to work. Is MPLS a prerequisite for mLACP?
Can I do it without MPLS (and how)? I only have MetroIPAccess license and not the AdvancedMetroIPAccess that has MPLS.
... View more
I'm have configured netflow and using it for some time on ASR9001, but i have a few questions that always appear and I don't have any answers:
I need to know the actual limits of the netflow engine on ASR9001. I have read Alexanders Thuijs post "ASR9000/XR Netflow Architecture and overview" which is an excellent post but I still need a few things clarified.
I'd like to understand the following:
1. Alexander says that netflow limit "Typhoon: 200kpps/LC". What exaclty is the 200Kpps.
a) Is it normal traffic packets passing by an interface with netflow enabled? Does this mean that the netflow engine can only examine 200kpps of real traffic and if i have more, i need to sample it? b) the 200kpps is refering to new cache entries created? Does this mean that packets that belong to flows that are already cached, don't count for the 200Kpps limit? c) the 200kpps is refering to the export capability in netflow packets and has nothing to do with actual traffic packets. d) something else
2. Is the netflow cache creation/update a hardware function? If i create a flow monitor *without* an exporter, with 1:1 sampling and enable it on a 100 interfaces/subinterfaces,in and out and pass 40G traffic, will the box collapse or increase its CPU usage? Don't try to convince me that there is not point in such a config, because I agree. I'm trying to understand the limits/impact, as I said.
3. when i 'sh flow monitor-map flm-input', it says 'CacheRateLimit: 2000'.
Does this mean that the netflow cannot create more than 2000 new entries/sec? If not, what does it mean?
I think this is enough for now.
... View more