why router is doing switching for packets inside a vpn

[Hythim Ali El Hadad]

Hi,

We have 2 sites connected by wan routers 3945

and we have ASA on each site in the inside of the wan router

we configured vpn between the 2 sites on the ASAs and it is working good

We found that when high traffic is passing through the vpn. it cause the router cpu to increase. you can say traffic with 20M increase the cpu to 64%

I found the interrupt value is very high

CPU utilization for five seconds: 66%/63%; one minute: 66%; five minutes: 65%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
 179   109559196   852516836        128  0.55%  0.30%  0.28%   0 IP Input

So I checked for what is the traffic is causing this high cpu

I found this on the incoming interface of the router

Protocol  IP                  
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
                 Process  522226690 116822805108  396148191 67755655895
            Cache misses          0          -          -          -
                    Fast 63343040080 14806815078202 68511337389 22130664616037
               Auton/SSE          0          0          0          0

I found also it is the same value on the outside of ASA

So the router is seeing the outgoing packets inside the vpn and count it in the fast switching and I think it is the cause of high cpu

The question here is why the router is doing fast switching or CEF switching for packets coming inside the vpn although it is encrypted and the router isn't required to route or switch these packets. Or it is a normal behavior ? and cpu is normal to be high like that ?

Thanks

[Joseph W. Doherty]

I'm a bit confused by what you describe.  If your ASAs are behind the routers, then the routers will route the encrypted VPN packets like any other packets passing through them.

On a 3945, the router's CPU load is driven by the amount of traffic passing through the router.  I.e. it sounds perfectly normal or as expected.

[Hythim Ali El Hadad]

Thanks a lot Joseph for your reply

Yes we have the internet line received on router 3945 then the router connected to the ASA outside

So the packets is going out from ASA outside to the input of the router then to internet

you say the router will route these packets and the cpu will increase and it is normal. And it is already what I'm seeing and I expect it also

But the strange thing is that there is a vpn from the ASA to another ASA on a remote site

So the outgoing packets from ASA will pass through the router with no need for routing as it is encrypted and the vpn is already created

So we though the router won't route these encrypted packets so the cpu should be normal

Or do you see the encrypted packets is containing a header or something like that from the ASA and the router will route of fast switch the packet to its outside interface based on this header ?? or how the router is routing these packets as they appear in the fast switching as below

Protocol  IP                  
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
                 Process  522226690 116822805108  396148191 67755655895
            Cache misses          0          -          -          -
                    Fast 63343040080 14806815078202 68511337389 22130664616037
               Auton/SSE          0          0          0          0

Finally is "Fast" mean CEF switching ?

Thanks again

[Joseph W. Doherty · Accepted solution]

VPN packets generally still need to be routed.

I.e., the packets coming out of each ASA, destined for the other ASA, have a destination address.  It's that address that needs to be routed.

The VPN, itself, might have addresses only known to it, and not visible to the external routers.

[Hythim Ali El Hadad]

Good

so what is the FAST mean when I do sh interface switching

is it CEF or fast ? That I think CEF and fast switching shouldn't raise the cpu that they are using cache

I found CEF enabled on all interfaces

[Richard Burts]

The FAST is in the output for backward compatibility with software that actually used FAST switching. It would be nice if Cisco would rewrite this command so that it did reflect the current reality of CEF switching. But so far the command is still formulated for the older output.

FAST and CEF are similar in that they are both based on using a cache for information used to forward a packet. But they are different in several ways so they are certainly not the same.

In software based devices like routers it takes some CPU cycles to forward a packet. CEF (and FAST) make the process more efficient and reduce the amount of CPU required. But they do not eliminate the requirement for CPU to work to forward packets.

HTH

Rick

[Hythim Ali El Hadad]

So if we have 200 call center agents in our site

and the call is going out the gateway in the remote site through vpn

when the router use CEF. In repeated packets ?   Or our case is normal to use cpu heavily ?

And is this voip calls will consume cpu more than data or no difference

[Richard Burts]

Perhaps I am not understanding what you are trying to ask when you say "when the router use CEF". The router will use CEF for every packet that it forwards. This is one of the differences between CEF and FAST because this would use process switching for the first packet in a flow to build the cache entry and then would use the cache for the following packets in the flow. But CEF builds the cache from the routing table (not from actual packets) and uses the cache entry for every packet.

I believe that your case is normal to have the high CPU when you have high traffic level. In the original post you indicate that the interrupt value is high. That is good and it indicates that most of the CPU load is for forwarding packets in interrupt mode (so it is using cache for all packets).

As far as forwarding packets is concerned forwarding voip packets is no different from forwarding data packets. If the router were configured for processing voip then it would be different. Without knowing details of your router config it is not possible to be certain. But my guess (based on the very high interrupt level) is that your router is not processing voip but is just forwarding the packets, so no difference for voi9.

HTH

Rick

[Hythim Ali El Hadad]

Thanks a lot Richard for these words

only you say

But CEF builds the cache from the routing table (not from actual packets) and uses the cache entry for every packet.

This previous sentence is my question

So the cache is used many times during the call 

Is using the cache consume cpu ?

I think if we push traffic. It could be one flow

but voip calls are many flows

so voip calls should consume cpu more than pushing data traffic for the same amount of bandwidth

Thanks

[Joseph W. Doherty · Accepted solution]

And is this voip calls will consume cpu more than data or no difference

Routers like the 3945 expend CPU forwarding packets.  As a general rule, VoIP packets are often smaller than data packets, so for the same bandwidth consumption used by VoIP or data packets, VoIP packets would consume more CPU than for data packets.  (NB: if the VoIP packets and the data packets were the same size, generally the CPU consumption would be the same for either, if their bandwidth consumption was the same.)

[Hythim Ali El Hadad]

Yes I try to confirm this one

thanks

only can we simulate voip calls traffic behvior using wan killer by reducing the packet size ? Or it is useless test ?

[Richard Burts]

There are a couple of things that I want to make comments about:

- With CEF the cache is used for every packet that the router forwards. There are CPU cycles used for every packet that is forwarded. Some of the CPU work is reading the cache but other work done by the CPU is managing queues, doing rewrite of the layer two header as the packet is forwarded, applying access lists if configured, doing address translation if configured, and other things like this.

- Joseph makes an interesting point about the size of packets and the impact on CPU use. But I do not agree that it means that voip traffic takes more CPU than data traffic. There are applications that use smaller packets (for example TFTP uses much less than maximum size packets and Telnet is another example of a data application that uses small packets).

- If you run a test to measure the impact on CPU it is certainly valid to test using smaller packet size. And it might be useful to look at the traffic flows on your router to determine what is the average packet size. (in the output of show interface look at the number of output packets per second and the number of output bits per second and you can find the average bits per packet).

- my understanding of the original question was whether there is anything about voip that makes the router work harder to forward that traffic than to forward other types of data traffic. And the answer to that is that packet by packet it makes no difference in how hard the CPU works whether it is a voip packet or some other type of traffic.

HTH

Rick 

[Joseph W. Doherty]

- Joseph makes an interesting point about the size of packets and the impact on CPU use. But I do not agree that it means that voip traffic takes more CPU than data traffic. There are applications that use smaller packets (for example TFTP uses much less than maximum size packets and Telnet is another example of a data application that uses small packets).

Rick, for the same bandwidth consumption.  I don't believe I proposed packets being VoIP packets, alone, using more CPU, just that as a rule they are often smaller than data packets.  If they are smaller, for the same bandwidth usage (with their higher PPS), often CPU usage is higher too.

- my understanding of the original question was whether there is anything about voip that makes the router work harder to forward that traffic than to forward other types of data traffic. And the answer to that is that packet by packet it makes no difference in how hard the CPU works whether it is a voip packet or some other type of traffic.

Yes, although again, if the VoIP packet is smaller, for the same bandwidth usage, for a 3945, I would expect higher CPU usage.

[Joseph W. Doherty]

No, I think that would be a good test to verify, or not, what I'm suggesting.

BTW, Cisco's white paper on performance, lists a 3945's as 982 Kpps for 64-byte packets and 8,025 Mbps for 1500-byte packets.  Rather odd they use Kpps for one measurement and Mbps for the other, but 1,845 Kpps times 64-byte times 8 bits per byte equals 502.784 Mbps about a 16x performance difference.