03-27-2011 07:43 AM
I have site-2-site IPSec VPNs as follows:
- Site A to Site C,
- Site B to Site C,
- Site A is in Raleigh. Site B is New York. Site C is New Jersey
- Latency is 25msec between Site A & Site C, 27msec between Site B and Site C,
- Latency is 29msec between Site A and Site C.
- Each location is connected to the Internet with OC-3 speed.
- Site A has Cisco VXR 7206 router with VAM+ encryption card,
- Site B has Cisco ASR1002 with GE5 engine,
- Site C has Cisco 3845 with AIM card,
All routers are behind checkpoint firewalls (NGx R71.30). All location are connected
to OC-3 Internet connection at each location. Site A Internet provider is Internap.
Site B Internap provider is also Intenap. Site C Intenap provider is Level 3.
I can push 120Mbps IPSec traffics (AES-256/SHA, DH-5 with PFS Group 5)
between Site A and Site C without any issues,
I can push 125Mbps IPSec traffics (AES-256/SHA, DH-5 with PFS Group 5)
between Site B and Site C,
Last week, when I tried to bring up VPN between Site A and Site B, the VPN came up just
fine except that I can only push at most 8Mbps between Site A and Site C using Iperf.
All of these VPN devices sit behind the Checkpoint firewalls. The checkpoint just
routes the traffics and not doing any NAT for IPSec traffics. The firewalls have very low
utilization. I have rule out the Checkpoint firewalls as the source of the problem because
IPSec VPN between Site A <-> C and B <-> C can push 125Mbps of VPN traffics. Furthermore,
I also have hosts directly connected behind the Checkpoint firewalls and those hosts can
download/upload almost 150Mbps. I've tried different options with Iperf between site A <-> B
and the most I can get is 8Mbps.
At this point, I am at a loss here. If anyone can suggest something else I can try, that would
be great.
Thanks in advance.
03-27-2011 10:02 AM
Hi,
First of all you need to understand why the performance problem persist. Check sniffer trace of that iperf you're doing.
Check crypto accelerator stats and CEF drops.(If you want I can give you some commands but I need to understand the versions of software).
I understand that it's a VAM2+ and not VAM+?
BTW PFS is only done once during rekey, so it's not very relevant to performance.
Marcin
03-27-2011 10:35 AM
I already checked the trace from iperf since these hosts are linux hosts. looking at tcpdump, I am not seeing a lot re-transmissions which indicate the communication is clean:
3845: c3845-adventerprisek9-mz.124-24.T4.bin
ASR 1002: asr1000rp1-adventerprisek9.02.06.02.122-33.XNF2.bin
7206VXR: c7200-ik9o3s-mz.124-25c.bin
VPN-1#show cef drop
% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'
IPv4 CEF Drop Statistics
Slot Encap_fail Unresolved Unsupported No_route No_adj ChkSum_Err
RP 0 0 0 0 0 0
VPN-1#
where else should I look next?
03-27-2011 10:53 AM
Well I assume we're focusing on A and B?
IOS
"show crypto engine accel stati"
"show ip cef switch stati"
IOS XE
show platform hardware qfp active statistics drop | e _0_
sh pl ha qf ac fe ipsec data drop
This will give us some basic info. It's important taht you run those commands a few times while running the test.
Marcin
03-27-2011 06:12 PM
I will provide them tomorrow. By the way, does the command "show ip cef switch stat" does NOT exist on the IOS of the VXR 7206 I am running:
V0N>sh ip cef ?
scanner CEF scanner trigger statistics
summary CEF table summary
| Output modifiers
V0N>sh ip cef
I am curious. What kind of info are you looking for from these output? If you still can not determine the root cause, what would be the next step?
03-28-2011 12:24 AM
We need to establlish where/if the packets are dropped on data path:
- CEF
- buffers
- crypto accelerator stats
And how does the exchange look like - sniffer trace.
- You said it's not latency...
- You metion that performance is fine on devices just in front VPN routers, it's time to see what happens just after VPN routers.
Logical next step is to confirm whether those packets are dropped/delayed/OoO'ed by VPN routers.
There's a limited scope of what we can do via forums and the amount of info we can collect/analyze.
Marcin
03-28-2011 12:27 AM
By the way, does the command "show ip cef switch stat" does NOT exist on the IOS of the VXR 7206 I am running:
V0N>sh ip cef ?
scanner CEF scanner trigger statistics
summary CEF table summary
| Output modifiers
V0N>sh ip cef
Ah indeed you're running mainline ...
"show cef drop"
"show cef not-cef"
would be a place to start.
Marcin
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide