Slow performace with FWSM in differerent DMZ

alap84 · ‎05-09-2011

Hi

i am feeling an issue related to 2 x 6509 with fwsm.

Mod Ports Card Type                              Model
--- ----- -------------------------------------- ------------------ -----------
1    6 Firewall Module                                                 WS-SVC-FWM-1
2   48 CEF720 48 port 1000mb SFP                            WS-X6748-SFP
3   48 CEF720 48 port 10/100/1000mb Ethernet            WS-X6748-GE-TX
4   48 CEF720 48 port 10/100/1000mb Ethernet            WS-X6748-GE-TX
5    5 Supervisor Engine 720 10GE (Active)                   VS-S720-10G

Hw    Fw           Sw           Status
------------ ------------ -------
4.3   7.2(1)       4.1(2)       Ok
1.12 12.2(14r)S5 12.2(33)SXH3 Ok
3.0   12.2(18r)S1 12.2(33)SXH3 Ok
3.0   12.2(18r)S1 12.2(33)SXH3 Ok
2.1   8.5(2)       12.2(33)SXH3 Ok

on fwsm we have different dmz for different application. mostly of Oracle (1521 port) to application means seperate one. problem is oracle people reported the slow performace when exporting dump from once vlan to another.

before that they are using 3com technology (network) and on this its ok. time difference is double from old to new.

i had sniffed the traffic also and found alot of TCP OUT OF ORDERS errors. i read that this is the bug which is resolved in 4.0

can anyone guide me whats the problem, anyone else experienced the same thing !

please do let us know what to do

Calin C. · ‎05-10-2011

Do you have there load balancing over line with different speeds / latency? Might be that you don't have a problem with FWSM but actully your packets arrive out of order due lines with different bandwidth or latency and FWSM is not able to reorder them.

For TCP this means retransmissions. More retransmissions = more delay giving you the impression of bad performance on FWSM.

Cheers,

Calin

Sent from Cisco Technical Support iPhone App

alap84 · ‎05-10-2011

Both switches are connected with 2 interfaces back to back with same speed n duplex ... check this

GigabitEthernet2/43 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0021.a07f.1312 (bia 0021.a07f.1312)
Description: *** Connected to Core Sw 1 Gig 2/43 ***
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is SX
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:51, output 00:00:05, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 699
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1419000 bits/sec, 394 packets/sec
5 minute output rate 2457000 bits/sec, 1001 packets/sec
     11197079129 packets input, 3393173916972 bytes, 0 no buffer
     Received 938440133 broadcasts (437783684 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     17805472555 packets output, 11455343923155 bytes, 0 underruns
     0 output errors, 0 collisions, 6 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

GigabitEthernet2/43 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0021.a0b4.1a72 (bia 0021.a0b4.1a72)
Description: *** Connected to Core Sw 2 Gig 2/43 ***
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is SX
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:24, output 00:00:12, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 2949901
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 2488000 bits/sec, 989 packets/sec
5 minute output rate 981000 bits/sec, 361 packets/sec
     17789026996 packets input, 11454150674566 bytes, 0 no buffer
     Received 2789903563 broadcasts (1540803399 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     11182986034 packets output, 3391898917788 bytes, 0 underruns
     0 output errors, 0 collisions, 3 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

Calin C. · ‎05-10-2011

OK, so the network part looks fine.

Second, I see you wrote:

before that they are using 3com technology (network) and on this its ok

I understand from this that before Cisco 65k + FWSM you used 3com network devices. Since you didn't specified, I don't know if with 3Com you were using any firewalling mechanism (packet inspection, ACL...), but please take note that there is a big difference between vlan routing (through SVI interfaces, with not FWSM or any other security device) and VLAN routing + firewalling (thorugh FWSM). Without consistent rules on the FWSM, you may encounter double time (end-to-end packet flow time) to transfer the same amount of data, when compare the transfer with / without FWSM.

The explanation is that with FWSM the packet encounter additional delay as it is inspected on the FWSM.

Please clarify if before switching to 65k+FWSM, you were still using some security device (e.g. firewall) from a different company.

Cheers,

Calin

alap84 · ‎05-10-2011

....

alap84 · ‎05-10-2011

alap84 · ‎05-10-2011

snapshot attached