The implications of flow control settings and traffic flow

[acameron12]

I have a pair of 4948's between a Juniper firewall cluster and F5 load blancer pair and traffic through this configuration appreas to be throttled or rate limmited at 100mb, maybe 120mb at most. Traffic is trunked from the Juniper firewalls and set as an access port to the F5 on each switch

The ports towards the Juniper's are set as:

Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000-TX
input flow-control is off, output flow-control is off

The ports towards the F5's are set as:

Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000-TX
input flow-control is on, output flow-control is off

The dual inter-switch connections are set

Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000-TX
input flow-control is on, output flow-control is off

Can these settings be cuasing a limitation in the flow of packets?

Is there iny value in flow-control in this confiouration?

[Edison Ortiz]

Flow control is only active inbound on the Cisco side so if the attached device isn't sending PAUSE frames, this feature is not the cause for any rate limitation.

I suggest checking for input/output errors or any QoS configuration on either side.

Don't discard any Layer 1 issue (cables / port / etc).

Regards,

Edison

[acameron12]

Thanks for the prompt response.

With respect to layer 1, we recently replaced all cables as part of a general cleanup and standardization on structureed wiring infrastructure.

It was thought cabling might be non-statndard, but, the interfaces in question show no errors.

For my switch "A" facing the firewall:

  Last input never, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 119725000 bits/sec, 28163 packets/sec
  5 minute output rate 101995000 bits/sec, 28698 packets/sec
     29504868089 packets input, 16000631092431 bytes, 0 no buffer
     Received 274868 broadcasts (6 multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     30607413011 packets output, 13502613809545 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

And switch "B" facing the fierwwall, I see no issues:

  Last input never, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 45000 bits/sec, 83 packets/sec
     11848444 packets input, 6112299860 bytes, 0 no buffer
     Received 212 broadcasts (1 multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     126890791 packets output, 12998404251 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

Similary with the port facing the F5 from switch "A"

  Last input never, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     149884047 packets input, 68855910512 bytes, 0 no buffer
     Received 1426 broadcasts (40 multicast)
     0 runts, 0 giants, 0 throttles
     1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     153994997 packets output, 84894526198 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

and from switch "B":

Last input never, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 97437000 bits/sec, 28257 packets/sec
  5 minute output rate 118333000 bits/sec, 27757 packets/sec
     30140728875 packets input, 13012226944724 bytes, 0 no buffer
     Received 4393 broadcasts (51 multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     29050078113 packets output, 15816070525953 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

Nothing obvious jumps out.

With respect to flow control behavior

Sw A to the Fw

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper
---------  -------- -------- -------- --------    ------- -------
Gi1/23     off      off      desired  off         0       0

Sw A to the F5

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper
---------  -------- -------- -------- --------    ------- -------
Gi1/13     off      off      desired  on          0       0

Sw A interswitch to Sw B

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper
---------  -------- -------- -------- --------    ------- -------
Gi1/43     off      off      desired  on          0       0

and

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper
---------  -------- -------- -------- --------    ------- -------
Gi1/44     off      off      desired  on          0       0

Sw B to the Fw

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper                      
---------  -------- -------- -------- --------    ------- -------
Gi1/23     off      off      desired  off         0       0    

Sw B to the F5

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper                      
---------  -------- -------- -------- --------    ------- -------
Gi1/13     off      off      desired  on          0       0   

Sw B interswitch to Sw A

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper                      
---------  -------- -------- -------- --------    ------- -------
Gi1/43     off      off      desired  on          0       0

and

Port       Send FlowControl  Receive FlowControl  RxPause TxPause
           admin    oper     admin    oper                      
---------  -------- -------- -------- --------    ------- -------
Gi1/44     off      off      desired  on          0       0     

[jorge.calvo]

Hello,

Flow control does not affect the traffic throughput, except between a switch and a server.

What I see from your interface outputs is that your traffic rate is very high, more than 28000 pps inbound and outbound. However you only achieve 100 - 120Mb because your average packet size is around 530 bytes.

What I always look at to check the throughput on my interfaces are the pps more than the mbps so, from my point of view your traffic flow seems normal to me.

Regards.

[alanbcameron]

Hello,

     Thanks for your insights and the informative response.

I will take this into consideration. Traffic has increased over time through switches and concurrenlty, therre have been

techology upgrades on either side of the 4948's with the upgraded firewalls and introduction of the F5 load balancers to handle increased traffic.

I expect I may be looking at the natural traffic pattern which would have been seen had all the required components been in place to begin with.