Application suddenly stopped working

[yeow28]

i have a few off site cisco 1900 series whereby some application suddenly stopped working.

eg Internet access is very slow or RDP to the site server fails.

i can only RDP to the site router. 

no changes made to router config.

i found this error on one of the routers

FW-4-GLOBAL_ALERT_ON: getting aggressive, count (50/5000) current 1-min rate: 5001

[Georg Pauwen]

Hello,

try and increase the maximum number of half-open sessions:

1900(config)#ip inspect max-incomplete high 10000
1900(config)#ip inspect max-incomplete low 8000

[yeow28]

my current configuration is this

parameter-map type inspect global
max-incomplete low 3000
max-incomplete high 4000
one-minute low 4000
one-minute high 5000

[Georg Pauwen]

Hello,

change the values to:

---

max-incomplete low 6000

max-incomplete high 8000

one-minute low 8000

one-minute high 10000

So essentially doubling the values...

---

---

---

[yeow28]

do i need to reboot the router after changing the values ?

[balaji.bandi]

you do not require technically, a check  is the config effective - show policy-firewall stats global

[Georg Pauwen]

Hello,

actually, I would add the lines marked in bold as well (and reboot the router to clear all existing half open sessions).

parameter-map type inspect global
max-incomplete low 6000
max-incomplete high 8000
one-minute low 8000
one-minute high 10000
tcp synwait-time 10
tcp half-open reset on
tcp idle-time 90
tcp idle reset on

[yeow28]

this issue have being bothering me the last few weeks as 1 by 1, the routers just dropped the connections.

simple connection like RDP just refused to work, email access also died off and some Internet app also refused to work.

i replaced router, switched from my local MPLS provider link to backup link using 4G mobile router.

when i switch to backup line, the issue did went away.

I have since configured this on one of the C1921 router and will monitor it for a few days 

[Georg Pauwen]

Hello,

if possible, post the full running configuration (sh run) of your 1900 router, maybe we can spot something in there that causes these problems...

[yeow28]

how can i check what kind of traffic/connections ?

the issue happen again after changing to 10k

000037: Jan 19 23:41:10.863 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up
000038: Jan 19 23:42:10.095 UTC: %FW-4-GLOBAL_ALERT_ON: getting aggressive, count (1148/10000) current 1-min rate: 10001
000039: Jan 19 23:42:43.263 UTC: %FW-4-GLOBAL_ALERT_OFF: calming down, count (1/10000) current 1-min rate: 4966

1#show policy-firewall stats
Global Stats:
Packet inspection statistics [process switch:fast switch]
tcp packets: [873:332815]
udp packets: [53:1656]
icmp packets: [2:1768]

Session creations since subsystem startup or last reset 12094
Current session counts (estab/half-open/terminating) [90:3:0]
Maxever session counts (estab/half-open/terminating) [2602:1228:73]
Last session created 00:00:00
Last statistic reset never
Last session creation rate 284
Maxever session creation rate 10234
Last half-open session total 3

[balaji.bandi]

you can only check span the port or using some advanced tools with netflow.

[Georg Pauwen]

Hello,

did you also configure:

tcp synwait-time 10
tcp half-open reset on
tcp idle-time 90
tcp idle reset on

Post the full running configuration of your router...