We're currently experiencing fairly bizarre behavior when attempting to set up http probes that expect a regexp. Namely, if we specify a regexp, the probe *always* passes, regardless of status code and regardless of whether or not the message actually matches the pattern. Doing 'no expect regexp' fixes this behavior (by which I mean that the 'expect status' rules work again).
We haven't noticed until now because this is the first time we've tried to set up a probe that does this. Are we missing something? Is this a known issue with our current firmware version?
Patrick T. Ramsey
# show run probe | begin HTTP-nfscheck | end regex
probe http HTTP-nfscheck
description Simple HTTP probe to check nfs mount health
passdetect interval 20
request method head url /nfs-health-check/
expect regex "^ureytgraeuikghfdjg$"
Are you trying to parse a large page? There were some problems with expect regex on ACE like CSCtg13892, so we introduced a new way of parsing content on A3(2.6). You should upgrade to this version and check ACE behavior on that version.
Can you check if the server response contains the Content-Length header filed? Without it the ACE will not attempt to parse the content.
The server response did not contain Content-Length. I was briefly hopeful, but I've altered the health check to return said header and the behavior of the probe has not changed: while both rservers in the server farm are reporting an error state, the probe is being marked successful.
Also, the documentation at http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA1_7_/command/reference/probe.html does not seem to say anything about the requirement that the Content-Length header be present (except for the hash option, which we're not using). Is this a new requirement? Also, why is it that the probe succeeds if the content isn't parsed? If there's no usable content, shouldn't the regex fail?
Patrick T. Ramsey
It is documented in the config guide and it is not a new requirement:
"For HTTP or HTTPS probes, the server response must include the Content-Length header for the expect regex command to function. Otherwise, the probe does not attempt to parse the regex."
So, if the Content-Length is missing, this functionality is not active and no decision is made based on it.
BTW, I am not sure what are you trying to do with HEAD method, as the server will not return message-body with it. For this type of probe you need to use method GET.
What if the size is large? I experienced that my HTTP probe, the server return content around 6K bytes, but the packets are split to 4 ethernet payload due to 1500 MTU. And the regexp is split to first and second packets. the ACE never match the regex and failed. I changed to only world in the first packet, it worked. So, the ACE seem to not assembly entire HTTP payload or having fixed buffer i.e. 2048 or 4096 bytes. It just grab the first TCP packet which has HTTP header (return code), then parse for regex.
Note: My OS is A2 (3.4) ACE20 module.
I have seen similar behavior where an HTTP probe still failed even when properly setting the Content-Length header using the probe config below:
probe http HA-script-second-15101
passdetect interval 30
passdetect count 2
request method get url /status.html
expect regex "responding"
I was able to get this to work by setting up a TCP probe using the "send-data" command to specify my GET.
probe tcp HA-script-15101
passdetect interval 2
passdetect count 1
send-data "GET /status.html"
expect regex "responding"
My suspicion is there are some incompatibility between HTTP servers (possibly HTTP/1.1 vs. HTTP/1.0?) where the HTTP probes don't always work. My suggestion is to always use the TCP with the send-data to guarantee compatibility.
Also - Cisco's documentation seems to differ a bit on whether the Content-Length is actually needed. One says it is needed when using the hash function:
The server response must include the Content-Length header for the hash command to function. Otherwise, the probe does not attempt to parse the hash value.
Another suggests in general this is required:
For HTTP or HTTPS probes, the server response must include the Content-Length header for the expect regex command to function. Otherwise, the probe does not attempt to parse the regex.
I would like to test this out in a lab to determine the actual definitive behavior.
I hope this helps.
Your applications need to enter the Content-Length.
Also you are missing the expect status, please try with "expect status 0 999"
Here you have an old thread which might help:
Hope this helps a little bit.
I also went through a similar issue in which we need to probe the real server PESERVER01 and if the real server replies with the keyword "PE Server" in the HTTP content then the probe should be passed successful.
In my case the real server was listening on port 32776 for HTTP service so we configured the serverfarm as below,
serverfarm host SF-TEST-32776
rserver PESERVER01 32776
And the TCP probe as below,
probe tcp PE-SERVER-STRING
send-data GET /IOR/ping HTTP/1.1 <<== command should not be in inverted commas
expect regex "PE Server"
The above probe worked really well and when we checked the probe status it was marking as success. I also tried changing the regex from "PE Server" to "Vishal12345" and it was failing as expected because there was no such keyword in the HTTP content.
T2-LB02# sh probe PE-SERVER-STRING
probe : PE-SERVER-STRING
type : TCP
state : ACTIVE
port : 32776 address : 0.0.0.0 addr type : -
interval : 15 pass intvl : 60 pass count : 3
fail count: 3 recv timeout: 10
------------------ probe results ------------------
associations ip-address port porttype probes failed passed health
serverfarm : SF-TEST-32776
real : PESERVER01
10.10.10.1 32776 PROBE 105 0 105 SUCCESS
I was struggling with this issue from long time. Even raised couple of Cisco TAC cases with no luck. The most important thing here is to identify the exact command to be send to real server like GET /IOR/ping HTTP/1.1 that we used here.
To collect this command I did packet capture on one of the client machine and then tried to open the URL from real server which can return the string "PE Server". Then analyzed the captures in Wireshark and checked the HTTP data with follow the TCP stream option in which I seen the below data, which gives the command to be send in probe as well as the string we should expect.
GET /IOR/ping HTTP/1.1
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/18.104.22.168 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
HTTP/1.0 200 OK
Please try this and see if it helps you.