After replacing a Cisco CSS/SSL Accelorator and PIX firewall with an ACE 4710 to do load balancing and SSL encryption behind an ASA firewall we started seeing mangled HTTP requests in the Apache access logs for the servers in the server farm. Here is one example:
XX.XX.XXX.XXX - - [21/Oct/2012:01:42:12 -0500] "heckoutFlag=true&verifyPassword=false&newsletter=false&emailaddress=&email2=&pass1=&pass2=&username=POST /register/LServlet HTTP/1.1" 501 3322 "https://www.ourwebsite.com/register/CServlet" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
Rather than appearing just after the timestamp, the "POST /register/LServlet" is tacked on to header information that shouldn't even appear in the log. Also the first letter in that header information is always missing (heckoutFlag instead of checkoutFlag in this example).
The mangled request always shows up as a 501 HTTP error and shows up late in the Apache access logs (timestamp is out of chronogical order) and always appears with several duplicate POSTs:
XX.XX.XXX.XXX - - [21/Oct/2012:01:42:23 -0500] "POST /register/LServlet HTTP/1.1" 200 8537 "https://www.ourwebsite/register/CServlet" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
XX.XX.XXX.XXX - - [21/Oct/2012:01:44:12 -0500] "POST /register/LServlet HTTP/1.1" 200 8537 "https://www.ourwebsite/register/CServlet" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
XX.XX.XX.XXX - - [21/Oct/2012:01:42:12 -0500] "heckoutFlag=true&verifyPassword=false&newsletter=false&emailaddress=&email2=&pass1=&pass2=&username=POST /register/LServlet HTTP/1.1" 501 3322 "https://www.ourwebsite.com/register/CServlet" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
XX.XX.XXX.XXX - - [21/Oct/2012:01:44:12 -0500] "POST /register/LServlet HTTP/1.1" 200 8537 "https://www.ourwebsite/register/CServlet" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
This is occurring for several different URLs and not just the one above and for multiple web browsers.
The ACE load balances to servers running Tomcat 7 with Apache HTTP server v. 2.2.14.
A recent ACE software upgrade to A5(2.1) has not fixed the problem.
Has anyone seen this before?
Thanks for any insight you can provide.
-Kari
