Intermittent auth with NTLM

rngai_ironport · ‎10-20-2009

This problem is quite subjective as it may be desktop setting vary from another but never or less, I hope you could share some insight how to get to the bottom of this.

There are few client intermittently get popup auth screen, which they should not because their PC join the domain and C360 is configure to use NTLM only. There 3 websites we sample and isolate which exhibit this problem. They are:

http://www.saptechnical.com/
http://myxcelsius.com/
http://www.forumtopics.com/

From access log, I could see the http request was made but all of sudden they get 407. Could it be http version IE use? What ver of http C360 recommend? 1.0 or 1.1? Here's a snapshot:

SAPTechnical website
1255990207.043 268 10.9.131.58 TCP_REFRESH_HIT/200 1072 GET http://www.saptechnical.com/images/sidebarbg.jpg "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.saptechnical.com image/jpeg OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Comp> - "0" "0" "0" "0" "0" "0" "264" "264"
1255990207.043 265 10.9.131.58 TCP_REFRESH_HIT/200 1209 GET http://www.saptechnical.com/images/bullet.gif "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.saptechnical.com image/gif OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Comp> - "0" "0" "0" "0" "0" "0" "261" "261"
1255990207.055 0 10.9.131.58 TCP_DENIED/407 3333 GET http://www.saptechnical.com/Tutorials/BI/Xcelsius/Index.1.jpg - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990207.059 0 10.9.131.58 TCP_DENIED/407 3333 GET http://www.saptechnical.com/Tutorials/BI/Xcelsius/Index.2.jpg - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990207.078 0 10.9.131.58 TCP_DENIED/407 467 GET http://www.saptechnical.com/Tutorials/BI/Xcelsius/Index.1.jpg - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990207.268 222 10.9.131.58 TCP_REFRESH_HIT/200 1115 GET http://www.saptechnical.com/images/textbg.jpg "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.saptechnical.com image/jpeg OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Comp> - "0" "0" "0" "0" "0" "0" "215" "215"
1255990207.307 226 10.9.131.58 TCP_REFRESH_HIT/200 14139 GET http://www.saptechnical.com/Tutorials/BI/Xcelsius/Index.1.jpg "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.saptechnical.com image/jpeg OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Comp> - "6" "0" "0" "0" "0" "0" "216" "216"

Forumtopics website
1255990242.668 197 10.9.131.58 TCP_REFRESH_HIT/200 770 GET http://www.forumtopics.com/busobj/templates/bob/formIE.css "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.forumtopics.com text/x-c OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Blog> - "0" "0" "0" "0" "0" "0" "192" "192"
1255990243.888 1200 10.9.131.58 TCP_MISS/200 82960 GET http://www.forumtopics.com/busobj/images/banners/xenon_top_banner_v2.swf "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.forumtopics.com application/x-shockwave-flash MONITOR_CUSTOMCAT_1090519042-GeneralGroup-AD_AUTH-NONE-NONE-DefaultRouting <C_Whit> - "0" "0" "0" "0" "0" "0" "1006" "192"
1255990244.218 0 10.9.131.58 TCP_DENIED/407 3333 GET http://www.forumtopics.com/busobj/templates/bob/images/nav_print.gif - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990244.219 0 10.9.131.58 TCP_DENIED/407 3333 GET http://www.forumtopics.com/busobj/templates/bob/images/nav_next.gif - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990244.231 0 10.9.131.58 TCP_DENIED/407 467 GET http://www.forumtopics.com/busobj/images/smiles/banghead.gif - NONE/- - OTHER-NONE-AD_AUTH-NONE-NONE-NONE <-,-,-,-,-,-,-,-,-,-,-,-,-,-,-> - "0" "0" "0" "0" "0" "0" "0" "0"
1255990244.427 197 10.9.131.58 TCP_REFRESH_HIT/200 1622 GET http://www.forumtopics.com/busobj/images/ranks/bobrank_06.gif "GASCOM\smith@GAS-AD-DOMAIN" DIRECT/www.forumtopics.com image/gif OTHER-NONE-AD_AUTH-NONE-NONE-DefaultRouting <Blog> - "0" "0" "0" "0" "0" "0" "191" "191"

RBC____CS · ‎10-20-2009

The sneaky browser tries to get through the proxy without providing authentication first. When the ironport replies with the request for authentication, the browser responds with the domain/user/password and the ironport checks that against your authentication source then delivers the content if the domain/user/password checks out.

I routinely see a Request / Deny / Request with auth / Allow in my logs.

If you use Wireshark on your pc or use it to look at a traffic capture from the ironport, you can see the 'authentication required' packet returned from the ironport.

If you are getting the popup box, you may want to look at the authlogs on the ironport and it can tell you why are failing primary authentication. IE

20/Oct/2009:14:22:54 -0500 INFO : PROX_AUTH : - : NTLM CRAP authentication for user [somedomain]\[someuser] returned NT_STATUS_ACCOUNT_LOCKED_OUT (PAM: 8)

rngai_ironport · ‎10-21-2009

Nothing found in authlogs, for 10 different sets of logs within that timeframe. Couldn't find the userid in that authlogs.

Any more hint?

serialmonkey · ‎10-23-2009

I'm getting the exact same thing with my users - both IE and Firefox. It seems to happen on websites that use AJAX (hence alot of concurrent adhoc requests ?).

I'm seeing things like

23/Oct/2009:14:36:59 +1100 INFO : PROX_AUTH : - : NTLM CRAP authentication for u
ser [OFFICE]\[MyUser] returned NT_STATUS_NO_LOGON_SERVERS (PAM: 12)
23/Oct/2009:14:36:59 +1100 CRITICAL : PROX_AUTH : - : NTLMSSP BH: NT_STATUS_NO_L
OGON_SERVERS

Both domain controllers are alive and well though.

RBC____CS · ‎10-24-2009

serialmonkey-

It is odd that you bring that up.

I am getting similar messages on all 4 of my production ironports. I have a ticket open with support escalated to the application engineers. One of my ironports was so bad I had to take it out of service, yet the AD servers they auth against continue to hum along.

serialmonkey · ‎10-26-2009

I might go ahead and raise a support ticket as well. Weight in numbers :-)

rngai_ironport · ‎10-29-2009

The problem didn't come back anymore. What had happen was intermittent. We did a Test Query to LDAP from our domain and saw time stamp variance between WSA and AD. Found out later NTP server where WSA point is not responding so we reset the NTP box and things are better.

Attempting to get TGT...

Failure: Error while fetching Kerberos Tickets from server 'server1.gas.com.au' :
kinit: krb5_get_init_creds: Clock skew too great

Failure: Error while fetching Kerberos Tickets from server 'server2.gas.com.au' :
kinit: krb5_get_init_creds: Clock skew too great

Failure: Error while fetching Kerberos Tickets from server 'server3.gas.com.au' :
kinit: krb5_get_init_creds: Clock skew too great

Checking local WSA time and server time difference...

Warning: Clock skew between WSA 'Mon Oct 12 14:30:52 2009' and AD server 'Mon Oct 12 14:36:27 2009' is too great

Warning: Clock skew between WSA 'Mon Oct 12 14:30:52 2009' and AD server 'Mon Oct 12 14:36:27 2009' is too great

Warning: Clock skew between WSA 'Mon Oct 12 14:30:52 2009' and AD server 'Mon Oct 12 14:36:27 2009' is too great