Cant Connect IP Phone While on Comcast via VPN?

stownsend · ‎02-24-2011

Can An ISP Filter Traffic within the VPN Tunnel? Sounds weird but...

We have a Avaya IP Office 500 Head end Phone Server. Several 5610 IP Phones.

I've setup a PIX 501 to Connect to our ASA 5510. In the Office, going from one Public IP Subnet to the Public IP on the ASA 5510 I'm able to connect up the 5610 IP Phone through the PIX 501 through the ASA 5510 to the IPOffice 500 Server and place calls.

I take the same setup home and connect it to my Comcast Internet connection anf it does not work. I can connect a Laptop behind the PIX501 and Connect to the HQ network just fine. I can see the Phone do a TFTP Transfer to the VM Server, though it stops short can cannot connec to the Call Server.

I then gave the unit to 4 other Comcast Users, all of them do not work.

I then gave it to a AT&T DSL user, works Great!

then another local DSL ISP (Sonic.Net) and it works great.

Same hardware, same VPN, Same everything except ISP.

Both With Comcast we tried directly to the Cable Modem, or behind a edge router. PCs connect, Phone does not.

The thing I do not understand is If Comcast is filtering something, how can they filter something that is in my VPN Tunnel?

Any Thoughts?

Jennifer Halim · ‎02-24-2011

Your statement:

I've setup a PIX 501 to Connect to our ASA 5510. In the Office, going from one Public IP Subnet to the Public IP on the ASA 5510 I'm able to connect up the 5610 IP Phone through the PIX 501 through the ASA 5510 to the IPOffice 500 Server and place calls.

--> Do you mean in clear text, or through VPN tunnel?

--> Also, do you always have PIX connected behind the Comcast to create the VPN tunnel? even when you gave the Comcast to 4 other users, do you give the PIX to them together with the Comcast connection, and establish VPN tunnel between the PIX to ASA at the HQ?

- With IPSec VPN between the 2 sites, definitely ISP is not able to detect the voice/data traffic because they are encrypted.

stownsend · ‎02-25-2011

Sorry if I was not Clear. Yes, the IP Phone is Directly Connected to the PIX 501 in ALL test cases. So when in the office, I wanted to go from one Public Subnet to the other to at least go through our Edge Router.

In All Cases, the PIX 501 fires up. I wait a couple minutes, then I connect the Phone. It Loads its Firmware, Gets a DHCP address from the PIX 501, The Tunnel Light comes on, the Phone Transferrs a file from a TFTP server, then Tries to contact the VoIP headend server.

It is only the Comcast Test cases where the IP Phone cannot connect to the VoIP headend server.

Thank you,

Jennifer Halim · ‎02-26-2011

Mmm.. sounds like a Comcast issue

They might be dropping packets hence the problem.

Can you please check if the IPSec is in ESP packet, or it is encapsulated in UDP/4500 packet?

If it's actually been encapsulated in UDP/4500, you can probably test it using IPSec encapsulated in TCP so at least it retransmits if there is any drop packets. However, if you are passing voice traffic via the VPN tunnel, it might not be such a good idea, but worth a test to prove it's an ISP issue, and you can then go back to Comcast with the evidence

Here is the command to encapsulate with TCP instead of UDP if it's actually in UDP/4500:

http://www.cisco.com/en/US/docs/security/asa/asa82/command/reference/c5.html#wp2236488

stownsend · ‎03-01-2011

so I have the crypto isakmp ipsec-over-tcp 1000

entered on the Head end, though I'm not sure if I need to disable UDP, or what I need to do on the PIX 501 to tell it to use TCP.

Thanks!

Scott<-

stownsend · ‎03-01-2011

So I also entered the following on the head end.

group-policy RemotePIX501 internal
group-policy RemotePIX501 attributes
 vpn-tunnel-protocol IPSec
 ipsec-udp disable
tunnel-group remote.home type ipsec-l2l
tunnel-group remote.home general-attributes
 default-group-policy RemotePIX501
tunnel-group remote.home ipsec-attributes
 pre-shared-key *****

Though I'm not sure how to tell if its using TCP or UDP.

Using ASDM I look at the IKE connection and it still says UDP/500

asa(config-group-policy)# sh cry ipsec sa sum

Current IPSec SA's:            Peak IPSec SA's:
IPSec            :     8         Peak Concurrent SA :    12
IPSec over UDP   :     0         Peak Concurrent L2L :    12
IPSec over NAT-T :     0         Peak Concurrent RA :     0
IPSec over TCP   :     0
IPSec VPN LB     :     0
Total            :     8

So I'm not sure where to go from here.

stownsend · ‎03-01-2011

So I also entered the following on the head end.

group-policy RemotePIX501 internal
group-policy RemotePIX501 attributes
 vpn-tunnel-protocol IPSec
 ipsec-udp disable
tunnel-group remote.home type ipsec-l2l
tunnel-group remote.home general-attributes
 default-group-policy RemotePIX501
tunnel-group remote.home ipsec-attributes
 pre-shared-key *****

Though I'm not sure how to tell if its using TCP or UDP.

Using ASDM I look at the IKE connection and it still says UDP/500

asa(config-group-policy)# sh cry ipsec sa sum

Current IPSec SA's:            Peak IPSec SA's:
IPSec            :     8         Peak Concurrent SA :    12
IPSec over UDP   :     0         Peak Concurrent L2L :    12
IPSec over NAT-T :     0         Peak Concurrent RA :     0
IPSec over TCP   :     0
IPSec VPN LB     :     0
Total            :     8

So I'm not sure where to go from here.

stownsend · ‎03-01-2011

So I've entered the following ont he head end.

group-policy RemotePIX501 internal
group-policy RemotePIX501 attributes
vpn-tunnel-protocol IPSec
ipsec-udp disable
tunnel-group remote.home type ipsec-l2l
tunnel-group remote.home general-attributes
default-group-policy RemotePIX501
tunnel-group remote.home ipsec-attributes
pre-shared-key *****

then I looked in ASDM and it still looks like its UDP/500.

Then did a show cry ipsec sa sum with the following results.

ASA(config-group-policy)# sh cry ipsec sa sum

Current IPSec SA's:            Peak IPSec SA's:
IPSec            :     8         Peak Concurrent SA :    12
IPSec over UDP   :     0         Peak Concurrent L2L :    12
IPSec over NAT-T :     0         Peak Concurrent RA :     0
IPSec over TCP   :     0
IPSec VPN LB     :     0
Total            :     8

Seems like the IPSec over TCP should have something.

Not sure where to go from here...

Tanks!

Jennifer Halim · ‎03-01-2011

No, so from that output, it doesn't look like it's going through any PAT router, so it will never use either UDP or TCP encapsulation. It will just use ESP

as the IPSec protocol. Only when it passes through a PAT router, it will encapsulate it to UDP or TCP, since ESP is not a TCP or UDP protocol that can be PATed.

If you really want to test further, you can place a PAT device in front of the PIX, then the ISAKMP will detect that it's behind a PAT device, and will automatically encapsulate that in UDP/TCP. But all evidence so far pointing that it's a Comcast issue. If you can provide them with all the test that you have performed with other providers, I am confident that Comcast will own up to the issue.

stownsend · ‎03-02-2011

You are reaching (or reached already) my knowledge limit. So is ESP like TCP in the retransmissions of dropped packets? Or like UDP, and jsut keeps going if it can?

I dont have a PAT device here that I can connect the router to. I have a DLINK unit, though I cant seem to get it configured to allow the IPSec though it even though the IPSec Pass-through is on.

At one of the Close Comcast sites I can try it. Though how does the PIX 501 know to use TCP vs UDP. There was no Config change on the PIX 501.

Thanks!

stownsend · ‎03-02-2011

So I put the PIX 501 at the remote location and now the sh cry ipsec sa sum looks like this:

Current IPSec SA's:            Peak IPSec SA's:
IPSec            :     4         Peak Concurrent SA :    12
IPSec over UDP   :     0         Peak Concurrent L2L :    12
IPSec over NAT-T :     2         Peak Concurrent RA :     0
IPSec over TCP   :     0
IPSec VPN LB     :     0
Total            :     6

I did some Packet Traces and captured the pcap info from the session.

I can see the TFTP transaction between the Phone and the Voicemail Server.

Then it tries to send a Request to the IPOffice 500 server. The Server gets it and on the HQ ASA I can see the Request from the IPPhone and the Reply from the IPOffice sent back to the phone's IP.

Though I dont see that Packet on the Remote PIX.

So my question is how can I be sure that the Packet received on the HQ Pix entered the VPN Tunnel and didn;t get dropped there somewhere?

Thanks!

Jennifer Halim · ‎03-02-2011

The best way to check it is the actual ESP packet because you would like to see if you are receiving the same amounts of ESP packet on both ASA and PIX.

Here it is:

1) Configure access-list that says to and from both peer address and vice versa on both ASA and PIX:

access-list cap-out permit esp host host

2) Clear the VPN tunnel from both ends to get a fresh capture.

3) Apply the capture on both outside interfaces of ASA and PIX:

capture capout access-list cap-out interface outside

4) Do the test - voice stuff

5) Grab the packet capture from both ASA and PIX, and view it in Wireshark

6) Also grab the output of "show cry ipsec sa" from both ASA and PIX at the same time. You should see the number of encrypted packets on ASA should match the decrypted packets on PIX, and vice versa.

stownsend · ‎03-03-2011

So Then I want to see more Packets Encrypted @ HQ than Decrypted @ Remote? That would say that the Packet from the IPOffice server is hitting the Tunnel and getting encrypted and then jsut not ever reaching the Remtoe?

I'm still having a hard time figuring out how the ISP would be able to pluck out these types of Packets that are encrypted. I could see if I was not using VPN, but the ISP shouldn't be able to tell what is in the Packets right? Why let everthing else (that I can find so far) through?

Thanks!

Jennifer Halim · ‎03-05-2011

It's more dropping a couple of packets here and there, not totally blocking every packets, and since ESP is not a stateful connection, then it does not retransmit.

The reason why others are working just fine is typically you will probably browse the internet which will be HTTP or HTTPS, email will be SMTP, and all of those are TCP, and TCP is a stateful connection so if there is packet drops here and there, it will try to retransmit the packet again until the destination end receives everything.

Again, your ISP will not know what is inside the encrypted packet. It's more the encrypted packet itself is stateless, hence there is no retransmission therefore packet drops affects the connection within the encrypted packet.