VCS-E Not forwarding provisioning request from Jabber to VCS-C

[abertram]

Hello, I recently had to move a VCS-C and VCS-E to new segments on the network.  VCS-C went to another internal vlan and the VCS-E went to a new DMZ interface of the firewall.  Everything worked fine prior to this change but to facilitate the change I had to obviously change the IPs on the appliances and also update the Traversal IP address on the VCS-C for the Traversal Zone.  After these changes I'm unable to register Jabber Video clients anymore.  They can register fine to the VCS-C but when I do a debug level log for network.sip on the VCS-E I see the provisioning@domain.com SUBSCRIBE request come to the VCS-E but it immediately rejects with a 404 Not Found.  I don't see the request sent to the VCS-C via Traversal Zone nor do I see the receipt of this request on the VCS-C.  H323/SIP registration is active for the traversal.  Also outbound communication to external codecs works.  I've deleted and re-created Traversal Zones on both appliances as well as recreated search rules with no luck.  Almost as if after the IP address change the VCS-E is holding on to something invalid or not listening to it's search rules.  I have the generic "any" "any" rule to send all requests to the traversal zone as well as the standard DNS search rule after that.  I've disabled all but the traversal zone search rule with no luck there either.  Appears to not even try to send to VCS-C.

Any suggestions are appreciated.

[Zac Colton]

I have read through this thread, and there are a few things I would like to point. I strongly suggest not using hard coded SIP routes on the Expressway. These are normally used on the VCS that also runs the provisioning services. These routes are there to route the provisioning and phone book request to the local service. The provisioning requests from the Expressway should really go through the traversal zone. SIP routes on the expressway that point to the control defeats the purpose of a firewall traversal zone. Also, the search rule for the DNS zone should not be set to any alias. This could cause SIP messaging that should stay local be sent out the DNS zone and routed back into the Expressway's default zone - basically a call loop. If you can share your complete xconfig and xstatus of both the Control and Expressway, I can assist on identifying and correcting the configuration that is causing the issue.

- Zac Colton

Sent from Cisco Technical Support iPhone App

[Paulo Souza]

Hi Zac,

Thanks for your reply.

Well, as I stated before, I suggested SIP Routes only for testing purposes. That's why I said clearly:

Paulo Souza wrote:
Of course, that is just for test, that is not the correct way to the communication work between VCSC and VCSE.

That was just for testing, as it seems to be an strange behavior, VCS not routing provisioning messages.

Paulo Souza

Was my response helpful? Please rate useful replies and remember to mark any solved questions as "answered".

[abertram]

Thanks for the response Zac.  As Paulo mentioned the sip route was only to force the hand of the VCSE which did not seem to change anything.  I have a TAC case open on the issue currently if you want to look at it.  You should see the xstat and xconfig attached to the notes if you have visability.  SR#627236283.

I'm going to be on the customer site tomorrow and as you mentioned Paulo my next step was to do a factory reset.  Won't take much to re-configure everything.

-Adam

[Zac Colton]

Adam,

There are numerous configuration issue, including the over all design of the deployment. Issues that you could (but may not have yet come accross) are numerous. The TAC engineer that is the current owner of the service request works at the same location as I. I recommend that we continue working on this issue through TAC. I will contact the TAC engineer tomorrow morning and discuss this case with her. I will ask her to schedule a WebEx meeting withi you so that we can discuss all f the config issues that I have found.

- Zac Colton

[abertram]

Zac,

When you say numerous configuration issues I'm curious what you are referring to as this is a fairly vanilla setup of VCS-C and VCS-E and configured per the X7.2 deployment guide for Control and expressway.  Feel free to PM me if there are customer sensitive specifics that I can look at this evening.

-Adam

[Paulo Souza]

Hi folks,

When you get a solution for the problem, please, let me know. I am really interested on this case, as it seems to be a strange behavior of VCS.

=)

Paulo Souza

Was my response helpful? Please rate useful replies and remember to mark any solved questions as "answered".

[Zac Colton]

The exact cause of the issue is that the VCS Expressway is configured with an internal IP address only, and that there is an external firewall that is preforming a NAT from the external IP address to the internal IP address. For this design to function, the VCS Expresway requires the Dual NIC Option key installed, and the external LAN interface would contain a NAT configuration. The reason that the SUBSCRIBE message gets a 404 return message is because the route header field contains the SIP route referenceoing the external IP address. The VCS is seeing in this and sending back a  warning: 399 "Policy Response" since it does not know that it is suposed to respond to messages destined for the exernal IP address.

As discussed earlier, there are other issues, and these should be handled within the TAC service request.

- Zac Colton

[Paulo Souza]

Great Zac!   =)

That is true! Now I am checking the SIP SUBSCRIBE message and it really has an external IP address in the "route" field. Then this address should be configured as NAT address in VCS (and that requires dual nic option key), otherwise VCS won't recognize the external NAT address.

Now it makes sense!

However, the strange point is, I have resolved a case just like that here in the community (I didn't find the case unfortunately), but on that case, the error message wasn't 404 not found. In fact, in that case, after receiving the provisioning message, VCSe tried to connect to the external IP address placed in the "route" field, then it received a "TCP connection error" and the registration failed, but the client didn't receive any responde from VCSe.

Here we have different behavior. Now I am confusing about how VCS handle this kind of situation... Can you clarify it? What is the expected behavior for that case?

Paulo Souza

Was my response helpful? Please rate useful replies and remember to mark any solved questions as "answered".

[Zac Colton]

Paulo,

To explain the behavior of that other case, I would need to see the xconfig/xstatus of the VCS Control, of the Expressway, the topology, and a diagnostic log with the failure.

- Zac Colton

[Paulo Souza]

Hi Zac,

No problem, don't worry about it.

Now I know that the "404 not found" message received from VCS can also be related to NAT issues. I didn't know that, once I have never seen a case with NAT issues where that 404 error was sent by VCSe.    =)

Thank you very much for sharing the solution with me. I hope to be able to help another people here with similar problems.

Great Zac!!!!  +5

Paulo Souza

Was my response helpful? Please rate useful replies and remember to mark any solved questions as "answered".

[Zac Colton]

Paulo,

The important thing to take out of this is that it is vital to have a complete understadning of the full topology to effectively diagnose call flow issues. Once you can see the full design, the messaging (and behavior) in the diagnostic logs are much easier to understand.

- Zac Colton

[abertram]

It still does not explain why this worked before in the DMZ.  Granted I realize maybe not ideally but it did work and the only element that was different was the ip was in the 172.16.x.x private block instead of the 10.x.x.x private block in the DMZ.

[Zac Colton]

Adam,

I would need a full topology of what it was. It is possible that the firewall in the previous deployment was also running ALG, and the external SIP communication was only tcp and not tls. The ALG would over-write the external IP with the internal IP address. This is only a theory, as, like I said, I would have needed to see it in action.

Keep in mind that ALG is not supported, and even though it appears to function, there could be issues down the road.

- Zac Colton