VCS-E v8.9 - certificate, SAN and cluster

gcriscuoli · ‎01-19-2017

Hello,
i'm setting up a Cisco VDC platform for a client and it's required a secured environment.
I've read both the guides:
- Cisco VCS Cluster Creation and Maintenance Deployment Guide (X8.9)
- Cisco VCS Certificate Creation and Use Deployment Guide (X8.9)
but still have some doubts about how to correct configure cluster with TLS enforcing mode, because of the following:

I have a cluster of 2 VCS-E, every VCS-E with LAN1 (internal) and LAN 2 (external), and with a local domain:

vcse1.local.platform.it
vcse2.local.platform.it

and a public domain:

vcse1.platform.it
vcse2.platform.it

I read on the Certificate Creation Guide (p.10) that:

"These service discovery domains may or may not match the SIP registration domains. It depends on the deployment, and they don't have to match. One example is a deployment that uses a .local or similar private domain with Unified CM on the internal network, and public domain names for the VCS Expressway FQDN and service discovery. In this case, you need to include the public domain names in the VCS Expressway certificate as SANs. There is no need to include the private domain names used on Unified CM. You only need to list the edge domain as a SAN."

So i create CSR, on every VCS-E, with the public domain's FQDN ( vcse1.platform.it and vcse2.platform.it) as a SAN.

But, on the other way, I read on the Cluster Creation Guide (p.15) that when adding a new peer in a cluster:

"If the new VCS peer has the Advanced Networking option, the PeerN address MUST specify the LAN 1 interface. This interface must not have NAT enabled on it."

Now, this is the first doubt i have:
on the clustering page, when i add a VCS-E and i configure TLS enforcing mode, in the "peer 2 address" field have to put the SAN of the second VCS-E, ma this SAN is the one with the public domain, resolved dy DNS with the LAN 2 ip address, not the LAN 1 ip address as specified in the Cluster Creation Guide...

so what happens is that:
if: peer 2 address = vcse2.platform.it , then: cluster not working because peer 2 address requires LAN 1 address (that is to say the .local fqdn)
if: peer 2 address = vcse2.local.platform.it , then: cluster not working because "vcse2.local.platform.it" is not the SAN of vcse2 server certificate

Has anyone experienced the same problem?

Have i to generate new CSR with, as additional SAN , the .local fqdn?

Second doubt:
which is the role of the "cluster name", in all this cluster & certification procedure?
i was not able to clarify this reading the guides..

thanks anyone who should reply,
Gianluca

Jonathan Unger · ‎01-19-2017

Hi Gianluca,

I have seen this type of dillema come up when X8.9 was released and TLS verification was announced.

There are a couple of approaches you could take in my view:

1. Do not enable TLS verification mode for the cluster.

This does NOT mean that the communication between cluster peers is insecure.
The communication between peers will still use TLS, it will just not be enforcing trust of the certificates.
If the DMZ in which the LAN#1 interfaces exist is under your control there is extremely little risk of not enforcing TLS verification for the clustering.

2. Use an additional 2 PUBLIC DNS names and SAN entries on your VCS-E certificates to enable TLS verification mode for the cluster:

PUBLIC DNS Records:

vcse1.platform.it - Should resolve to LAN#2 IP Address
lan1.vcse1.platform.it - Should resolve to LAN#1 IP Address
vcse2.platform.it - Should resolve to LAN#2 IP Address
lan1.vcse2.platform.it - Should resolve to LAN#1 IP Address

Also add the "lan1" FQDNs to the SANs on your certificate request as additional names. This will allow you to configure the cluster to point at the "lan1" name which resolve to LAN#1. In turn, since the "lan1" FQDNs are in the SANs of the certificate, you can turn on TLS verification for the cluster.

I would recommend option 1 since it is simpler and there is very little extra benefit to option #2.

To answer your cluster name question, the cluster name is required to setup the VCS cluster, and is also required to be present in the list of certificate SANs. Since it is possible to direct client traffic to the cluster name, a SAN entry must exist for it on both VCS node certificates.

Does that help?

gcriscuoli · ‎01-20-2017

Jonathan,
thanks for your reply, good to know i was not the only one with those doubts!

Actually, as for the solution #2, i didn't understand your hint about the "lan1.vcs.platform.it" entry on public (?, or i misunderstood?) DNS. Tha LAN1 ip address is a local one, not routed on internet. I'm going to do like this:

local DNS: resolves "xxxx.local.platform.it" FQDNs into LAN1 ip address
public DNS: resolves "xxxx.platform.it" FQDNs into LAN 2 (NATted) ip address
create certificates on each VCS-E with the following SANs:

PUBLIC : vcseN.platform.it,
PRIVATE: vcseN.local.platform.it,
CLUSTER :vcsecluster.platform.it

create the cluster with the following PEERs names:

PEER 1 (primary): vcse1.platform.it,
PEER 2 : vcse2.local.platform.it,

In this way, it should works (i'm waiting for the certs from CA so i still don't know) 'cause both PEER1 and PEER2 fields are resolved by public/ and locale DNS, respectively, and verified by certificates.

What do you think about? Maybe it is just the same you wrote and i didn't understand that :)

Jonathan Unger · ‎01-20-2017

What you mentioned is similar to what I was trying to convey.

However, you just need to make sure that the DNS server configured on the EXP-E can resolve the LAN#1 and the LAN#2 DNS records.

Additionally I think you would need to have the following peer names specified on the VCS.

PEER 1 (primary): vcse1.local.platform.it,
PEER 2 : vcse2.local.platform.it,

gcriscuoli · ‎01-23-2017

Actually it keeps not working, i don't understand why.. at the moment i have:

on VCS-E1 :

server certificate with:

DNS:vcse01.platform.it, DNS:vcse01.local.platform.it, DNS:vcsecluster.platform.it

trusted CA certificate:

CA root
CA intermediate (company that is our client)
VCS-E2 server certificate

DNS public configured to resolve "platform.it" domain
DNS private configured to resolve "local.platform.it" domain
on Maintenance>Tools>Network Utils>DNS lookup i've verified that all FQDN involved are correctly resolved
on Maintenance>Tools>Network Utils>ping i've verified that all FQDN involved are correctly reachable

on VCS-E2 :

server certificate with:

DNS:vcse02.platform.it, DNS:vcse02.local.platform.it, DNS:vcsecluster.platform.it

trusted CA certificate:

CA root
CA intermediate (company that is our client)
VCS-E1 server certificate

DNS public configured to resolve "platform.it" domain
DNS private configured to resolve "local.platform.it" domain
on Maintenance>Tools>Network Utils>DNS lookup i've verified that all FQDN involved are correctly resolved
on Maintenance>Tools>Network Utils>ping i've verified that all FQDN involved are correctly reachable

I was thinking that this should be the correct configuration, (and i've restarted both vcse) but: nevertheless what I have now is that:

IF on VCS-E1 Ii configure:

Cluster Name = vcsecluster
TLS verif = permissive
PEER 1 (primary): vcse01.platform.it,
PEER 2 : vcse02.local.platform.it,

then I obtain:

PEER 1 (primary): OK
Certificate: Valid
Clustering: This system
PEER 2 : FAILED

Certificate: Invalid
Clustering: Failed

IF on VCS-E1 Ii configure:

Cluster Name = vcsecluster
TLS verif = permissive
PEER 1 (primary): vcse01.local.platform.it,
PEER 2 : vcse02.local.platform.it,

then I obtain:

PEER 1 (primary): X
PEER 2 : X

Any idea about that?

Thanks a lot for you replies

Jonathan Unger · ‎01-23-2017

Hi There,

Did you follow all of the steps as outlined in the VCS Cluster Creation and Maintenance guide?
http://www.cisco.com/c/dam/en/us/td/docs/telepresence/infrastructure/vcs/config_guide/X8-9/Cisco-VCS-Cluster-Creation-and-Maintenance-Deployment-Guide-X8-9.pdf

The configuration on VCS-E1 of the following should be correct:

PEER 1 (primary): vcse01.local.platform.it
PEER 2 : vcse02.local.platform.it

Both peers should be specified by their LAN#1 IP address.

I am not sure what you mean by:

PEER 1 (primary): X

PEER 2 : X

Can you post a screenshot of this?

On another note, You do not need to explicitly import the other cluster peer certificates into the trusted CA certificate section (you only need to import CA certs here).

gcriscuoli · ‎01-25-2017

Yes, i followed what's written in both guides:
- Cisco VCS Cluster Creation and Maintenance Deployment Guide (X8.9)
- Cisco VCS Certificate Creation and Use Deployment Guide (X8.9)
With the "double blue X" I intend right what it is rappresented, that is to say that in the VCS-E clustering configuration page , in that scenario, there are only 2 blue x near the peers fields :)

PEER 1 ADDRESS:

X Certificate:
X Clustering:
PEER 2 ADDRESS :

X Certificate:
X Clustering:

Nothing more than this.
(No "failed" message, no error message.. but only those blue X)

At the moment (due to some functionality tests that we have to do) i've restored default certificates, used IP ADDRESSES instead of FQDN in the Clustering > Peer Address field , specified no cluster name, TLS mode = permissive, and the cluster is UP (but obviuosly it is not secured)

PEER 1 ADDRESS:

X Certificate: Invalid (Common Name in certificate does not match)
V Clustering: This system

PEER 2 ADDRESS:

X Certificate: Invalid (Common Name in certificate does not match)
V Clustering: Active as vcse02 (Resolved 'A.B.C.D' as 'A.B.C.D')

(where A.B.C.D is IP Address of VCS-E02 LAN 1)

It works as a workaround, but doubts still remain :(

Jonathan Unger · ‎01-30-2017

Hi Gianluca,

I labbed this up to check it out for myself and I am getting a similar result (although the x's I see are red, and not blue).

The scenario I tried is similar to yours.

DNS Hostname: exp-node01
DNS Domain Name: example.com
Cluster FQDN: exp-cluster01.example.com
Peer 1: exp-node01.local.example.com

Module="developer.management.databasemanager" Level="ERROR" CodeLocation="databasemanager(537)" Detail="Cluster peer configuration does not match interface configuration and status" Peers="[u'exp-node01.local.example.com']" IP-Status="['192.168.0.230']" IP-Config="['192.168.0.100', '192.168.0.230']" Hostnames-Config="[u'exp-node01.example.com']"

Based on the error message in the log, it looks like the cluster peer name must match the combination of the hostname and the domain (on the System > DNS page) for it to be valid.

I was able to get the cluster working if I configured split DNS on my domain and pointed the EXP-E at the Internal DNS server (so that LAN#1 would be resolved).

DNS Hostname: exp-node01
DNS Domain Name: example.com
Cluster FQDN: exp-cluster01.example.com
Peer 1: exp-node01.example.com

Do you have split DNS available in your deployment?

gcriscuoli · ‎02-01-2017

Hi Jonathan,

thanks for you reply. What do you mean for "split DNS" ?

Jonathan Unger · ‎02-01-2017

Split DNS is the concept that you can resolve an FQDN to one IP address on the internal network, and a different IP address on the external network.

For example, you have a DNS zone for "domain.com" configured on your internal DNS servers and also on your external DNS servers.

You can configure host1.domain.com to resolve to 192.168.0.10 on the internal DNS server and 158.55.22.15 on the external DNS server.

When clients on your internal network are pointed to the internal DNS server, they will resolve the internal address of "host1.domain.com". Clients from the internet will resolve the external IP address.

If you configure your EXP-E FQDN to resolve to the LAN#1 IP address on internal DNS, and LAN#2 on external DNS, you should be able to point the EXP-E at the internal DNS server and enable TLS clustering.

Does that make sense?

gcriscuoli · ‎02-02-2017

Thanks for your explanation!
Yes we manage the internal DNS so we have the possibility to configure it as we prefer. (and to resolve the public VCS-E FQDN (vcse.domain.com) to its private LAN1 IP), so i will consider it (i've just to think about having VCS-E pointing only to internal DNS if is ok for our scenario) and try as soon as possible.
Thanks a lot,
G.