DNAC Installation IP connectivity issues

[Rajesh Kongath]

Hello Cisco Community

We are building a new DNAC and facing some issues. configuration as follows

• Appliance model : DN2-HW-APL-XL
• Software Version : Version 1.3.3.9
• Enterprise IP / enp69s0f0: 10.18.4.85 / 28 with Default gateway 
• Cluster Subnet IP / enp69s0f1: 10.18.4.101 / 28
• Management IP / enp53s0f0: 10.18.4.117 / 28 With Static route 10.0.0.0 255.0.0.0 10.18.4.113
• All the switch port are access with corresponding vlan

Note : I have updated the IPs with masks

the issue is , DNAC always using management interface for all communications, even to reach default gateway of enterprise network. we can not access GUI thru enterprise port, only through the management. 

Appreciate your support to solve the issue.. thanks in advance. 

[balaji.bandi]

how come all the network IP are in the same IP range? is this subnetted in the same /24 address?

all are 10.18.4.X  range? ( bare in mind once the installation is done, you can not change any IP address as per I know, you need to re-image again from start) - so that is the reason IP plan very important for DNAC installation or SD-Access implementation.

(Required) 10-Gbps Enterprise Port (enp94s0f0/enp69s0f0, Network Adapter 3): The purpose of this port is to enable Cisco DNA Center to communicate with and manage your network. Connect this port to a switch with connections to the enterprise network and configure one IP address with subnet mask for the port.

Have you looked at the installation guide :

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/1-3-3-0/install_guide/2ndGen/b_cisco_dna_center_install_guide_1_3_3_0_2ndGen/b_cisco_dna_center_install_guide_1_3_2_0_M5_chapter_01.html

[Rajesh Kongath · Accepted solution]

Thank you @balaji.bandi &  @Tomas de Leon for your response. Apologies for not properly giving the subnet details. the subnet mask is /28 hence interface IPs were not overlapping. However, we found the static route is having an overlap, as we have given 10.0.0.0 / 8 thru management, which was always choosing as the preferred interface as it got more specific route. I have removed the /8 route and added multiple /24 to as static route and problem got resolved.

Thank you for your support, apart from your helps, I found once more helpful community posts in this regard

https://community.cisco.com/t5/cisco-digital-network/dna-installation-gateway-static-route-configuration/td-p/3816332

Good Day!

[trapasso]

Can you give us a breakdown of how you configured each interface, similar to how you wrote it up in the original description of your problem.  Would like to see how you have the IP, subnet mask, gateway and static routes configured for each interface.

Thanks

[Tomas de Leon]

FOR DN2 Appliance (M5) - Model XL
---------------------------------
NETWORK ADAPTER #5 (enp53s0f0) - Management (recommended)
NETWORK ADAPTER #6 (enp53s0f1) - Cloud Update Connectivity (recommended)
NETWORK ADAPTER #7 (enp69s0f0) - Enterprise Network (recommended)
NETWORK ADAPTER #8 (enp69s0f1) - Intra Cluster Link (recommended)

So, the first thing I noticed is that you have overlapping IP addresses. Each of the Interfaces should be in separate Subnets and VLANs. I am making the assumption that you are using a /24.

Some comments:

• Whichever interface is going to be used to access the Internet will be configured with the default gateway.
• The other interface (either Management or Enterpise) will have static routes.
• The "Cluster Port" & Intraclusterlink is a "Private" network and vlan so this can be anything like 1.1.1.0/24.

Important notes:

• The IP subnets should not overlap. This also includes the services & cluster internal subnets that you configured during the installation.
• Once the Cluster interface is configured, it cannot be changed. To change any configuration to this interface requires a complete rebuild from ISO File.

EXAMPLE OF AN IP NETWORKING CONFIGURATION & ROUTES SETUP
### GET CISCO DNA CENTER CLUSTER CONTAINER & SERVICES SUBNET
#
$ etcdctl get /maglev/config/cluster/cluster_network | jq '.container_subnet' 
$ etcdctl get /maglev/config/cluster/cluster_network | jq '.cluster_subnet'


For example:
------------
$ etcdctl get /maglev/config/cluster/cluster_network | jq '.container_subnet'
"172.31.240.0/21"

$ etcdctl get /maglev/config/cluster/cluster_network | jq '.cluster_subnet'
"172.31.248.0/21"




### GET CISCO DNA CENTER INTERFACE CONFIGURATION
#
$ etcdctl get /maglev/config/node-< member.ip.address >/network | python -mjson.tool


$ ip a | egrep "enp|eno|management|internet|cluster|enterprise"
2: enp69s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master enterprise state UP group default qlen 1000
3: enp69s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master cluster state UP group default qlen 1000
6: enp53s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master management state UP group default qlen 1000
8: enp53s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master internet state UP group default qlen 1000
10: management: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 2.2.2.207/24 brd 2.2.2.255 scope global management
inet 2.2.2.208/32 scope global management
11: internet: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 3.3.3.207/24 brd 3.3.3.255 scope global internet
inet 3.3.3.208/32 scope global internet
12: enterprise: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.211.207/24 brd 192.168.211.255 scope global enterprise
inet 192.168.211.208/32 scope global enterprise
13: cluster: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 1.1.1.207/24 brd 1.1.1.255 scope global cluster
inet 1.1.1.208/32 scope global cluster


$ etcdctl get /maglev/config/node-1.1.1.207/network | python -mjson.tool
[
{
"inet": {
"dns_servers": [
"192.168.211.213"
],
"gateway": "",
"host_ip": "192.168.211.207",
"netmask": "255.255.255.0",
"routes": [
{
"gateway": "192.168.211.1",
"netmask": "255.255.0.0",
"network": "192.168.0.0"
},
{
"gateway": "192.168.211.1",
"netmask": "255.255.0.0",
"network": "172.16.0.0"
}
]
},
"inet6": {
"host_ip": "",
"netmask": ""
},
"interface": "enterprise",
"intra_cluster_link": false,
"lacp_mode": false,
"lacp_supported": true,
"slave": [
"enp69s0f0",
"enp237s0f2"
]
},
{
"inet": {
"dns_servers": [],
"gateway": "",
"host_ip": "1.1.1.207",
"netmask": "255.255.255.0",
"routes": []
},
"inet6": {
"host_ip": "",
"netmask": ""
},
"interface": "cluster",
"intra_cluster_link": true,
"lacp_mode": false,
"lacp_supported": true,
"slave": [
"enp69s0f1",
"enp237s0f3"
]
},
{
"inet": {
"dns_servers": [
"8.8.8.8"
],
"gateway": "",
"host_ip": "2.2.2.207",
"netmask": "255.255.255.0",
"routes": [
{
"gateway": "2.2.2.1",
"netmask": "255.0.0.0",
"network": "10.0.0.0"
}
]
},
"inet6": {
"host_ip": "",
"netmask": ""
},
"interface": "management",
"intra_cluster_link": false,
"lacp_supported": true,
"slave": [
"enp53s0f0",
"enp237s0f0"
]
},
{
"inet": {
"dns_servers": [
"8.8.8.8"
],
"gateway": "3.3.3.1",
"host_ip": "3.3.3.207",
"netmask": "255.255.255.0",
"routes": []
},
"inet6": {
"host_ip": "",
"netmask": ""
},
"interface": "internet",
"intra_cluster_link": false,
"lacp_supported": true,
"slave": [
"enp53s0f1",
"enp237s0f1"
]
}
]