Policy Extended Node Configuration Guide

[jeaves@cisco.com]

• Introduction
• Configure Cisco DNA Center for Policy Extended Node
• Site Level Credentials
• IP Pools for Policy Extended Node
• Create Port Channel
• Host Onboarding
• Assign Port - Fabric Edge to Policy Extended Node
• DHCP Requirements
• Example DHCP Scope for Linux/Unix Systems
• Example DHCP Scope for Windows Systems
• Example DHCP Scope for Cisco Routers/Switches
• Verify Policy Extended Node Bringup
• Use-Case Example
• Verify CLI Configuration
• Fabric Edge Device
• Policy Extended Node (IE3400)

Introduction

Prior to release 1.3.3, Extended Node configuration was not all automated, segmentation was static per VLAN and not dynamically assigned per endpoint and SGTs were not assigned on the Extended Nodes themselves so East-West traffic on an individual Node was not enforced.

Release 1.3.3 introduces Policy Extended Node:

• As well as the AAA and interface configuration being orchestrated on the Policy Extended Node, ISE assigns both the VLAN and SGT upon authentication/authorization.
• This allows full fabric segmentation to be extended down to the Policy Extended Node.
• Inline Tagging is enabled between the Policy Extended Node and the connected Fabric Edge.
• Supported only on the IE3400 and IE3400H (Note: Cisco DNA Center 2.1.2.x extends PEN functionality to most Catalyst 9000 Series Switching platforms: 9200/L, 9300/L, 9400, 9500/H)
• Multicast support on Policy Extended node enables endpoints such as video surveillance to be part of the fabric

Starting from Cisco SD-Access 1.3.3 release, Policy Extended Node for the IE3400 and IE3400H is supported connected to Fabric Edge Nodes. In the backend Autoconf is used for Host-onboarding.

Previously enabled “Extended Node” with reduced functionality will continue to be supported on the IE3300, IE4000, IE4010, IE5000, 3560-CX and the Catalyst Digital Building (CDB) switches.

Note:

Policy Extended node should not have any existing configuration for the plug and play to work. If there is any existing configuration please do “write erase” and reload the Policy Extended Node so that it is at this prompt:

Would you like to enter the initial configuration dialog? [yes/no]:

Fabric Edge switches require from release 16.12.2 for any 1.3.3 Cisco DNA Center code. The IE3400/IE3400H Policy Extended Node requires from release 17.1.1

A device is onboarded according to the license of its Extended Node neighbor and its own license:

• If the neighbor is operating with an Essentials license, the device is onboarded as a standard Extended Node, regardless of its license.

• If the neighbor is operating with an Advantage license (network-advantage with or without dna-advantage addon), the device is onboarded as a standard Extended Node if it has an Essentials license.

• If the neighbor is operating with an Advantage license (network-advantage with dna-advantage addon), the device is onboarded as a Policy Extended Node if it has an Advantage license (network-advantage with dna-advantage addon).

• If the device has more than one neighbor, and those neighbors have different license levels, the device is onboarded as a standard Extended Node, regardless of its license.

Licensing information: Cisco Catalyst and Cisco DNA Software Subscription Matrix for Switching

Please see official CCO documentation for latest Extended Node updates. For example, Catalyst Center 2.3.7 user guide.

Configure Cisco DNA Center for Policy Extended Node

 

Site Level Credentials

Make sure credentials in Design > Network Settings > Device Credentials are applied to every site and not just at the Global level. Navigate to the site, select the credentials and hit SAVE.

IP Pools for Policy Extended Node

Make sure IP Address pool for Policy Extended Node is configured. This has to be an IPv4 Pool.

Navigate to Design > Network Settings > IP Address Pools > Global and add ExtNode Pool(s).

Reserve the IP Address Pool(s) under the site/building that the Policy Extended Node will be added to.

Create Port Channel

From release 1.3, SD-Access supports port channel between Policy Extended Node and Fabric Edge. Policy Extended Node and Fabric Edge device is always connected using port-channel (even for single port).

If any fabric level Authentication mode is set, user needs to create a PAGP port-channel on fabric edge in port-channel tab. For no-authentication mode, port-channel will be created automatically.

Follow the following steps if the auth mode is something other than no-authentication to create the port-channel manually:

1) Click on the Fabric Edge device to which the Policy Extended Node is connected and go to port-channel tab, providing the port(s) information and selecting the protocol

2) In the Port Channel tab, click on Create PortChannel

3) In the Port Channel create section, select the interface that is connected to the Policy Extended Node that you want to be part of the port channel.

Note: Even if only one interface is connected to the Policy Extended Node from FE, port channel still needs to be created.

There are three options. For the IE3400/IE3400H choose PAGP.

Please verify that port channel gets created successfully

Host Onboarding

1) Configure Auth Template as “No Authentication” for the Policy Extended Node to come up automatically when connected

If the Auth Template is configured for anything other than “No Authentication” then follow the below steps:

2) Select IP Pool for Policy Extended Node’s management IP in Provision > Fabric > Host Onboarding > Infra_VN.

The Policy Extended Node will be part of the INFRA_VN for Cisco DNA Center’s PnP host onboarding feature.  Click on the INFRA-VN.

Click on Add in the INFRA_VN to add ExtNode Pool created during IP Pool reservation

Click on the Pool Type and select Extended and click Update.

Assign Port - Fabric Edge to Policy Extended Node

Now we need to assign the port channel as an extended node.

1) To assign the port channel as a extended node, go to Provision > Fabric > Host Onboarding > Select Port Assignment > FE connected to the Policy Extended Node.

You should see the port channel that you just created

2) Click on Port-channel1 and click on Assign to assign it as an extended node

3) In the Port Assignments, select extended node for the Connected Device Type from the drop down menu and click on Update
4) Click on Save to push the configuration to the Fabric Edge device and start the Policy Extended Node bringup

DHCP Requirements

The Policy Extended Node requires an IP address provided by a DHCP server to start the PNP process. This request is forwarded in the underlay via the INFRA_VN. As well as providing the management IP of the Policy Extended Node, the DHCP server must provide required information to allow the Policy Extended Node to contact Cisco DNA Center to start the PNP process.

Example DHCP scopes are shown below which includes Option 43 and details information required to contact Cisco DNA Center.

Option 43 includes three type-length-values (TLV). The first value is 5A1D;B2;K4; which specifies the PNP option. The second is the Cisco DNA Center IP address. The third is the port which could be 80 or 443. In total an example is 5A1D;B2;K4;I<Cisco DNA Center IP>;J80;

Example DHCP Scope for Linux/Unix Systems

option dnac code 43 = string;

subnet 10.4.6.0 netmask 255.255.255.0 {

  range 10.4.6.100 10.4.6.200;

  option domain-name-servers 10.1.100.2;

  option domain-name "kernow.com";

  option subnet-mask 255.255.255.0;

  option routers 10.4.6.254;

  option broadcast-address 10.4.6.255;

  option dnac "5A1D;B2;K4;I10.1.150.20;J80;";

  default-lease-time 3600;

  max-lease-time 7200;

}

N.B 10.1.150.20 is the Cisco DNA Center IP address.

Example DHCP Scope for Windows Systems

For Windows systems, the same Option 43 ASCII content is required, you just need to enter the equivalent Hex values for the characters, as shown below

N.B. 10.1.200.26 is the Cisco DNA Center IP address.

Example DHCP Scope for Cisco Routers/Switches

ip dhcp pool ExtendedNodeNW

 network 192.168.17.0 255.255.255.0

 option 43 ascii 5A1D;B2;K4;I10.5.132.10;J80;

 default-router 192.168.17.1

Here, DHCP pool name is ExtendedNodeNW, IP Pool range is 192.168.17.0/24, and in option 43, 10.5.132.10 is the Cisco DNA Center IP Address.

Verify Policy Extended Node Bringup

Policy Extended node should not have any existing configuration for the plug and play to work. If there is any existing configuration please do “write erase” and reload the Policy Extended Node so that it is at this prompt:

Would you like to enter the initial configuration dialog? [yes/no]:

If the IE3400/IE3400H has previously been used for any PNP purposes, the following may be required to completely erase the configuration:

del /f flash:private-config.text

del /f sdflash:private-config.text

del /f flash:config.text

del /f sdflash:config.text

del /f flash:vlan.dat

del /f sdflash:vlan.dat

del /f sdflash:pnp.dat

del /f flash:pnp.dat

del /f flash:*pnp*

del /f sdflash:*pnp*

conf t

no pnp profile pnp-zero-touch

exit

delete /for nvram:*.cer

delete /f flash:pnp-reset-config.cfg

delete /f sdflash:pnp-reset-config.cfg

conf t

crypto key zeroize

yes

no crypto pki certificate pool

yes

no crypto pki trustpoint pnplabel

yes

Vtp mode off

Vtp mode transparent

end

write erase

To check the status of the Policy Extended Node, go to Provision > Devices > Plug and Play

You should see the devices show up in the Plug and Play window and the devices will be in Provisioned state

Once the Policy Extended Nodes are provisioned in Plug and Play, they will start showing up in Fabric inventory and added to site in Managed State. Also, it will be added to Fabric Topology.

Authentication templates will have been provisioned onto the Policy Extended Node and all access ports will be provisioned with the default authentication profile – just as it were a Fabric Edge. Client devices can be connected to these access ports and authentication/authorization should succeed with ISE (ensure the ISE authorization rules and profiles are set just as you would with requests originating from a Fabric Edge i.e. assigning the VLAN and SGT).

Rather than relying on the default authentication profile you can go to Host Onboarding page, select the Policy Extended Node and configure the ports to be connected to AP or other IoT devices.

Use-Case Example

Host 2 is connected to a Fabric Edge (FE2), authenticated and authorized with ISE, placed into the correct VLAN/VN and assigned SGT4 Employees

The company policy is to block Employees from accessing PLC devices. A policy is added in Cisco DNA Center to block traffic from SGT Employees to SGT PLC

Cisco DNA Center pushes this policy to ISE.

A PLC device is connected to a Policy Extended Node (IE3400), authenticated and authorized with ISE, placed into the correct VLAN/VN and assigned SGT19 PLC.

As soon as the Policy Extended Node learns of this new SGT it needs to protect, it downloads the required policies from ISE:

SN-FOC2121Y0WB#sh cts role-based permissions

IPv4 Role-based permissions default:

        Permit IP-00

IPv4 Role-based permissions from group 4:Employees to group 19:PLC:

        Deny_IP_Log-00

RBACL Monitor All for Dynamic Policies : FALSE

RBACL Monitor All for Configured Policies : FALSE

If the Employee tries to access the PLC device, the source SGT (SGT4 Employees) is carried over VXLAN to FE1 and then carried in line from FE1 to the Policy Extended Node. The Policy Extended Node does a source SGT lookup, finds SGT4 Employees (received inline), then does a destination SGT lookup, finds SGT19 PLC (assigned by ISE). That flow is then enforced by the policy:

SN-FOC2121Y0WB#show cts role-based counters

Role-based IPv4 counters

From    To      SW-Denied  HW-Denied  SW-Permitt HW-Permitt SW-Monitor HW-Monitor

*       *       0          0          0          0          0          0    

4       19      0          53         0          0          0          0  

Verify CLI Configuration

Fabric Edge Device

 

Check the port channel and interface configuration (inline tagging enabled on the trunk):

FE2-9300-04#sh run int gig 1/0/2

!

interface GigabitEthernet1/0/2

 switchport mode trunk

 cts manual

  policy static sgt 8000 trusted

 channel-group 1 mode desirable

end

FE2-9300-04#sh run int port-channel 1

Building configuration...

Current configuration : 54 bytes

!

interface Port-channel1

 switchport mode trunk

end

ExtPool Management VLAN on FE

FE2-9300-04#sh run int Vlan1024

Building configuration...

Current configuration : 292 bytes

!

interface Vlan1024

 description Configured from Cisco DNA-Center

 mac-address 0000.0c9f.f45f

 ip address 192.168.17.1 255.255.255.0

 ip helper-address 10.5.130.12

 no ip redirects

 ip route-cache same-interface

 no lisp mobility liveness test

 lisp mobility 192_168_17_0-INFRA_VN-IPV4

end

Host IP VLAN on FE

FE2-9300-04#sh run int Vlan1021

Building configuration...

Current configuration : 590 bytes

!

interface Vlan1021

 description Configured from Cisco DNA-Center

 mac-address 0000.0c9f.f45c

 vrf forwarding Campus_VN

 ip address 192.168.11.1 255.255.255.0

 ip helper-address 10.5.130.12

 no ip redirects

 ip route-cache same-interface

 no lisp mobility liveness test

 lisp mobility 192_168_11_0-Campus_VN-IPV4

 lisp mobility 192_168_11_0-Campus_VN-IPV6

 ipv6 address 2003::1/96

 ipv6 enable

 ipv6 nd managed-config-flag

 ipv6 nd other-config-flag

 ipv6 nd router-preference High

 ipv6 dhcp relay destination ACE::1

 ipv6 dhcp relay source-interface Vlan1021

 ipv6 dhcp relay trust

end

Inline tagging authorized and trusted on Fabric Edge interface towards the Policy Extended Node:

FE2-9300-04#sh cts interface

Global Dot1x feature is Disabled

Interface GigabitEthernet1/0/2:

    CTS is enabled, mode:    MANUAL

    IFC state:               OPEN

    Interface Active for 00:57:45.274

    Authentication Status:   NOT APPLICABLE

        Peer identity:       "unknown"

        Peer's advertised capabilities: ""

    Authorization Status:    SUCCEEDED

        Peer SGT:            8000

        Peer SGT assignment: Trusted

    SAP Status:              NOT APPLICABLE

    Propagate SGT:           Enabled

    Cache Info:

        Expiration            : N/A

        Cache applied to link : NONE

    Statistics:

        authc success:              0

        authc reject:               0

        authc failure:              0

        authc no response:          0

        authc logoff:               0

        sap success:                0

        sap fail:                   0

        authz success:              0

        authz fail:                 0

        port auth fail:             0

    L3 IPM:   disabled.

Interface Port-channel1:

    CTS is disabled.

    L3 IPM:   disabled.

Policy Extended Node (IE3400)

 

Uplink on the Policy Extended Node (with inline tagging enabled):

SN-FOC2121Y0WB#sh run int gig 1/10

!

interface GigabitEthernet1/10

 description PNP STARTUP VLAN

 switchport mode trunk

 cts manual

  policy static sgt 8000 trusted

 channel-group 1 mode desirable

end

SN-FOC2121Y0WB#sh run int port-channel 1

Building configuration...

Current configuration : 54 bytes

!

interface Port-channel1

 switchport mode trunk

end

CTS PAC and Environment-data downloaded from ISE on Policy Extended Node:

SN-FOC2121Y0WB#show cts pacs

AID: B34FA41307051BDEFC62F317FF5DBEC6

PAC-Info:

  PAC-type = Cisco Trustsec

  AID: B34FA41307051BDEFC62F317FF5DBEC6

  I-ID: FOC2121Y0WB

  A-ID-Info: Identity Services Engine

  Credential Lifetime: 15:45:46 UTC Wed Mar 18 2020

PAC-Opaque: 000200B80003000100040010B34FA41307051BDEFC62F317FF5DBEC60006009C000301002DD510E92D3FC4134FA57716B009AABF000000135DF41CD400093A807D3BE2BD82AE0D427EEB7FE5B2CA9465272F31DB47CC670BB06457B45502EDE1EF94B7F3F6720A833E4F26CB82053D11398168E701CA1C1B2B0D968207A800ADCCC0D3B8CB5DE2080166DAD9520403E2A3900CF8AA14C47FBFE923DA825675E99E89B09BD483DE92C2FD477D511049BDB2186CC264817D2EC64DFC2D

Refresh timer is set for 12w4d

SN-FOC2121Y0WB#show cts environment-data

CTS Environment Data

====================

Current state = COMPLETE

Last status = Successful

Local Device SGT:

  SGT tag = 2-00:TrustSec_Devices

Server List Info:

Installed list: CTSServerList1-0001, 1 server(s):

 *Server: 10.1.200.127, port 1812, A-ID B34FA41307051BDEFC62F317FF5DBEC6

          Status = ALIVE

          auto-test = TRUE, keywrap-enable = FALSE, idle-time = 60 mins, deadtime = 20 secs

Security Group Name Table:

    0-00:Unknown

    2-00:TrustSec_Devices

    3-00:Network_Services

    4-00:Employees

    5-00:Contractors

    6-00:Guests

    7-00:Production_Users

    8-00:Developers

    9-01:Auditors

    10-00:Point_of_Sale_Systems

    11-00:Production_Servers

    12-00:Development_Servers

    13-00:Test_Servers

    14-00:PCI_Servers

    15-00:BYOD

    16-00:Intranet

    17-00:Extranet

    18-00:Doctors

    255-00:Quarantined_Systems

Environment Data Lifetime = 86400 secs

Last update time = 15:45:45 UTC Thu Dec 19 2019

Env-data expires in   0:22:59:50 (dd:hr:mm:sec)

Env-data refreshes in 0:22:59:50 (dd:hr:mm:sec)

Cache data applied           = NONE

State Machine is running

Inline tagging authorized and trusted on Policy Extended Node interface towards the Fabric Edge:

SN-FOC2121Y0WB#sh cts interface

Global Dot1x feature is Disabled

Interface GigabitEthernet1/10:

    CTS is enabled, mode:    MANUAL

    IFC state:               OPEN

    Interface Active for 01:06:37.024

    Authentication Status:   NOT APPLICABLE

        Peer identity:       "unknown"

        Peer's advertised capabilities: ""

    Authorization Status:    SUCCEEDED

        Peer SGT:            8000

        Peer SGT assignment: Trusted

    SAP Status:              NOT APPLICABLE

    Propagate SGT:           Enabled

    Cache Info:

        Expiration            : N/A

        Cache applied to link : NONE

    Statistics:

        authc success:              0

        authc reject:               0

        authc failure:              0

        authc no response:          0

        authc logoff:               0

        sap success:                0

        sap fail:                   0

        authz success:              0

        authz fail:                 0

        port auth fail:             0

    L3 IPM:   disabled.

Interface Port-channel1:

    CTS is disabled.

    L3 IPM:   disabled.

[kis8fe_cisco]

I have just tried bringing up an IE3400 connected to a pair of stacked 9300L following these instructions.

Sadly, the IE3400 came up as a regular Extended Node, not as Policy Extended Node. First of all, the license level needs to be Network-Advantage with DNA-Advantage enabled for the PEN role to even show up, without DNA-A the DNAC (mine currently running Ghost release) will show a warning.

After enabling DNA-A on my IE3400 and running a resync, the fabric role selection is showing the PEN role under the default EN role, but after moving the switch of PEN to on, I am not able to click on Add. Which means it is not possible to actually assign the PEN role.

[HOLGER ALIX]

same behavior as @kis8fe_cisco mentioned 
DNAC Version 2.3.5 IE3400 17.9.4a , C9300 17.9.4a 
All running Network/DNA Advantage (before adding to fabric)

Closed-Auth profiles missing, CTS Commands on Trunk missing (both sides) 
Gui shows IE3400 as ext.-Node (PEN Switch automaticaly on)

Has anyone currently depolyed a PEN with DNAC Version 2.3.5 successful? 

[kis8fe_cisco]

@HOLGER ALIX 

I have done some more research and testing regarding this. I'm on 2.3.5.4 and have a few IE3400 running as EN and as PEN.

The issue I had was that I missed the little fine print in the IE3400 data sheet saying that PEN functionality is not supported with certain expansion modules. Namely the IEM-3300 modules. Here is the exact quote from the datasheet: "IE3300 expansion modules can also be plugged with IE3400 base switch. However, this combination prevents support for advanced security feature such as SGT/SGACL on the IE3400 base switch."

How I realized my mistake was that I tried running one of the steps in DNAC that are usually ran with edge nodes. I can't remember if it was provisioning or if I tried changing the fabric role. But the result was that the task failed because some cts related command was not found on my IE3400 with an IEM-3300-16P module. After some digging and asking Cisco I have figured out the limitation mentioned above.

The only strange thing is that you need to enable the right DNA license depending on whether you have or have not attached a 3300 expansion module. Without a module, you're fine with running dna-advantage and the switch will run as PEN. With a module, the license should be downgraded to dna-essentials, because otherwise the DNAC still tries to run it as a PEN which obviously will not work.