1000v / VSG - Packets not being sent to policy engine

Paul Marks · ‎09-25-2012

Hi All,

I've got my 1000v, VNMC and VSG installed and all registered with one another. I've created a simply policy in VNMC for testing of allow all. However, as soon as I assign it to my port-profile all traffic flow stops, yet nothing is ever hitting the policy engine.

This is my port profile:

port-profile type vethernet Test-VM

vmware port-group

switchport mode access

switchport access vlan 10

org root/TestTenant

vservice node VSG profile MyProfile

no shutdown

state enabled

And this is the output of "show vservice brief":

N1000v# show vservice brief

--------------------------------------------------------------------------------

License Information

--------------------------------------------------------------------------------

Type In-Use-Lic-Count UnLicensed-Mod

vsg 2

asa 0

--------------------------------------------------------------------------------

Node Information

--------------------------------------------------------------------------------

ID Name Type IP-Address Mode State Module

1 VSG vsg 10.1.20.99 v-20 Alive 3,

--------------------------------------------------------------------------------

Path Information

--------------------------------------------------------------------------------

Port Information

--------------------------------------------------------------------------------

PortProfile:Test-VM

Org:root/TestTenant

Node:VSG(10.1.20.99) Profile(Id):MyProfile(3)

Veth Mod VM-Name vNIC IP-Address

1 3 2k3 1 10.1.10.200,

Show it appears the 1000v is happy it's received valid config for the VSG. Seems fine too, it has the rule:

firewall# show run rule

rule MyACL/PermitAll@root/TestTenant

action 10 permit

rule default/default-rule@root

action 10 permit

firewall# show run policy

Policy MyACLSet@root/TestTenant

rule MyACL/PermitAll@root/TestTenant order 201

Policy default@root

rule default/default-rule@root order 2

However... Nothing ever gets punted up for policy inspection:

firewall# show policy-engine stats

Policy Match Stats:

default@root : 0

default/default-rule@root : 0 (Permit)

NOT_APPLICABLE : 0 (Drop)

MyACLSet@root/TestTenant : 0

MyACL/PermitAll@root/TestTenant : 0 (Permit)

NOT_APPLICABLE : 0 (Drop)

And I just can't fathom why.... any thoughts greatly appreciated!

Paul Marks · ‎09-25-2012

OK, I'd forgotten my vPath config, I've added that:

vservice path vpath

node VSG profile MyProfile order 1

And the corresponding node config that was there before:

vservice node VSG type vsg

ip address 10.1.20.99

adjacency l2 vlan 20

fail-mode open

And traffic is flowing... but nothing is hitting the policy engine and nothing is denied even if I ask the rules to deny all.

Update: Traffic is flowing only thanks to the "fail open" setting in the VSG setting. That's why counters aren't going up, but I can't work out where I've gone wrong. Any thoughts?

Paul Marks · ‎09-25-2012

OK, I'm confused...

If I use:

vservice node VSG profile MyProfile

In my port-profile I can see packets on the 1000v matching:

N1000v# show vservice statistics

#VSN VLAN: 20, IP-ADDR: 10.1.20.99

Module: 3

#VPath Packet Statistics Ingress Egress Total

Total Seen 26 29 55

Policy Redirects 26 29 55

But no counters going up on the VSG and devices behind that port profile are unreachable.

If I switch to:

vservice path vpath

N1000v# show vservice statistics

#VSN VLAN: 20, IP-ADDR: 10.1.20.99

Module: 3

#VPath Packet Statistics Ingress Egress Total

Total Seen 26 29 55

Policy Redirects 26 29 55

With the above vpath specified, I get just counters on fail-open going up, and nothing on the VSG again, but devices are reachable.

Any thoughts?

Paul Marks · ‎09-25-2012

A small update...

Looking in VNMC, it says the VM/Port-Profile is associated with the IP "10.1.20.99" which it says is a non-existent firewall. That's the managment address of the VSG in question.

I then realised when you add the VSG to VNMC it asks you to create a data address, which I did and set to "10.1.20.100". I tried changing my node settings in the 1000v to use that address instead but it then says that the VSG is down (it's up when set to 10.1.20.99)

What is that data address used for? Or am I going in the wrong direction?

skalje · ‎09-26-2012

Paul,

From your port-profile config :

org root/TestTenant

vservice node VSG profile MyProfile

You have used Security Profile MyProfile. In VNMC have you created Security Profile with same name - MyProfile(name is case sensitive), within Tenant- TestTenant associated with your compute firewall. All the Policy Sets will be part of this security profile.

To check other configurations look fine, From VSM can you get the output for following commands :

Module vem 3 execute vemcmd show vsn binding

Module vem 3 execute vemcmd show vsn config

darren.frowen · ‎07-24-2013

Dear Skalje,

I am having the same issue. I have a ASA managed via ASDM with VService Node configuration iusing 10.1.1.1 for its inside address and for the VSG 10.1.1.2 for the data0 interface on the VSG. However the ASA is reachable but the VSG is not. Previously I had a the vservice node IP address for the VSG in the same subnet as the Management0 address of the Nexus1000v and this worked fine, with service chaining enable and applied to a port profile.

What I am trying to understand is does the VSM and VEM need to communicate with the VSG at Layer2 or Layer3. As the ASA does not seem to need to do, as the ASA is alive with the insdie IP set to 10.1.1.1?

Here is the output you request from the previous post?

gs2-cldnexus-01# Module vem 3 execute vemcmd show vsn binding

VSG Services Enabled | VSG Licenses Available 2

ASA Services Enabled | ASA Licenses Available 2

LTL PATH VSN SWBD IP P-TYPE P-ID

50 1 3 501 10.1.1.2 1 7

50 1 1 501 10.1.1.1 2 3

56 1 3 501 10.1.1.2 1 7

56 1 1 501 10.1.1.1 2 3

66 2 2 506 10.1.2.1 2 5

69 2 2 506 10.1.2.1 2 5

gs2-cldnexus-01#

gs2-cldnexus-01# Module vem 3 execute vemcmd show vsn config

VSG Services Enabled | VSG Licenses Available 2

ASA Services Enabled | ASA Licenses Available 2

VSN# SWBD IP MAC LTLs VER VER-BITMAP

1 501 10.1.1.1 00:50:56:ba:1a:61 2 2 1,2

2 506 10.1.2.1 00:50:56:ba:76:8c 2 2 1,2

3 501 10.1.1.2 00:00:00:00:00:00 2 1 1

Also from show vservice brief

gs2-cldnexus-01# show vservice brief

--------------------------------------------------------------------------------

License Information

--------------------------------------------------------------------------------

Type In-Use-Lic-Count UnLicensed-Mod

vsg 2

asa 2

--------------------------------------------------------------------------------

Node Information

--------------------------------------------------------------------------------

ID Name Type IP-Address Mode State Module

1 CUST01-ASA asa 10.1.1.1 v-501 Alive 3,

2 CUST02-ASA asa 10.1.2.1 v-506 Alive 3,

3 CUST01-VSG-01 vsg 10.1.1.2 v-501 Unreach 3,

--------------------------------------------------------------------------------

Path Information

--------------------------------------------------------------------------------

Name:CUST01-Chain NumOfSvc:2 Mod:3,

Node Order Profile

CUST01-VSG-01 1 CUST01-Server-Compute-Profile

CUST01-ASA 2 Profile-CUST01-Server

--------------------------------------------------------------------------------

Port Information

--------------------------------------------------------------------------------

PortProfile:Profile-CUST01-Server

Org:root/CUST01

Path:CUST01-Chain

Node Profile(Id)

CUST01-VSG-01(10.1.1.2) CUST01-Server-Compute-Profile(7)

CUST01-ASA(10.1.1.1) Profile-CUST01-Server(3)

Veth Mod VM-Name vNIC IP-Address

3 3 cust01-vm01 1 10.1.1.10

10 3 cust01-router01 3 10.1.1.254

PortProfile:Profile-CUST02-Server

Org:root/CUST02

Node:CUST02-ASA(10.1.2.1) Profile(Id):Profile-CUST02-Server(5)

Veth Mod VM-Name vNIC IP-Address

5 3 cust02-vm01 1 10.1.2.10

12 3 cust02-router01 3 10.1.2.254

Regards

Darren Frowen

darren.frowen · ‎07-24-2013

Hi All,

This issue is now resolved. Although I had defined the VSG security Profiles, and assigned the VSG to the Tennat. What I had failed to perform is the Assisgn to VSG in the same Window, seen at the top right hand corner.

It does appear that there is only a requirement for L2 or L3 communication between the VEM and the VSM. As long as you have that using Layer2 address for the VSG data interface or the ASA inside interface for the VSN config is fine, as I belience that the Tunnel created for the vPath communication is between the vmk on the VEM and the Mgmt of the VSM, mac in mac or UDP at L3.

Configuration now looks great and I have service chaining working perfectly;

gs2-cldnexus-01# sh vservice brief

--------------------------------------------------------------------------------

License Information

--------------------------------------------------------------------------------

Type In-Use-Lic-Count UnLicensed-Mod

vsg 2

asa 2

--------------------------------------------------------------------------------

Node Information

--------------------------------------------------------------------------------

ID Name Type IP-Address Mode State Module

1 CUST01-ASA-01 asa 10.1.1.1 v-501 Alive 3,

2 CUST02-ASA-01 asa 10.1.2.1 v-506 Alive 3,

3 CUST01-VSG-01 vsg 10.1.1.2 v-501 Alive 3,

4 CUST02-VSG-01 vsg 10.1.2.2 v-506 Alive 3,

--------------------------------------------------------------------------------

Path Information

--------------------------------------------------------------------------------

Name:CUST01-CHAIN NumOfSvc:2 Mod:3,

Node Order Profile

CUST01-VSG-01 1 CUST01-Server-Comp-Profile-01

CUST01-ASA-01 2 CUST01-Server-Edge-Profile-01

Name:CUST02-CHAIN NumOfSvc:2 Mod:3,

Node Order Profile

CUST02-VSG-01 1 CUST02-Server-Comp-Profile-01

CUST02-ASA-01 2 CUST02-Server-Edge-Profile-01

--------------------------------------------------------------------------------

Port Information

--------------------------------------------------------------------------------

PortProfile:CUST02-Server-Chained-Profile-01

Org:root/CUST02

Path:CUST02-CHAIN

Node Profile(Id)

CUST02-VSG-01(10.1.2.2) CUST02-Server-Comp-Profile-01(16)

CUST02-ASA-01(10.1.2.1) CUST02-Server-Edge-Profile-01(11)

Veth Mod VM-Name vNIC IP-Address

5 3 cust02-vm01 1

12 3 cust02-router01 3

PortProfile:CUST01-Server-Chained-Profile-01

Org:root/CUST01

Path:CUST01-CHAIN

Node Profile(Id)

CUST01-VSG-01(10.1.1.2) CUST01-Server-Comp-Profile-01(12)

CUST01-ASA-01(10.1.1.1) CUST01-Server-Edge-Profile-01(10)

Veth Mod VM-Name vNIC IP-Address

3 3 cust01-vm01 1

10 3 cust01-router01 3

Regards

Darren Frowen