Nexus 3ks to Dell storage server

[Little Bunny]

Hello

We are hooking up a Dell storage server to our Nexus 3k pair and would like a sanity check on the config. We propose using a VPC on the Nexus side and standard NIC bonding on the Dell side. Here is the draft Nexus config:

interface port-channel32

  description VPC to Dell Storage

  switchport mode trunk

  switchport trunk allowed vlan 2001,2008,2254,4093

  spanning-tree port type edge trunk

  vpc 32

interface Ethernet1/32

  description VPC to Dell Storage

  switchport mode trunk

  switchport trunk allowed vlan 2001,2008,2254,4093

  spanning-tree port type edge trunk

  channel-group 32 mode active

  no shutdown

We tried first without setting the native vlan but it didn't work and this showed up in the logs:

"%ETH_PORT_CHANNEL-3-COMPAT_CHECK_FAILURE: port native VLAN is not compatible"

We haven't tested with the native vlan set yet but are planning on adding "switchport trunk native vlan 4093". Is it possible that was the cause of it not working or are we missing something else? We'd like to make sure we're prepared for our next attempt.

Let me know if you need more config snippets.

Thanks

LB

[Reza Sharifi]

Hi,

Native vlan should not be the cause of this issue.  Is the PO up and running?

What is the output of "sh portc summary" from both Nexus?

Is the storage head configured correctly?

Where is the SVI for the storage vlan?

HTH 

[Little Bunny]

So it turns out its all configured correctly but there is an issue with ARP within the VPC.

To answer your questions Reza:

- The PO comes up fine

- The storage side is configured correctly

- The SVI resides on the same device

We will be trying to bring up the storage on a single link without vpc to see if it works and narrow down the cause.

This is the VPC configuration:

!Primary side:

vpc domain 100
  role priority 100
  system-priority 5000
  peer-keepalive destination 192.168.255.2 source 192.168.255.1
  delay restore 60
  auto-recovery
  ip arp synchronize

!Secondary side:

vpc domain 100
  system-priority 5000
  peer-keepalive destination 192.168.255.1 source 192.168.255.2
  delay restore 60
  auto-recovery
  ip arp synchronize

Are we missing something in the VPC config?

Thanks

LB

[Reza Sharifi]

The vPC config looks good.  You may want to add peer-gateway to the vPC config, so both switches can forward when using HSRP and not just the active one.

One other thing, can you try connecting the storage to both Nexus without any vPC or Portcahnnle (simply as an access or trunk port) and test again?

HTH

[Little Bunny]

Thanks for the tip, we will add vpc peer-gateway in. Is there any chance it might be service impacting or have any adverse effects? Our configuration isn't in the best shape in general so if doing this might break something else we'll need to do it in a maintenance window.

We just tried connecting the storage as a single trunk without port-channel or vpc and still couldn't reach the server. We see the mac address on the port but ARP is empty. Any other suggestions to try? The IP is definitely configured correctly, we just tested moving it to another NIC and connected it to a different switch and it worked so it is something wrong between the Nexus and server.

Thanks

LB

[Reza Sharifi]

Is there any chance it might be service impacting or have any adverse effects? Our configuration isn't in the best shape in general so if doing this might break something else we'll need to do it in a maintenance window.

I would recommend doing it in a maintenance window as it may cause an STP reconvergence and since the STP priorities need to be the same for both switches.

We just tried connecting the storage as a single trunk without port-channel or vpc and still couldn't reach the server. We see the mac address on the port but ARP is empty. Any other suggestions to try? The IP is definitely configured correctly, we just tested moving it to another NIC and connected it to a different switch and it worked so it is something wrong between the Nexus and server.

Not that this should have any effect but for the sake of testing, can you hard set the ports on the nexus to Gig or whatever speed the storage is using?

HTH 

[Little Bunny]

Would it help if I send the spanning-tree priority info for both sides?

The port-channel and interface are negotiating correctly at 10G-full:

CA1CORE1(config-if)# show int status | i 32
Eth1/32 VPC to Dell Storag connected trunk full 10G SFP-H10GB-CU3M
Po32 VPC to Dell Storag connected trunk full a-10G --

I just tried hard-coding the speed to 10G but it made no difference.

On the server side the LACP link shows as up.

Thanks

Amy

[Reza Sharifi]

How about just simply using the physical interfaces with no Portchannel and no LACP, etc?

[Reza Sharifi]

Example with no PO or vPC. And the same on the storage side.

interface Ethernet1/32

  description VPC to Dell Storage

  switchport mode trunk

  switchport trunk allowed vlan 2001,2008,2254,4093

   no shutdown

[Little Bunny]

Tested a couple more things and it only works when in access mode. Trunk mode does not work.

[Reza Sharifi]

ok, I was going to ask about that earlier.

Why do you have trunk link to the storage?

The storage is usually all in one vlan, and so the port should be configured as an access port.

HTH

[Little Bunny]

Ok perhaps we can just leave it as it is then :)

Thanks

Amy