Routing help between cisco nexus and unix server

[asteroid4u]

Hi Team,

I have Nexus 5k and I have connected port eth1/7 to unix host (freebsd). It should communicate with each other using vlan 50.

But when i try to ping from host it is not pinging hopefully i have done wrong configuration because i am very new to network world.

I have assigned 50.50.50.10/24 to free bsd host (I have tried both with vlan tag and without vlan tag )

I have created Port channel po11 for eth1/7 and below is the config details

interface port-channel11
  inherit port-profile freenas-uplink
  description cxl0
  switchport mode trunk
  switchport access vlan 50
  switchport trunk allowed vlan 50
  vpc 11

SCC-TEST-N5K-A(config-if)# show running-config interface eth1/7

!Command: show running-config interface Ethernet1/7
!Time: Fri May 25 21:02:24 2007

version 5.2(1)N1(9a)

interface Ethernet1/7
  description freenas:cxl0
  switchport mode trunk
  switchport access vlan 50
  switchport trunk native vlan 2
  switchport trunk allowed vlan 50
  load-interval counter 3 60
  channel-group 11 mode active

SCC-TEST-N5K-A(config-if)# show running-config

!Command: show running-config
!Time: Fri May 25 21:04:04 2007

version 5.2(1)N1(9a)
feature fcoe

hostname SCC-TEST-N5K-A

feature npiv
no feature telnet
cfs eth distribute
feature interface-vlan
feature lacp
feature vpc
feature lldp

username admin password 5 $1$1HInVQx/$tRRlCCVdCV..ijaMUriGW.  role network-admin

banner motd #Nexus 5000 Switch
#

ip domain-lookup
policy-map type network-qos jumbo
  class type network-qos class-fcoe
    pause no-drop
    mtu 2158
  class type network-qos class-default
    mtu 9216
system qos
  service-policy type network-qos jumbo
slot 1
slot 2
snmp-server user admin network-admin auth md5 0xf2046d99828f820a5a7c955cbfbb9c1d
 priv 0xf2046d99828f820a5a7c955cbfbb9c1d localizedkey

vrf context management
  ip route 0.0.0.0/0 10.104.205.1
vlan configuration 50
vlan 1
vlan 2
  name native-vlan
vlan 3-9
vlan 10
  name management-vlan
vlan 11-19
vlan 20
  name nfs-vlan
vlan 21-29
vlan 30
  name vmotion-vlan
vlan 31-39
vlan 40
  name vm-vlan
vlan 41-49
vlan 50
  name iscsi
vlan 51-100
spanning-tree port type edge bpduguard default
spanning-tree port type network default
port-channel load-balance ethernet source-dest-port
vpc domain 10
  role priority 10
  peer-keepalive destination 10.104.205.73 source 10.104.205.74
  auto-recovery
port-profile default max-ports 512
port-profile type port-channel freenas-uplink
  switchport mode trunk
  switchport trunk native vlan 2
  switchport trunk allowed vlan 50
  spanning-tree port type edge trunk
  load-interval counter 3 60
  state enabled
port-profile type port-channel ucs-ethernet
  switchport mode trunk
  switchport trunk native vlan 2
  switchport trunk allowed vlan 10, 20, 30, 40, 50
  spanning-tree port type edge trunk
  state enabled
port-profile type port-channel vpc-peer-link
  switchport mode trunk
  switchport trunk allowed vlan 10, 20, 30, 40, 50
  spanning-tree port type network
  state enabled


interface Vlan1

interface Vlan50
  no shutdown
  ip address 50.50.50.1/24

interface port-channel10
  inherit port-profile vpc-peer-link
  description vpc peer-link
  vpc peer-link

interface port-channel11
  inherit port-profile freenas-uplink
  description cxl0
  switchport mode trunk
  switchport access vlan 50
  switchport trunk allowed vlan 50
  vpc 11

interface port-channel13
  inherit port-profile ucs-ethernet
  description bheema-a
  vpc 13

interface port-channel14
  inherit port-profile ucs-ethernet
  description bheema-b
  vpc 14

interface port-channel15
  description bheema-a:1/17
  vpc 15

interface port-channel16
  description bheema-b:1/17
  vpc 16

interface fc2/1

interface fc2/2

interface fc2/3

interface fc2/4

interface fc2/5

interface fc2/6

interface fc2/7

interface fc2/8

interface Ethernet1/1

interface Ethernet1/2

interface Ethernet1/3
  description bheema-A:1/19
  switchport mode trunk
  switchport trunk native vlan 2
  switchport trunk allowed vlan 10,20,30,40,50
  channel-group 13 mode active

interface Ethernet1/4
  description bheema-B:1/19
  switchport mode trunk
  switchport trunk native vlan 2
  switchport trunk allowed vlan 10,20,30,40,50
  channel-group 14 mode active

interface Ethernet1/5
  description iscsi-A:1/17
  channel-group 15 mode active

interface Ethernet1/6
  description iscsi-B:1/17
  channel-group 16 mode active

interface Ethernet1/7
  description freenas:cxl0
  switchport mode trunk
  switchport access vlan 50
  switchport trunk native vlan 2
  switchport trunk allowed vlan 50
  load-interval counter 3 60
  channel-group 11 mode active

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

interface Ethernet1/13
  description nexus-b:1\13
  switchport mode trunk
  switchport trunk allowed vlan 10,20,30,40,50
  channel-group 10 mode active

interface Ethernet1/14
  description nexus-b:1/14
  switchport mode trunk
  switchport trunk allowed vlan 10,20,30,40,50
  channel-group 10 mode active

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface mgmt0
  ip address 10.104.205.74/24
line console
line vty
interface fc2/1
interface fc2/2
interface fc2/3
interface fc2/4
interface fc2/5
interface fc2/6
interface fc2/7
interface fc2/8

[willwetherman]

Hi,

With your current configuration you are tagging VLAN 50 on the port-channel interfaces that are connecting to the FreeBSD host. Is the FreeBSD host also tagging vlan 50?

Also I can see that port-channel 11 is using LACP. Can you check if port E1/7 has bundled correctly by posting the output of 'show port-channel summary' from the Nexus 5K switch?

[asteroid4u]

Hi willwetherman,

I Just removed Port channel and LACP Everything.

Now Nexus 5k Eth 1/7 is connected to freebsd host.

Created Vlan 50

assigned 50.50.50.1/24 for vlan 50

Made it interface eth 1/7 as access vlan 50

and assigned 50.50.50.10/24 to  freebsd host  (No gate way configured)

trying to ping from host to vlan ip 50.50.50.1 and from nexus to freebsd ping 50.50.50.10

Both are are failing.

Is any other configuration I need to be done?

[willwetherman]

Hi,

No, you shouldn’t need to configure anything else on the Nexus 5K unless there is a configuration mismatch between the devices.

Can you post the configuration of port Eth 1/7 again as well as the 'ifconfig' output from your FreeBSD host?

Edit: Can you also post the output of 'show spanning-tree vlan 50' from the Nexus 5K as well please?

[asteroid4u]

Hi WillWetherman,

SCC-TEST-N5K-A(config-if)# show running-config interface eth1/7

!Command: show running-config interface Ethernet1/7
!Time: Sun May 27 01:37:06 2007

version 5.2(1)N1(9a)

interface Ethernet1/7
  description freenas:cxl0
  switchport access vlan 50

[root@freenas] ~# ifconfig
cxl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:07:43:2d:e0:a0
    inet 50.50.50.10 netmask 0xffffff00 broadcast 50.50.50.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet 10Gbase-Twinax <full-duplex>
    status: active

SCC-TEST-N5K-A(config-if)# sh running-config interface vlan 50

!Command: show running-config interface Vlan50
!Time: Sun May 27 01:45:17 2007

version 5.2(1)N1(9a)

interface Vlan50
  ip address 50.50.50.1/24

Thanks,

[willwetherman]

Thanks

Can you also post the output of the following show commands from the N5K switch?

show spanning-tree vlan 50
show interface ethernet 1/7 switchport  
show interface vlan 50

[asteroid4u]

Hi willwetherman ,

SCC-TEST-N5K-A# show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol rstp
  Root ID    Priority    32818
             Address     0005.9b24.2f3c
             Cost        1
             Port        4105 (port-channel10)
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     000d.ecb1.d83c
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po10             Root FWD 1         128.4105 (vPC peer-link) Network P2p
Po13             Desg FWD 1         128.4108 (vPC) Edge P2p
Po14             Desg FWD 1         128.4109 (vPC) Edge P2p
Eth1/7           Desg BKN*2         128.135  Network P2p *BA_Inc

SCC-TEST-N5K-A# show interface ethernet 1/7 switchport
Name: Ethernet1/7
  Switchport: Enabled
  Switchport Monitor: Not enabled
  Operational Mode: access
  Access Mode VLAN: 50 (iscsi)
  Trunking Native Mode VLAN: 1 (default)
  Trunking VLANs Allowed: 1-4094
  Voice VLAN: none
  Extended Trust State : not trusted [COS = 0]
  Administrative private-vlan primary host-association: none
  Administrative private-vlan secondary host-association: none
  Administrative private-vlan primary mapping: none
  Administrative private-vlan secondary mapping: none
  Administrative private-vlan trunk native VLAN: none
  Administrative private-vlan trunk encapsulation: dot1q
  Administrative private-vlan trunk normal VLANs: none
  Administrative private-vlan trunk private VLANs: none(0 none)
  Operational private-vlan: none
  Unknown unicast blocked: disabled
  Unknown multicast blocked: disabled

SCC-TEST-N5K-A# show interface vlan 50
Vlan50 is up, line protocol is up
  Hardware is EtherSVI, address is  000d.ecb1.d83c
  Internet Address is 50.50.50.1/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec

Thanks

[willwetherman · Accepted solution]

Ok I can see that ethernet 1/7 is in the Bridge Assurance Inconsistent state.

Can you configure the following under ethernet 1/7 and test again?

interface Ethernet1/7

 spanning-tree port type edge

[asteroid4u]

Hi Willwetherman,

Wow, Thanks a lot it is working fine.

Just wanted to understand

Eth1/7           Desg BKN*2         128.135  Network P2p *BA_Inc

shows Bridge Assurance Inconsistent state?

Could you please tell me test case for spanning tree types edge, default and networking.

I am very new to networking.

[willwetherman]

Hi

You can configure a spanning tree enabled port as either a network port, edge port or a normal port.

'spanning-tree port type network' enables an enhanced feature called Bridge Assurance which monitors the status of the connected device, most commonly another switch. Spanning tree BPDUs are sent out of the port, and if it doesnt receive a BPDU back for a specific period of time, it places the port into the blocking state. Both ends of the link must have Bridge Assurance enabled. This was the cause of your issue as 'spanning-tree port type network default' was enabled globally on the N5K switch. As the FreeBSD host doesn’t support Bridge Assurance, the N5K placed E1/7 into the blocking state as we observed.

'spanning-tree port type edge' is used when the port connects directly to a host like in your situation. The edge ports immediately transitions to the spanning tree forwarding state bypassing the blocking and learning states.

'spanning-tree port type normal' configures the port to use normal spanning tree and is used when connecting to a switch that doesn’t run Bridge Assurance.

This is a good document if you wish to learn about this further

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nx-os/layer2/configuration/guide/b_Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide/config_stp.pdf

[asteroid4u]

Great Explanation! and I will the read document.