Solved: Basic trunking and vlans nexus 3132Q

FashionNStyle · ‎09-25-2018

Hello

I need some help setting NEXUS 3132Q-V with basic switching and vlans

1 site I have router/firewall with DHCP and DNS

other site server with esxi with few VMs Windows Servers

between them I have NEXUS 3132Q-V

I am trying to get NEXUS to do trunking between the router and the server

Let say on NEXUS port 1 is connected to the router and port 32 to the server

Here is my configuration on the nexus

config
interface eth1/1

speed 40000
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 10,20,30,40,50,60,70,80,90,100
description TRUNK_PF
no shutdown
exit

config
interface eth1/32

speed 40000
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 10,20,30,40,50,60,70,80,90,100
description TRUNK_ESX
no shutdown
exit

That should pass all defined vlans trough? but I can't ping any host on esxi

Thank you

ADP_89 · ‎10-04-2018

Do you have windows firewall or linux iptables on the server? Can you use Wireshark or iptables to see if the packets are actually getting there. As ping from server to router/firewall works I will exclude problems on the L2.. Looks more something is blocking icmp echo towards the server..

View solution in original post

ADP_89 · ‎09-25-2018

Hello,

Yes that should work. Can you post the following?

- show interface status

- show interface trunk

- show mac address-table

Thanks,

ADP

FashionNStyle · ‎09-26-2018

switch# sh ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

switch# sh ip route static
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

switch# sh ip int brief
IP Interface Status for VRF "default"(1)
Interface IP Address Interface Status

ADP_89 · ‎09-26-2018

Hello,

Those commands are not helping as you switch is only performing L2, please provide the ones I asked you earlier:

- show interface status

- show interface trunk

- show mac address-table

Thanks,

ADP

FashionNStyle · ‎09-26-2018

hello
sorry for inconvenience the server I've used as a router has some firmware issue so I put it on an other server with esxi and the same router but visualized only 1 vlan is used just to test it if it go trough the switch

here are the new configs on the switch to match the new router

config
interface eth1/29-32
speed 40000
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 140
no shutdown
exit

Both servers are connected to the 3132Q switch via Mellanox ConnectX-3 adapter and cisco QSFP-40G-SR4 transceiver

here is what you had requested :

switch# show interface status

Eth1/29 UPLINK_PF connected trunk full 40G QSFP-40G-SR4
Eth1/30 UPLINK_PF connected trunk full 40G QSFP-40G-SR4
Eth1/31 UPLINK_ESX connected trunk full 40G QSFP-40G-SR4
Eth1/32 UPLINK_ESX connected trunk full 40G QSFP-40G-SR4

switch# show interface trunk

--------------------------------------------------------------------------------
Port Native Status Port
Vlan Channel
--------------------------------------------------------------------------------

Eth1/29 1 trunking --
Eth1/30 1 trunking --
Eth1/31 1 trunking --
Eth1/32 1 trunking --

--------------------------------------------------------------------------------
Port Vlans Allowed on Trunk
--------------------------------------------------------------------------------

Eth1/29 140
Eth1/30 140
Eth1/31 140
Eth1/32 140

--------------------------------------------------------------------------------
Port Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------

Eth1/29 none
Eth1/30 none
Eth1/31 none
Eth1/32 none

--------------------------------------------------------------------------------
Port STP Forwarding
--------------------------------------------------------------------------------

Eth1/29 none
Eth1/30 none
Eth1/31 none
Eth1/32 none

--------------------------------------------------------------------------------
Port Vlans in spanning tree forwarding state and not pruned
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Port Vlans Forwarding on FabricPath
--------------------------------------------------------------------------------

Eth1/29 none
Eth1/30 none
Eth1/31 none
Eth1/32 none

switch# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since first seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------

ADP_89 · ‎09-26-2018

Hello,

As you can see vlan 140 is not forwarding in any of the ports, let's see if there is anything wrong with STP:

- show spann vlan 140 detail

**Edit, have you created vlan 140 at all?

conf t

vlan 140

name XX

exit

Thanks,

ADP

paul driver · ‎09-25-2018

Hello
It could be due to various ressons, As the router/fw is performing the routing it correct you should have a trunk on the nx-os facing the router.

Can you first validate you connectivity from the nx-os towards the router/fw?

From the nx-os:

sh ip route
sh ip route static
sh ip int brief

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

FashionNStyle · ‎09-29-2018

hello

Sorry I forgot to create the vlan 140 and after I did I've got connection, now I can ping from the router/fw to server, but not the way around something is still missing

here is again
switch# show interface status

Eth1/29       UPLINK_PF          connected trunk     full    40G     QSFP-40G-SR4
Eth1/30       UPLINK_PF          connected trunk     full    40G     QSFP-40G-SR4
Eth1/31       UPLINK_ESX         connected trunk     full    40G     QSFP-40G-SR4
Eth1/32       UPLINK_ESX         connected trunk     full    40G     QSFP-40G-SR4

switch# show interface trunk

--------------------------------------------------------------------------------
Port          Native Status        Port
              Vlan                  Channel
--------------------------------------------------------------------------------

Eth1/29       1       trunking      --
Eth1/30       1       trunking      --
Eth1/31       1       trunking      --
Eth1/32       1       trunking      --

--------------------------------------------------------------------------------
Port          Vlans Allowed on Trunk
--------------------------------------------------------------------------------

Eth1/29       140
Eth1/30       140
Eth1/31       140
Eth1/32       140

--------------------------------------------------------------------------------
Port          Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------

Eth1/29       none
Eth1/30       none
Eth1/31       none
Eth1/32       none

--------------------------------------------------------------------------------
Port          STP Forwarding
--------------------------------------------------------------------------------

Eth1/29       140
Eth1/30       140
Eth1/31       140
Eth1/32       140

--------------------------------------------------------------------------------
Port          Vlans in spanning tree forwarding state and not pruned
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Port          Vlans Forwarding on FabricPath
--------------------------------------------------------------------------------

Eth1/29       none
Eth1/30       none
Eth1/31       none
Eth1/32       none

switch# show mac address-table
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since first seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY   Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 140      0050.XXXX.XXXX    dynamic   159430     F    F Eth1/31
* 140      0050.XXXX.XXXX    dynamic   159680     F    F Eth1/29

ADP_89 · ‎09-30-2018

Hello,

Ping is bidirectional, if you can ping from A to B on the same LAN but not from B to A it must be some sort of ACL/COPP limiting this kind of traffic. I would say you have to check if the firewall has the interface enabled for ping or if the ACLs/policy are allowing the traffic.

I don't think there are any other problems.

HTH,

ADP

paul driver · ‎09-30-2018

Hello

Does the server have the correct subnet and D/g?

sh ip arp x.x.x. ( server )

Can you post the config of the switch

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

FashionNStyle · ‎10-02-2018

hello

currently trying to get from int eth1/1-2 ROUTER to int eth1/16 and eth1/26 SERVER

the server is set to DHCP and on eth1/16 have to get ip on vlan10 and eth1/26 to get ip from vlan11

nothing is happen

!Time: Tue Oct  2 16:40:35 2018

version 6.0(2)U6(7)
hostname xxxxxxxxxx

no feature telnet
cfs eth distribute

username admin password 5 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  role network-admin

banner motd #xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx#

ssh key rsa 2048
ip domain-lookup
ip access-list copp-system-acl-eigrp
  10 permit eigrp any 224.0.0.10/32
ipv6 access-list copp-system-acl-eigrp6
  10 permit 88 any ff02::000a/128
ip access-list copp-system-acl-icmp
  10 permit icmp any any
ip access-list copp-system-acl-igmp
  10 permit igmp any any
ip access-list copp-system-acl-ntp
  10 permit udp any any eq ntp
  20 permit udp any eq ntp any
ip access-list copp-system-acl-pimreg
  10 permit pim any any
ip access-list copp-system-acl-ping
  10 permit icmp any any echo
  20 permit icmp any any echo-reply
ip access-list copp-system-acl-routingproto1
  10 permit tcp any gt 1024 any eq bgp
  20 permit tcp any eq bgp any gt 1024
  30 permit udp any 224.0.0.0/24 eq rip
  40 permit tcp any gt 1024 any eq 639
  50 permit tcp any eq 639 any gt 1024
  70 permit ospf any any
  80 permit ospf any 224.0.0.5/32
  90 permit ospf any 224.0.0.6/32
ip access-list copp-system-acl-routingproto2
  10 permit udp any 224.0.0.0/24 eq 1985
  20 permit 112 any 224.0.0.0/24
ip access-list copp-system-acl-snmp
  10 permit udp any any eq snmp
  20 permit udp any any eq snmptrap
ip access-list copp-system-acl-ssh
  10 permit tcp any any eq 22
  20 permit tcp any eq 22 any
ip access-list copp-system-acl-stftp
  10 permit udp any any eq tftp
  20 permit udp any any eq 1758
  30 permit udp any eq tftp any
  40 permit udp any eq 1758 any
  50 permit tcp any any eq 115
  60 permit tcp any eq 115 any
ip access-list copp-system-acl-tacacsradius
  10 permit tcp any any eq tacacs
  20 permit tcp any eq tacacs any
  30 permit udp any any eq 1812
  40 permit udp any any eq 1813
  50 permit udp any any eq 1645
  60 permit udp any any eq 1646
  70 permit udp any eq 1812 any
  80 permit udp any eq 1813 any
  90 permit udp any eq 1645 any
  100 permit udp any eq 1646 any
ip access-list copp-system-acl-telnet
  10 permit tcp any any eq telnet
  20 permit tcp any any eq 107
  30 permit tcp any eq telnet any
  40 permit tcp any eq 107 any
ipv6 access-list copp-system-acl-v6routingProto2
  10 permit udp any ff02::0066/128 eq 2029
  20 permit udp any ff02::00fb/128 eq 5353
  30 permit 112 any ff02::0012/128
ipv6 access-list copp-system-acl-v6routingproto1
  10 permit 89 any ff02::0005/128
  20 permit 89 any ff02::0006/128
  30 permit udp any ff02::0009/128 eq 521
ip access-list copp-system-dhcp-relay
  10 permit udp any eq bootps any eq bootps
class-map type control-plane match-any copp-icmp
  match access-group name copp-system-acl-icmp
class-map type control-plane match-any copp-ntp
  match access-group name copp-system-acl-ntp
class-map type control-plane match-any copp-s-arp
class-map type control-plane match-any copp-s-bfd
class-map type control-plane match-any copp-s-bpdu
class-map type control-plane match-any copp-s-dai
class-map type control-plane match-any copp-s-default
class-map type control-plane match-any copp-s-dhcpreq
class-map type control-plane match-any copp-s-dhcpresp
  match access-group name copp-system-dhcp-relay
class-map type control-plane match-any copp-s-dpss
class-map type control-plane match-any copp-s-eigrp
  match access-group name copp-system-acl-eigrp
  match access-group name copp-system-acl-eigrp6
class-map type control-plane match-any copp-s-glean
class-map type control-plane match-any copp-s-igmp
  match access-group name copp-system-acl-igmp
class-map type control-plane match-any copp-s-ipmcmiss
class-map type control-plane match-any copp-s-l2switched
class-map type control-plane match-any copp-s-l3destmiss
class-map type control-plane match-any copp-s-l3mtufail
class-map type control-plane match-any copp-s-l3slowpath
class-map type control-plane match-any copp-s-mpls
class-map type control-plane match-any copp-s-pimautorp
class-map type control-plane match-any copp-s-pimreg
  match access-group name copp-system-acl-pimreg
class-map type control-plane match-any copp-s-ping
  match access-group name copp-system-acl-ping
class-map type control-plane match-any copp-s-ptp
class-map type control-plane match-any copp-s-routingProto1
  match access-group name copp-system-acl-routingproto1
  match access-group name copp-system-acl-v6routingproto1
class-map type control-plane match-any copp-s-routingProto2
  match access-group name copp-system-acl-routingproto2
class-map type control-plane match-any copp-s-selfIp
class-map type control-plane match-any copp-s-ttl1
class-map type control-plane match-any copp-s-v6routingProto2
  match access-group name copp-system-acl-v6routingProto2
class-map type control-plane match-any copp-s-vxlan
class-map type control-plane match-any copp-snmp
  match access-group name copp-system-acl-snmp
class-map type control-plane match-any copp-ssh
  match access-group name copp-system-acl-ssh
class-map type control-plane match-any copp-stftp
  match access-group name copp-system-acl-stftp
class-map type control-plane match-any copp-tacacsradius
  match access-group name copp-system-acl-tacacsradius
class-map type control-plane match-any copp-telnet
  match access-group name copp-system-acl-telnet
policy-map type control-plane copp-system-policy
  class copp-s-selfIp
    police pps 500
  class copp-s-default
    police pps 400
  class copp-s-l2switched
    police pps 200
  class copp-s-ping
    police pps 100
  class copp-s-l3destmiss
    police pps 100
  class copp-s-glean
    police pps 500
  class copp-s-l3mtufail
    police pps 100
  class copp-s-ttl1
    police pps 100
  class copp-s-ipmcmiss
    police pps 400
  class copp-s-l3slowpath
    police pps 100
  class copp-s-dhcpreq
    police pps 300
  class copp-s-dhcpresp
    police pps 300
  class copp-s-dai
    police pps 300
  class copp-s-igmp
    police pps 400
  class copp-s-eigrp
    police pps 200
  class copp-s-pimreg
    police pps 200
  class copp-s-pimautorp
    police pps 200
  class copp-s-routingProto2
    police pps 1300
  class copp-s-v6routingProto2
    police pps 1300
  class copp-s-routingProto1
    police pps 1000
  class copp-s-arp
    police pps 200
  class copp-s-ptp
    police pps 1000
  class copp-s-vxlan
    police pps 1000
  class copp-s-bfd
    police pps 900
  class copp-s-bpdu
    police pps 12000
  class copp-s-dpss
    police pps 1000
  class copp-s-mpls
    police pps 100
  class copp-icmp
    police pps 200
  class copp-telnet
    police pps 500
  class copp-ssh
    police pps 500
  class copp-snmp
    police pps 500
  class copp-ntp
    police pps 100
  class copp-tacacsradius
    police pps 400
  class copp-stftp
    police pps 400
control-plane
  service-policy input copp-system-policy
hardware profile portmode 32x40G

hardware profile front portmode qsfp
snmp-server user admin network-admin auth md5 xxxxxxxxxxxxxxxxxxxxxxxxxx priv xxxxxxxxxxxxxxxxxxxxxxxx localizedkey

vlan 1
vlan 2
  name LAN
vlan 10
  name MGMT_P
vlan 11
  name MGMT_S
vlan 12
  name MGMT_SERV
vlan 140
  name TEST
vrf context management
  ip route 0.0.0.0/0 192.168.xx.xx

interface Ethernet1/1
  description ROUTER
  switchport mode trunk

interface Ethernet1/2
  description ROUTER
  switchport mode trunk

interface Ethernet1/3
  description UPLINK_R320

interface Ethernet1/4
  description UPLINK_R320

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11
  description UPLINK_SERVER_1
  switchport mode trunk

interface Ethernet1/12
  description UPLINK_SERVER_1
  switchport mode trunk

interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16
  description UPLINK_WIN_2016
  switchport access vlan 10

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23/1
  speed 10000
  description TRUNK_X1052P
  switchport mode trunk
  no shutdown

interface Ethernet1/23/2
  description TRUNK_X1052P
  switchport mode trunk
  no shutdown

interface Ethernet1/23/3
  description TRUNK_X1052P
  switchport mode trunk
  no shutdown

interface Ethernet1/23/4
  description TRUNK_X1052P
  switchport mode trunk
  no shutdown

interface Ethernet1/24/1
  speed 10000
  description TRUNK_SG500X_48
  switchport mode trunk
  no shutdown

interface Ethernet1/24/2
  description TRUNK_SG500X_48
  switchport mode trunk
  no shutdown

interface Ethernet1/24/3
  description TRUNK_SG500X_48
  switchport mode trunk
  no shutdown

interface Ethernet1/24/4
  description TRUNK_SG500X_48
  switchport mode trunk
  no shutdown

interface Ethernet1/25

interface Ethernet1/26
  description UPLINK_WIN_2016
  switchport access vlan 11

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29
  description UPLINK_3064_X
  switchport mode trunk

interface Ethernet1/30
  description UPLINK_3064_X
  switchport mode trunk

interface Ethernet1/31
  description UPLINK_3064_X
  switchport mode trunk

interface Ethernet1/32
  description UPLINK_3064_X
  switchport mode trunk

interface mgmt0
  description MGMT_0
  vrf member management
  ip address 192.168.xx.xx/24
line console
line vty
boot kickstart bootflash:/n3000-uk9-kickstart.6.0.2.U6.7.bin
boot system bootflash:/n3000-uk9.6.0.2.U6.7.bin

FashionNStyle · ‎10-02-2018

is there a way I can do any any to any or whatever there is lots of stuff I don't need that will mess me out for now
I need to pass all kind of traffic trough this switch Just need L2 switching that's all for now. Might be in the future when I learn what all of that stuff mean I will turn them on but for now if I can be able to turn that stuff off somehow that's all i need

ADP_89 · ‎10-03-2018

A few questions:

- Why do you have 2 interfaces going to the router without a Port-Chanel?

- How is your router configured? If you have a command line on it post the show run

- Post a show mac address-table

I think you are almost done with this, just need to better connect/configre the router

ADP

FashionNStyle · ‎10-04-2018

Hello

My Router/Firewall is pfSense Installed on Dell R320 and Chelsio T580-SO-CR network controller with 2x40G ports

Some VLANs are going trough NIC1 and some trough NIC2 they are not in LAGG.

NIC1 - cxl0

NIC2 - cxl1

Do I still have to create a port-channel on the switch?

ADP_89 · ‎10-04-2018

No in this case a Port-Channel is not required, but as a best practice I would trunk only the required vlans on eth 1/1 and eth1/2.

Are you still in the case where the pfsense can ping the server but the server cannot ping the pfsense?