problem ping vlan address on nexus 3k

[malmsteen81]

hi, i have configured vpc on 2 nexus 3064. That i have configured a port-channel with cisco 3750 and seems ok. But now i have problem with traffic. So i configured an ip address on vlan 1

n3k-01 -> 10.39.0.110/21

n3k-02 -> 10.39.0.110/21

i have create a vrf context withj ip route 0.0.0.0/0 10.39.0.5 (that is the firewall ip address) but i can't ping this ip address from the lan and also directly from cli's switches.

n3k-ced-01(config)# ping 10.39.0.111 vrf lan

PING 10.39.0.111 (10.39.0.111): 56 data bytes
ping: sendto 10.39.0.111 64 chars, No route to host
Request 0 timed out
ping: sendto 10.39.0.111 64 chars, No route to host
Request 1 timed out
ping: sendto 10.39.0.111 64 chars, No route to host
Request 2 timed out
ping: sendto 10.39.0.111 64 chars, No route to host
Request 3 timed out
ping: sendto 10.39.0.111 64 chars, No route to host
Request 4 timed out

i also connected a server on a nexus and network in/out network traffic from/to this server dosen't work

so i'm new into nx-os, where is the problem? 

thanks

Andrea

[Mark Malone]

Hi
......That i have configured a port-channel with cisco 3750 and seems ok.
You can confirm this with show vpc brief , it will show if the VPC has any issues and if its a success

Is this a typo as you cant have same ip on both switches or you will have issues as its clustered environment

n3k-01 -> 10.39.0.110/21
n3k-02 -> 10.39.0.110/21

Dont use vlan 1 its used for control traffic too , shut it down and use another number , you want to separate the control and data plane traffic

do you require traffic to be in a VRF ? you dont have to use a vrf , only for mgmt traffic on the mgmt0 interface , if your trying to ping in the vrf everything else has to be in the same vrf too as its isolated

can you post the config ,take a look just remove the passwords

[malmsteen81]

Hi, no i don't need vrf because i will use only layer2 (maybe in future i will configure inter-vlan routing).

I need to use vlan1 because all other switches use vlan1 (we have a flat enviroement). 

this is the configuration 

!Command: show running-config
!Running configuration last done at: Fri Jan 17 14:20:38 2020
!Time: Fri Jan 17 16:15:38 2020

version 7.0(3)I7(6) Bios:version 4.5.0
hostname n3k-ced-01
vdc n3k-ced-01 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 104
limit-resource u4route-mem minimum 128 maximum 128
limit-resource u6route-mem minimum 96 maximum 96
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8

cfs eth distribute
feature interface-vlan
feature lacp
feature vpc
feature lldp

ssh key rsa 2048
no ip domain-lookup
service unsupported-transceiver
ip access-list copp-system-acl-eigrp
10 permit eigrp any 224.0.0.10/32
ipv6 access-list copp-system-acl-eigrp6
10 permit eigrp any ff02::a/128
ip access-list copp-system-acl-icmp
10 permit icmp any any
ip access-list copp-system-acl-igmp
10 permit igmp any any
ip access-list copp-system-acl-ntp
10 permit udp any any eq ntp
20 permit udp any eq ntp any
ip access-list copp-system-acl-pimreg
10 permit pim any any
ip access-list copp-system-acl-ping
10 permit icmp any any echo
20 permit icmp any any echo-reply
ip access-list copp-system-acl-routingproto1
10 permit tcp any gt 1024 any eq bgp
20 permit tcp any eq bgp any gt 1024
30 permit udp any 224.0.0.0/24 eq rip
40 permit tcp any gt 1024 any eq 639
50 permit tcp any eq 639 any gt 1024
70 permit ospf any any
80 permit ospf any 224.0.0.5/32
90 permit ospf any 224.0.0.6/32
ip access-list copp-system-acl-routingproto2
10 permit udp any 224.0.0.0/24 eq 1985
20 permit 112 any 224.0.0.0/24
ip access-list copp-system-acl-snmp
10 permit udp any any eq snmp
20 permit udp any any eq snmptrap
ip access-list copp-system-acl-ssh
10 permit tcp any any eq 22
20 permit tcp any eq 22 any
ip access-list copp-system-acl-stftp
10 permit udp any any eq tftp
20 permit udp any any eq 1758
30 permit udp any eq tftp any
40 permit udp any eq 1758 any
50 permit tcp any any eq 115
60 permit tcp any eq 115 any
ip access-list copp-system-acl-tacacsradius
10 permit tcp any any eq tacacs
20 permit tcp any eq tacacs any
30 permit udp any any eq 1812
40 permit udp any any eq 1813
50 permit udp any any eq 1645
60 permit udp any any eq 1646
70 permit udp any eq 1812 any
80 permit udp any eq 1813 any
90 permit udp any eq 1645 any
100 permit udp any eq 1646 any
ip access-list copp-system-acl-telnet
10 permit tcp any any eq telnet
20 permit tcp any any eq 107
30 permit tcp any eq telnet any
40 permit tcp any eq 107 any
ipv6 access-list copp-system-acl-v6routingProto2
10 permit udp any ff02::66/128 eq 2029
20 permit udp any ff02::fb/128 eq 5353
30 permit 112 any ff02::12/128
40 permit pim any ff02::d/128
ipv6 access-list copp-system-acl-v6routingproto1
10 permit 89 any ff02::5/128
20 permit 89 any ff02::6/128
30 permit udp any ff02::9/128 eq 521
ip access-list copp-system-dhcp-relay
10 permit udp any eq bootps any eq bootps
class-map type control-plane match-any copp-icmp
match access-group name copp-system-acl-icmp
class-map type control-plane match-any copp-ntp
match access-group name copp-system-acl-ntp
class-map type control-plane match-any copp-s-arp
class-map type control-plane match-any copp-s-bfd
class-map type control-plane match-any copp-s-bpdu
class-map type control-plane match-any copp-s-dai
class-map type control-plane match-any copp-s-default
class-map type control-plane match-any copp-s-dhcpreq
class-map type control-plane match-any copp-s-dhcpresp
match access-group name copp-system-dhcp-relay
class-map type control-plane match-any copp-s-dpss
class-map type control-plane match-any copp-s-eigrp
match access-group name copp-system-acl-eigrp
match access-group name copp-system-acl-eigrp6
class-map type control-plane match-any copp-s-glean
class-map type control-plane match-any copp-s-igmp
match access-group name copp-system-acl-igmp
class-map type control-plane match-any copp-s-ipmcmiss
class-map type control-plane match-any copp-s-l2switched
class-map type control-plane match-any copp-s-l3destmiss
class-map type control-plane match-any copp-s-l3mtufail
class-map type control-plane match-any copp-s-l3slowpath
class-map type control-plane match-any copp-s-mpls
class-map type control-plane match-any copp-s-pimautorp
class-map type control-plane match-any copp-s-pimreg
match access-group name copp-system-acl-pimreg
class-map type control-plane match-any copp-s-ping
match access-group name copp-system-acl-ping
m class-map type control-plane match-any copp-s-ptp
class-map type control-plane match-any copp-s-routingProto1
match access-group name copp-system-acl-routingproto1
match access-group name copp-system-acl-v6routingproto1
class-map type control-plane match-any copp-s-routingProto2
match access-group name copp-system-acl-routingproto2
class-map type control-plane match-any copp-s-selfIp
class-map type control-plane match-any copp-s-ttl1
class-map type control-plane match-any copp-s-v6routingProto2
match access-group name copp-system-acl-v6routingProto2
class-map type control-plane match-any copp-s-vxlan
class-map type control-plane match-any copp-snmp
match access-group name copp-system-acl-snmp
class-map type control-plane match-any copp-ssh
match access-group name copp-system-acl-ssh
class-map type control-plane match-any copp-stftp
match access-group name copp-system-acl-stftp
class-map type control-plane match-any copp-tacacsradius
match access-group name copp-system-acl-tacacsradius
class-map type control-plane match-any copp-telnet
match access-group name copp-system-acl-telnet
policy-map type control-plane copp-system-policy
class copp-s-default
police pps 400
class copp-s-l2switched
police pps 200
class copp-s-ping
police pps 100
class copp-s-l3destmiss
police pps 100
class copp-s-glean
police pps 500
class copp-s-selfIp
police pps 500
class copp-s-l3mtufail
police pps 100
class copp-s-ttl1
police pps 100
class copp-s-ipmcmiss
police pps 400
class copp-s-l3slowpath
police pps 100
class copp-s-dhcpreq
police pps 300
class copp-s-dhcpresp
police pps 300
class copp-s-dai
police pps 300
class copp-s-igmp
police pps 400
class copp-s-eigrp
police pps 200
class copp-s-pimreg
police pps 200
class copp-s-pimautorp
police pps 200
class copp-s-routingProto2
police pps 1300
class copp-s-v6routingProto2
police pps 1300
class copp-s-routingProto1
police pps 1000
class copp-s-arp
police pps 200
class copp-s-ptp
police pps 1000
class copp-s-vxlan
police pps 1000
class copp-s-bfd
police pps 350
class copp-s-bpdu
police pps 12000
class copp-s-dpss
police pps 1000
class copp-s-mpls
police pps 100
class copp-icmp
police pps 200
class copp-telnet
police pps 500
class copp-ssh
police pps 500
class copp-snmp
police pps 500
class copp-ntp
police pps 100
class copp-tacacsradius
police pps 400
class copp-stftp
police pps 400
control-plane
service-policy input copp-system-policy

snmp-server community monitor group network-operator
ntp server 10.39.1.208 use-vrf management

vlan 1
vrf context keepalive
vrf context lan
ip route 0.0.0.0/0 10.39.0.5
vrf context management
ip route 0.0.0.0/0 10.39.10.254
no port-channel load-balance resilient
hardware profile portmode 48x10G+4x40G

no hardware profile ecmp resilient
vpc domain 1
peer-switch
peer-keepalive destination 192.168.23.2 source 192.168.23.1 vrf keepalive
delay restore 360
peer-gateway
auto-recovery
ip arp synchronize

interface Vlan1
vrf member lan
ip address 10.39.0.110/21

interface port-channel14
description *** LACP 10GB CATALYST 3750 ****
switchport mode trunk
vpc 14

interface port-channel123
description *** VPC PEER LINKS ***
switchport mode trunk
spanning-tree port type network
vpc peer-link

interface Ethernet1/1

interface Ethernet1/2

interface Ethernet1/3

interface Ethernet1/4

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23

interface Ethernet1/24

interface Ethernet1/25

interface Ethernet1/26

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29

interface Ethernet1/30

interface Ethernet1/31

interface Ethernet1/32

interface Ethernet1/33

interface Ethernet1/34

interface Ethernet1/35

interface Ethernet1/36

interface Ethernet1/37

interface Ethernet1/38

interface Ethernet1/39

interface Ethernet1/40
description *** Connessione cisco 3750 ***
switchport mode trunk
channel-group 14

interface Ethernet1/41

interface Ethernet1/42

interface Ethernet1/43

interface Ethernet1/44

interface Ethernet1/45

interface Ethernet1/46

interface Ethernet1/47

interface Ethernet1/48
no switchport
vrf member keepalive
ip address 192.168.23.1/24

interface Ethernet1/49
description *** VPC PEER LINKS ***
switchport mode trunk
channel-group 123 mode active

interface Ethernet1/50
description *** VPC PEER LINKS ***
switchport mode trunk
channel-group 123 mode active

interface Ethernet1/51

interface Ethernet1/52

interface mgmt0
vrf member management
ip address 10.39.10.2/24
line console
line vty

the second switch have vlan ip address 10.39.0.111 and same configuration

thanks

Andrea

[ChuckMcF]

If I understand your post correctly you have two Nexus switches that are port channeled together each with one uplink to a 3750 switch. I assume that the 3750 switch is directly connected to the firewall. If this is correct please answer the following. If incorrect please reply with how the devices are actually connected.

- Can the 3750 ping the firewall sourced from vlan 1?

- Can you ping the VLAN 1 interface on the 3750 switch from the Nexus switches?

- Is vlan 1 allowed across the trunks between all devices?

Appreciate your reply,

Chuck

[Mark Malone]

Hi

Your VPC setup looks fine just confirm you dont have inconsistencies in show vpc brief  , thats fine if you have to keep vlan1 it but if you ever get a chance move off it leave it shutdown at interface level , leave it to control traffic

 what are the status of the trunks int the po14 back to the 3750 switch , is vlan 1in FWD and not being blocked -- show int trunk -- will show you this

remove the vrf lan from under the vlan 1 interface , not required

remove the vrf context lan as well and just have a normal routes to break out

also put vlan 1 into HSRP and turn on the feature on both 3ks  , then set the Cisco 3750 default gateway as the HSRP VIP , this will give you proper resiliency across both switches

Set a vlan 1 interface on the 3750 switch bring it up same subnet as 3ks and see if you can ping back then

Also its better to use lacp rather than on mode in a vpc cluster design , if your adding more switches or servers

This is just an example below ive altered from one of my 5k setups  , you only need the HSRP bit on vlan 1 in your setup  , then point the 3750 to .254 IP

interface Vlan24
description xxxxxxxxxxxx
no shutdown
no ip redirects
ip address 10.150.4.251/24
no ipv6 redirects
ip router eigrp xxxxxx
no ip passive-interface eigrp xxxxxx
hsrp version 2
hsrp 24
authentication text secret
preempt
priority 250
ip 10.150.4.254

interface Vlan24
description xxxxxxxxxxxx
no shutdown
no ip redirects
ip address 10.150.4.252/24
no ipv6 redirects
ip router eigrp xxxxxx
no ip passive-interface eigrp xxxxxx
hsrp version 2
hsrp 24
authentication text secret
preempt
priority 240
ip 10.150.4.254

Check HSRP formed with show hsrp brief , let me know if that works for you

[paul driver]

Hello

Just like to add for validation can you post the ouput of following please:
sh vpc
sh vpc peer keepalive
sh vpc consistency-parameters global
sh vpc consistency-parameters  vpc 14

[malmsteen81]

Hi, thank you for replies. i double checked the configuration and i found (i don't know why) but the interface vlan 1 was down. i bring it to up and the ping now works.

Now i will connect some server and check the connectivity.

sorry if I made you waste time

thanks

Andrea

[Mark Malone]

no problem , i would still take it out of the vrf or when you expand it will all need to be in vrf lan , i would also set the HSRP feature on vlan 1

[malmsteen81]

hi, now i removed vlan1 from vrf member and ping between two vlan1 ip addresses works fine.

[Mark Malone]

great your up :)

[malmsteen81]

yes is up.

I take advantage of the open discussion for this, the jumbo frame configuration. I saw the documentation and i understand that i can create a qos policy or set mtu per interface. So which is better?

thanks

Andrea

[Mark Malone]

depends on what control you want, we just set it globally as its 5k thats all it can take and its a storage network , but if you want to be more granular and not blast every port then use per port option , only some 3ks i believe support this per port method