Reza, Yes uplinks are working fine. The only issue with first 24 ports and other remaining ports are working fine. ... View more

Reza, thanks for the response. It's a working laptop so no issue with the laptop. I am planning to reload the switch this weekend. If reload didn't work I will follow your suggestion and update you. Thank you so much. ... View more

Hi Philip, Sorry for late response. Yes, ports are not coming up and tried with new cables but no luck. switch 2 has 52 ports but only issue with ports (2/0/1 to 2/024). ... View more

Hi Reza, Sorry for late response and missed to mention switch connected in stack. i tried connecting laptop with switch 2 affected port but port is not coming up and also tried changing the cable as well but no luck. ... View more

Thanks for the response. Logs captured as you advice. Any suggestion.    Logs before when tunnel is in UP status and  before reboot:    #show crypto isa sa IPv4 Crypto ISAKMP SA dst src state conn-id status 207.99.106.X 75.99.252.X QM_IDLE 1007 ACTIVE 108.58.212.X 75.99.252.X MM_NO_STATE 0 ACTIVE 108.58.212.X 75.99.252.X MM_NO_STATE 0 ACTIVE (deleted) 69.46.229.X 75.99.252.X QM_IDLE 1003 ACTIVE 216.105.104.X 75.99.252.X QM_IDLE 1002 ACTIVE IPv6 Crypto ISAKMP SA   1#show crypto ipsec sa interface: Tunnel10 Crypto map tag: Tunnel10-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (216.105.104.X/255.255.255.255/47/0) current_peer 216.105.104.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 216.105.104.X plaintext mtu 1438, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/0 current outbound spi: 0x3E9F13FC(1050612732) PFS (Y/N): N, DH group: none inbound esp sas: spi: 0x8ADC6CB7(2329701559) transform: esp-256-aes esp-sha-hmac , in use settings ={Tunnel, } conn id: 2027, flow_id: ESG:27, sibling_flags FFFFFFFF80004048, crypto map: Tunnel10-head-0 sa timing: remaining key lifetime (sec): 1562 Kilobyte Volume Rekey has been disabled IV size: 16 bytes replay detection support: Y Status: ACTIVE(ACTIVE) inbound ah sas: inbound pcp sas: outbound esp sas: spi: 0x3E9F13FC(1050612732) transform: esp-256-aes esp-sha-hmac , in use settings ={Tunnel, } conn id: 2028, flow_id: ESG:28, sibling_flags FFFFFFFF80004048, crypto map: Tunnel10-head-0 sa timing: remaining key lifetime (sec): 1562 Kilobyte Volume Rekey has been disabled IV size: 16 bytes replay detection support: Y Status: ACTIVE(ACTIVE) outbound ah sas: outbound pcp sas: interface: Tunnel0 Crypto map tag: Tunnel0-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (108.58.212.X/255.255.255.255/47/0) current_peer 108.58.212.X port 500 PERMIT, flags={origin_is_acl,ipsec_sa_request_sent} #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 1260, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 108.58.212.X plaintext mtu 1400, path mtu 1400, ip mtu 1400, ip mtu idb Tunnel0 current outbound spi: 0x0(0) PFS (Y/N): N, DH group: none inbound esp sas: inbound ah sas: inbound pcp sas: outbound esp sas: outbound ah sas: outbound pcp sas: interface: Tunnel1 Crypto map tag: Tunnel1-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (207.99.106.X/255.255.255.255/47/0) current_peer 207.99.106.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 6610, #pkts encrypt: 6610, #pkts digest: 6610 #pkts decaps: 5651, #pkts decrypt: 5651, #pkts verify: 5651 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 207.99.106.X plaintext mtu 1378, path mtu 1400, ip mtu 1400, ip mtu idb Tunnel1 current outbound spi: 0x645B6C29(1683713065) PFS (Y/N): N, DH group: none inbound esp sas: spi: 0xFB9D18FF(4221376767) transform: esp-3des , in use settings ={Transport, } conn id: 2017, flow_id: ESG:17, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 45 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) spi: 0xC88DF61A(3364746778) transform: esp-3des , in use settings ={Transport, } conn id: 2019, flow_id: ESG:19, sibling_flags FFFFFFFF80000008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 45 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) inbound ah sas: inbound pcp sas: outbound esp sas: spi: 0x25A199E2(631347682) transform: esp-3des , in use settings ={Transport, } conn id: 2018, flow_id: ESG:18, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 45 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) spi: 0x645B6C29(1683713065) transform: esp-3des , in use settings ={Transport, } conn id: 2020, flow_id: ESG:20, sibling_flags FFFFFFFF80000008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 45 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) outbound ah sas: outbound pcp sas: protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (69.46.229.X/255.255.255.255/47/0) current_peer 69.46.229.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 15094, #pkts encrypt: 15094, #pkts digest: 15094 #pkts decaps: 11021, #pkts decrypt: 11021, #pkts verify: 11021 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 69.46.229.X plaintext mtu 1378, path mtu 1400, ip mtu 1400, ip mtu idb Tunnel1 current outbound spi: 0xD3F6081A(3556116506) PFS (Y/N): N, DH group: none inbound esp sas: spi: 0x9AB40336(2595488566) transform: esp-3des , in use settings ={Transport, } conn id: 2009, flow_id: ESG:9, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 42 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) inbound ah sas: inbound pcp sas: outbound esp sas: spi: 0xD3F6081A(3556116506) transform: esp-3des , in use settings ={Transport, } conn id: 2010, flow_id: ESG:10, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 19 hours, 42 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) outbound ah sas: outbound pcp sas:   Logs after spoke reboot and Tunnel is in NHRP status:   Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.X 172.16.254.1 NHRP 00:00:05 S #show crypto isa sa IPv4 Crypto ISAKMP SA dst src state conn-id status 69.46.229.X 75.99.252.X QM_IDLE 1001 ACTIVE 108.58.212.X 75.99.252.X MM_NO_STATE 0 ACTIVE 216.105.104.X 75.99.252.X QM_IDLE 1002 ACTIVE IPv6 Crypto ISAKMP SA     #show crypto ipsec sa interface: Tunnel10 Crypto map tag: Tunnel10-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (216.105.104.X/255.255.255.255/47/0) current_peer 216.105.104.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 216.105.104.X plaintext mtu 1438, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/0 current outbound spi: 0x9E69D2EA(2657735402) PFS (Y/N): N, DH group: none inbound esp sas: spi: 0xB3743BB(188171195) transform: esp-256-aes esp-sha-hmac , in use settings ={Tunnel, } conn id: 2003, flow_id: ESG:3, sibling_flags FFFFFFFF80004048, crypto map: Tunnel10-head-0 sa timing: remaining key lifetime (sec): 3545 Kilobyte Volume Rekey has been disabled IV size: 16 bytes replay detection support: Y Status: ACTIVE(ACTIVE) inbound ah sas: inbound pcp sas: outbound esp sas: spi: 0x9E69D2EA(2657735402) transform: esp-256-aes esp-sha-hmac , in use settings ={Tunnel, } conn id: 2004, flow_id: ESG:4, sibling_flags FFFFFFFF80004048, crypto map: Tunnel10-head-0 sa timing: remaining key lifetime (sec): 3545 Kilobyte Volume Rekey has been disabled IV size: 16 bytes replay detection support: Y Status: ACTIVE(ACTIVE) outbound ah sas: outbound pcp sas: interface: Tunnel1 Crypto map tag: Tunnel1-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (69.46.229.X/255.255.255.255/47/0) current_peer 69.46.229.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 17, #pkts encrypt: 17, #pkts digest: 17 #pkts decaps: 30, #pkts decrypt: 30, #pkts verify: 30 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 69.46.229.X plaintext mtu 1378, path mtu 1400, ip mtu 1400, ip mtu idb Tunnel1 current outbound spi: 0x663B3256(1715155542) PFS (Y/N): N, DH group: none inbound esp sas: spi: 0x7D70493F(2104510783) transform: esp-3des , in use settings ={Transport, } conn id: 2001, flow_id: ESG:1, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 23 hours, 59 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) inbound ah sas: inbound pcp sas: outbound esp sas: spi: 0x663B3256(1715155542) transform: esp-3des , in use settings ={Transport, } conn id: 2002, flow_id: ESG:2, sibling_flags FFFFFFFF80004008, crypto map: Tunnel1-head-0 sa timing: remaining key lifetime 23 hours, 59 mins Kilobyte Volume Rekey has been disabled IV size: 8 bytes replay detection support: N Status: ACTIVE(ACTIVE) outbound ah sas: outbound pcp sas: interface: Tunnel0 Crypto map tag: Tunnel0-head-0, local addr 75.99.252.X protected vrf: (none) local ident (addr/mask/prot/port): (75.99.252.X/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (108.58.212.X/255.255.255.255/47/0) current_peer 108.58.212.X port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 4, #recv errors 0 local crypto endpt.: 75.99.252.X, remote crypto endpt.: 108.58.212.X plaintext mtu 1400, path mtu 1400, ip mtu 1400, ip mtu idb Tunnel0 current outbound spi: 0x0(0) PFS (Y/N): N, DH group: none inbound esp sas: inbound ah sas: inbound pcp sas: outbound esp sas: outbound ah sas: outbound pcp sas:     ... View more

Hi Georg,   Thanks for your response. I tired below config on tunnel 1 but no luck went to NHRP after spoke reboot. Any other suggestion.    if-state nhrp   Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.X 172.16.254.1 NHRP 00:05:06 S ... View more

Hi    ulrickfr2001,   Thanks for your response. Please find below logs.    1. Logs when tunnel is up & before reboot: Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.X 172.16.254.1 UP 18:52:27 S 1 207.99.106.X 172.16.254.15 UP 18:48:15 D OOD-RTR-20-001#sh ip route 69.46.229.X % Network not in table OOD-RTR-20-001#sh ip cef 69.46.229.X 69.46.229.X/32 nexthop 75.99.252.X GigabitEthernet0/0/0 OOD-RTR-20-001# OOD-RTR-20-001#ping 69.46.229.X Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 69.46.229.X, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/12 ms OOD-RTR-20-001# 2. logs after spoke reboot & DMVPN is in NHRP state Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.X 172.16.254.1 NHRP 00:00:45 S OOD-RTR-20-001#sh ip route 69.46.229.X % Network not in table OOD-RTR-20-001# OOD-RTR-20-001#sh ip cef 69.46.229.X 69.46.229.X/32 nexthop 75.99.252.X GigabitEthernet0/0/0 OOD-RTR-20-001# OOD-RTR-20-001#ping 69.46.229.X Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 69.46.229.X, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 9/9/10 ms OOD-RTR-20-001# 3. logs after spoke reboot & Tunnel 1 reset & DMVPN is in UP. Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 2 69.46.229.X 172.16.254.1 UP 00:01:52 S 172.16.254.15 UP 00:01:49 D OOD-RTR-20-001#sh ip route 69.46.229.X % Network not in table OOD-RTR-20-001# OOD-RTR-20-001#sh ip cef 69.46.229.X 69.46.229.X/32 nexthop 75.99.252.X GigabitEthernet0/0/0 OOD-RTR-20-001# OOD-RTR-20-001#ping 69.46.229.X Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 69.46.229.X, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/13 ms OOD-RTR-20-001# ... View more

Hi ulrickfr2001   Thanks for the response. Sorry right now production started i unable to do changes on spoke end. I will share you logs once production ended.    many spokes are connected to that hub, if i deploy key on tunnel interface on HUB end it will get impact for other spokes is it?   ... View more

Sorry for this. I wrongly pasted the tunnel 0 configuration. Spoke end tunnel config is tunnel 1.   Current configuration : 498 bytes ! interface Tunnel1 description "DMVPN SPOKE 20 - DMVPN-1" ip address 172.16.254.20 255.255.255.0 no ip redirects ip mtu 1400 ip pim nbma-mode ip pim sparse-dense-mode no ip next-hop-self eigrp 50 ip nhrp map 172.16.254.1 69.46.229.X ip nhrp map multicast 69.46.229.X ip nhrp network-id 101 ip nhrp holdtime 300 ip nhrp nhs 172.16.254.1 ip tcp adjust-mss 1360 delay 120 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel protection ipsec profile ODMVPN2 end   #sh run int GigabitEthernet0/0/0 Building configuration... Current configuration : 162 bytes ! interface GigabitEthernet0/0/0 description external to internet (CableVision) ip address 75.99.252.X 255.255.255.248 ip nat outside negotiation auto end   ... View more

Hi Georg,   Thanks for the response. Please find below logs after applied static route and  rebooted the router. DMVPN state went to NHRP state and after reset tunnel interface 1 DMVPN state back to normal. Any other suggestion.    Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.X 172.16.254.1 NHRP 00:00:06 S   OOD-RTR-20-001#sh crypto isakmp sa IPv4 Crypto ISAKMP SA dst src state conn-id status 216.105.104.195 75.99.252.X QM_IDLE 1002 ACTIVE 69.46.229.182 75.99.252.X QM_IDLE 1001 ACTIVE IPv6 Crypto ISAKMP SA       ... View more

Hi Georg,   After applying static routes on spoke router i'm unable to ping NHS IP's  but DMVPN status is normal. Any suggestion.   Note: I didn't reboot and check whether DMVPN status is moving to NHRP status.   #ping 172.16.254.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 172.16.254.1, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) #ping 10.254.254.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.254.254.1, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) #ping 108.58.212.26 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 108.58.212.26, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket T1 - Route Installed, T2 - Nexthop-override C - CTS Capable # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ==========================================================================   Interface: Tunnel1, IPv4 NHRP Details Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 69.46.229.182 172.16.254.1 UP 16:03:06 S 1 207.99.106.25 172.16.254.15 UP 15:59:26 D   ... View more