Gre-tunnel Issue between CSR1000v and R881-K9

[Kwuenph]

Hi everybody,

I've run into strange issue while setting up GRE tunnel between CSR1000v and R881-K9. The tunnel is locally up without " keepalive" but endpoints can not ping each other via tunnel IP.

The strange thing is that I tried to setup GRE tunnel between this CSR1000v with ISR-4331, it worked well. Then, another tried to setup GRE tunnel between R881-K9 and ISR-4331, it went well too.

Has anybody had this problems ? Is a bug with R881-K9 with GRE tunnel to CSR1000v ?

==== R881 show version ====

System returned to ROM by reload
System image file is "flash:c880data-universalk9-mz.152-4.M3.bin"
Last reload type: Normal Reload
Last reload reason: power-on

Cisco 881 (MPC8300) processor (revision 1.0) with 236544K/25600K bytes of memory.
5 FastEthernet interfaces
1 Virtual Private Network (VPN) Module
256K bytes of non-volatile configuration memory.
125440K bytes of ATA CompactFlash (Read/Write)
License Info:
License UDI:
-------------------------------------------------
Device#   PID                   SN
-------------------------------------------------
*0        CISCO881-SEC-K9       ##########
License Information for 'c880-data'
    License Level: advipservices   Type: Permanent
    Next reboot license Level: advipservices
==== CSR1000v show version ====

License Level: ax
License Type: Default. No valid license found.
Next reload license Level: ax
cisco CSR1000V (VXE) processor (revision VXE) with 2190074K/3075K bytes of memory.
Processor board ID 99B6FIZPQSK
3 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
3984468K bytes of physical memory.
7774207K bytes of virtual hard disk at bootflash:.
0K bytes of WebUI ODM Files at webui:.

==== GRE Configuration ====

# CSR1000v

interface Tunnel1
 description TUNNEL TO R881
 bandwidth 1000
 ip address 192.168.1.1 255.255.255.0
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 10 3
 tunnel source 123.33.154.14
 tunnel destination 203.163.55.24
 tunnel key 2018
 tunnel path-mtu-discovery
end

# R881

interface Tunnel1
 description TUNNEL TO CSR
 bandwidth 1000
 ip address 192.168.1.2 255.255.255.0
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 10 3
 tunnel source 203.163.55.24
 tunnel destination 123.33.154.14
 tunnel key 2018
 tunnel path-mtu-discovery
end

======

Troubleshooting actions:

- Ping dst 203.163.55.24 with src 123.33.154.14 in R881 --> OK

- Ping dst 123.33.154.14 with src 203.163.55.24  in CSR1000v --> OK

- Check ACL --> not the issue

- add/remove tunnel key; change MTU/bandwidth; change tunnel IP Subnet to 192.168.100.0/24 --> there no different result

[Francesco Molino]

Hi

Can you share please output of sh ip int brie and sh int tunnel 1?

What's in between the 2 routers? Firewalls?

When you tested gre tunnel with an isr you were locally on your Lab or it was sitting in the same far end as the 881?

[Kwuenph]

For ISR4331, it's using Public IP too.

[Kwuenph]

Thank for your response.

There isn't any firewall between routers. They are using public IP

# CRS1000v ============================

show interfaces tunnel 1
Tunnel1 is up, line protocol is down
  Hardware is Tunnel
  Description: TUNNEL TO R881
  Internet address is 192.168.1.1/24
  MTU 9972 bytes, BW 100 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (10 sec), retries 3
  Tunnel linestate evaluation down - keepalive down
  Tunnel source 123.33.154.14, destination 203.163.55.24
  Tunnel protocol/transport GRE/IP
    Key 0x7E2, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1472 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output never, output hang never
  Last clearing of "show interface" counters 00:59:02
  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     354 packets input, 9912 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     359 packets output, 20464 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
---
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet1       123.33.154.14   YES NVRAM  up                    up      
GigabitEthernet2       unassigned      YES NVRAM  administratively down down    
GigabitEthernet3       10.4.1.1        YES NVRAM  up                    up      
Tunnel1                192.168.1.1     YES manual up                    down   

# R881==============================================

show int tun 1
Tunnel1 is up, line protocol is down
  Hardware is Tunnel
  Description: TUNNEL TO CSR
  Internet address is 192.168.1.2/24
  MTU 17912 bytes, BW 100 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (10 sec), retries 3
  Tunnel source 203.163.55.24, destination 123.33.154.14
  Tunnel protocol/transport GRE/IP
    Key 0x7E2, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1472 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output 00:00:06, output hang never
  Last clearing of "show interface" counters 01:03:56
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     384 packets output, 21504 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
----
show ip int br
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0              unassigned      YES unset  up                    up      
FastEthernet1              unassigned      YES unset  up                    up      
FastEthernet2              unassigned      YES unset  administratively down down    
FastEthernet3              unassigned      YES unset  administratively down down    
FastEthernet4              203.163.55.24   YES manual up                    up      
NVI0                       203.163.55.24   YES unset  up                    up      
Tunnel1                    192.168.1.2     YES manual up                    down    
Vlan1                      172.16.1.1      YES manual up                    up     

[Francesco Molino]

Can you try lowering down the MTU?
Andi can you run the debug tunnel and debug tunnel keepalive? Please share the output.

[Kwuenph]

---

@Francesco Molino wrote:
Can you try lowering down the MTU?
Andi can you run the debug tunnel and debug tunnel keepalive? Please share the output.

---

Thank Francesco for your response

I tried lowered down MTU to 1000 with CLI "ip MTU 1000" in tunnel interface.

But show interface tunnel 1 show "Tunnel transport MTU 1472 bytes"

Debug output for

=== CSR 1000v ===

Nov 22 06:59:16.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9415
*Nov 22 06:59:26.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9416
*Nov 22 06:59:36.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9417
*Nov 22 06:59:46.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9418
*Nov 22 06:59:56.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9419
*Nov 22 07:00:06.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9420
*Nov 22 07:00:16.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9421
*Nov 22 07:00:26.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9422
*Nov 22 07:00:36.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9423
*Nov 22 07:00:46.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9424
*Nov 22 07:00:56.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9425
*Nov 22 07:01:06.711: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9426
*Nov 22 07:01:16.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9427
*Nov 22 07:01:26.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9428
*Nov 22 07:01:36.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9429
*Nov 22 07:01:46.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9430
*Nov 22 07:01:56.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9431
*Nov 22 07:02:06.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9432
*Nov 22 07:02:16.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9433
*Nov 22 07:02:26.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9434
*Nov 22 07:02:36.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9435
*Nov 22 07:02:46.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9436
*Nov 22 07:02:56.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9437
*Nov 22 07:03:06.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9438
*Nov 22 07:03:16.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9439
*Nov 22 07:03:26.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9440
*Nov 22 07:03:36.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9441
*Nov 22 07:03:46.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9442
*Nov 22 07:03:56.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9443
*Nov 22 07:04:06.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9444
*Nov 22 07:04:16.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9445
*Nov 22 07:04:26.713: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9446
*Nov 22 07:04:36.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9447
*Nov 22 07:04:46.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9448
*Nov 22 07:04:56.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9449
*Nov 22 07:05:06.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9450
*Nov 22 07:05:16.712: Tunnel1: sending keepalive, 203.163.55.24->123.33.154.14 (len=28 ttl=255), counter=9451
----
 show int tun 1
Tunnel1 is up, line protocol is down
  Hardware is Tunnel
  Description: TUNNEL TO R881
  Internet address is 192.168.1.1/24
  MTU 9972 bytes, BW 100 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (10 sec), retries 3
  Tunnel linestate evaluation down - keepalive down
  Tunnel source 123.33.154.14, destination 203.163.55.24
  Tunnel protocol/transport GRE/IP
    Key 0x7E2, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1472 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output 00:16:29, output hang never
  Last clearing of "show interface" counters 1d02h
  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     9438 packets input, 264264 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     9497 packets output, 533832 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

=== R881 ===

Nov 22 07:08:19.076: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:08:19.076: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:08:29.128: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9462
Nov 22 07:08:29.128: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:08:29.128: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:08:39.128: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9463
Nov 22 07:08:39.128: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:08:39.128: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:08:49.129: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9464
Nov 22 07:08:49.129: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:08:49.129: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:08:59.129: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9465
Nov 22 07:08:59.129: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:08:59.129: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:09.129: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9466
Nov 22 07:09:09.129: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:09.129: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:19.129: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9467
Nov 22 07:09:19.129: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:19.129: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:29.178: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9468
Nov 22 07:09:29.178: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:29.178: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:39.178: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9469
Nov 22 07:09:39.178: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:39.178: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:49.178: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9470
Nov 22 07:09:49.178: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:49.178: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:09:59.178: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9471
Nov 22 07:09:59.178: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:09:59.178: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:10:09.179: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9472
Nov 22 07:10:09.179: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:10:09.179: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:10:19.179: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9473
Nov 22 07:10:19.179: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
Nov 22 07:10:19.179: Tunnel1 count tx, adding 0 encap bytes
Nov 22 07:10:29.227: Tunnel1: sending keepalive, 123.33.154.14->203.163.55.24 (len=28 ttl=255), counter=9474
Nov 22 07:10:29.227: Tunnel1: GRE/IP encapsulated 203.163.55.24->123.33.154.14 (linktype=7, len=56)
----
Tunnel1 is up, line protocol is down
  Hardware is Tunnel
  Description: TUNNEL TO CSR
  Internet address is 192.168.1.2/24
  MTU 17912 bytes, BW 100 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (10 sec), retries 3
  Tunnel source 203.163.55.24, destination 123.33.154.14
  Tunnel protocol/transport GRE/IP
    Key 0x7E2, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1472 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output 00:00:05, output hang never
  Last clearing of "show interface" counters 1d02h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     9477 packets output, 530712 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

[Francesco Molino]

Please share your config, i will lab it to see.