Issue Building an IP GRE Tunnel

[John Saldana]

I'm trying to build a tunnel accross the internet and seem to be having an issue with actually being able to ping accross the tunnel. Router 1 is a terrestrial link; Router 2 is a satlink with ping times avg'ing 600ms. Both routers are able to ping each other without issue. Not sure if someone can give me some pointers?

Router 1 - 1.1.1.1

interface Tunnel0
 bandwidth 1024
 ip address 172.0.0.1 255.255.255.0
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 5 2
 tunnel source FastEthernet0/0
 tunnel destination 2.2.2.2

Tunnel0 is up, line protocol is down
  Hardware is Tunnel
  Internet address is 172.0.0.1/24
  MTU 1514 bytes, BW 1024 Kbit/sec, DLY 500000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (5 sec), retries 2
  Tunnel source 1.1.1.1 (FastEthernet0/0), destination 2.2.2.2
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255
  Fast tunneling enabled
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input 00:00:02, output 00:00:01, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     38426 packets input, 1847564 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     131069 packets output, 6297012 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out


Nov 27 00:33:23: Tunnel0: sending keepalive, 2.2.2.2->1.1.1.1 (len=24 ttl=255), counter=25
Nov 27 00:33:23: Tunnel0: GRE/IP encapsulated 1.1.1.1->2.2.2.2 (linktype=7, len=48)
Nov 27 00:33:23: Tunnel0 count tx, adding 24 encap bytes

Router 2 - 2.2.2.2

interface Tunnel0
 description ### VPN Tunnel ###
 bandwidth 1024
 ip address 172.0.0.2 255.255.255.0
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 5 2
 tunnel source GigabitEthernet0
 tunnel destination 1.1.1.1

Tunnel0 is up, line protocol is up
  Hardware is Tunnel
  Description: ### VPN Tunnel ###
  Internet address is 172.0.0.2/24
  MTU 17916 bytes, BW 1024 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (5 sec), retries 2
  Tunnel source 2.2.2.2 (GigabitEthernet0), destination 1.1.1.1
   Tunnel Subblocks:
      src-track:
         Tunnel0 source tracking subblock associated with GigabitEthernet0
          Set of tunnels with source GigabitEthernet0, 1 member (includes iterators), on interface <OK>
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1476 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output 00:00:04, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     1338 packets output, 64224 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out


*Nov 27 04:12:05.019: Tunnel0: sending keepalive, 1.1.1.1->2.2.2.2 (len=24 ttl=255), counter=1
*Nov 27 04:12:05.019: Tunnel0: GRE/IP encapsulated 2.2.2.2->1.1.1.1 (linktype=7, len=48)
*Nov 27 04:12:05.019: Tunnel0 count tx, adding 0 encap bytes
*Nov 27 04:12:05.635: Tunnel0: GRE/IP (PS) to decaps 1.1.1.1->2.2.2.2 (tbl=0,"default" len=24 ttl=242)
*Nov 27 04:12:05.635: Tunnel0: keepalive received, 1.1.1.1->2.2.2.2 (len=24 ttl=242), resetting counter


Thanks,


John

[Tagir Temirgaliyev]

Router 1 - 1.1.1.1

interface Tunnel0
bandwidth 1024
ip address 172.0.0.1 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
keepalive 5 2
tunnel source 1.1.1.1
tunnel destination 2.2.2.2

Router 2 - 2.2.2.2

interface Tunnel0
description ### VPN Tunnel ###
bandwidth 1024
ip address 172.0.0.2 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
keepalive 5 2

tunnel source 2.2.2.2
tunnel destination 1.1.1.1

[John Saldana]

Made the changes, still not able to ping across the tunnel.

==============================================================

Router 1 = 1.1.1.1
interface Tunnel0
 bandwidth 1024
 ip address 172.0.0.1 255.255.255.252
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 5 2
 tunnel source 1.1.1.1
 tunnel destination 2.2.2.2

Tunnel0 is up, line protocol is down
  Hardware is Tunnel
  Internet address is 172.0.0.1/30
  MTU 1514 bytes, BW 1024 Kbit/sec, DLY 500000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (5 sec), retries 2
  Tunnel source 1.1.1.1, destination 2.2.2.2
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255
  Fast tunneling enabled
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input 00:00:00, output 00:00:03, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     40196 packets input, 1932524 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     132839 packets output, 6381972 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

Nov 27 03:28:52: Tunnel0: sending keepalive, 2.2.2.2->1.1.1.1 (len=24 ttl=255), counter=2131
Nov 27 03:28:52: Tunnel0: GRE/IP encapsulated 1.1.1.1->2.2.2.2 (linktype=7, len=48)
Nov 27 03:28:52: Tunnel0 count tx, adding 24 encap bytes
Nov 27 03:28:55: Tunnel0: GRE/IP classify 2.2.2.2->1.1.1.1 failed, tunnel down
Nov 27 03:28:55: Tunnel0: GRE/IP to decaps 2.2.2.2->1.1.1.1 (len=48 ttl=245)
Nov 27 03:28:55: Tunnel0: GRE decapsulated IP packet (linktype=7, len=24)
==============================================================
Router 2 = 2.2.2.2
interface Tunnel0
 description ### VPN Tunnel ###
 bandwidth 1024
 ip address 172.0.0.2 255.255.255.252
 ip mtu 1400
 ip tcp adjust-mss 1360
 keepalive 5 2
 tunnel source 2.2.2.2
 tunnel destination 1.1.1.1

Tunnel0 is up, line protocol is up
  Hardware is Tunnel
  Description: ### VPN Tunnel ###
  Internet address is 172.0.0.2/30
  MTU 17916 bytes, BW 1024 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (5 sec), retries 2
  Tunnel source 2.2.2.2, destination 1.1.1.1
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Tunnel transport MTU 1476 bytes
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input never, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     3122 packets output, 149856 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

*Nov 27 06:35:17.918: Tunnel0: sending keepalive, 1.1.1.1->2.2.2.2 (len=24 ttl=255), counter=1
*Nov 27 06:35:17.918: Tunnel0: GRE/IP encapsulated 2.2.2.2->1.1.1.1 (linktype=7, len=48)
*Nov 27 06:35:17.918: Tunnel0 count tx, adding 0 encap bytes
*Nov 27 06:35:18.530: Tunnel0: GRE/IP (PS) to decaps 1.1.1.1->2.2.2.2 (tbl=0,"default" len=24 ttl=242)
*Nov 27 06:35:18.530: Tunnel0: keepalive received, 1.1.1.1->2.2.2.2 (len=24 ttl=242), resetting counter
*Nov 27 06:35:22.918: Tunnel0: sending keepalive, 1.1.1.1->2.2.2.2 (len=24 ttl=255), counter=1
*Nov 27 06:35:22.918: Tunnel0: GRE/IP encapsulated 2.2.2.2->1.1.1.1 (linktype=7, len=48)
*Nov 27 06:35:22.918: Tunnel0 count tx, adding 0 encap bytes
*Nov 27 06:35:23.514: Tunnel0: GRE/IP (PS) to decaps 1.1.1.1->2.2.2.2 (tbl=0,"default" len=24 ttl=242)
*Nov 27 06:35:23.514: Tunnel0: keepalive received, 1.1.1.1->2.2.2.2 (len=24 ttl=242), resetting counter

==============================================================

[paul driver]

Hello

This is just a basic GRE tunnel so it should be straight forward to establish connectivity unless you have Fw prohibiting GRE

How are you providing NLRI between host 1.1.1.1 and 2.2.2.2.
    



Can you ping these between these
      ping 2.2.2.2 source 1.1.1.1

Sh ip route 2.2.2.2




res
Paul

[John Saldana]

Paul,

No firewall and yes both routers can ping each other with no probs.

[Tagir Temirgaliyev]

share full configs

[Richard Burts]

We find this in the output of router 1

Tunnel0 is up, line protocol is down

which suggests that part of the issue may be the keepalive function. It is interesting that the output from router 2 is different

Tunnel0 is up, line protocol is up

So we need to figure out why keepalives seem to work in one direction but not in the other direction. I agree that more complete configs would be helpful. I would also like to see output showing each router pinging the tunnel destination address and specifying the source address of the ping as the tunnel source address.

HTH

Rick

[Tagir Temirgaliyev]

or try

keepalive 10 4

[John Saldana]

Paul,

I couldn't agree more, this is a basic GRE Tunnel.. I have done this before, its just interesting that I do the same thing this time and it doesnt work.. very interesting!

Richard & Paul,

I have posted the full configs below. Also, I dont think this would matter but worth mentioning at this point. The NVI0 Interface is Admin down, not sure how to no shut it.

Tagir,

I modified the keeplive as requested, no change. With regards to the debugs, still getting the same debugs as before. The Keepalives didnt change anything with the debug messages.

Also, my appoligies for not replying sooner.. its the holidays and I'm a bit mobile.

Thanks again ya'll for helping me out with this! :)

--- Router # 1 --- Cisco 1841
version 12.4
service timestamps debug datetime localtime
service timestamps log datetime localtime
service password-encryption
!
hostname router1
!
boot-start-marker
boot-end-marker
!
no aaa new-model
ip cef
!
!
no ip dhcp use vrf connected
!
!
ip dhcp pool VLAN2
network 192.168.1.0 255.255.255.252
default-router 192.168.1.1
dns-server 8.8.8.8 8.8.4.4
lease infinite
!
!
!
ip name-server 8.8.8.8
ip name-server 8.8.4.4
!
!
!
!
!
!
!
!
interface Tunnel0
bandwidth 1024
ip address 172.0.0.1 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
keepalive 10 4
tunnel source 1.1.1.38
tunnel destination 2.2.2.2
!
interface FastEthernet0/0
bandwidth 10000
ip address 1.1.1.38 255.255.255.224
ip nbar protocol-discovery
ip nat outside
logging event subif-link-status
load-interval 30
duplex auto
speed auto
no cdp enable
!
interface FastEthernet0/1
bandwidth 60000
no ip address
ip nbar protocol-discovery
load-interval 30
duplex auto
speed auto
!
interface FastEthernet0/1.1
encapsulation dot1Q 1 native
ip address 10.0.0.1 255.255.255.0
!
interface FastEthernet0/1.2
encapsulation dot1Q 2
ip address 192.168.1.1 255.255.255.252
ip nat inside
no cdp enable
!
interface FastEthernet0/1.3
encapsulation dot1Q 3
ip address 192.168.0.1 255.255.255.0
ip flow ingress
ip flow egress
ip nat inside
no cdp enable
!
interface FastEthernet0/1.4
encapsulation dot1Q 4
ip address 192.168.3.1 255.255.255.0
ip access-group 104 in
ip access-group 104 out
ip flow ingress
ip flow egress
ip nat inside
no cdp enable
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 1.1.1.33 name Default_Gateway
!

!
no ip http server
no ip http secure-server
ip nat inside source list 100 interface FastEthernet0/0 overload
!
!
access-list 100 remark ### NAT ###
access-list 100 permit ip 192.168.0.0 0.0.0.255 any
access-list 100 permit ip 192.168.1.0 0.0.0.3 any
access-list 100 permit ip 192.168.3.0 0.0.0.255 any
!
!
tftp-server flash:P00307020300.bin
!
control-plane
!
!
!
line con 0
line aux 0
line vty 0 4
exec-timeout 60 0
logging synchronous
login local
history size 256
!
scheduler allocate 20000 1000

### PING TEST FROM ROUTER 1 TO ROUTER 2 ###
ROUTER1#ping 172.0.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

========================================

--- Router 2 --- c891

version 15.0
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname Router2
!
boot-start-marker
boot-end-marker
!
logging buffered 20480
!
no aaa new-model
!
!
!
!
!
!
ip dhcp pool VLAN2
network 192.168.10.0 255.255.255.0
dns-server 8.8.8.8
default-router 192.168.10.1
!
!
ip cef
ip name-server 8.8.8.8
ip name-server 8.8.4.4
!
no ipv6 cef
!
!
multilink bundle-name authenticated
!
!
!
vlan 2-4,7
!
!
!
!
!
!
!
!
interface Tunnel0
description ### VPN Tunnel ###
bandwidth 1024
ip address 172.0.0.2 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
keepalive 10 4
tunnel source 2.2.2.2
tunnel destination 1.1.1.38
!
!
interface FastEthernet0
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet1
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet2
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet3
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet4
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet5
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet6
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet7
description ### INTERNET ###
switchport access vlan 2
!
!
interface FastEthernet8
description ### NOT USED ###
no ip address
duplex auto
speed auto
!
!
interface GigabitEthernet0
ip address dhcp
ip nat outside
ip virtual-reassembly
load-interval 30
duplex auto
speed auto
no cdp enable
!
!
interface Vlan1
no ip address
!
!
interface Vlan2
description ### INTERNET ###
ip address 192.168.10.1 255.255.255.0
ip nat inside
ip virtual-reassembly
!
!
interface Async1
no ip address
encapsulation slip
!
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
!
ip nat inside source list PUBLIC_NAT interface GigabitEthernet0 overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet0
!
ip access-list standard PUBLIC_NAT
remark ### PUBLIC NAT ACL ###
permit 192.168.10.0 0.0.0.255
!
!
!
!
!
!
!
control-plane
!
!
!
line con 0
line 1
modem InOut
stopbits 1
speed 115200
flowcontrol hardware
line aux 0
line vty 0 4
exec-timeout 60 0
logging synchronous
login local
history size 256
!
exception data-corruption buffer truncate
scheduler max-task-time 5000
end

### PING TEST FROM ROUTER 2 TO ROUTER 1 ###
ROUTER2#ping 172.0.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.0.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

[paul driver]

Hello John

Where is 1.1.1.1 and 2.2.2.2 , Which is your source and destination address for either side of the tunnel?

On rtr 1
ip route 0.0.0.0 0.0.0.0 1.1.1.33 name Default_Gateway

Rtr2
ip route 0.0.0.0 0.0.0.0 GigabitEthernet0 ( whats th public for this)

Please post the ping output  between 1.1.1.1 and 2.2.2.2

res

paul

[Richard Burts]

I agree with Paul that a very likely cause of this issue is some kind of basic connectivity issue between the peer routers. So I would very much like to see the output of this command on router 1

ping 2.2.2.2 source 1.1.1.38

note that depending on version of code this command may work or it may require that you use the extended ping command to be able to specify the source address. and I would like to see the results of this command on router 2

ping 1.1.1.38 source 2.2.2.2

Thank you for posting the configs. There are two things that concern me as possible problems. On router 2 the default route

ip route 0.0.0.0 0.0.0.0 GigabitEthernet0

might be a problem. Configuring a static route which specifies only the outbound interface is, at best, a suboptimal config and may cause real problems. The crux of this issue is that this configuration requires the router to ARP for every remote destination to which it attempts to forward a packet. And this requires that proxy ARP be enabled on the next hop router. If proxy ARP is not enabled then the ARP will fail and the packet can not be forwarded. It would be much better to configure it this way

ip route 0.0.0.0 0.0.0.0 DHCP

The other thing that might be a concern is the comment about interface NVIO being down. Is this on router 1, router 2, or on both routers?

HTH

Rick

[paul driver]

Hello richard

would indifferent keepalive affect establishment of a basic gre tunnel?

I am not sure this would be a cause?

Youve  got me curious now but am not able to test at present - lol

res

paul

[Richard Burts]

Paul

Perhaps you are suggesting something that I am not understanding. While we do not yet understand the underlying problem part of the behavior seems quite clear to me. R1 is configured on the GRE tunnel with keepalives. The keepalives are failong on R1. This causes R1 to have its GRE tunnel in the UP/DOWN state. When the tunnel interface is UP/DOWN the router will not be able to send anything through that interface which would certainly impact establishment of the GRE tunnel.

As a test the original poster could remove keepalives from the configuration. If he does remove keepalives then I believe that both GRE tunnels will be in the UP/UP state. But I believe that until we identify and solve the underlying problem that he still would not be able to send data end to end (even though both tunnels are UP/UP).

HTH

Rick

[Richard Burts]

Adding a bit to my own post:

It might help us if we remember that for basic GRE tunnels the default behavior is for the router to disable keepalives on the tunnel interface and to report the tunnel as UP/UP as long as the router has a viable route in its routing table to the tunnel destination. Being UP/UP does not tell us anything about end to end connectivity. Cisco added the optional capability to run keepalives on GRE tunnels to deal with the possibility that there is not end to end connectivity and to mark the tunnel interface as protocol down if keepalives fail.

HTH

Rick

[paul driver]

Hello Richard

FYI mate I wasn’t suggesting you were incorrect in anyway – I just was saying per my understanding was I didn’t think the keepalive(s) would negate a FULL gre establishment  ( meaning connectivity over the tunnel)

I have since tested this( one side with keepalive and the other with none) and the tunnel became active with full reachability, So it seems a correct assumption.

Lastly , My believe was that the tunnel status on one end showing down could be due to the reachability between the sip/dip of these tunnels.and this testing I have just performed looks like this is indeed a possible cause.

Example:

R1
interface Tunnel12
ip address 100.100.100.1 255.255.255.0
keepalive 10 5
tunnel source FastEthernet0/0
tunnel destination 10.1.12.2

sh int tun 12 | in Keep
Keepalive set (10 sec), retries 5


R2
interface Tunnel12
ip address 100.100.100.2 255.255.255.0
no keepalive
tunnel source FastEthernet0/0
tunnel destination 10.1.12.1

sh int tun 12 | in Keep
Keepalive not set

R2
ping 100.100.100.1 source tun 12 repeat 2
Type escape sequence to abort.
Sending 2, 100-byte ICMP Echos to 100.100.100.1, timeout is 2 seconds:
Packet sent with a source address of 100.100.100.2
!!
Success rate is 100 percent (2/2), round-trip min/avg/max = 40/50/60 ms

 
Sip/DIp reachability test

R2
int fa0/0 < -----sip for R2 tun12 and dip for R1 tun12
shut

sh int tun 12 | in line
Tunnel12 is up, line protocol is down

R1
sh int tun 12 | in line
Tunnel12 is up, line protocol is up



res
Paul