Solved: Re: OSPF and VPN - Page 2

Dima Dvorcovoy · ‎03-20-2020

Problem: OSPF does not injects routes from VPN interfaces
I have an ASR router and linux PC with network connected to it.
ASR:

Cisco IOS XE Software, Version 16.06.02 Cisco IOS Software [Everest], ASR1000 Software (X86_64_LINUX_IOSD-UNIVERSALK9_NOLI-M), Version 16.6.2, RELEASE SOFTWARE (fc2)

What I want: dial to ASR and then all devices () have to communicate in our campus network without problems.

There are no problems with l2tp connection and PC itself, but although it announces it's network to ASR, I can't see it in route table - so no net for devices.
configuration:

router ospf 10
router-id 217.21.43.0
no capability lls
area 0 range 10.0.0.0 255.0.0.0
area 1 stub
network 10.144.0.0 0.0.255.255 area 1
network 10.149.8.0 0.0.0.255 area 0
network 10.161.0.0 0.0.255.255 area 1
neighbor 10.175.100.0
default-information originate
distance 15

!

interface Virtual-Template1
ip address 10.161.0.1 255.255.0.0
ip nat inside
> ip ospf network point-to-point
peer default ip address dhcp-pool DVPN
no keepalive
ppp authentication pap LDAPA
ip virtual-reassembly

-----

Status:

Vi2.29 o10a-route PPPoVPDN - 10.161.0.124
Virtual-Access2.29 is up, line protocol is up
Hardware is Virtual Access interface
Internet address is 10.161.0.1/16
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100000 usec,
reliability 255/255, txload 21/255, rxload 36/255
Encapsulation PPP, LCP Open
Open: IPCP
PPPoVPDN vaccess, cloned from Virtual-Template1
Vaccess status 0x0
Protocol l2tp, tunnel id 64801, session id 35903
Keepalive not set

border#show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
10.194.0.0 0 2WAY/DROTHER 00:00:10 10.149.8.40 Port-channel2
10.255.255.255 128 FULL/DR 00:00:30 10.149.8.2 Port-channel2
10.175.100.0 0 FULL/ - 00:00:31 10.161.0.124 Virtual-Access2.29
show ip ospf database | inc 10.175
10.175.100.0 10.175.100.0 1750 0x8000000A 0x004A7D 3
border#show ip route | inc 10.175.
# (nothing)

But I see correct routing table on PC linux side.

What I really want is simple: to automatically add route 10.175.100.0/24 to Vi XXX every time THIS user calls VPN. There are many other users. Because every time interface and IP changes,

Dima Dvorcovoy · ‎03-23-2020

Thanks, it helps a lot. After I changed both ppp0 and vty1 to ip ospf network broadcast, route to Linux appears on Cisco.
And all starts to work...

the end, credits rolled.

(part 2)

....for 5-10 minutes.

After it, cisco stops to route packets from linux.

border#show ip route ospf | inc 10.175.100
O 10.175.100.0/24 [15/11] via 10.161.1.134, 00:02:09, Virtual-Access2.6

border#ping 10.175.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.175.100.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
And if I add the same route manually, all works (of course, until restart ppp)
border#conf t
Enter configuration commands, one per line. End with CNTL/Z.
border(config)#ip route 10.175.100.0 255.255.255.0 10.161.1.134
border(config)#^Z

border#show ip route | inc 10.175.100
S 10.175.100.0/24 [1/0] via 10.161.1.134

border#ping 10.175.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.175.100.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 31/43/68 ms
border#conf t
Enter configuration commands, one per line. End with CNTL/Z.
border(config)#no ip route 10.175.100.0 255.255.255.0 10.161.1.134
border(config)#^Z
border#ping 10.175.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.175.100.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
border#show ip route ospf | inc 10.175.100
O 10.175.100.0/24 [15/11] via 10.161.1.134, 00:01:03, Virtual-Access2.6

WTF??? I think the problem is in difference, extra Virtual-Access2.6, but why? Something with arp?

interface Virtual-Template1
ip address 10.161.0.1 255.255.0.0
ip nat inside
ip ospf network broadcast
peer default ip address dhcp-pool DVPN
no keepalive
ppp authentication pap LDAPA
ip virtual-reassembly
end

Cristian Matei · ‎03-23-2020

Hi,

When you can't reach the remote network but have the route via OSPF in the RIB, can you ping the next-hop, like 10.161.1.134, or whatever the next-hop is? Also, at that point in time, what does "show ppp all" and "show ppp interface xyz detail" say?

Regards,

Cristian Matei.

Dima Dvorcovoy · ‎03-23-2020

checked with timer...
Yes, 10 minutes exactly

Pings to neighbours are ok

border# ping 10.161.1.147
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.161.1.147, timeout is 2 seconds:
!!!!!

border#show ip ospf neighbor

10.175.100.0 1 FULL/BDR 00:00:31 10.161.1.147 Virtual-Access2.9

border#show ppp all

Vi2.9 LCP+ PAP+ IPCP+ LocalT 10.161.1.147 o10a-route

border#show ppp interface Vi 2.9

Vi2.9 No PPP serial context
PPP Session Info
----------------
Interface : Vi2.9
PPP ID : 0xBA0002F7
Phase : UP
Stage : Local Termination
Peer Name : o10a-route
Peer Address : 10.161.1.147
Control Protocols: LCP[Open] PAP+ IPCP[Open]
Session ID : 759
AAA Unique ID : 267328
SSS Manager ID : 0x820005EE
SIP ID : 0xB50005ED
PPP_IN_USE : 0x11

Vi2.9 LCP: [Open]
Our Negotiated Options
Vi2.9 LCP: ACCM 0x000A0000 (0x0206000A0000)
Vi2.9 LCP: AuthProto PAP (0x0304C023)
Vi2.9 LCP: MagicNumber 0x0892B0D1 (0x05060892B0D1)
Vi2.9 LCP: PFC (0x0702)
Vi2.9 LCP: ACFC (0x0802)
Peer's Negotiated Options
Vi2.9 LCP: ACCM 0x00000000 (0x020600000000)
Vi2.9 LCP: MagicNumber 0xAAFABE7B (0x0506AAFABE7B)

Vi2.9 IPCP: [Open]
Our Negotiated Options
Vi2.9 IPCP: Address 10.161.0.1 (0x03060AA10001)
Peer's Negotiated Options
Vi2.9 IPCP: Address 10.161.1.147 (0x03060AA10193)
Vi2.9 IPCP: PrimaryDNS 10.0.0.66 (0x81060A000042)
Vi2.9 IPCP: SecondaryDNS 10.0.0.67 (0x83060A000043)

Cristian Matei · ‎03-23-2020

Hi,

When it no longer works, after 10 minutes, do the following debug and post the output:

access-list 145 permit imp any any

debug ip packet 145

ping next-hop(10.161.1.x)

ping protected-network (10.175.100.x)

Regards,

Cristian Matei.

Dima Dvorcovoy · ‎03-23-2020

ping 10.161.1.147
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.161.1.147, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/40/48 ms
border#
Mar 23 17:30:04.697: IP: s=10.161.0.1 (local), d=10.161.1.147, len 100, local feature, feature skipped, NAT(2), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.697: IP: tableid=0, s=10.161.0.1 (local), d=10.161.1.147 (Virtual-Access2.9), routed via FIB
Mar 23 17:30:04.697: IP: s=10.161.0.1 (local), d=10.161.1.147 (Virtual-Access2.9), len 100, sending
Mar 23 17:30:04.697: IP: s=10.161.0.1 (local), d=10.161.1.147 (Virtual-Access2.9), len 100, output feature, feature skipped, NAT Inside(8), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.697: IP: s=10.161.0.1 (local), d=10.161.1.147 (Virtual-Access2.9), len 100, output feature, feature skipped, iEdge(16), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.745: IP: s=10.161.1.147 (Virtual-Access2.9), d=10.161.0.1, len 100, input feature, Virtual Fragment Reassembly(38), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.745: IP: s=10.161.1.147 (Virtual-Access2.9), d=10.161.0.1, len 100, input feature, iEdge(97), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.745: IP: s=10.161.1.147 (Virtual-Access2.9), d=10.161.0.1, len 100, input feature, MCI Check(108), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:30:04.745: IP: s=10.161.1.147 (Virtual-Access2.9), d=10.161.0.1, len 100, rcvd 2
Mar 23 17:30:04.745: IP: s=10.161.1.147 (Virtual-Access2.9), d=10.161.0.1, len 100, stop process pak for forus packet

----------------

border#ping 10.175.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.175.100.1, timeout is 2 seconds:

Mar 23 17:31:53.401: IP: s=10.161.0.1 (local), d=10.175.100.1, len 100, local feature, feature skipped, NAT(2), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:53.401: IP: tableid=0, s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), routed via FIB
Mar 23 17:31:53.401: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, sending
Mar 23 17:31:53.401: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, output feature, feature skipped, NAT Inside(8), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:53.401: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, output feature, feature skipped, iEdge(16), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE.
Mar 23 17:31:55.414: IP: s=10.161.0.1 (local), d=10.175.100.1, len 100, local feature, feature skipped, NAT(2), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:55.414: IP: tableid=0, s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), routed via FIB
Mar 23 17:31:55.414: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, sending
Mar 23 17:31:55.414: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, output feature, feature skipped, NAT Inside(8), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:55.414: IP: s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), len 100, output feature, feature skipped, iEdge(16), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:57.417: IP: s=10.161.0.1 (local), d=10.175.100.1, len 100, local feature, feature skipped, NAT(2), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Mar 23 17:31:57.418: IP: tableid=0, s=10.161.0.1 (local), d=10.175.100.1 (Virtual-Access2.10), routed via FIB

-----
Vi2.10? Why? there are no 2.10 here!

Vi2.9 o10a-route PPPoVPDN - 10.161.1.147
Vi2.11 shcharbakova PPPoVPDN - 10.144.0.68

-----

But, of course, I used on linux tcpdump - i ppp0 and have seen incoming packets to 10.161 not to 10.175.

Extended IP access list 154
11 permit icmp any 10.161.0.0 0.0.255.255 (288 matches)
12 permit icmp any 10.175.100.0 0.0.0.255 (120 matches)
13 permit icmp 10.161.0.0 0.0.255.255 any
14 permit icmp 10.175.100.0 0.0.0.255 any
border#

Dima Dvorcovoy · ‎03-23-2020

show ip cef

....

10.175.100.0/24 10.161.1.147 Virtual-Access2.3

...

later:

show ip cef

10.175.100.0/24 10.161.1.147 Virtual-Access2.5

Giuseppe Larosa · ‎03-23-2020

Hello @Dima Dvorcovoy ,

after the debug commands suggested by @Cristian Matei we can say :

>> But, of course, I used on linux tcpdump - i ppp0 and have seen incoming packets to 10.161 not to 10.175.

The IOS XE device is not working properly, after 10 minutes the CEF FIB becomes "corrupted" and the prefix 10.175.100.0/24 is seen via a non existing virtual-access interface vi2.10.

You can ask a maintenance window to reload the IOS XE device but it is likely that you should try to upgrade the IOS XE device in the hope to solve.

Hope to help

Giuseppe

Dima Dvorcovoy · ‎03-23-2020

We can't afford it. The ASR1000 with license was our university main investment in Cisco. If I ask to replace it because of poor coding, nobody understand me and there will be new device from another brand. :-((((

Are there some bypass?

Dima Dvorcovoy · ‎03-24-2020

I did solution.

It is: to make my own dynamic routing. This script has to be run after successfull connection to edit routeing table.

#!/bin/perl
# The super-dynamic router because Cisco one OSPF does not works because of error in CIFS in CISCO IOS
# V1.1 (c) by Inry, Minsk 2020
$DEVICE='ppp0'; # has to be discovered too
$LOGIN='mjprouter'; # need a special user to do this
$PASSWORD='******'; #no, I did not used THIS password
$ROUTER='';
$MYIP='';
$MYROUTE='10.175.100.0 255.255.255.0';

use Net::Telnet;

#-------------------------------------------------------- find my IP and provider
sub FINDIP {
open C,"ifconfig $DEVICE|" or die "?E-no interface!";
while (<C>){
$MYIP=$1 if m/inet ([\d\.]+)/;
$ROUTER=$1 if m/destination ([\d\.]+)/
};
close C;
$MYIP&&$ROUTER
};

if (FINDIP) { print STDERR "?I-MJP: All ok\n"; exit};

open C,">/var/run/xl2tpd/l2tp-control"; # reconnect
print C "c myconnect";
close C;
sleep(60);

unless (FINDIP) { print STDERR "?W-MJP: Can't reconnect, waiting\n"; exit};

#------------------------------------------------------- find route
$RETRIES=4;
do{$RETRIES-- or die "?E-MJP: can't telnet"} until ($T=new Net::Telnet (Host=>$ROUTER,
Prompt=>'/border[^#]*#/',
# dump_log=>'mjp.log', # debug!
Timeout=>10
));
$T->login($LOGIN,$PASSWORD);
#$T->cmd('terminal length 0'); #no need - its already short
$ESCAPEDROUTE=$MYROUTE;
$ESCAPEDROUTE=~s/\./\\./g;
@cfg=$T->cmd("show run | inc ip route $ESCAPEDROUTE"); #filter router side

#------------------------------------------------------- edit routing table
$ALLOK=0;
$T->cmd('configure terminal');
for (@CFG) {
if (m"$MYIP") {$ALLOK=1} #delete only extra routes
else {$T->cmd("no $_")};
};
$T->cmd("ip route $MYROUTE $MYIP") unless $ALLOK; #do not insert if already has one
$B=$T->cmd("exit");
$T->close;
print STDERR "?I-MJP: router table fixed\n";
___END___

Cristian Matei · ‎03-24-2020

Hi,

1. Ensure there is no other remote Linux such device, injecting the same prefix.

2. If the above is not true, you hit a CEF bug which needs to be fixed through a software upgrade, as CEF bugs may impeded further router forwarding capability anyways; if you can't upgrade, at least perform a reload, and see if the problem is gone, sometimes, a reload fixes CEF small hangs.

Regards,

Cristian Matei.

Dima Dvorcovoy · ‎03-24-2020

Of course, if I install another one, I''l use another subnet.

Cisco IOS XE Software, Version 16.06.02
Cisco IOS Software [Everest], ASR1000 Software (X86_64_LINUX_IOSD-UNIVERSALK9_NOLI-M), Version 16.6.2, RELEASE SOFTWARE (fc2)
uptime is 5 days, 5 hours, 45 minutes

cisco ASR1002-X (2RU-X) processor (revision 2KP) with 1127566K/6147K bytes of memory.
Processor board ID FOX*******
8 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
Try to download this one?
https://software.cisco.com/download/home/284146581/type/282046477/release/Fuji-16.9.5?i=!pp