Any Specification about AM if it functions as "medium p2p" with OSPF?

mhiyoshi · ‎10-25-2023

Dear all,

It is pretty curious if it configures below and after ping Nexus9K creates AM route. I appreciate if there is any speficiation or tips.

interface Ethernet1/3
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

[Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide, Release 9.3(x)]
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/93x/unicast/configuration/guide/b-cisco-nexus-9000-series-nx-os-unicast-routing-configuration-guide-93x/b-cisco-nexus-9000-series-nx-os-unicast-routing-configuration-guide-93x_cha...

In rare instances, an inconsistency can occur between the unicast RIB and the FIB on each module. Cisco NX-OS supports the Layer 3 consistency checker. This feature detects inconsistencies between the unicast IPv4 RIB on the supervisor module and the FIB on each interface module. Inconsistencies include the following:

Missing prefix
Extra prefix
Wrong next-hop address
Incorrect Layer 2 rewrite string in the ARP or neighbor discovery (ND) cache

The Layer 3 consistency checker compares the FIB entries to the latest adjacency information from the Adjacency Manager (AM) and logs any inconsistencies. The consistency checker then compares the unicast RIB prefixes to the module FIB and logs any inconsistencies. See the Triggering the Layer 3 Consistency Checker section.

The following is test output.

LF4# sh ip route ospf-1
*omit
22.22.22.22/32, ubest/mbest: 1/0
    *via 22.22.22.22, Eth1/3, [110/2], 00:00:44, ospf-1, intra

LF4# show forwarding 22.22.22.22
slot  1
=======
IPv4 routes for table default/base
------------------+-----------------------------------------+-------------------
---+-----------------+-----------------
Prefix            | Next-hop                                | Interface
   | Labels          | Partial Install
------------------+-----------------------------------------+-------------------
---+-----------------+-----------------
22.22.22.22/32       22.22.22.22                               Ethernet1/3

After ping you can see am (adacency manager?) route.

LF4# ping 22.22.22.22 count 1
PING 22.22.22.22 (22.22.22.22): 56 data bytes
64 bytes from 22.22.22.22: icmp_seq=0 ttl=254 time=1.652 ms

--- 22.22.22.22 ping statistics ---
1 packets transmitted, 1 packets received, 0.00% packet loss
round-trip min/avg/max = 1.652/1.651/1.652 ms
LF4#

LF4# sh ip route ospf-1
*omit
22.22.22.22/32, ubest/mbest: 1/0
    *via 22.22.22.22, Eth1/3, [110/2], 00:01:12, ospf-1, intra
     via 22.22.22.22, Eth1/3, [250/0], 00:00:56, am

LF4# show forwarding 22.22.22.22
slot  1
=======
IPv4 routes for table default/base
------------------+-----------------------------------------+-------------------
---+-----------------+-----------------
Prefix            | Next-hop                                | Interface
   | Labels          | Partial Install
------------------+-----------------------------------------+-------------------
---+-----------------+-----------------
22.22.22.22/32       22.22.22.22                               Ethernet1/3

LF4# show consistency-checker forwarding
IPV4 Consistency check :table_id(0x1)
Execution time : 14 ms ()
No inconsistent adjacencies.
No inconsistent routes.
Consistency-Checker: PASS for ALL

f00z · ‎10-31-2023

medium p2p has nothing to do with ospf, but it is required on nexus to use unnumbered interfaces. It's kind of a workaround to auto-install routes to the interface with the nexthop when it's unnumbered instead of requiring a route with a nexthop to interface like in IOS where you had to have the route such as 'ip route 1.1.1.1/32 interface gi1/1 3.3.3.3' whereas with medium p2p it makes it auto install nexthop so route can just be 'ip route 1.1.1.1/32 interface eth1/1' when it's unnumbered. That's why you see an autoinstalled adjacency route. As far as I know this is a nx-os(and also newer IOS-XE/XR) specific thing. medium p2p will also make ospf default to point to point although the command itself doesn't pertain to ospf, it allows routing protocols to work by auto installing nexthop adjacencies.

To add a little clarification to what this command does, it disables broadcast on the interface, thus anything that requires broadcast to form adjacency (such as ARP, such as EIGRP) won't work unless changes are made to the protocol to support it, so OSPF code had some changes to support this, AM(adjacency manager) installs FIB adj entry , etc.. Lots of things in the code to support it so it's seamless with unnumbered. This makes OSPF send multicast only (i.e. the point to point setting which the medium p2p would make OSPF default to). I don't think EIGRP has a multicast/unicast only setting with no broadcast (correct me if I'm wrong but afaik it doesn't work over p2p). OSPF/IS-IS should work fine and then the BGP can form through that and then no need for any static routing anywhere.

To add a 2nd clarification now that I slept on it a little (sucks to always be dreaming about networking but hey it's my job).. Your question is a little vague but I think you are asking why there are two entries. One of the entries is the learned route from OSPF:
*via 22.22.22.22, Eth1/3, [110/2], 00:01:12, ospf-1, intra
And, the other entry: via 22.22.22.22, Eth1/3, [250/0], 00:00:56, am (this is the ARP entry). So the first entry is the route learned from OSPF (via multicast) and then the 2nd entry is there because the router has to know what MAC address to send the ethernet frames to (i.e. eedb/adjacency entry). The OSPF route is learned from multicast, OSPF code is modified to work with unnumbered to generate proper LSA's and info, and AM code is modified to send ARP broadcast even on p2p link to generate AM entry for egress encapsulation (what mac do we send it to). This is not a bug, it is how the system works. Since it doesn't need the ARP entry because there's 0 traffic on the link other than OSPF (which is multicast only so no need to resolve ARP entry), the ARP entry gets created on first traffic sent to the destination like usual, and that happens to be your ping.