Introduction

Frederic-Cuiller · ‎03-10-2015

BGP is one of the most critical routing protocol for a Service Provider. Although this protocol is very robust and scalable, it is crucial for them to monitor it. BGP Monitoring Protocol has been built for this purpose.

In this blog post, you will learn:

What is BGP Monitoring Protocol (BMP)
How to configure BMP on Cisco IOS-XR
How to install OpenBMP collector
How to collect BMP data and what are the potential usage

Introduction

Cisco IOS-XR 5.2.2 release introduced the support of BGP Monitoring Protocol (BMP). In a nutshell, BMP aims to:

Monitor BGP session state
Check BGP prefixes activity (currently IPv4, IPv6, VPNv4)
Provide access to the Adj-RIB-in table (pre inbound BGP route-policy filtering)

BMP architecture relies on two key components: a client (e.g IOS-XR router) and a collector (e.g OpenBMP).

BMP client copies each BGP update it receives from a peer and sends it to the BMP collector. The BMP collector decodes and stores BMP records, which may be later used.

BMP is currently based on draft-ietf-grow-bmp-07.

IOS-XR configuration

The configuration on IOS-XR is straightforward. You first need to configure a BMP server:

RP/0/RSP0/CPU0:ASR9010(config)#bmp server <1-8> ?

description BMP server specific description

dscp Set IP DSCP (DiffServ CodePoint)

host Hostname of BMP Server

initial-delay Initial connect delay in sending BMP server updates

initial-refresh Initial refresh to generate BGP updates

precedence Set precedence

shutdown Shutdown connection to BMP server

stats-reporting-period Stats reporting period for BMP server

update-source Source to reach BMP server

vrf VRF for the BMP server

<cr>

Then activate the BMP server under a BGP peer:

router bgp <ASN>

neighbor <neighbor>

bmp-activate server <1-8>

There are few commands available to monitor BMP:

RP/0/RSP0/CPU0:ASR9010#sh bgp bmp summary

Tue Mar 3 17:33:48.404 CET

ID Host Port State Time NBRs

1 172.17.254.123 5555 ESTAB 00:35:34 1

RP/0/RSP0/CPU0:ASR9010#

RP/0/RSP0/CPU0:ASR9010#sh bgp bmp server 1 detail

Tue Mar 3 17:33:09.372 CET

BMP server 1

Host 172.17.254.123 Port 5555

Connected for 00:34:55

Precedence: internet

BGP neighbors: 1

VRF: - (0x60000000)

Update Source: 172.17.255.35 (Mg0/RSP0/CPU0/0)

Update Source Vrf ID: 0x60000000

Message Stats:

Total messages sent: 10

INITIATION: 4

TERMINATION: 1

STATS-REPORT: 0

PER-PEER messages: 5

Neighbor 99.0.1.7

Messages pending: 0

Messages sent: 5

PEER-UP: 2

PEER-DOWN: 1

ROUTE-MON: 2

RP/0/RSP0/CPU0:ASR9010#

And some debugs:

RP/0/RSP0/CPU0:ASR9010#debug bgp bmp

RP/0/RSP0/CPU0:Mar 3 16:58:13.819 : bgp[1053]: [default-ext]: [1/172.17.254.123] vsocket -> default vrf

RP/0/RSP0/CPU0:Mar 3 16:58:13.819 : bgp[1053]: [default-ext]: [1/172.17.254.123] set tos 0x0 / 0x6 afi 0x2 fd 153

RP/0/RSP0/CPU0:Mar 3 16:58:13.820 : bgp[1053]: [default-ext]: [1/172.17.254.123] set tos result 0

RP/0/RSP0/CPU0:Mar 3 16:58:13.820 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp connect: -1

RP/0/RSP0/CPU0:Mar 3 16:58:13.820 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp connect: delayed

RP/0/RSP0/CPU0:Mar 3 16:58:13.820 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp statechange: NONE->CONNECT

RP/0/RSP0/CPU0:Mar 3 16:58:13.821 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp select: fd: 153 rc: 0

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp callback: fd: 153 cond: 1

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp callback: fd: 153 cond: 2

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp statechange: CONNECT->ESTAB

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [BMP Message Dump : Server 172.17.254.123, nbr (None), type Initiation Message]:

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 0300 0000 6c04 0001 0057 4369 7363 6f20

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 494f 5320 5852 2053 6f66 7477 6172 652c

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 2056 6572 7369 6f6e 2035 2e32 2e32 5b44

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 6566 6175 6c74 5d0a 436f 7079 7269 6768

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 7420 2863 2920 3230 3134 2062 7920 4369

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 7363 6f20 5379 7374 656d 732c 2049 6e63

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: 2e00 0200 0741 5352 3930 3130

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp select: fd: 153 rc: 0

RP/0/RSP0/CPU0:Mar 3 16:58:13.873 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp callback: fd: 153 cond: 2

RP/0/RSP0/CPU0:Mar 3 16:58:13.876 : bgp[1053]: [default-ext]: [1/172.17.254.123] tcp callback: fd: 153 cond: 2

OpenBMP installation

OpenBMP is an open-source BMP collector written by Tim Evens, Technical Leader at Cisco.

OpenBMP collects BMP updates, which are then stored in a MySQL database. Installation is easy and the procedure is documented here.

Once installed, we can check the BMP exchanges between the router and the collector:

root@openbmp:/home/cisco# tail -f /var/log/openbmpd.log

2015-02-20T10:33:14.185583 | INFO | runServer | Client Connected => 172.17.255.35:22078, sock = 8

2015-02-20T10:33:14.185847 | INFO | runServer | Waiting for new connection, active connections = 2

2015-02-20T10:33:14.195967 | INFO | runServer | Client Connected => 172.17.255.35:45828, sock = 5

2015-02-20T10:33:14.195998 | INFO | runServer | Waiting for new connection, active connections = 2

2015-02-20T10:33:14.279303 | INFO | ClientThread | Thread started to monitor BMP from router 172.17.255.35 using socket 8

2015-02-20T10:33:14.279445 | INFO | ClientThread | Thread started to monitor BMP from router 172.17.255.35 using socket 5

2015-02-20T10:33:14.335657 | INFO | ReadIncomingMsg | 172.17.255.35: Init message received with length of 102

2015-02-20T10:33:14.335679 | INFO | handleInitMsg | Init message type 1 and length 87 parsed

2015-02-20T10:33:14.335690 | INFO | handleInitMsg | Init message type 1 = Cisco IOS XR Software, Version 5.2.2[Default]

2015-02-20T10:33:14.335695 | INFO | handleInitMsg | Init message type 2 and length 7 parsed

2015-02-20T10:33:14.335699 | INFO | handleInitMsg | Init message type 2 = ASR9010

2015-02-20T10:33:14.377702 | INFO | ReadIncomingMsg | 172.17.255.35: Init message received with length of 102

2015-02-20T10:33:14.377721 | INFO | handleInitMsg | Init message type 1 and length 87 parsed

2015-02-20T10:33:14.377739 | INFO | handleInitMsg | Init message type 1 = Cisco IOS XR Software, Version 5.2.2[Default]

2015-02-20T10:33:14.377744 | INFO | handleInitMsg | Init message type 2 and length 7 parsed

2015-02-20T10:33:14.377748 | INFO | handleInitMsg | Init message type 2 = ASR9010

2015-02-20T10:36:00.912343 | INFO | ReadIncomingMsg | 172.17.255.35: PEER UP Received, local addr=99.0.1.6:41962 remote addr=99.0.1.7:179

2015-02-20T10:36:02.976462 | INFO | parseUpdateMsg | 99.0.1.7: rtr=172.17.255.35: End-Of-RIB marker

2015-02-20T10:38:11.172338 | INFO | parseUpdateMsg | 99.0.1.7: rtr=172.17.255.35: End-Of-RIB marker

Collecting BGP data

Once your IOS-XR router and OpenBMP collector are ready, it’s time to use those BMP records you collected.

The only way today is to run raw SQL queries against the OpenBMP database. Several tables are available:

mysql> show tables;

+---------------------------+

| Tables_in_openBMP |

+---------------------------+

| bgp_peers |

| gen_asn_stats |

| gen_whois_asn |

| path_attr_log |

| path_attrs |

| peer_down_events |

| peer_up_events |

| prefix_aspath |

| rib |

| routers |

| stat_reports |

| v_peer_prefix_report |

| v_peer_prefix_report_last |

| v_peers |

| v_routes |

| v_routes_history |

| v_routes_history2 |

| withdrawn_log |

+---------------------------+

18 rows in set (0.00 sec)

From which you can collect different information. For example, it’s possible to gather details for specific prefixes:

mysql> select RouterName,PeerName,Prefix,PrefixLen,Origin,Origin_AS,MED,LocalPref,NH,AS_Path from v_routes where prefix like '2.2.2%';

+------------+----------+---------+-----------+------------+-----------+------+-----------+----------+---------+

+------------+----------+---------+-----------+------------+-----------+------+-----------+----------+---------+

| ASR9010 | 99.0.1.7 | 2.2.2.2 | 32 | incomplete | 1337 | 0 | 0 | 99.0.1.7 | 1337 |

+------------+----------+---------+-----------+------------+-----------+------+-----------+----------+---------+

1 row in set (0.00 sec)

We can also collect the withdraw history:

mysql> select * from withdrawn_log;

+----------+----------------------------------+---------+------------+---------------------+

+----------+----------------------------------+---------+------------+---------------------+

| 21849170 | 3b0a6a0b8790682b3e5014618ad81589 | 1.1.1.1 | 32 | 2015-02-20 10:38:53 |

+----------+----------------------------------+---------+------------+---------------------+

1 row in set (0.00 sec)

mysql>

If you want to see what’s possible to do with BMP records, a demo is available here:

http://demo.openbmp.org:8000/ (demo/demo)

It currently uses DB_REST. However, OpenBMP is working on OpenDaylight integration. This looks very promising and would be the occasion of a second post once available.

You can also interact directly with the OpenBMP MySQL database and build your own applications.

BMP use cases

BMP can be used for many applications. Here are some examples of BMP use cases:

BGP looking glass:

Classic looking glasses software are not always secure. With BMP, no need to give access to the router or use an alternative BGP daemon. Use the BMP collector to provide information on BGP prefixes.

Removal of “soft-reconfiguration-inbound always” feature and route-policy troubleshooting:

I still see many customers using BGP soft-reconfiguration-inbound. The reason I’m given is RPL troubleshooting purpose, as they can access the Adj-RIB-In and see how are the prefixes they receive BEFORE filtering. The drawback with this feature is memory consumption: router keeps a copy of the BGP table for each peer it’s enabled. With BMP, this information can be retrieved on the BMP collector directly. You can compare the view before and after the RPL application without impacting the router performance.

Route analytics:

Track BGP prefixes activity, convergence time, attributes modifications, etc.

Conclusion

You are now ready to deploy BMP on Cisco IOS-XR to monitor the critical piece BGP is in your Service Provider infrastructure.

You should now understand what BMP is, how to configure it on Cisco IOS-XR and make it work with OpenBMP.

Credits: I would like to thank Bertrand Duvivier and Tim Evens for the valuable information they shared with me.

Frederic CUILLER – Network Consulting Engineer

@fcuiller

References

BGP Monitoring Protocol draft v07

Cisco IOS-XR 5.2.2 Release Notes

BGP Monitoring Protocol presentation by Bertrand Duvivier, Product Manager @ Cisco

OpenBMP by Tim Evens, Technical Leader @ Cisco

amit.bhagat · ‎02-04-2016

Hi Fred,

Does IOS-XR support sending post-policy Adj-RIB-in too (L flag set to 1 as per BMP draft v17)?

Amit

luis.cunha · ‎04-29-2016

Hi Fred,

user demo/demo are not working.

ravisha4 · ‎10-25-2016

Hello,

I have one case where peer CPU is getting 90% utilized by sending Route Refresh messages from ASR.

In logs i found

default-bgp/spkr-tr2-gen 0/RSP0/CPU0 # t28 [GEN]:73: BMP Initiated route refresh for <>

Below is Config

remote-as *****

password encrypted *****

description *****

bmp-activate server 1

address-family ipv4 unicast

route-policy ***** in

maximum-prefix 100000 75

route-policy ***** out

remove-private-AS

soft-reconfiguration inbound always

Config for BMP

bmp server 1

host ***** port 5000

description OpenBMP

update-source Loopback0

initial-delay 300

stats-reporting-period 300

initial-refresh delay 900 spread 450

My questions is initial-refresh delay 900 then what would be frequency of sending RR messages

chintan-shah · ‎03-15-2017

same for me, it's not working.

bgiorgadze · ‎05-17-2017

Hello,

I try configure bmp connection established but bmp not sent any update.

please Help to solve this issue

Installed IOS-XR 5.3.4

bmp server 1

host x.x.x.x port 5000

update-source Loopback0

initial-delay 60

stats-reporting-period 300

initial-refresh delay 60 spread 1

neighbor x.x.x.x

remote-as ZZ (my LOCAL_AS)

bmp-activate server 1

address-family ipv4 unicast

route-policy PASS in

route-policy PASS out

!

route-policy PASS

pass

end-policy

!

RP/0/RSP0/CPU0:IOS-XR#show bgp bmp server 1 detail

Wed May 17 19:14:51.450 gmt

BMP server 1

Host x.x.x.x Port 5000

Connected for 00:18:23

Last Dosconnect event received : 00:00:00

Precedence: internet

BGP neighbors: 1

VRF: - (0x60000000)

Update Source: x.x.x.x (Lo0)

Update Source Vrf ID: 0x60000000

Message Stats:

Total msgs dropped : 0

Total messages sent: 3

INITIATION: 3

TERMINATION: 0

STATS-REPORT: 0

PER-PEER messages: 0

Neighbor x.x.x.x

Messages pending: 0

Messages dropped: 0

Messages sent: 0

PEER-UP: 0

PEER-DOWN: 0

ROUTE-MON: 0

ZHR_123 · ‎03-03-2019

Hi Fred,

Does IOS-XR support sending post-policy Adj-RIB-in too ?

fcuiller · ‎03-04-2019

Hi,

IOS-XR 6.2.2 support Post-Policy Adj-RIB-In

You can turn it off/on with:

bmp server all
[no] route-monitoring policy post inbound

Release notes is here: https://www.cisco.com/c/en/us/td/docs/iosxr/ncs5500/general/62x/release/notes/b-release-notes-ncs55k-622.html#id_53888

Fred

ZHR_123 · ‎03-04-2019

Thanks ：）

Observing BGP activity with BGP Monitoring Protocol

Introduction

IOS-XR configuration

OpenBMP installation

Collecting BGP data

BMP use cases

Conclusion

References

Chalk Talk: Troubleshooting DTMF-Relay over MGCP

Configuring and Troubleshooting Basic MPLS Layer3 VPN

Upgrading to 6.3.2 plus releases coming from 43x 51x 52x need to knows!