XRd Sandbox - BGP Status Active Not Able to Form TCP Connection

[mood977]

Hi Guys,

I've lunched the XRd Sandbox which include multiple XRd devices as a docker containers .. 

I'm not able to form BGP neighborship, I assume the TCP port 179 is not open ..

Do anyone faced the same issue and how to solve it ?

Noting that I don't have much experience with docker.

Thanks

RP/0/RP0/CPU0:xrd-2#show bgp vpnv4 unicast summary

Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 6 6 6 6 6 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
100.100.100.108 0 100 0 0 0 0 0 00:00:00 Active

RP/0/RP0/CPU0:xrd-2#sho tcp brief
0x00007fc8b00086b0 0x60000000 0 0 100.100.100.102:32388 100.100.100.107:4189 SYNSENT
0x00007fc8a800d940 0x60000000 0 0 100.100.100.102:24368 100.100.100.107:179 SYNSENT
0x00007fc8c000c740 0x60000000 0 0 100.100.100.102:56718 100.100.100.108:179 SYNSENT

[bigevilbeard]

@mood977 the question would be, is this designed to be open? As much as a great resource the sandbox can be, they are set up/designed with security posture in mind which makes this often not possible to change the set up here. 

[mood977]

Thanks for your response, but the purpose of the lab is to do some segment routing with a sample L3VPN service on top .. and that will not work without BGP.

Moreover, the lab includes an XRd node acts as a PCE and pce peering is not forming as well.

Snap from the lab instructions:

Thanks

[bigevilbeard]

@mood977 right... in which case it is designed for such. I looked at the set up in the sandbox guide, is this all up correctly, I see the compose has https://github.com/CiscoDevNet/XRd-Sandbox/blob/main/docker-compose.xr.yml which should bring the up peering when applied?

[mood977]

Hi,

Yes all the containers are up with the startup conf following up the instructions guide..

Even I can see that the isis is forming adjacency as expected.

Thanks

[bigevilbeard]

@mood977 on the devbox, try netstat -tuln | grep 179 or docker network inspect <network_name> might show you some info.

[mood977]

@bigevilbeard I appreciate your valuable time trying to help .. 

Unfortunately, the net-tools package is not installed in the VM.

One way to check it yourself is to Lunch the XRd Sandbox, it takes less than 10 min to setup, but it requires AnyConnect to connect to the VPN.

No specific conf is done from my side, just the default conf ..

Again, thank you very much for your kind support .. appreciated!

[Harold Ritter]

Hi @mood977 ,

The issue is that the IPv4 dataplane does not work properly. I identified a potential issue with the XRd sandbox configuration and contacted the sandbox owner for him to fix it. I will let you know once it is fixed.

Regards,

[bigevilbeard]

Amazing @Harold Ritter 

[mood977]

Great @Harold Ritter , waiting for your feedback.

[Jesus Illescas]

Thanks @mood977 for reporting the issue and thanks @Harold Ritter for pinging.

To fix the issue we need to disable the bridge IP tables. This is actually displayed when running the xrd-host script (expand the output summary on https://github.com/CiscoDevNet/XRd-Sandbox?tab=readme-ov-file#host-check ).

---

For xr-compose to be able to use Docker bridges, bridge IP tables must be disabled. Note that there may be security considerations associated with doing so.
Bridge IP tables can be disabled by setting the kernel parameters net.bridge.bridge-nf-call-iptables and net.bridge.bridge-nf-call-ip6tables to 0. These can be modified by adding 'net.bridge.bridge-nf-call-iptables=0' and 'net.bridge.bridge-nf-call-ip6tables=0' to /etc/sysctl.conf or in a dedicated conf file under /etc/sysctl.d/.

---

I tested the fix and works. Next is to work with the sandbox team to push the change along with other details found by Harold. Changes happen on Thursday, so expect next week to have it done if no issues happen.

[Harold Ritter]

Thanks @Jesus Illescas for fixing the issue

[mood977]

---

@Jesus Illescas wrote:

Thanks @mood977 for reporting the issue and thanks @Harold Ritter for pinging.

To fix the issue we need to disable the bridge IP tables. This is actually displayed when running the xrd-host script (expand the output summary on https://github.com/CiscoDevNet/XRd-Sandbox?tab=readme-ov-file#host-check ).

---

For xr-compose to be able to use Docker bridges, bridge IP tables must be disabled. Note that there may be security considerations associated with doing so.
Bridge IP tables can be disabled by setting the kernel parameters net.bridge.bridge-nf-call-iptables and net.bridge.bridge-nf-call-ip6tables to 0. These can be modified by adding 'net.bridge.bridge-nf-call-iptables=0' and 'net.bridge.bridge-nf-call-ip6tables=0' to /etc/sysctl.conf or in a dedicated conf file under /etc/sysctl.d/.

---

I tested the fix and works. Next is to work with the sandbox team to push the change along with other details found by Harold. Changes happen on Thursday, so expect next week to have it done if no issues happen.

---

Thank @Jesus Illescas & @Harold Ritter 

Appreciated!

[Harold Ritter]

Hi @mood977 ,

I just launched the XRd sandbox and the issues have been fixed. I would like to thank @Jesus Illescas for his help to get this fixed.

Regards,

[Jesus Illescas · Accepted solution]

Hi, as @Harold Ritter shared, the fix is deployed.

It was curious for me how to fix it. The lines to disable the iptables where already present on the sandbox, but I noticed they were not persistent across reboots. So the changes were not applied when a new reservation was done.

The fix was to load the br_netfilter module, so the iptables were loaded and then disabled.

echo "br_netfilter" | sudo tee /etc/modules-load.d/br_netfilter.conf

This make the changes to iptables persistent. I documented this change on the repo https://github.com/CiscoDevNet/XRd-Sandbox?tab=readme-ov-file#how-the-sandbox-was-built