as you may have read recently I was playing with open source routing protocol packages again. from pure CLI familiarity reasons, I kept myself to FRRouting, which is evolution of Quagga, which itself is evolution of Zebra. and Zebra syntax and CLI is based on Cisco IOS.
FRRouting
thanks to Job Snijders for correcting me on the name - it’s no longer OpenFRR, it’s FRRouting. sorry! :)
unfortunately, while it worked very well for my home network (FRRouting that is), when deploying in AS112 I hit some unexpected behaviors quite quickly after starting the project. FRRouting was forgetting ARP entries to directly connected next-hops, which in turn was invalidating some of the routes… and traffic was no longer returning over paths advertised by IXP members. that was bad, as it meant while we’re still sinkholing private DNS requests, we were not responding to them. that may have meant slow responses and Customers noticing delay in “no such host” response.
initially I thought it was something related to my sysctl.conf config that had networking stack tuned, or pf.conf configuration that is responsible for filtering traffic coming in and out.
after clearing out all customizations, unfortunately the problem didn’t go away - FRRouting was still dropping next-hops shortly after restart. I suspect it’s somehow related to interaction between bgpd and zebra that talk over TCP connections in BSD systems, but debugging this further meant digging in the code myself. FRRouting devs seems to be currently focused more on Linux it seems.
OpenBGPd
so I turned back to my old friend - OpenBSD project’s OpenBGPd. I even did patch it years ago to be able to run in on FreeBSD and in some scenarios, I’m using it until today (for example, BGP Blackholing PL feeders are running on it currently).
currently, in FreeBSD ports tree there are two packages - OpenBGPd based on 5.5 and on 6.6 (so called portable version). I got stuck at 5.5 release, and happily noticed 6.6 is available.
after preparing configuration and launching it in AS112 deployment, I noticed however OpenBGPd 6.6 won’t install best paths (and therefore - announce them to upstream BGP speakers). after quick debug I realized that OpenBGPd simply doesn’t install any routes in FreeBSD FIB (aka kernel) table. that got me thinking that maybe I’m doing something wrong, but casual reading (RTFM!!!) of ports package explained the behavior. let me quote:
root@as112:/usr/local/etc # pkg info openbgpd6
openbgpd6-6.6p0_1
Name : openbgpd6
Version : 6.6p0_1
[...]
Description :
OpenBGPD is a FREE implementation of the Border Gateway Protocol, Version 4.
It allows ordinary machines to be used as routers exchanging routes with
other systems speaking the BGP protocol.
This is the portable version and it does not have the means to
influence kernel routing tables. It is only suitable for route
servers/collectors.
well… if it can’t actually interact with kernel FIB, then it’s no use for scenarios where I actually need to install routes (typical IXP scenario, running default-less). it can however be of great use for typical control-plane only scenarios.
but I needed ideally solution to cover both scenarios, as our AS112 deployment uses them.
enter BIRD
my initial BIRD first steps were back in version 1, where bird was still using two different daemons for IPv4 and IPv6 separately. also, the configuration file syntax was kind of… complex, so I simply skipped learning it, as I had better things to do.
right now BIRD seemed a good fit and option to consider, given FRRouting defective behavior.
long story short, BIRDv2 uses single configuration file, and after reading some intro manuals you can get it working. there’s some interesting naming conflict (BIRD for example uses keyword protocol to mean internal communication channel, not exactly routing protocol itself), but after playing with it for a moment, things became clearer.
so, for AS112 my configuration looks like that (compare this to my original FRRouting bgpd and staticd configuration:
log syslog all;
router id A.B.C.D; # your BGP router-id
filter as112_v4
prefix set allnet4;
{
allnet4 = [
192.175.48.0/24,
192.31.196.0/24
];
if ! (net ~ allnet4) then reject;
accept;
# this filter will be used later to control what we advertise
# towards IPv4 neighbors - those are AS112 prefixes
}
filter as112_v6
prefix set allnet6;
{
allnet6 = [
2001:4:112::/48,
2620:4f:8000::/48
];
if ! (net ~ allnet6) then reject;
accept;
# this filter will be used later to control what we advertise
# towards IPv6 neighbors - those are AS112 prefixes
}
protocol device {
interface "vmx0" {
preferred A.B.C.D;
preferred 2001:A:B:C::1;
# if you have multiple addresses defined on interface
# (i'm not using loopback0) this is handy to make sure
# which source addresses are used by BIRD
};
}
protocol direct IF_VMX0 {
interface "vmx0";
ipv4;
ipv6;
}
protocol kernel kipv4 {
ipv4 {
table master4;
import all; # you may not want to do that, as it means
# BIRD will install in your OS kernel RIB
# prefixes learnt over BGP - but we do want
# that to happen
export all; # we also want to export OS kernel RIB
# to your BGP
};
learn;
}
protocol kernel kipv6 {
ipv6 {
table master6;
import all;
export all; # same as above for IPv6
};
learn;
}
protocol static sipv4 {
ipv4; # we're nailing down static IPv4 routes
route 192.175.48.0/24 via A.B.C.D;
route 192.31.196.0/24 via A.B.C.D;
}
protocol static sipv6 {
ipv6; # we're nailing down static IPv6 routes
route 2001:4:112::/48 via 2001:A:B:C::D;
route 2620:4f:8000::/48 via 2001:A:B:C::D;
}
template bgp RS_v4 { # template for IPv4 BGP neighbors
local A.B.C.D as 112;
ipv4 {
import all;
export filter as112_v4; # here we're using filter defined
# previously
};
};
template bgp RS_v6 { # template for IPv6 BGP neighbors
local 2001:A:B:C::D as 112;
ipv6 {
import all;
export filter as112_v6; # here we're using filter defined
# previously
};
};
protocol bgp RS1_IPv4 from RS_v4 {
description "RS IPv4 #1";
neighbor C.D.E.F as 65055; # insert your peer IPv4 and ASN
};
protocol bgp RS1_IPv6 from RS_v6 {
description "RS IPv6 #1";
neighbor 2001:C:D:E::F as 65055; # insert your peer IPv6 and ASN
};
once that done, you can verify if connections are established either from interactive CLI (birdc) or from shell command like below. we’re looking for routes advertised towards specific neighbor:
$ birdc show route export RS1_IPv4
BIRD 2.0.7 ready.
Table master4:
192.175.48.0/24 unicast [sipv4 14:06:50.346] * (200)
via A.B.C.D on vmx0
192.31.196.0/24 unicast [sipv4 14:06:50.346] * (200)
via A.B.C.D on vmx0
listing all configured channels is done via:
$ birdc show protocols
BIRD 2.0.7 ready.
Name Proto Table State Since Info
device1 Device --- up 14:06:50.346
IF_VMX0 Direct --- up 14:06:50.346
kipv4 Kernel master4 up 14:06:50.346
kipv6 Kernel master6 up 14:06:50.346
sipv4 Static master4 up 14:06:50.346
sipv6 Static master6 up 14:06:50.346
RS_IPv4 BGP --- up 14:06:55.213 Established
RS_IPv6 BGP --- up 14:06:55.212 Established
if you want to check route counts advertised and received:
$ birdc show route count
BIRD 2.0.7 ready.
13 of 13 routes for 9 networks in table master4
14 of 14 routes for 10 networks in table master6
Total: 27 of 27 routes for 19 networks in 2 tables
summary
I can highly recommend BIRDv2 as working routing package if you’re looking for open source solutions. configuration may look complex and while there’s interactive CLI option it’s not quite even remotely similar to well known CLI parsers.