Today I want to show how to set up the FRR package in pfSense. In the past I used for Routed IPsec (VTI) also the OpenBGPD package to advertise the routes automatically to other connected peers.

OpenBGPD is now depricated in pfSense since version 2.5.0 and only available till version 2.4.5.

If you upgrade from 2.4.5 and OpenBGPD package is installed, it will be removed automatically in version 2.5.0.

So keep in mind and take care, that after upgrading to 2.5.0, you first have to install and configure the FRR package with BGP, if not still moved to, otherwise routing won’t work as used to.

https://www.provya.com/blog/pfsense-2-5-0-bugs-and-fixes-after-upgrade/#openbgpd-packages-removed-pfsense


Since late 2017 the FRRouting (FRR) package is available in pfSense and supports BGP, OSPF and OSPF6.

It is designed to replace the OpenBGPD and Quagga OSPF package and allows pfSense users to run both BGPD and OSPF simultaneously.

If you install FRR, pfSense will automatically uninstall OpenBGPD and Quagga OSPF, if either of these packages are already installed.

Source: Mastering pfSense (Second Edition) by David Zientara



Install the FRR Package

Go to System -> Package Manager -> Available Packages and search for bgp or frr to find and install the package.

You will see that in the description also is mentioned, that the FRR package cannot be installed side by side with the OpenBGPD package.

FRR routing daemon for BGP, OSPF, and OSPF6 Conflicts with Quagga OSPF and OpenBGPD. These packages cannot be installed at the same time.

After the package is installed, we can see under the Services menu a few FRR … sub-bullets as follows.



Configure FRR and BGP

First we click and go to the FRR Global/Zebra configuration.

Zebra
http://docs.frrouting.org/en/latest/zebra.html#zebra

zebra is an IP routing manager. It provides kernel routing table updates, interface lookups, and redistribution of routes between different routing protocols.

In the Global Settings section we have to enable FRR, if FRR is not enabled, all other FRR daemons/services will not work as this is the main service under which all others runs and will be managed.

You can of course also first configure all settings and daemons you want to use and finally enable FRR in the Global Settings.


For the Default Router ID in the Global Settings section, I will use here the local IP from my Tunnel Interface (VTI), configured for the Routed IPSec VTI. This IP is also the next to the opposite side and BGP peer.

A Router ID is the unique identifier of a BGP router in an AS.

The router identifier is used by BGP and OSPF to identify the routing device from which a packet originated. The router identifier usually is the IP address of the local routing device. If you do not configure a router identifier, the IP address of the first interface to come online is used. This is usually the loopback interface. Otherwise, the first hardware interface with an IP address is used.

Source: https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/router-id-edit-routing-options.html


You can also use the IP from the internal interface or leave it blank, in this case FRR is using for the Router ID automatically one of the assigned IPs from pfSense.

If the bgpd (BGP Service) connects to zebra it gets interface and address information. In that case default router ID value is selected as the largest IP Address of the interfaces.

Source: https://docs.frrouting.org/en/latest/bgp.html#bgp-router-configuration

Instead assigning the Router ID in the Global Settings section, you can also assign it in the BGP section, in this case it is only used for BGP in case you also running one of the other services like OSPF side by side.

But as still mentioned, you can also leave the Router ID blank and FRR is assigning one regarding the above mentioned conditions automatically.

Further we need to enter a Master Password , this password is only used internally in pfSense and between zebra and the other management services.

Mostly it is used from the Status tab which will need this password to process and request informations from all the FRR services.



Now we can switch to the BGP configuration, therefore you can click in the Global Settings sections menu on BGP with the square brackets or from the main menu under Services -> FRR BGP.

The square brackets shows you that you leave the FRR global configuration section and switch to another FRR service. The same when you reside in one of the other FRR services, here also menu items with square brackets signals you that this belongs to another FRR service and you leaving the actual configuration service section and switch to another.




Here we also first have to enable BGP Routing and further we need to assign our local AS number.

You can use here a number from the reserved private use range which is from 64512 – 65534.

More about Autonomous system (Internet)
https://en.wikipedia.org/wiki/Autonomous_system_(Internet)


The Router ID I still set in the Global Settings section.


Also we need to configure here in the Network Distribution section which networks we want to advertise to the other BGP peer.

In my case I want to advertise the kernel routing table with the static routes from pfSense. You can also restrict the networks which should be advertised and adding them just manual in Networks to Distribute.


Update from 09.07.2022:
Above I configured to only redistribute and advertise the kernel routing table with the static configured routes from pfSense. This week I was running into an issue with the below old configuration were I configured and advertised also the connected networks to pfSense as show in the screenshot below.

With this configuration also the IP range (subnet) from the configured public WAN IP from pfSense will be advertised over BGP to the BGP Peer on the other site, in my case an Azure VNet which is connected to the on-premise network over an IPSec route based VPN tunnel. With that configuration all VMs which resides in that VNet, was getting the route to this public IP subnet with the next hop from the internal network interface from pfSense. Therefore when you try to connect from the on-premise network to some services on the VMs public interface and IP, like a public website, the traffic from the on-premise network is routed correctly over the internet instead the vpn tunnel (only internal IP traffic should be routed through the tunnel), but the reply traffic from the VMs in the Azure VNet was instead routed through the VPN tunnel which ended in a broken connection of course. Because the VMs now had a route entry for the public IP range from the on-premise network with the next hop from the internal pfSense interface, they will route back the reply traffic through the vpn tunnel instead over the internet.

To also advertise the local connected network (LAN Interface) from pfSense, you can use instead the Networks to Distribute field.

First falsely also advertised the connected networks as mentioned.



The result for these settings you will see under Services -> FRR -> Status -> BGP in the BGP table.

Here you can see the routes pfSense will advertise + the routes advertised from other BGP peers.

Btw. here you can see, that the 4. entry with the route to the 172.16.0.0/16 network is advertised from a different connected BGP peer, which is my lab network in Azure. Under Path you can see the AS number from the Azure BGP peer which is 65515 (Private ASN reserved by Azure).

As this is the old screenshot before the update from 22.07.2022, you can also see the advertised public ip subnet 217.110.216.128/25 as mentioned above, which I wouldn’t advertise to the other peer.
The reason why this is advertised and not only the internal static routes, is because I first configured to also advertise connected networks from pfsense.



Now we move on to the Neighbors menu.

Here we have to add all other peers connected to our with BGP.

I will add here the BGP peer from my Azure lab VNet with the AS 65515 number.

In contrast to OpenBGPD, we do not have to create and using a Peer Group.


Enter the Remote AS number from the other BGP peer.


More besides Peer Filtering we do not need to configure for neighbors and nearly been through with the configuration.

This last step with Peer Filtering costs me a lot of time and I struggled with as I forgotten it simply, respectively not knowing I have to configure it as it was nowhere mentioned in sources on the web I found.

Without configuring this step, my BGP peers indeed established the connection between them and also sends BGP Open Messages, Keepalive Messages, Notification Messages and even Update Messages, but the Update Messages normally should contain the advertised routes and subnets, unfortunately not in my case and the reason for are these missing Peer Filtering settings.



In contrast to OpenBGPD, in FRR you have to configure Inbound and Outbound path policies in order FRR will accept and send inbound and outbound updates for the advertised routes.

Without configuring them, you will notice under Services -> FRR -> Status -> BGP in the BGP Neighbors section the following informations.

Inbound updates discarded due to missing policy
Outbound updates discarded due to missing policy
0 accepted prefixes

You can also check the raw config regarding the policy setting when you switch to the Configuration tab in the Status -> BGP menu.


Therefore, the final step before the BGP peers can exchange successfully the routing information between them, is to configure a Prefix List in the Global Settings section from FRR and using them in the Peer Filtering section for the Neighbors in the BGP configuration.


So we go to Services -> FRR -> Global Settings -> Prefix Lists and create a new Prefix Lists type.

In my case I want to advertise all IPv4 routes without any filtering, so the entry will look like this. You can also restrict here what routes you want to advertise using a set of prefixes and sequences to control the order they will be applied.


This prefix I have to set for the inbound and outbound policy in the BGP Neighbors under Services -> FRR -> BGP -> Neighbors in the Peer Filtering settings as follows and mentioned above with the last step.


Now under Services -> FRR -> Status -> BGP in the BGP Neighbors Section, we can see that inbound and outbound policies are configured.

Btw. the one accepted prefix is the 172.16.0.0/16 subnet advertised from the Azure BGP peer and is my Azure lab VNet.


We will also find the policies in the raw config now as follows.

Further we can see this advertised route (prefix) from the Azure BGP peer, under Diagnostics -> Routes

All advertised routes from other peers will have the number 1 as suffix in the Flags column as follows.


Even you configured the above settings correct and nevertheless won’t see any routes from the other peer are advertised and in the BGP Neighbors section under the status page 0 accepted prefixes is shown up, please also check your firewall rules for the IPsec tunnel, the BGP protocol/service is using and listening on TCP Port 179 which should not be blocked by the firewall.




Links

Free Range Routing or FRRouting or FRR
https://en.wikipedia.org/wiki/FRRouting

Free Range Routing or FRRouting or FRR is a network routing software suite running on Unix-like platforms, particularly Linux, Solaris, OpenBSD, FreeBSD and NetBSD. It was created as a fork from Quagga. FRRouting is distributed under the terms of the GNU General Public License v2 (GPL2).

FRROUTING
https://frrouting.org/

FRRouting (FRR) is an IP routing protocol suite for Linux and Unix platforms which includes protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, and RIP.

FRR’s seamless integration with the native Linux/Unix IP networking stacks makes it applicable to a wide variety of use cases including connecting hosts/VMs/containers to the network, advertising network services, LAN switching and routing, Internet access routers, and Internet peering.

FRR has its roots in the Quagga project. In fact, it was started by many long-time Quagga developers who combined their efforts to improve on Quagga’s well-established foundation in order to create the best routing protocol stack available. We invite you to participate in the FRRouting community and help shape the future of networking.

FRRouting User Guide
http://docs.frrouting.org/en/latest/

Zebra
http://docs.frrouting.org/en/latest/zebra.html#zebra

zebra is an IP routing manager. It provides kernel routing table updates, interface lookups, and redistribution of routes between different routing protocols.

Prevent route leaks by explicitly defining policy
https://blog.donatas.net/blog/2019/02/19/ebgp-requires-policy/

https://twitter.com/pfsense/status/941310923096182786
FRR is a different package and more featured solution, we recommend using it instead of the old OpenBGPD package. You’re welcome to attempt fixing OpenBGPD, however we recommend FRR instead.

Dynamic Routing with FRR on pfSense
https://www.youtube.com/watch?v=4IlKcB17rWk&feature=youtu.be