In my post below I will show how to set up a lean branch office network without any servers and DCs inside, by using an IPSec S2S VPN tunnel connected with the headquarters network.

Set up a lean Branch Office Network without any Servers and DCs inside by using an IPSec S2S VPN Tunnel connected with the Headquarters Network


In order to provide a WPA2/3-Enterprise WLAN by using the RADIUS server from the headquarters network, over an IPSec S2S route-based VPN tunnel with pfSense on both sites, I was running into the following issue.

The EAP authentication exchange for wireless clients through the access point and RADIUS server homed in the headquarters network, stuck by using a route-based IPSec with pfSense during the SSL handshake and here exactly after the Client Hello as shown below.

Here you will see the IEEE 802.11 EAP authentication exchange on data link layer 2 between the wireless client (notebook/wlan card) and the access point.
Access Point -> MAC 94:A6:7E:00:7F:AA
Notebook -> MAC 20:1E:88:D2:61:67


Below you will see the UPD/IP traffic between the RADIUS server and the Access Point regarding the above EAP authentication exchange. You will see several duplicate Access-Request and Access-Challange messages and fragmented UDP packets, they all exceed the maximum transmission unit (MTU).

Filter only RADIUS traffic and fragmented traffic
Fragmentation has occured when either the more fragment bit is set or the fragmentation offset is greater than zero

ip.flags.mf ==1 or ip.frag_offset gt 0 or radius


When using a policy-based IPSec in pfSense the EAP authentication exchange works as expected.


Here you can see finally the Access-Accept message from the RADIUS server to the Access Point and only one fragmented UDP packet and no duplicate response or request message.

All other traffic works for both, route-based and policy-based without any issues, besides the EAP authentication exchange with the RADIUS server.


This issue is related to the RADIUS attribute Framed-MTU size on the RADIUS server (NPS server role on Windows server) itself. By default this will be 1.500 bytes for the Microsoft Network Policy Server (NPS).

The solution is to add on the NPS server the RADIUS attribute Framed-MTU with a maximum size of 1.344 bytes, as mentioned further below in the Microsoft article about recommended Framed-MTU value.


After that the EAP authentication exchange for wireless clients through the access point and RADIUS server will also works by using the route-based IPSec VPN tunnel in pfSense.

Configure the Framed-MTU Attribute
Use this procedure to lower the maximum EAP payload size by using the Framed-MTU attribute in an NPS network policy. You can lower the EAP payload size by configuring the Framed-MTU attribute in network policy settings properties in the NPS console.

Perform this procedure if you have routers or firewalls that are not capable of performing fragmentation. The recommended Framed-MTU value in this circumstance is 1344 bytes or less.

Source: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc771164(v=ws.10)?redirectedfrom=MSDN


So by using a route-based IPSec VPN tunnel with pfSense, in my case I had to adjust the default Framed-MTU size from 1.500 bytes into 1.344 bytes on the NPS server role.

I have no clue why this issue only happens by using the route-based S2S IPSec


As the IPsec encapsulation overhead, along with the IP and frame overhead, can cause the resulting Encapsulating Security Payload (ESP) packet to exceed the MTU, I double checked both, phase 1 and phase 2 encryption settings, to be sure that there is no difference between the route-based and policy based IPSec tunnel in pfSense, both where identical.

As you can see further above from the capture by using the Microsoft Network Monitor on the NPS server, the EAP authentication exange will stuck after the SSL handshake and here exactly after the Client Hello, the next step would be normally the Server Hello from the NPS server, by returning the full certificate chain from the server certificate, to proof its identity to the client.

So it seems that the response packets from the NPS server including the full certificate chain are too large and have to be fragmented which isn’t tolerate by EAP itself recording the following article.

EAP itself cannot tolerate fragmentation and must receive the complete message intact, but certain EAP methods have a way of dealing with frags.  Within the context of EAP, the Framed-MTU doesn’t control the MTU of the outer packet that gets fired down the WAN, it controls the EAP maximum message size.  If the EAP-TLS message is larger than the Framed-MTU, then the message is broken up locally, and a “fragmentation” flag is added to each message.  Before the Network Access Device (NAD) will send subsequent fragments to the RADIUS server (or vice versa), the previous one needs to be ACK’ed, and the fragmentation flag tells the other side this needs to be done.
Source: https://community.cisco.com/t5/network-access-control/radius-framed-mtu-attribute/m-p/3542070



About how to set up a RADIUS server on a Windows Server 2019 for 802.1X connections and here especially for 802.11 wireless connections, you can read my following post.

Set up a RADIUS Server on Windows Server 2019 for 802.1X Wireless Connections



Links

Configure the Framed-MTU Attribute
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc771164(v=ws.10)?redirectedfrom=MSDN

EAP-Fragmentierungsimplementierungen und -Verhalten
https://www.cisco.com/c/de_de/support/docs/lan-switching/8021x/118634-technote-eap-00.html

Problem with fragmentation? NPS won’t authenticate clients
https://docs.microsoft.com/en-us/answers/questions/329662/problem-with-fragmentation-nps-won39t-authenticate.html