In this post we will see how to set up a perimeter network with public IPv4 Addresses and pfSense. The configuration and routing for the back firewall which separates the perimeter network with the internal LAN, I will not go through and is not a topic of this post.

Generally you can use private or public IPv4 addresses for your perimeter network. When using private addresses you have to configure Network Address Translation (NAT) to translate the private addresses behind the PERIMETER interface into one of the public addresses assigned to the WAN Interface on pfSense.

If you want to use public IPv4 addresses for your perimeter network and don’t want or cannot deploy NAT with private IPv4 addresses, for what ever reason, some applications are also very NAT unfriendly, we will first need to divide our public IPv4 network (address block), our ISP has assigned to us into more than one subnet.

At least we need two subnets, so that we get two separate network IDs, one for the external WAN interface and subnet and one for the PERIMETER interface and subnet.

For this post I will use my public IPv4 213.61.93.80/29 address block from my lab network and assigned by Colt.


Btw. if you have also only a /29 subnet from your ISP, it doesn’t really make sense to split this into two /30 subnets in order to use public IPv4 addresses (actually only one address as one of the two available is used for the internal interface on pfSense) in your perimeter network.

But for this post to show what we have to configure, the /29 subnet is good enough.


Dividing this /29 network into two /30 will give me the following subnets

Subnet ID 1: 213.61.93.80/30 with a host range 213.61.93.81 – 213.61.93.82
Subnet ID 2: 213.61.93.84/30 with a host range 213.61.93.85 – 213.61.93.86

Colt has still assigned the IP 213.61.93.81 as gateway for the origin /29 network, so I will use the first subnet id 1 for the WAN interface at pfSense and the subnet id 2 for the perimeter network.

For larger networks and multiple subnets you can use a subnet calculator like the following
https://www.site24x7.com/tools/ipv4-subnetcalculator.html



As mentioned I will use the first subnet id 1 for the WAN interface and subnet. From this subnet only one IP address 213.61.93.82 is left to use for the WAN interface. So the addresses from the subnet id 1 are as follows.

  • 213.61.93.80 Subnet Address
  • 213.61.93.81 Gateway Address
  • 213.61.93.82 WAN Interface pfSense
  • 213.61.93.83 Broadcast Address



For the IPv4 Upstream gateway in pfSense, we have to use the 213.61.93.81 address which is configured from Colt as the gateway for the whole /29 network.

This Gateway IP is still in the range of the new first /30 subnet we will use for the WAN interface.



For the PERIMETER subnet I will use the second subnet id 2. From this subnet I can use two IP addresses, one for the PERIMETER interface on pfSense and one for a virtual machine in the perimeter network. So the addresses from the subnet id 2 are as follows.

  • 213.61.93.84 Subnet Address
  • 213.61.93.85 Perimeter Interface pfSense
  • 213.61.93.86 Virtual Machine inside the Perimeter Network
  • 213.61.93.87 Broadcast Address




For the internal perimeter interface we do not assign a gateway as usual, here you can assign static routes to your internal LAN.


Next we should disable NAT Outbound if we want to use public IPv4 addresses inside our perimeter network. With public IPv4 addresses, we just have to route the traffic without the need to translate the addresses.

Btw. you can of course also use NAT for your public IPv4 addresses inside the perimeter network, which then will be translated into the public IPv4 addresses from the WAN interface, but this doesn’t really make sense and would waste public IPv4 addresses.

So goto Firewall – NAT – Outbound and select Disable Outbound NAT. Not to mention that we also doesn’t need Port Forwarding anymore. From now on we just route and filter the traffic from the WAN interface and subnet to the PERIMETER interface and subnet and vice versa.


Finally, that the routing from and into our perimeter network with the public IPv4 subnet will work, we have to consider and configure one more thing and can therefore choose between two options.

At the moment, the ISP Router from Colt doesn’t know how to route traffic for the second subnet id 2 (213.61.93.84/30), we created from the origin /29 subnet for our perimeter network.

The first subnet id 1 with our WAN interface from pfSense is still in the range from the ISP gateway with the IP 213.61.93.81, despite the fact that the subnet mask has changed from 29/ into /30.



So what are the two options, to tell the ISP Router the next hop for the second subnet id 2, which have to be the WAN interface and IP 213.61.93.82 from pfSense?


The first and best method is to tell your ISP, to what IP and next hop he should route the second subnet id 2. So in my case, I should tell the ISP that he should route the subnet 213.61.93.84/30 to my WAN interface on pfSense with the IP 213.61.93.82.


The second is to configure pfSense to tell the ISP Router the next hop for the second subnet id 2. So if you cannot tell your ISP to set a static route to the second perimeter subnet, you can use this method.

In pfSense you can create different types of virtual IPs on the WAN interface.

Virtual IP Address Feature Comparison
https://pfsense-docs.readthedocs.io/en/latest/firewall/virtual-ip-address-feature-comparison.html

To publish more than the assigned public IPv4 on the WAN interface, you can use virtual IPs from the type IP Alias.

They will be used in a classic NAT deployment where you use private IPv4 addresses in your perimeter or LAN network. So with the IP Aliases, you have more than one public IPv4 on the WAN interface assigned and can therefore publish multiple systems listening on the same public port number.


What we are interested, to tell the ISP Router the next hop for our perimeter subnet id 2, are virtual IPs from the type Proxy ARP.

As I only have one public IPv4 address in my perimeter network, besides the PERIMETER interface from pfSense, I just need to add my public IPv4 address 213.61.93.86, which will use the VM in the perimeter network, as Proxy ARP address.

You can also add the whole /30 subnet as Proxy ARP address



From now on pfSense replies to ARP requests on Layer 2 from the ISP Router and therefore the ISP Router knows that he have to route traffic for the perimeter subnet directly to the WAN interface and IP 213.61.93.82 from pfSense.


Therefore the routing and connection to and from the internet also works from now on.


But why does this work and what’s all behind this two methods?

Therefore we should know how hosts (nodes) on a network segment at layer 2, Gigabit Ethernet Switch in our case, determine the destination port at the switch to which they should send traffic for another host (node) connected to this port.


The following figure shows our configuration with the perimeter network.

To better understand what’s happening under the hood, we sends out an ICMP Echo Request (Ping command) to google.com from our Windows 10 VM homed in the perimeter network.

First we try this without configuring one of the two options with the new route at the ISP Router from Colt for our perimeter subnet id 2 or the ARP Proxy setting. So in this case, the Colt Router doesn’t know the route to our perimeter network.

Let’s see what happens.

Here you can see the IP configuration from the Windows 10 VM.

Before we fire up the PING command, we first need to start the packet capture at pfSense, to capture the traffic from the VM to the Colt ISP Router use the following settings.

So at pfSense we goto Diagnostics – Packet Capture and start the capture with the following settings.

Click on Start

Now we can fire up the PING (ICMP Echo Request) command at our Windows 10 VM in the perimeter network.

As you can see we didn’t get any ICMP Echo Reply messages back, at least pfSense resolved the FQDN google.com for us into an valid IP address.


And the reason is not related to the firewall settings, outbound traffic from the perimeter network to WAN is generally allowed.

So as mentioned the reason is the missing route to the perimeter network on the ISP Colt Router.

To check this we can now download the Capture from our test and open the cap file with WireShark.


Here we can see that pfSense correctly forwards our ICMP Echo Request packets to the Colt ISP Router but doesn’t get any response back.


We also can see the destination JuniperN_b7:3f:f0 (28:c0:da:b7:3f:f0), pfSense forwards our ICMP Echo Request packets towards, which is our ISP Colt Router with the gateway IP 213.61.93.81.

To be sure we can check this with the ARP Table at pfSense to determine the corresponding IP address to this MAC address as follows.


So the Colt ISP Router doesn’t send back the ICMP Echo Request packets, because he doesn’t knows the correct route and next hop for our perimeter network 213.61.93.84/30 and the source IP inside from the Windows 10 VM 213.61.93.86/30.

To show you how the Colt ISP Router with the gateway IP 213.61.93.81/29 and MAC Address 28:c0:da:b7:3f:f0 tries to determine the next hop for the perimeter network, I will have to start a second packet capture at pfSense during the Windows 10 VM is starting up.

Therefore I will first shutdown the VM and after it is turned off, I will start a second packet capture at pfSense and start the VM again.

The settings for the packet capture btw. will be exactly the same as for the first time and above.

After downloading and opening the cap file with WireShark, we can filter the traffic for ARP messages.

Here we can see two interesting ARP Broadcast messages sends out from the Colt ISP Router 213.61.93.81 for the IP addresses 213.61.93.86 (our Windows 10 VM) and 213.61.93.85 (the PERIMETER interface from pfSense)

With ARP request messages, the Colt ISP Router tries to determine the MAC address for the requested IPs and therefore the final destination port in his own network segment (WAN Switch) he should submit the packets directly.

You can see that the Colt ISP Router sends the ARP request to the Broadcast address and therefore to all nodes in this network segment (WAN Switch).


Normally the node with the requested IP should send an ARP reply message back like 213.61.93.86 is at 00:15:5d:32:aa:42


As this IP is homed on the internal side and perimeter network of pfSense, pfSense wouldn’t reply by design and for security reasons to such requests for internal IPs and therefore the ISP Router doesn’t get any reply to his requests and don’t know where to route the packets back.

On the other side, if at the Colt ISP Router where configured a static route with the next hop for the perimeter network where this IP is homed, pointed to the WAN interface from pfSense, pfSense would accept the packets and route them to the perimeter network.


If we configured the mentioned ARP Proxy at the WAN interface, pfSense will reply to ARP Requests for this IP.


To show you how pfSense in this case would reply with ARP reply messages, we have to capture another time the packets from the WAN interface. This time we have to wait a little bit longer as for the ARP request messages. The reason for is the default arp timeout value from Juniper with 21600 seconds (6 hours) 🙁 . So in this case the Colt ISP Router holds the MAC address for this IP in his cache before he sends out a further ARP request message.

Juniper NETWORKS – TechLibrary
arp timeout
https://www.juniper.net/documentation/en_US/junose15.1/topics/reference/command-summary/arp-timeout.html


Finally I captured an ARP reply message back to the Colt ISP Router from the WAN interface from pfSense as follows. As mentioned, pfSense only replies here because we added the public IPv4 address as Proxy ARP address.


Here you can see that the MAC address in the ARP reply message belongs to the IP from the WAN interface from pfSense.



Links

Virtual IP Address Feature Comparison
https://pfsense-docs.readthedocs.io/en/latest/firewall/virtual-ip-address-feature-comparison.html

Proxy ARP 
https://en.wikipedia.org/wiki/Proxy_ARP

Common Address Redundancy Protocol (CARP)
https://de.wikipedia.org/wiki/Common_Address_Redundancy_Protocol

IP aliasing
https://en.wikipedia.org/wiki/IP_aliasing

Colt Technology Services
https://www.colt.net/