You’re standing in the cold aisle of the data center, or perhaps just staring at a flickering rack in your home office, and the realization hits: if that primary firewall power supply pops, your entire operation goes dark. It’s not just about the internet being “down.” It’s about the dropped Zoom calls, the severed VPN tunnels, and the frantic Slack messages from users who don’t care about your uptime statistics—they just want to work. High Availability (HA) in OPNsense isn’t a luxury; it’s the difference between a restful night’s sleep and a 2:00 AM emergency drive to the office. The Common Address Redundancy Protocol (CARP) is the magic trick that makes two physical firewalls look like one to the rest of your network. But here’s the reality: most engineers treat CARP like a “set it and forget it” checkbox. They miss the nuance of state synchronization (pfsync) or the subtle switch-level traps that lead to the dreaded “split-brain” scenario, where both firewalls think they’re the boss. We’re going to fix that today. We’re moving beyond basic tutorials and into the forensic details of building a cluster that actually works when the smoke starts rising from Node A.

Why Does My OPNsense Failover Drop Active Connections?

The most common frustration I hear from admins is that while their secondary firewall takes over, every active session—SSH, VoIP, database queries—dies instantly. This happens because you’ve configured CARP (the “who is in charge” part) but neglected pfsync (the “what do we know” part). CARP handles the Virtual IP (VIP). It’s a heartbeat. If Node B stops hearing from Node A, it grabs the IP. But if Node B doesn’t have a copy of the state table—the ledger of every active connection passing through the firewall—it will see incoming traffic for an existing session and drop it because it has no record of the initial handshake. This is why a dedicated “Sync” interface is non-negotiable. You are asking your firewalls to replicate thousands of states per second. If you try to cram that data over your busy LAN or WAN interfaces, you’re introducing latency into the very mechanism meant to provide stability. Furthermore, the configuration itself must stay in lockstep. This is where XMLRPC comes in. You don’t want to manually recreate firewall rules or aliases on the backup node. However, a “Costly Trap” here is syncing too much. If you sync DHCP daemon settings to a backup node that doesn’t have the same physical hardware mapping, or if you sync interfaces that aren’t identical, you can lock yourself out of the backup entirely.

What Are the Hidden Infrastructure Requirements for CARP?

Before you even touch the OPNsense WebGUI, your physical (or virtual) switching environment must be ready. CARP relies on the Common Address Redundancy Protocol, which uses multicast address 224.0.0.18 to announce its presence. If you are running OPNsense on managed switches (Cisco, Juniper, Ubiquiti Pro), “IGMP Snooping” is often your enemy. The switch sees the multicast traffic and, trying to be helpful, limits where it sends those packets. If the backup node stops receiving those heartbeats because the switch pruned the multicast group, the backup node will promote itself to Master. Now you have two firewalls fighting over the same IP. This is “Split-Brain,” and it will take your network down faster than a hardware failure ever could. If you are virtualizing OPNsense (Proxmox, ESXi), you have an additional hurdle: **Promiscuous Mode**. CARP uses a virtual MAC address (00:00:5e:00:01:XX). Standard virtual switches will block a VM from using a MAC address that wasn’t assigned to it by the hypervisor. You must enable Promiscuous Mode and “Forged Transmits” on the vSwitch or Port Group, or CARP will simply fail to communicate.

How Do I Configure XMLRPC Without Breaking My DHCP Scopes?

XMLRPC is the engine that copies your configuration from the Master to the Backup. In OPNsense 24.7, this is found under System > High Availability > Settings. The mentor’s advice here: **Be surgical.** When you enable the “XMLRPC Sync” option on the Master node, you’ll see a long list of checkboxes. It is tempting to check everything. Don’t. If you sync “Interfaces,” and your Backup node has a different NIC driver (e.g., the Master uses igb0 and the Backup uses re0), you will break the Backup node’s connectivity the moment you hit “Save.” Focus on syncing: 1. **Firewall Rules:** Essential for consistent security posture. 2. **Firewall Aliases:** Rules are useless without the underlying aliases. 3. **NAT Configuration:** Port forwards must exist on both nodes. 4. **Virtual IPs:** The Backup needs to know which CARP IPs it is responsible for. Avoid syncing **DHCP Server** settings unless you have a very specific reason and have configured DHCP Failover (which is a separate protocol from CARP). Usually, it’s safer to manually configure a non-overlapping DHCP pool on the secondary or use a dedicated DHCP relay.

The Actionable Playbook: Implementing CARP and State Sync

Follow this sequence exactly. Do not skip the verification steps, or you’ll find yourself troubleshooting in the dark. ### Step 1: The Physical Layer (The Heartbeat) Connect a dedicated cable between a free NIC on Node A and Node B. If you’re in a VM environment, create a dedicated “Sync” VLAN. – **Node A Sync IP:** 10.99.99.1/30 – **Node B Sync IP:** 10.99.99.2/30 – Ensure you can ping between these two IPs before proceeding. ### Step 2: Configure pfsync (The Memory) On both nodes, navigate to System > High Availability > Settings. – **Synchronize States:** Check this. – **Synchronize Interface:** Select your dedicated “Sync” interface. – **Synchronize Peer IP:** On Node A, enter 10.99.99.2. On Node B, enter 10.99.99.1. – **Save.** This ensures that if Node A has an active state for a user, Node B knows about it. ### Step 3: Configure XMLRPC (The Brain) **Perform this ONLY on the Master node.** – **Remote Copy Config:** Check this. – **Remote IP:** 10.99.99.2 (The Backup’s Sync IP). – **Remote Username:** root (or an admin account). – **Remote Password:** The password of the Backup node. – **Select Categories:** Choose Rules, Aliases, NAT, and Virtual IPs. – **Save.** Check the System > Firmware > Log to ensure the sync was successful. ### Step 4: Create the Virtual IPs (The Face) On the **Master Node**, go to Interfaces > Virtual IPs > Settings. – **Type:** CARP. – **Interface:** LAN (and later, repeat for WAN). – **IP Address:** This is your “Gateway” IP that clients will use. – **VHID Group:** A unique integer (e.g., 1). This must match on both nodes. – **Advertising Frequency:** – **Master:** Base 1, Skew 0. – **Backup:** Base 1, Skew 100. – **Password:** A shared secret for CARP. ### Step 5: The Switch Hardening Log into your physical switch. If it’s a Cisco-style CLI: – Find the ports connected to your OPNsense nodes. – Disable IGMP Snooping for those specific VLANs: no ip igmp snooping vlan [ID]. – If using Port Security, ensure the switch allows multiple MAC addresses on those ports (the physical MAC + the CARP virtual MAC).

Why Is My Secondary Node Stuck in a Master-Master Conflict?

If both nodes show “MASTER” in the Interfaces > Virtual IPs > Status page, you have a communication breakdown. The Backup node isn’t “hearing” the Master’s advertisements. Check these three things immediately: 1. **Firewall Rules on the Sync/LAN Interface:** You must have a rule on both nodes allowing “Protocol: CARP” from “Any” to “Any.” If the firewall blocks the heartbeat, it will trigger a failover. 2. **Outbound NAT:** On the WAN side, ensure your Outbound NAT is set to “Hybrid” or “Manual” and that it is using the **WAN CARP VIP** as the translation address, not the “Interface Address.” If the Master goes out as its physical IP and the Backup goes out as its physical IP, external services (like banking sites or VPNs) will drop the connection during failover because the source IP changed. 3. **The VHID:** Ensure you aren’t using the same VHID on two different subnets unless you absolutely know your switch handles it. Keep them unique across your entire infrastructure to avoid MAC address collisions.

Frequently Asked Questions

**Can I use CARP with a single public WAN IP?** Technically, no. CARP requires three IPs per subnet: one for Node A, one for Node B, and one for the Virtual IP. If your ISP only provides a single static IP via DHCP, you cannot use CARP on the WAN. You would need to put a basic router/modem in front of the OPNsense cluster or ask your ISP for a /29 subnet. **Does CARP work over Wi-Fi?** Absolutely not. The latency and packet loss inherent in Wi-Fi will cause constant CARP flapping. High Availability requires stable, low-latency wired connections. **What happens if the Sync cable unplugged?** This is a “Split-Brain” trigger. Both nodes will assume the other is dead and become Master. This is why some admins prefer to run the Sync traffic over a dedicated VLAN on their main switches (with redundancy) rather than a single point-to-point cable.