Bypassing the AT&T Fiber Gateway with an XGS-PON SFP+ Module

Published on

XGS-PON AT&T FreeBSD Networking Homelab

The Goal

Take the AT&T BGW320-500 fiber gateway out of the path entirely, and let the FreeBSD edge router pull the public DHCP lease itself. No passthrough mode, no double NAT, no extra hop, no vendor box quietly running its own software between me and the wire. Just fiber, an SFP+ ONT, and FreeBSD.

The hardware that makes this practical is the X-ONU-SFPP, an XGS-PON SFP+ module (a WAS-110 alternative) sold pre-flashed with the 8311 community firmware and the Azores bootloader. It speaks XGS-PON on the fiber side and presents itself as a 10G Ethernet link on the SFP+ side. Drop it into a 10G port on a router or switch and the host gets the WAN. That's it.

Before and After

Before
======
AT&T fiber  ->  BGW320-500 (gateway)  ->  Ethernet  ->  FreeBSD (LAN-side IP)
                  ^ vendor NAT, vendor firewall, vendor admin UI
                  ^ public IP lives here

After
=====
AT&T fiber  ->  X-ONU-SFPP  ->  ix0 SFP+ on FreeBSD (homefw)
                                  ^ FreeBSD pulls the public DHCP lease itself
                                  ^ pf is the only firewall in the path

Bill of Materials

The Prep: What to Pull Off the AT&T Gateway

AT&T's PON authentication needs three things tied to your BGW320-500: the GPON / XGS-PON serial number, the MAC address it advertises, and an EAP-TLS certificate / private key pair. Without those, OLT-side authentication will quietly refuse the new module and you'll see no link.

The 8311 community has built up a thorough, regularly updated set of guides for extracting these from the BGW320-500. Rather than re-paste a procedure that changes with each AT&T firmware push, I'll point at the source:

The exact button-mash sequence on the BGW depends on its current firmware version, so check the latest pon.wiki post for your specific case. What you'll come away with is a small bundle of files and identifiers that get loaded onto the X-ONU-SFPP via its 8311 web UI.

Configuring the SFP Module

The X-ONU-SFPP boots into an OpenWRT-derived environment with a 8311 web UI. Once it's powered up (an SFP+ slot supplies enough power; the USB-C cooler fan needs its own cable) you reach the UI by setting your laptop or workstation to the same management subnet (the module ships with 192.168.11.1 by default) and opening a browser.

Inside the UI, paste in the cert/key pair, the PON serial, and the MAC address you pulled from the BGW. The 8311 firmware exposes the right fields directly; pon.wiki's screenshots make this hard to mess up. Save, reboot the module, and confirm in the UI logs that authentication succeeds and a PON link comes up.

At this point you can already cable the SFP module to the fiber and watch for an OLT-authenticated link, before involving FreeBSD at all. That's the single best test point in the whole project.

Wiring It In

Pull the BGW320 out of the path:

  1. Disconnect the fiber drop from the BGW.
  2. Plug the X-ONU-SFPP into the SFP+ port on the FreeBSD box (in my case ix0).
  3. Plug the fiber drop into the X-ONU-SFPP's SC/APC port.
  4. Power the USB-C cooler.

The order matters: the module should be in the host before the fiber goes in, so you can watch the link come up on the host and aren't squinting at the module's LEDs in a closet.

FreeBSD-Side Config

With the module installed in ix0, the FreeBSD config to pull a public DHCP lease is small. The pieces below are the WAN-relevant lines from my /etc/rc.conf; the rest of the router build is unchanged. 

# /etc/rc.conf, WAN-side excerpt

# WAN: SFP+ port hosting the X-ONU-SFPP
ifconfig_ix0="DHCP"
ifconfig_ix0_ipv6="inet6 accept_rtadv"
background_dhclient_ix0="YES"   # XGS-PON DHCP can be slow at boot

The background_dhclient_ix0="YES" line is the one that fixed a class of boot-time bugs for me, and it's worth its own paragraph. The X-ONU-SFPP doesn't always finish PON authentication and DHCP in the second or two between when the kernel brings ix0 up and when rc.d wants to start the rest of the network-dependent services. Without backgrounding the dhclient call, boot blocks waiting for a lease that's about to arrive, and the wait takes long enough that ntpd, unbound, and even sshd all try to come up against a half-configured stack. Backgrounding the WAN dhclient lets boot continue normally; the lease lands a beat later and everything settles.

Bringing the Link Up

$ service netif restart
$ ifconfig ix0
ix0: flags=8943<UP,BROADCAST,RUNNING,...,MULTICAST> metric 0 mtu 1500
        inet 104.10.x.y netmask 0xfffffc00 broadcast 104.10.x.255
        inet6 fe80::%ix0 prefixlen 64 scopeid 0x1
        ...
        media: Ethernet autoselect (10Gbase-LR <full-duplex>)
        status: active

$ netstat -rn | head -5
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            104.10.x.1         UGS         ix0

That's a working WAN. From here, every other piece of the homelab tour is the same as it was when the BGW was upstream: pf rules, DHCP server on the LAN side, unbound, IPv6 RA acceptance.

What Could Go Wrong (and Where to Look)

Rollback Plan

Keep the BGW320 in the closet, untouched, until you've been on the bypass for a couple of weeks. Reverting is just: power down the FreeBSD box, pull the X-ONU-SFPP, plug the fiber back into the BGW, power the BGW. The whole round-trip is on the order of five minutes. The new setup is reliable, but nothing about a residential ISP is friendly to "I broke the WAN at 11pm", so the safety net is cheap.

Why It's Worth It

The replacement isn't about saving money (the X-ONU-SFPP costs more than the BGW does to rent for a year), and it isn't about throughput (XGS-PON was already 10G capable behind the BGW). It's about removing a piece of opaque vendor software from the data path on my own home network, and putting the public IP, the firewall, and the routing in one place I actually control. After the swap, every packet between my LAN and the internet crosses exactly one box I built, and that's the whole point.

Where to Go Next

Running the same module on a different ISP, or doing the same kind of bypass on a different gateway? Drop me a note; I want to hear how the FreeBSD-side story changes.

$ subscribe --to newsletter

FreeBSD, pf, and SRE notes, straight to your inbox. No spam, just signal.

Powered by Buttondown. Unsubscribe anytime. ~2 emails/month. Or grab the RSS feed.

Related Posts

← Back to Blog