[Prompt]
custom: Let's talk about the purpose and functionality of tunables in Opensense. These are custom advanced p | Hosts: corn, herman

[Response]
Corn: Daniel sent us this one — he's asking about OPNsense tunables, what they actually are under the hood, and why you'd ever want to mess with kernel-level variables on a firewall that already ships with sensible defaults. It's a good question, because the feature sits right there in the GUI looking like just another settings page, but what you're actually doing is reaching past the entire web interface and telling the FreeBSD kernel how to behave.

Herman: The timing on this is interesting. OPNsense twenty-four point seven is due next month — July twenty twenty-six — and one of the things they're expected to ship is a new tunables helper that surfaces inline sysctl descriptions directly in the GUI. So the feature's about to get more accessible, which is great for discoverability, but also means more people are going to stumble into it and think, "well, let me just bump this number up, bigger must be better.

Corn: Which is how you turn a perfectly functional firewall into a very expensive space heater with packet loss.

Herman: That's the danger, yeah. The descriptions being right there lowers the barrier, and lowering the barrier is good when you know what you're doing and catastrophic when you don't. I've already seen forum threads where someone sets kern dot ipc dot nmbclusters to two million because they read somewhere that more mbufs means more throughput, and then their box runs out of wired memory and the state table collapses.

Corn: So they allocated roughly four gigabytes of RAM to mbuf clusters and then wondered why things got worse.

Herman: Each cluster is two kilobytes on modern FreeBSD — thirteen and fourteen — so the math is straightforward, but people skip the math. And that's really what Daniel's question gets at. What are these things, what do they actually control, and when is it not just safe but genuinely useful to touch them?

Corn: Let's get into it. Where do we start — the mechanism?

Herman: Because if you don't understand what a tunable is at the kernel level, the GUI page just looks like a spreadsheet of intimidating variable names and numbers, and that's exactly how people get themselves into trouble.

Herman: A tunable, stripped down to its bones, is a sysctl MIB — Management Information Base entry. That's the naming convention FreeBSD uses for kernel state variables. Every knob the kernel exposes, from TCP buffer sizes to how many packets can queue up on a network interface, lives in this hierarchical namespace. You've seen them — net dot inet dot tcp dot sendspace, kern dot ipc dot nmbclusters. Those dotted names are the OIDs, the object identifiers.

Corn: OPNsense takes those and puts a web form on top of them.

Herman: The GUI page at System, Settings, Tunables is essentially a friendly editor for slash etc slash sysctl dot conf. But here's the specific plumbing — OPNsense doesn't write to that file directly. It writes to slash etc slash sysctl dot conf dot local, which gets sourced at boot by the sysctl service. So when you add a tunable through the web interface, OPNsense validates that the OID actually exists against the running kernel, stores it in its config XML, and then on next boot the rc scripts apply it.

Corn: If the OID doesn't exist — if you typo it or you're on a kernel version that doesn't have that particular variable — it just fails silently on boot?

Herman: It'll log an error, but the boot continues. Which is actually safer than the alternative, but it means you might think your tunable is active when it's not. That's one of those things you learn the hard way if you're not checking with sysctl dash a after boot.

Corn: The scope here matters. These are not firewall rules. You're not deciding what traffic passes. You're deciding how the kernel allocates memory for packet processing, how aggressively it acknowledges TCP segments, how many connections it'll queue before it starts dropping.

Herman: And that's the thing I want to nail down before we go deeper. Tunables are kernel-level. They affect the TCP slash IP stack behavior, memory allocation, concurrency limits, hardware offload parameters. They are not NAT rules, not plugin settings, not anything that lives in the pf packet filter. You could have a perfectly written firewall ruleset and still cripple your throughput by setting the wrong tunable. Or vice versa — you could fix a performance problem that no amount of ruleset tweaking would ever touch.

Herman: Let's walk the lifecycle on a real one. Take net dot inet dot tcp dot sendspace. That's the maximum amount of data the kernel will buffer in the TCP send socket buffer before it tells the application "hold on, the receiver's not keeping up." You go into the GUI, you add it as a tunable, you set it to, say, two hundred sixty-two thousand one hundred forty-four bytes. OPNsense immediately checks — does this OID exist in the running kernel? It does, because every FreeBSD kernel exposes the TCP stack variables. So it accepts the entry, writes it to config XML, and on next boot the rc script sources slash etc slash sysctl dot conf dot local and applies it.

Corn: If you run sysctl net dot inet dot tcp dot sendspace right now on a default OPNsense box, you'll see sixty-five thousand five hundred thirty-five. That's OPNsense's own baseline.

Herman: Which is already bumped up from FreeBSD's stock default. FreeBSD ships with something much more conservative — I want to say around thirty-two thousand on older releases — because the upstream project has no idea whether you're installing on a machine with two gigs of RAM or two hundred and fifty-six. OPNsense applies its own tuned baseline during install, and sixty-five K is a reasonable middle ground for a firewall that's mostly passing packets through, not terminating connections.

Corn: The moment that firewall becomes a VPN concentrator, it's not just passing packets anymore. It's terminating thousands of TLS sessions, doing encryption, buffering data for clients on slow links.

Herman: That's exactly where the default stops being your friend. An OpenVPN server running on OPNsense with the default sixty-five K sendspace will hit TCP window scaling issues under load — the sender can't fill the pipe because the buffer's too small to keep data in flight while waiting for acknowledgments. Bump that to two hundred sixty-two thousand one hundred forty-four, and suddenly the window can scale properly. Throughput improves dramatically. But — and this is the part people skip — you've just quadrupled the per-connection memory overhead for send buffers alone.

Corn: Which on a box handling ten thousand concurrent connections is the difference between a few hundred megs of buffer memory and over a gig.

Herman: That memory is wired. The kernel can't page it out. It's gone from the pool available for connection tracking, for the state table, for userspace processes. So you fix VPN throughput and then two weeks later your state table starts dropping entries under load, and you can't figure out why because you already "fixed" the problem.

Corn: This is the mbuf cluster problem in a different hat. You mentioned kern dot ipc dot nmbclusters earlier — walk me through that math.

Herman: Mbufs are the kernel's fundamental unit of network memory — every packet coming in or going out gets stored in one or more mbufs, and when the kernel needs to attach data to an mbuf it allocates a cluster. Each cluster is two kilobytes on FreeBSD thirteen and fourteen. The nmbclusters tunable sets the hard ceiling on how many of those clusters the kernel will allocate. Default on OPNsense is typically two hundred fifty-six thousand. That's five hundred twelve megabytes of wired memory just for packet buffers.

Corn: Someone sees packet drops under load, reads a forum post that says "increase nmbclusters," and sets it to a million.

Herman: Now they've wired two gigabytes of RAM into mbuf clusters. On a firewall with four gigs total, that leaves two gigs for everything else — the operating system, the state table, pf's rule processing, any running services. If the state table needs more than what's left, it starts dropping states. Which looks exactly like the original problem but is actually worse, because now you're dropping connections instead of packets. And the user thinks "well I increased the buffer pool, that can't be the issue," so they go hunting somewhere else entirely.

Corn: The classic "I fixed X so why is Y on fire" debugging spiral.

Herman: It all traces back to not measuring first. The kernel tells you exactly how many mbuf clusters are in use — vmstat dash z shows you the usage counters for every memory zone. If you're not hitting the ceiling, increasing it does nothing except reduce available RAM. If you are hitting it, you need to figure out why before you just throw more clusters at the problem. Maybe you have a driver that's leaking mbufs. Maybe you're under a volumetric attack. The number going up is a symptom, not a diagnosis.

Herman: That measurement-before-touching philosophy gets even more important when you realize tunables are global. Every single one applies system-wide. You change net dot inet dot tcp dot delayed underscore ack, and you've just changed it for every TCP connection that touches that firewall — not just the one service you're trying to fix.

Corn: You're trying to speed up a sluggish SMB share, you disable delayed ACK because you read it reduces latency, and suddenly your VoIP phones start stuttering because they were relying on the kernel batching acknowledgments to keep overhead low.

Herman: That exact scenario has played out on the OPNsense forums more times than I can count. Someone disables delayed ACK, throughput on their file transfers improves by maybe fifteen percent, and then three days later they notice SIP registration is flaky and DNS queries are taking longer. The tunable doesn't know you only meant to apply it to port four forty-five. It applies to everything.

Corn: The "I fixed X but broke Y" genre of forum post.

Herman: It's practically its own literary form at this point. And that global scope makes certain tunables especially treacherous in mixed-use environments. Take net dot inet dot tcp dot fastopen — TFO cookies let a client send data in the SYN packet instead of waiting for the three-way handshake to complete. Set that to one, and HTTPS connections to web servers that support it get noticeably faster.

Corn: I'm hearing a "but.

Herman: The SIP phone on your desk probably doesn't handle TFO cookies. The old network printer definitely doesn't. The embedded controller for your HVAC system? It sees a SYN packet with unexpected data and doesn't know what to do with it. So it just drops the connection. And because TFO failures are silent — the retransmission falls back to a normal handshake eventually — you don't get an error log. You get a phone that intermittently fails to register, and you spend three days chasing VLAN configs and firewall rules before you even think to check whether you turned on fastopen last Tuesday.

Corn: This is the tunable that keeps on giving. You set it, forget it, and then six months later you're troubleshooting a problem that makes no sense because the cause and effect are separated by an entire TCP fallback mechanism you forgot exists.

Herman: That's the subtlety with global kernel parameters. The blast radius is the entire system, and the symptoms don't always point back to the tunable. Now let me give you a few specific scenarios where tunables are actually the right tool, because it's not all danger. High-latency WAN links are one. If you're running OPNsense at a branch office with a satellite link — we're talking six hundred milliseconds round trip — the default TCP maximum segment size and path MTU discovery behavior can leave you with terrible throughput.

Corn: Because the window scaling we talked about earlier interacts with the latency, and the defaults assume a LAN-like environment.

Herman: You'd look at net dot inet dot tcp dot mssdflt — the default maximum segment size — and potentially lower it to avoid fragmentation across the satellite hop. You might also need to adjust net dot inet dot tcp dot recvspace upward to give the window enough room to scale across that delay. But you measure first. You run iperf3 between the sites before touching anything, establish your baseline, make one change, measure again.

Corn: Which brings us to the measurement problem. OPNsense's built-in traffic graphs and system activity dashboard are fine for spotting that something is wrong. They're useless for telling you whether a tunable change actually helped.

Herman: The performance tuning page on the OPNsense wiki — last updated March twenty twenty-six — is explicit about this. It tells you to benchmark with iperf3 before and after every change. And beyond that, you need the command-line tools. Netstat dash w gives you live packet and error counters per interface. Systat dash ifstat shows you throughput and queue drops in real time. Vmstat dash z shows you exactly how many mbuf clusters are in use versus the ceiling. If you're not looking at those, you're tuning blind.

Corn: The workflow is: measure with iperf3 and netstat, identify the actual bottleneck, test a tunable live with the sysctl command, observe for at least twenty-four hours, and only then make it permanent in the GUI.

Herman: That live-test step is critical because the GUI has no test mode. You add a tunable, save, apply, and it's permanent at next boot. There's no "try this for an hour and revert." You test with the sysctl command first because that changes the running kernel immediately but doesn't survive a reboot. If you break something, you reboot and you're back to baseline.

Corn: Which is a much better failure mode than "my firewall is broken and I have to SSH in from the cold.

Herman: And one more scenario worth touching — hardware offload conflicts. Certain Intel NICs, the igb driver family, have a feature called adaptive interrupt moderation. The tunable is hw dot igb dot enable underscore aim. It's supposed to dynamically adjust interrupt rates based on traffic load. On most hardware it works fine. On some specific chipsets, it causes periodic throughput collapses — the NIC will just drop to near zero for a few seconds, then recover.

Corn: That looks like a cable problem or a switch problem, so you go replacing hardware before you think to check a kernel tunable.

Herman: Disabling AIM by setting that tunable to zero fixes it instantly. And the only way you'd know to look there is by finding a forum thread where someone with the exact same NIC revision posted the exact same throughput graph. That's where the OPNsense community documentation becomes indispensable — the ~200 tunables listed in the GUI with descriptions are a starting point, but the real knowledge lives in those "I had this exact problem" threads.

Corn: Which is also the difference between OPNsense and pfSense on this front. Same sysctl backend, but pfSense exposes fewer tunables in the GUI by default. OPNsense surfaces more of them with descriptions, which is useful for discoverability — but it also means more people are going to find a knob, turn it, and learn the hard way that global kernel parameters don't come with guardrails.

Herman: That's the tension heading into the twenty-four point seven release. More descriptions, easier access, a possible reset-to-default button — all good things if you already understand the measurement-first discipline. If you don't, it's a faster path to a broken firewall.

Herman: We've covered the mechanism, the tradeoffs, the ways this can go wrong. Let's land on something practical. What should someone actually do the next time they're staring at the tunables page?

Corn: Daniel's question was "what are these and why would you use them" — and I think the honest answer to "why would you use them" starts with "only after you've proven you need to." So give me the workflow.

Herman: Step one: before you make anything permanent, you test it live. Run sysctl dash a pipe grep the OID to see the current value. Then sysctl OID equals new value to apply it to the running kernel. That change takes effect immediately and it's gone on reboot. If your network survives twenty-four hours — no weird call drops, no state table collapse, throughput actually improved — then and only then do you add it through the GUI.

Corn: The reboot safety net. If you broke something, power cycle and you're back to baseline.

Herman: Step two is knowing which tunables are actually worth your attention. For a typical home or lab OPNsense box, I'd put five on the shortlist. Net dot inet dot tcp dot sendspace and recvspace — those are your TCP buffer sizes, the first thing to look at when throughput on high-latency links is underwhelming. Kern dot ipc dot nmbclusters — your mbuf ceiling, only touch it if vmstat dash z shows you're actually hitting the limit. Net dot inet dot ip dot intr underscore queue underscore maxlen — that's the IP input queue, relevant when you're dropping packets under interrupt load. And net dot link dot ifqmaxlen — the interface output queue length, useful when a single interface is a bottleneck.

Corn: Everything else on that page of two hundred, you probably don't need unless you have a very specific problem and a forum thread from someone with your exact hardware.

Herman: The third thing is the one that costs nothing and breaks nothing. Open the GUI, go to System, Settings, Tunables, and just read the descriptions for ten minutes. Don't change anything. Just understand what's there. Most people will never need to touch a single tunable on that page. But knowing what's available — knowing that there is a knob for interrupt moderation, a knob for TCP buffer sizing, a knob for mbuf limits — means that when you do hit a performance wall, you'll remember where to look.

Corn: The map before the journey. You're not memorizing every street, you're learning the shape of the terrain so you know which direction to walk when something breaks.

Herman: And that ten minutes of reading is what separates "I found a forum post and blindly copied the settings" from "I understand what I'm changing and why.

Corn: That's the practical toolkit. But there's a bigger question lurking here, and I think it's worth landing on before we go. OPNsense is adding more hardware offload support — netmap, VPP, the kind of stuff that moves packet processing out of the kernel and into dedicated silicon or userspace frameworks. As that trend continues, do tunables become less relevant because the kernel's doing less of the heavy lifting, or do they become more dangerous because the interactions get harder to reason about?

Herman: I think it's both, and that's what makes it interesting. When you offload packet processing to netmap or a vector packet processor, a whole class of tunables — the mbuf tuning, the interrupt moderation, the queue lengths — those stop mattering for the offloaded traffic. The hardware or the userspace framework handles its own buffering and scheduling. So in one sense, tunables get less relevant. But the stuff that stays in the kernel — the TCP stack for management traffic, the state table, the control plane — still runs through those same variables.

Corn: You end up with a split brain. The data plane is offloaded and auto-tuned, the control plane is still running on FreeBSD's defaults, and now you've got two different tuning domains that interact in ways nobody's fully documented.

Herman: That's where the danger comes in. Someone sees that their offloaded throughput is great but their management interface is sluggish, so they start tweaking kernel tunables that they think only affect the management side — but some of those variables still have hooks into the offload driver's fallback paths. The blast radius gets weirder, not smaller.

Corn: The twenty-four point seven release is interesting in that context. A "reset to OPNsense default" button on the tunables page — that's not just a convenience feature. That's the project acknowledging that people are going to paint themselves into corners and need an escape hatch.

Herman: It's a safety net, and I'm glad they're adding it. The fact that they're building it into the same release that makes tunables more discoverable tells you they understand the dynamic. Lower the barrier to entry, but also lower the cost of getting it wrong.

Corn: For the power users who actually know what they're doing — the ones who've benchmarked, measured, tested live for a week — the escape hatch is still there. They'll always want the knobs. Auto-tuning can handle the common cases, but it can't anticipate a branch office on a satellite link with a specific traffic profile and a NIC that needs adaptive interrupt moderation disabled.

Herman: The knobs aren't going anywhere. The project's just trying to make them safer to find. And honestly, that's the right posture for a firewall distribution — expose the power, document the risks, and give people a way back when they oversteer.

Corn: Daniel's question — "what are these and why would you use them" — the answer turns out to be: they're the kernel's own configuration language, and you use them when you've proven the defaults are wrong for your specific hardware and workload, not a moment before.

Herman: Now: Hilbert's daily fun fact.

Hilbert: In the Tang dynasty, the Bureau of Forestry measured timber output in a unit called the "cun" — roughly one point two modern inches — and mid-level officials were required to personally verify log diameters against a standardized iron ruler. Filing a false measurement carried a penalty of sixty lashes.

Corn: Sixty lashes for fudging a lumber report. I'm not saying modern procurement could use some of that energy, but I'm not not saying it.

Herman: That's a level of bureaucratic commitment I was not prepared for.

Corn: This has been My Weird Prompts. Thanks to our producer Hilbert Flumingtop. If you enjoyed this, do us a favor and leave a review wherever you listen — it helps. We're back next week.