My old setup was:
VSDL modem -> pfsense on mini J1900 Celeron (2 GHz) -> CISCO SG300 10MPP switch -> Rukus R310 wifi -> Laptop
Currnet setup
Fiber model -> pfsense on mini J1900 Celeron (2 GHz) -> CISCO SG300 10MPP switch -> Rukus R310 wifi -> Laptop
Today i got my 1GBit fiber installed (big deal for those like me living in rural areas) only to discover that my current network setup is not allowing me to benefit from it.
I was on VSDL copper wire before and was probably in the region of 50-60 MBit/s with my above current setup. Even when removing the wifi bottle and linking with Cat5 UTP wire directly to switch, I'm not getting major improvements.
When I got the fiber installed this morning I was disappointed when I saw only marginal gain running at 80 MBit/s (c. +30 MBit). So I decided to connect the laptop via LAN cable directly to modem. I got a starkling 900MBit/s. So, along my network I have bottlenecks.
THe first one I tested was my little pfsense machine. I installed the speedtext-cli command and was surprised to find that it was giving my around 300 MBit/s. So a lot better than my laptop on its usual wifi connection but still only 33% of what I get directly off the modem.
So my first question is how can it be that my little mini J1900 Celeron (2 GHz) with 4 GB RAM cannot handle this bandwith? Do I need an upgrade for my pfsense machine? I noticed that the peak CPU demand as speedtest-cli was running was in the 60% region, far from a saturated CPU and RAM only occupied for about 30%. If it is my little pfsense machine, how far do I have to go with finding the right little machine that can handle 1 GBit/s.
The next question is if I'm getting 300 MBit/s on the WAN connection of the pfSense machine, how is it that I only see a small percentage of this on my laptop? i.e. a drop from 300 MBit/s to 80 MBit/s? I guess I would have to test the switch to start and then move to the wifi access points ...
The process is to go step-by-step. First direct connect to modem you have, bridged connection if possible, and test with multiple bandwidth measurements (speedtest, fast.com, downloading a big file from some university ftp...) and work your way downstream of the network. And on every step test multiple scenarios where it's possible, preferably with multiple devices.
When I got a 1Gbit fiber connection few years back I got an Ubiquiti Edgerouter-X with PoE-options. On paper that should've been plenty for my network, but in theory with NAT, DNAT, firewall rules and things like that it capped on 6-700Mbps depending on what I used it for. With small packets and VPN it dropped even more. So now that thing acts as an glorified PoE switch and the main routing is handled with Mikrotik device, which on manufacturers tests should be able to push 7Gbps on optimal conditions. I only have 1/1Gbps, so there's plenty of room, but with very specific loads that thing still is still pushed to the limit (mostly small packet size with other stuff on top of it) but it can manage the full duplex 1000Base-T. And on normal everyday use it's running at 20% (or so) load, but I like the fact that it can manage even the more challenging scenarios.
Ok, starting to think I need a new little device for my pfSense. I was thinking of going OpenSense and buying one of their devices to support the project.
Regarding my switch, the ports where my Rukus APs are connected are showing 1000M on the interface. But I think a step by step testing is what is needed as suggested above.
They are expensive but I run a OPNsense DEC740 and have no issues with my Gigabit fiber, even without modem and the PPPoE overhead.
You can still try playing with hardware offload on/off and if you use PPPoE, it runs on a single core by default.
Very nice but looks expensive. Do you think I could upload the pfSense configuration to it? I dread the pain of having to configure the whole thing from scratch.
I don't think you can import pfSense configurations into OPNsense. I switched from a DIY pfSense box as well and redid the config.
You can look for a converter or install pfSense onto it though.
Run iperf internally to see if your bottleneck is switch/ap or fw. I set up a j1900 pfsense for my sisters family a while back to do qos (gamer bois in the house) amd it had no problem staying at 500mbps. No ids or other stuff.
Not built any opn/pf-sense in a while, but i always use intel server-nic’s. Used to have way better support than other stuff on bsd