Why are PeeCee Unix routers doomed?
It's fun to use PeeCee's running Linux or BSD as routers. I did it
for many years, and still do it for specific things. It can save a
lot of money on the low end---router/firewall vendors tend to cripple
their devices and then gouge you for basic feature sets and tiny
capacity increases. With PeeCee's, a flat $1500 buys everyone
mid-range capacity, provided you have time to get the features you
need working properly, which you probably don't.
This list is somewhat redundant to the
L3 switch comparison table.
Release engineering. The zeroth reason they're
doomed: exotic features are usually broken because there's no decent
release process for a BSD router with PF, ipsec-tools racoon, Quagga,
Asterisk, some obscure Sangoma T1 card, and a Hifn IPsec accelerator.
You'll never get all that stuff working again. If you do, upgrade BSD
to the next BSD formal release, and it'll take a year to find
everything that broke, including weird performance regressions, and
another year to fix it. And there's no unified config file to do
change management all in one place or to save your work across
Low forwarding capacity. I expect brand new fancy
PeeCees can forward not more than about 5Gbit/s, and that's with big
packets. My old Alpine 3808 can, according to other people's tests,
forward small packets in and out of its twenty-four gigabit
High jitter. Since everything's process-switched in a
PeeCee, a packet being forwarded has to contend with all the other
complex software running on the PeeCee's control plane. Sometimes
it'll get stuck behind some other computation.
High forwarding delay. I think even the best case
forwarding delay is pretty long, although I'm not sure this honestly
matters. If I were to test, I'd try iSCSI or seek-heavy NFS (small
Low port density. With old L3 switches costing less
than $100 including shipping, I have 48 100Mbit/s ports, and I can
assign them to separate VLAN's on 48 different subnets if I want to.
My PeeCee router with fancy quad-port Ethernet cards (which seem to be
quite slow) only has eight ports.
No line-rate QoS. We would like at least RED for TCP
flows. Two or three levels of priority might be nice, too, or maybe
even WRED. These are basic features needed by high-performance
ECN might be nice, too---I'm not yet sold on its value, but the idea
sure has been around a long time to still not have it.
The Ethernet MAC chips used in PeeCee's already have priority levels,
although drivers don't seem to do anything with them (Solaris seems to
be looking into it). I don't think they have RED, but it's not such a
demanding feature---the problem is that chip designers often don't
understand the features they're implementing, but if they learn to
give two or three specific knobs to the driver, I think it can't be
done wrongly. The MAC chips are already calculating IP checksums
(``offloading'' features: of IP, TCP, UDP checksums and TCP
segmentation are already in the Unix drivers of a couple gigabit
chips), so asking them to do ECN is not crazy---they could calculate
the IP checksum the instant before transmission, in case the mark
BUT, none of this is happening yet! And L3 switches all do
at least RED and priority queueing if not more.
All that said, PeeCee's are still much cheaper, so
eBay is the real reason why I am changing to L3 switches at this
moment. Here are the pieces of my Extreme Alpine 3808 and their
costs, including shipping:
includes 4 gigabit MTRJ and 4 1000BaseT and 64 100Mbit/s.
4 1000BaseX GBIC slots
GM-4Xi x 2|
4 1000BaseX GBIC slots each
L3 switches / map / carton's page / Miles Nordin <carton@Ivy.NET>
Last update (UTC timezone): $Id: switch-unixrouterdoomed.html,v 1.1 2008/01/11 03:31:01 carton Exp $