Pictures

It is really fun. I like having enough stuff around to tinker. It's easy to forget how hard it is to do anything without so much entrenchment and tools, then go to an ordinary house and try to solve some computer problem, and it feels like living on a desert island. Computers don't work that well in general.

However, it's noisy. That may kill the whole thing soon. I don't know.

Future things we ought to work on:

• virtual storage. We should have some big disk array that has, if not actual backups, at least snapshots and failed-disk-protection. All the machines should take LUN's out of the big array. The big array should have disk encryption, and should require manual keying if it loses power. I believe pretty much every site should be built around something like this.
• power saving. I don't know if it's using $100/mo of electricity or $400/mo of electricity, but either way I think spending a lot of money and doing a lot of work could give us the same capability in 1/5 the energy budget. Each person gets his own box, which is some piece of junk he found abandoned in a hallway or bought on eBay for $100. The old computers don't exactly guzzle power, but they are so slow they could be combined into VM's on one fast computer that uses less power than ten slow ones. or we could buy low-power boards like Via C7 (space.net does this. too bad their prices are so insulting.).
• proper UPS with a bypass switch. Our UPS goes out more often than ConEd. For it to actually help rather than harm overall reliability, we should have a bypass switch so we can maintain the UPS without shutting down all the computers. As is, when I changed the batteries I soldered one DC lead at a time with an ungrounded butane iron and try not to touch anything metal and grounded while I'm doing it. We should also have a way to load-test the UPS with a space heater or something before throwing the bypass switch back to the normal position. And maybe we should have two UPS's for machines that have two power supplies. We do have already two separate clean power circuits wired back to the breaker panel.
• better Internet connection. T1 is slow and expensive. There doesn't seem to be anything cheaper for the same amount of upstream bandwidth, though. We are thinking of getting a worse Internet connection soon (half as much upstream bandwidth).

---

Ladder

It's actually a really bad idea to put computers near the ceiling, we've found. A layer of hot air accumulates up there. It does keep them out of the way of clumsy people, but I'd like to move them now, and it would cause like a one day outage to do that which we can't easily afford.

---

Typical maintenance

That lamp needs to be moved.

---

Blower

This is the blower part of a window aircon (maybe an in-wall aircon actually, like in cheap motels) from the East Village. We got one with the orange CFC's-emptied sticker on it, threw it in a cab to bring home, and then ripped out the compressor and copper coils. It sucks air through that tube. There's another weaker one inside Lauren's room.

---

Duct

duct to Lauren's room.

---

Switch

this Extreme Alpine 3808 is the new main switch that almost all the network stuff plugs into. There are also some ExtremeWare 4.x switches in the living room that I would like to remove because they are so noisy, but it'll probably never happen until I leave. (should I leave?) This switch does OSPF which is usually an expensive capability worth bragging about.

It has lots of fiber gigabit ports, because in the used market these are cheaper to get than copper ports (the NIC's are cheaper, too).

the main reason for buying it was to support some iSCSI stuff I am doing with terabithia and fishsticks. for the next generation music and movie storage thing, /arrchive.

---

ezln, kadmirra, antioch, castrovalva, fishsticks

To the left is ezln, a Digital 'avanti' (PWS 433au) which is a NetBSD PF firewall, doing HFSC with some PF/ALTQ patches which after a couple years I think are still not integrated into NetBSD. I will replace it with some FreeBSD/sparc64 box as soon as I can---so sick of this NetBSD shit (but NetBSD 5.0 is going to be awesome. no, I'm not joking. It will be! but it still won't have device polling, so for the router, FreeBSD it is, forever.). On top of ezln is a C1720, the T1 router. I had to buy more memory for it so it could run an IPv6 IOS image.

The fishsticks are the grey things on the extreme right. They are full of about seven hard disks each. At least one hard disk in each fishstick is an iSCSI target used by terabithia for mirrored ZFS. The fishsticks netboot from terabithia using a custom 'genkernel' script, and Etherboot burned into a FLASH ROM that I bought on eBay and added to the Compaq NC6136 (i82543) gigabit NIC's on which I've standardized here. (there is no Intel PXE image for these NIC's, and they don't ship with FLASH.)

We built them by getting a cheap tower case marketed for loading up with seven CD-ROM's (SCSI lets you have seven, remember?). Then we put seven PATA disks in the old fishstick, seven SATA disks in the new fishstick. I'm surprised they still make cases for this niche, but it turns out to be the perfect cheap way to mount disks, in my opinion, and give them proper ventillation. Each disk goes into a 3.5"-to-5.25" bracket and has a vented metal faceplate in front of it. In one of them we dremmeled out the top of the case and added a filtered, high-flow (~$10) fan from Allied. We never got around to adding a fan to the other one, so we leave it without filters, and it gets a bit dustier. They're running Gentoo Linux, which I hate, but BSD doesn't have any filesystems I feel are appropriate for large volume sizes. Each disk has a separate filesystem on it, no RAID. This no-RAID is a carefully-thought-out decision which I will defend pretty vigorously. It's not an unconditional preference, but in short we can't afford to spend what it takes to do it safely.

In the pizzabox tower there's castrovalva, another shell server and a hackint.org irc server. castrovalva is a Digital DS10 running an old version of NetBSD with horrible pthread bugs. There's antioch, another shell server, running I think NetBSD on a SPARCstation 10. ``shell'' means web, mail, DNS, 'screen' sessions, a working C compiler, things like that from the old Internet.

Above them unplugged are a Netra X1 I want to prepare and send to Germany for hosting, and a Cisco MC3810 that was supposed to do VoIP FXS for us so we could have an Australian phone number months ago, but I fail.

---

sakima, foster, eltoro

foster, just to the left of ezln, is a fairly modern AMD PeeCee running Gentoo Linux.

On top of foster is eltoro, a small desktop PeeCee running FreeBSD.

You can see part of sakima on the left.

---

sakima, ariel, setient's box

On the bottom of this second pizzabox pile is setient's box. I don't know the name of setient's box. I think it's a SPARCstation 4 running Solaris 8 or 9.

ariel is on top of that. ariel should replace ezln soon, and consume less electricity.

have a look at sakima, the 1U machine with no cover. It's power supply is bad, so it's running off the one in the foreground. something you can only do at home, not in a proper data center. I think it's fun.

sakima is running CentOS. sakima is one of the oldest machines on the Internet.

---

Coiled wire

I cannot terminate fiber, so there are a lot of humiliating coils everywhere.

---

Fiber drop

abuse of MMF. that's part of a chopstick lashed against the fiber.

---

Nylon string

The wiring system is based on high-tension nylon string supporting CAT5 or MMF or RG58 at lower tension.

---

terabithia

terabithia will replace castrovalva if i manage to finish it.

look closely, and you can see a tiny USB key fob disk plugged into it. This is because of a rather prickly problem with SVM and the quorum rules of metadb ``state database replicas''. Does this sound complicated? The rule is, ``you must have more than half of them available.'' And they give you two disks. so it's impossible.

---

Downlink

downlink to the rest of the house.

Here is what the shelf used to look like.