The Lost Art of System Administration

Sep 8, 2022

I spent a good chunk of my adolescence learning Linux systems administration. I attribute some of my success as a software engineer to the scripting and systems knowledge I picked up while installing, maintaining, and tinkering with Linux distros.

Now, one rarely needs to know things like that – runtimes are "serverless," and ssh is disappearing. Macbooks are the developer machine. Init systems and daemons have been replaced by single-process containers. General-purpose operating systems have been replaced with small (e.g., alpine) or even smaller ones (e.g., microkernels). We rarely manage our own headless bare metal (I ditched my home server years ago).

This is the natural progression of things. Here are a few things I learned from tinkering over the years.

  • Learning how to effectively use the command line. Unix philosophy. Pipes, scripting, and tooling. So much of programming is stitching things together.
  • Running and observing databases, proxies, and other self-hosted software. Sometimes it's quicker to learn something when you can watch it crash and burn up close.
  • Operating system primitives – filesystems, kernels, bootloaders, display managers, partitions. Dual booting or setting up a minimal distro like Arch Linux teaches you a lot about what the operating system is doing. Installing (and debugging) a display manager.
  • Debugging. Lots of things go wrong. It's up to you to fix it.

What is the "systems administration" for developers today? Maybe it's playing with models like Stable Diffusion. Or maybe it's running services in the cloud.