Confessions of an Internal Tool Builder

Aug 6, 2021

Engineering is an iceberg. A small amount of business logic sits above a towering but hidden iceberg of infrastructure. High leverage software engineers spend a non-trivial amount of time building internal dashboards and admin panels. I know because even though I was making open-source developer tools, I wasn't immune from working on internal tools.

And here's the catch – sometimes I even enjoyed it.

Beowulf slaying the dragon (1908). At Google, I managed a small Beowulf cluster of physical Windows, macOS, and Linux machines to run nested virtualization integration tests for minikube.

Every company has internal tools. Customer support applications, approval queues, or data entry forms. Business users need them. Developers don't have time for them. Developers who work on internal tools can have trouble showing quantifiable impact (despite how vital these tools are).

These tools show manifest in two ways: custom-built solutions and spreadsheets.

Developers begrudgingly prefer custom-built solutions. Internal applications nearly always touch internal data sources or integrate with third-party services. Moreover, engineering organizations need to own all the code that runs inside a company — any siloed scripts or jobs are liabilities. It's like if people outside the sales team were writing enterprise contracts.

Technical debt and future maintenance make the net present cost of building internal tools significantly higher than it looks. The worst part about custom-built solutions is that code compounds, and usually not in a good way.

A new category of software tools aims to make developing these tools more accessible. The best ones are rightly focused on catering to developers. But most are built on leaky abstractions and fail to understand just how deep the iceberg goes.

The other option is spreadsheets. Every business user universally understands spreadsheets (enough). The reactive programming paradigm (cells update automatically) and basic programming language of functions are expressive for complex logic. Yet, they fall short when it comes to reporting and automatically accessing the internal data and services that are core to internal tools.

Spreadsheets are just enough code to be dangerous, and that's a good and bad thing. Good, in that business users are programmers, and they don't even realize it. Bad, in that the code and logic in the spreadsheets live outside the engineering organization.

Surely there's a happier path than taking on serious technical debt or siloing logic in spreadsheets.

Internal applications are the first step in the journey. Software wants to be simple. Internal applications are usually easier, but not fundamentally different, than most software. Ray Kurzweil wrote that The Singularity is Near, but instead of artificial intelligence, the singularity that we're nearing is something a bit more mundane yet still exciting. What happens when we have software platforms that are easy and expressive enough so that anyone can build what's in their mind?