The Brittleness of Data Infrastructure: A Call for a New General-Purpose Model
By
larelli
1mo ago· 17 min readenOpinion
90/100
Golden Brown
Bagelometer↗
If you only eat one bagel today, this is the bagel.
Score90TypeopinionSentimentnegative
Summary
The author reflects on over a decade building data infrastructure at major tech companies (Twitter, Google, Snowflake), identifying a persistent gap between the conceptual elegance of programming languages/databases and the brittle reality of operating real systems. Infrastructure engineers develop a paranoia around change because systems are hard to modify and easy to break. The article argues that fragmented systems are brittle while coherent systems are too special-purpose, calling for a new general-purpose model to build coherent, multi-domain applications.
Key quotes
· 3 pulledAll of the systems I've ever worked on have felt brittle in one way or another: hard to change, and easy to break.
Infrastructure engineers develop paranoia around change.
Fragmented systems are brittle. Coherent systems are special-purpose. We need a new, general-purpose model so we can build coherent, multi-domain applications.
Fragmented systems are brittle. Coherent systems are special-purpose. We need a new, general-purpose model so we can build coherent, multi-domain applications.
