Docker's Image Pulling Mechanism Is Inefficient for Small Changes
By
a_t48
Leave it on the tray for the seagulls.
Summary
The article criticizes Docker's current image pulling mechanism, which requires downloading entire layers even when only small changes are made. It highlights the inefficiency of this approach, particularly for users with many dependencies or those working in bandwidth-constrained environments like remote robots or slow connections. The author argues that adding a single byte to one layer invalidates all subsequent layers, forcing unnecessary downloads of gigabytes of unchanged files.
Key quotes
· 5 pulledAdding a single byte to a single layer invalidates it and every following layer, even if every other file in those layers is byte-for-byte identical.
For anyone with many dependencies (looking at you, ROS!), it could mean gigabytes of unrelated files dragged along with your small change.
And you'd better hope you're not pulling on a robot out on the farm, or a device at a client's warehouse at dial-up speed.
We can do better.
Adding apt install vim shouldn't cause every other dependency to get pull
You might also wanna read
Building Deptool: A custom declarative deployment tool for personal infrastructure
Ruud van Asseldonk describes building Deptool, a custom declarative configuration deployment tool he created to address frustrations with ex

GitHub's Implementation of eBPF for Deployment Safety and Circular Dependency Prevention
GitHub uses eBPF (Extended Berkeley Packet Filter) technology to solve circular dependency challenges in their deployment process. Since Git
Stripe's Canonical Log Line Pattern: Consolidating Logging into Single Structured Events
The article discusses Stripe's approach to logging called 'canonical log lines' or 'wide events,' which involves emitting one structured rec
Three Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role
A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio
Three Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role
A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio
Bijou64: A variable-length integer encoding that's both correct and accidentally fast
This article describes the development of bijou64, a variable-length integer (varint) encoding created for the Subduction CRDT sync protocol
