All Topics
All Topics
Technology
Technology
Design
Design
Programming
Programming
Science
Science
News
News
Gaming
Gaming
Entertainment
Entertainment
Business
Business
Finance
Finance
Sports
Sports
Health
Health
Food
Food
Travel
Travel
Art
Art
Music
Music
Books
Books
Education
Education
Politics
Politics
Personal
Personal
No algorithm. No AI slop. No ads. Just RSS. Pro-human. Indie writers. Real journalism. Open web. Chronological. Hand toasted.

Docker's Image Pulling Mechanism Is Inefficient for Small Changes

By

a_t48

2mo ago· 1 min readenOpinion

Summary

The article criticizes Docker's current image pulling mechanism, which requires downloading entire layers even when only small changes are made. It highlights the inefficiency of this approach, particularly for users with many dependencies or those working in bandwidth-constrained environments like remote robots or slow connections. The author argues that adding a single byte to one layer invalidates all subsequent layers, forcing unnecessary downloads of gigabytes of unchanged files.

Key quotes

· 5 pulled
Adding a single byte to a single layer invalidates it and every following layer, even if every other file in those layers is byte-for-byte identical.
For anyone with many dependencies (looking at you, ROS!), it could mean gigabytes of unrelated files dragged along with your small change.
And you'd better hope you're not pulling on a robot out on the farm, or a device at a client's warehouse at dial-up speed.
We can do better.
Adding apt install vim shouldn't cause every other dependency to get pull
Snippet from the RSS feed
See how much bandwidth content-aware pulling saves over standard docker pull.

You might also wanna read

Building Deptool: A custom declarative deployment tool for personal infrastructure

Ruud van Asseldonk describes building Deptool, a custom declarative configuration deployment tool he created to address frustrations with ex

ruuda.nl·25d ago

GitHub's Implementation of eBPF for Deployment Safety and Circular Dependency Prevention

GitHub uses eBPF (Extended Berkeley Packet Filter) technology to solve circular dependency challenges in their deployment process. Since Git

GitHub·1mo ago

Stripe's Canonical Log Line Pattern: Consolidating Logging into Single Structured Events

The article discusses Stripe's approach to logging called 'canonical log lines' or 'wide events,' which involves emitting one structured rec

blog.alcazarsec.com·2mo ago

Three Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role

A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio

jamiehurst.co.uk·15h ago

Three Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role

A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio

jamiehurst.co.uk·15h ago

Bijou64: A variable-length integer encoding that's both correct and accidentally fast

This article describes the development of bijou64, a variable-length integer (varint) encoding created for the Subduction CRDT sync protocol

inkandswitch.com·1d ago