Memory Maps (mmap) Achieve 25x Faster File Access in Go Applications
By
ingve
A bagel you'd recommend to a friend without hedging.
Summary
The article discusses how memory maps (mmap) can deliver significantly faster file access in Go applications, with benchmarks showing up to 25x performance improvements over traditional file reading methods. It explains the technical implementation differences between mmap (memory-mapped files) and ReaderAt (pread) approaches, highlighting how mmap reduces latency by mapping files directly into memory. The author shares practical experience from developing an HTTP-backed filesystem for AI Storage Acceleration solutions, demonstrating real-world applications in high-performance computing environments.
Key quotes
· 3 pulledTo show what kind of performance you can get using memory maps, I've written a little Go library that allows you to read from a file using a memory map or a ReaderAt.
ReaderAt will do a pread(), which is a seek/read combo, while mmap will just read from the memory map.
The last couple of weeks I've been working on an HTTP-backed filesystem. This is part of our AI Storage Acceleration solution, geared towards high performance computing environments.
You might also wanna read
Restartable Sequences: A Linux Kernel Feature for Lock-Free Thread-Safe Programming
This article explores restartable sequences (rseq), a Linux kernel feature introduced in version 4.18 (circa 2018) that enables creation of
justine.lol·17h ago
Zig Days: Collaborative Full-Day Programming Events for the Zig Community
Zig Days are full-day collaborative programming events organized by the Zig community, typically held on Saturdays. Participants meet in the
Spectre Programming Language Documentation: A Contract-Based Approach to Safe Low-Level Systems Programming
Spectre is a contract-based programming language designed for safe low-level systems programming. It enforces correctness through type-level
A Grounded Conceptual Model for Ownership Types in Rust Programming
This article presents a grounded conceptual model for ownership types in Rust, the programming language known for memory safety without garb
cacm.acm.org·1mo ago
Rail: A Self-Hosting Systems Programming Language with Pure-Rail TLS 1.3 Implementation
Rail is a self-hosting systems programming language that compiles itself without any C dependencies. The compiler is written in Rail and pro
github.com·1mo ago
Solod: A Go Subset That Compiles to C for Systems Programming
Solod is a strict subset of the Go programming language that compiles to regular C code, designed for systems programming with manual memory
github.com·1mo ago