How avoiding SQL migrations in tests sped up a Go test suite by 2x
The bagel they save for the regulars. Don't skim, savour.
Summary
The article discusses a performance optimization technique for test suites that require pristine databases. The author explains how running SQL migrations at the start of each test is extremely CPU-intensive, especially when thousands of migrations have accumulated over time. By restructuring tests to avoid re-running migrations for every test case, the author achieved a 2x speedup in the test suite. The approach involves setting up the database schema once and then resetting only the data between tests, rather than reapplying all migrations from scratch.
Key quotes
· 3 pulledWe are spending a ton of CPU time just applying again and again the same SQL migrations at the start of each test.
Thousands (!) of SQL migrations have accumulated over the years, and I had to fix a performance issue where we spent a lot of time simply gathering all migration
The test coverage is great, but it means that it's not that fast to run, and it only will get slower over time as new tests are added.
You might also wanna read
Lessons from a Rushed Scala 2.13 to Scala 3 Migration: Performance Challenges in Data Ingestion Services
The article discusses a developer's experience migrating a service from Scala 2.13 to Scala 3, focusing on performance challenges encountere
Common SQL Anti-Patterns That Impact Performance and Maintainability
This article discusses common SQL anti-patterns that negatively impact query performance and maintainability in enterprise software environm
Testing PostgreSQL Race Conditions Using Synchronization Barriers
This technical article discusses race conditions in PostgreSQL database systems and introduces synchronization barriers as a testing methodo
Managing Test Fixture Dependencies in Large Software Projects
The article discusses the problem of 'frozen test fixtures' in large software projects, where test fixtures become difficult to modify becau

Go to Rust Migration: A Backend-Focused Guide on Correctness, Runtime Tradeoffs, and Developer Ergonomics
This article discusses the considerations and tradeoffs involved in migrating backend services from Go to Rust. It highlights that unlike ot

Go to Rust Migration: A Backend-Focused Guide on Correctness, Runtime Tradeoffs, and Developer Ergonomics
This article discusses the considerations and tradeoffs involved in migrating backend services from Go to Rust. It highlights that unlike ot
Understanding a Subtle Bug in Go's errgroup and the Importance of Testing
The article discusses a subtle bug encountered while working with Go's errgroup in a project called Kratos. The author shares their experien
