The Step-Size Trap: How Numerical Differentiation Fails with Floating-Point Arithmetic
By
davedx
Pure flour-power. Hearty enough to carry you through lunch.
Summary
The article explores the mathematical and computational challenges of numerical differentiation, specifically the step-size trap in finite difference approximations. It explains how using floating-point arithmetic to compute derivatives by evaluating functions at nearby points works well with moderate step sizes but breaks down when step sizes become too small due to rounding errors and machine precision limitations. The content uses the example of sin(x) at x=1 to illustrate how derivative approximations improve then deteriorate as step size decreases, highlighting fundamental issues in numerical analysis and computational mathematics.
Key quotes
· 5 pulledThat's not a metaphor. That's literally what happens when you compute a derivative with floating point arithmetic
Pick a small h. Evaluate twice. Divide. It worked. Then I made h smaller and it worked better. Then I made h way smaller and it got worse.
You can measure a curve with a ruler. Lay it across two points, read the slope. Zoom in and the ruler shrinks — better approximation. Keep zooming and your hands start shaking.
At some point you're measuring the shake, not the curve.
The step-size trap
You might also wanna read
Think Linear Algebra: A Code-First, Case-Based Introduction to Linear Algebra with Python
Think Linear Algebra is a code-first, case-based educational book that teaches linear algebra through real-world problems and Python program
Sostactic: Lean4 Tactics for Proving Polynomial Inequalities with Sum-of-Squares Certificates
Sostactic is a Lean4 mathematical proof assistant tool that provides tactics for proving polynomial inequalities using sum-of-squares (SOS)
github.com·1mo ago
Understanding Lagrange Interpolating Polynomials for Data Fitting
This article provides an educational explanation of Lagrange interpolating polynomials, a mathematical method for finding polynomial functio
eli.thegreenplace.net·3mo ago
Simulating the Ladybug Clock Puzzle: A Computational Approach to a Mathematical Problem
The article discusses a mathematical puzzle about a ladybug moving around a clock face, visiting all hour markers exactly once, and asks for
Using Quantifier Elimination to Solve Mathematical Competition Problems
The article discusses using quantifier elimination, a powerful computational technique from mathematical logic, to solve competition math pr
grossack.site·5mo ago
Mathlib4: The Mathematical Library for Lean 4 Theorem Prover
Mathlib4 is the mathematical library for the Lean 4 theorem prover, containing programming infrastructure, mathematical content, and develop
github.com·5mo ago