Theoretical Constraints of Embedding-Based Retrieval: Orthogonality Limits in Vector Spaces
By
sonabinu
An everything bagel for the brain. Substantive, layered, well-seasoned.
Summary
This article discusses the theoretical limitations of embedding-based retrieval systems, focusing on the mathematical constraints of vector representations in high-dimensional spaces. It explains the concept of strict orthogonality in n-dimensional Euclidean space where vectors must have exact 90-degree angles (dot product of 0), limiting the maximum number of orthogonal vectors to n. The article then explores approximate orthogonality where vectors can have angles close to 90 degrees (e.g., 89-91 degrees), which allows for more vectors while maintaining useful mathematical properties for retrieval systems.
Key quotes
· 4 pulled在 n 维欧几里得空间 R^n 中，一个向量组如果两两严格正交（夹角精确为90°，或点积为0），那么这个向量组的大小（基数）最多为 n
如果我们放宽这个条件，只要求向量之间的夹角"接近"90°，例如在你说的89°到91°之间
这构成了该空间的一组正交基
要求任意两个不同的单位向量 v...
You might also wanna read
A visual introduction to differential geometry and Maxwell's equations through pictures
This article presents a pictorial introduction to differential geometry, aimed at making the mathematical foundation accessible to pre-unive
arxiv.org·1d ago
Mathematical Model Identifies the Optimal Threshold for Human Ambition
A collaborative mathematical study reconciled conflicting pieces of cultural advice by mapping the exact parameters of human ambition. Using
neurosciencenews.com·1d ago
Weak and Block-Equitable Colourings in Uniform Group Divisible Designs and Maximum Packings
This article presents a mathematical study of colourings in uniform group divisible designs and maximum packings. It defines weak c-colourin
doi.org·4d agoVC Dimension and the Fundamental Theorem of Statistical Learning: A Complete Mathematical Derivation
This article explains the theoretical foundations of statistical learning theory, specifically addressing when learning from data is guarant
A Good Lemma is Worth a Thousand Theorems: Doron Zeilberger on Mathematical Impact
Doron Zeilberger's 82nd opinion piece argues that good lemmas are more valuable than theorems, using Szemerédi's Regularity Lemma as his pri
Collection of 939 Two-Dimensional Mathematical Curves
A collection of 939 two-dimensional mathematical curves is presented, organized alphabetically by name for easy browsing and discovery.