ClickHouse Rebuilds Full-Text Search with Native Columnar Integration
By
samaysharma
Baker's choice. Dense with flavour, light on filler.
Summary
ClickHouse has completely rebuilt its full-text search (FTS) capability from scratch, moving beyond its older implementation to deliver significantly improved performance, space efficiency, and deeper integration with its columnar architecture. The article provides a technical deep dive into the new data structures including inverted indexes, finite state transducers, and posting lists, explaining how the redesigned query pipeline reduces I/O and improves search speed. It covers the core technical implementation details and offers insights into how developers can leverage the new FTS capabilities.
Key quotes
· 3 pulledFull-text search (FTS) isn't new to ClickHouse, but until now it ran on an older implementation with limits in performance and flexibility
We've gone back to the drawing board and re-engineered it from scratch, making search faster, more space-efficient, and deeply integrated with ClickHouse's columnar design
This is your under-the-hood tour, showing the core data structures — inverted indexes, finite state transducers, posting lists — and how the query pipeline was redesigned to cut I/O
You might also wanna read
Why You Should Never Disable Asserts in Production
The article argues that disabling asserts in production is a harmful practice, contrary to common belief. It explores how asserts serve as c
A critique of misusing "backpressure" in AI code-generation system design
This article critiques Lucas Costa's piece on building systems for code-generating AI robots, arguing that Costa misuses the term "backpress
entropicthoughts.com·14h ago
Three Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role
A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio
jamiehurst.co.uk·1d agoThree Years In: A Senior Engineer's Reflection on AI's Impact on the Software Development Role
A senior engineer reflects on the long-term sustainability of AI tools in software development, three years into deep organizational adoptio
jamiehurst.co.uk·1d ago
Bijou64: A variable-length integer encoding that's both correct and accidentally fast
This article describes the development of bijou64, a variable-length integer (varint) encoding created for the Subduction CRDT sync protocol
inkandswitch.com·1d agoBijou64: A variable-length integer encoding that's both correct and accidentally fast
This article describes the development of bijou64, a variable-length integer (varint) encoding created for the Subduction CRDT sync protocol
inkandswitch.com·1d ago