SICK: Deduplicated Binary Storage Format for JSON with Efficient Binary Aggregate (EBA)
By
pshirshov
Sesame, salt, and substance. A flagship bake.
Summary
SICK (Streams of Independent Constant Keys) is a representation of JSON-like structures, and this repository provides EBA (Efficient Binary Aggregate) - a deduplicated binary storage format for JSON based on SICK. The technology enables storing JSON-like data in efficient indexed binary form, allowing access to nested data without deserializing entire structures, avoiding reading whole JSON files, and implementing lazy loading. The project includes implementations for Scala, C#, and JavaScript, with UEBA as a sister project for tagless binary encoding.
Key quotes
· 5 pulledSICK: Streams of Independent Constant Keys
SICK is a representation of JSON-like structures
Efficient Binary Aggregate (EBA) - a deduplicated binary storage format for JSON based on the SICK representation
Store JSON-like data in efficient indexed binary form - Access nested data without deserializing the entire structure
Avoid reading whole JSON files - Access only the data you need with lazy loading
You might also wanna read
RX: Embedded Data Store for JSON with In-Place Querying and Automatic Deduplication
RX is an embedded data store for JSON-shaped data that enables querying encoded documents in place without parsing. It functions as a 'no-SQ
github.com·2mo ago
Why Average LLM Use Is Likely Destroying Value in Software Development
The author argues that, contrary to prevailing hype, the average use of Large Language Models (LLMs) is likely destroying value rather than
unessays.substack.com·16h ago
How AI Accelerated Prototyping: From Idea to Tangible in Record Time
The author reflects on how AI has transformed their prototyping workflow. Previously, the biggest bottleneck was the time needed to scaffold
darylcecile.net·16h ago
GitLab 19.0 launches with Secrets Manager, agentic workflows, and self-hosted AI models
GitLab 19.0 has been released, positioning itself as an intelligent orchestration platform for DevSecOps. The release includes expanded secr
bit.ly·1d ago
Centralizing Error Handling in Rust with Custom AppError Enums
This article discusses the importance of centralizing error handling in Rust applications using a custom AppError enum combined with map_err
Zig Devlog: Build System Rework Separates Maker and Configurer Processes
This devlog entry from the Zig programming language project announces a major rework of the build system, separating the maker process from
ziglang.org·1d ago