ts_zip: Text Compression Tool Using Large Language Models Achieves High Compression Ratios
By
everlier
Lightly browned and well buttered. A solid pick from the rack.
Summary
ts_zip is a text compression utility that uses Large Language Models (LLMs) to achieve significantly higher compression ratios than traditional compression tools like xz. The article presents comparative data showing ts_zip achieving compression ratios of 1.142-1.431 bits per byte (bpb) on text files, outperforming xz's 2.551-2.717 bpb. The tool can compress and decompress text files, though the author notes there are some caveats to its use.
Key quotes
· 5 pulledThe ts_zip utility can compress (and hopefully decompress) text files using a Large Language Model.
The compression ratio is much higher than with other compression tools.
The compression ratio is given in bits per byte (bpb).
ts_zip(bytes) (bpb) 21713 1.142
ts_zip(bytes) (bpb) 137477 1.431
You might also wanna read
TurboQuant: Compressing AI Vectors to 2-4 Bits Using Random Rotations
TurboQuant is a novel compression technique for AI vectors (KV caches, embeddings, attention keys) that compresses each coordinate to 2-4 bi
Themery: AI-Powered IDE Color Theme Builder for Developers
Themery is an AI-powered tool that enables developers to create and customize beautiful color themes for their integrated development enviro
Clide: Native macOS Terminal with Integrated AI Assistant and Grid Layout
Clide is a native macOS terminal application that features an AI pair-developer assistant integrated into a side panel. The terminal offers
Codentis: AI-Powered Developer Assistant for Terminal Workflows
Codentis is an AI-powered developer assistant that runs directly in the terminal, featuring a CLI-based agent with real-time streaming, synt
SnapRewrite: macOS AI Writing Assistant for In-Place Text Rewriting
SnapRewrite is a macOS AI writing assistant that allows users to highlight text in any application and rewrite it in place without switching
imgcmd: Secure CLI Tool for Generating PNG Images via Gemini AI
imgcmd is a secure command-line interface tool that generates PNG images directly to disk using Google's Gemini AI. It addresses issues with
