Cross-seed explainability using Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoders
arXiv:2607.08499v1 Announce Type: new Abstract: We present a Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoder (SAE) for extracting cross-seed universal features from independently…
Read the full articleYou might also wanna read
Anthropic researchers extract interpretable features from Claude 3 Sonnet using sparse autoencoders
We demonstrate that sparse autoencoders can extract interpretable features from Claude 3 Sonnet, a production-scale language model, addressi
arxiv.org·1mo agoSemantic-Based Distributed Learning for Diverse and Discriminative Representations
In large-scale distributed scenarios, increasingly complex tasks demand more intelligent collaboration across networks, requiring the joint
Fast-dLLM: Training-Free Acceleration Method for Diffusion Language Models Using KV Cache and Parallel Decoding
Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation with parallel decoding capa
arxiv.org·8mo ago
NSA: A Hardware-Aligned and Natively Trainable Sparse Attention Mechanism for Efficient Long-Context Modeling
Long-context modeling is crucial for next-generation language models, yet the high computational cost of standard attention mechanisms poses
arxiv.org·11mo ago
Study Reveals Convergent Evolution in How Language Models Learn Number Representations
Language models trained on natural text learn to represent numbers using periodic features with dominant periods at $T=2, 5, 10$. In this pa
arxiv.org·2mo agoUnsupervised Algorithm for Language Model Fine-Tuning Introduced
To steer pretrained language models for downstream tasks, today's post-training paradigm relies on humans to specify desired behaviors. Howe
arxiv.org·1y ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.