SmolLM2-135M-Instruct Atlas: Sparse Autoencoder Features Across All Layers

552,960 sparse features were extracted from the SmolLM2-135M-Instruct model using 30 custom sparse autoencoders. The atlas covers every layer and component, including MLP, gate, up-projection, and attention heads, with a mean explained variance of 0.9519. It was created by juiceb0xc0de and last updated on June 16, 2026.

Use Cases

• Analyzing feature activation patterns across all 30 model layers.
• Studying the composition of model components like attention heads and MLPs based on sparse features.
• Investigating the properties of zero-dead-feature sparse autoencoders.
• Benchmarking sparse autoencoder performance using the reported mean explained variance metric.

Strengths

• Covers all 30 layers and every major component of the specified model.
• Reports zero dead features at convergence and a mean explained variance of 0.9519.
• Contains 552,960 sparse features for analysis.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and dataset size are unknown, which may limit suitability assessment.
• Freshness should be verified as the last update was June 16, 2026.

Provenance

Source

huggingface

Collection Method

Extracted from the HuggingFaceTB/SmolLM2-135M-Instruct model using custom sparse autoencoders.

Freshness

Last updated 2026-06-16 23:29:19