EMBER Features: Sparse Factorizations for Concept Erasure in Language Models

EMBER precomputed features for the EMBedding ERasure method, a plug-and-play module using Sparse Matrix Factorization to erase concept-related features from token embeddings. The dataset provides two factorizations per concept: embedding features and MLP features, derived from the google/gemma-2-2b-it model. It was authored by ClSu and last updated on June 20, 2026.

Use Cases

• Improving the robustness of existing erasure methods based on the provided sparse factorizations.
• Analyzing concept-related features in token embeddings using the EMBER factorization.
• Studying MLP activations for concept representation via the Semi-NMF (SNMF) features.
• Testing plug-and-play modules for precise feature removal in language model embeddings.

Strengths

• Provides precomputed features for a specific concept erasure method (EMBER).
• Includes two distinct factorization types per concept: embedding and MLP features.
• Features are derived from a specific, named model (google/gemma-2-2b-it).
• Dataset was updated on 2026-06-20, suggesting recent maintenance.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count, file formats, and dataset size are unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Precomputed via Sparse Matrix Factorization and Semi-NMF on model activations.

Freshness

Last updated 2026-06-20 12:01:48; freshness should be verified.