AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

C$^{2}$R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders

Haoran Jin, Xiting Wang, Shijie Ren, Hong Xie, Defu Lian

ICML 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Prototype and concept dictionary models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add C2R regularization to Batch TopK SAEBatch TopK SAE · Gemma-2-2B layer 12, SAEBenchfeature absorption and splittingImproved
add C2R regularization to Batch TopK SAEBatch TopK SAE · Gemma-2-2B layer 12, SAEBenchreconstruction fidelity (loss recovered, KL score)No clear effect
add C2R regularization to Batch TopK SAEBatch TopK SAE · Gemma-2-2B layer 12, AutoInterpAutoInterp scoreNo clear effect
add C2R regularization to Batch TopK SAEBatch TopK SAE · RAVEL disentanglementdisentanglement scoreNo clear effect
add C2R regularization to Batch TopK SAEBatch TopK SAE · Gemma-2-2B layer 12feature composition (max cosine similarity)Improved
use Matryoshka SAE instead of C2R-regularized Batch TopKSAE architectures · Gemma-2-2B layer 12reconstruction fidelityHurt
remove nearest-neighbor restriction (penalize all latent pairs) (ablation)C2R · Gemma-2-2B layer 12, k=100reconstruction fidelity and compositionHurt
remove ReLU cosine gate (ablation)C2R · Gemma-2-2B layer 12, k=100reconstruction fidelity and compositionHurt
increase lambda_C2R from 0.1 up to 5 (ablation)C2R · Gemma-2-2B layer 12, k=100composition and absorptionImproved
increase lambda_C2R to 10 (ablation)C2R · Gemma-2-2B layer 12, k=100reconstruction fidelityHurt
apply C2R on top of TopK, OrtSAE and AbsTopK backbonesSAE backbones · Gemma-2-2B layer 12, k=100composition and absorptionImproved
apply C2R to larger models (Qwen3-8B, Llama-3-8B layer 20)Batch TopK SAE · Qwen3-8B and Llama-3-8B, k=100composition and absorptionImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy