Research map / Label-efficient and robust learning / Self-supervised and contrastive
Task-Optimized Convolutional Recurrent Networks Align with Tactile Processing in the Rodent Brain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces an Encoder-Attender-Decoder (EAD) framework to search task-optimized temporal networks trained on simulated rodent whisker force/torque sequences, finding ConvRNN encoders best match mouse barrel cortex responses.
Problem
Tactile processing is poorly understood in neuroscience and weakly performing in artificial systems; there is little computational modeling of rodent somatosensory cortex and no prior check of whether such models match neural population responses.
Approach
An Encoder-Attender-Decoder parameterization that combines convolutional recurrent encoders (IntersectionRNN, Zhuang's ConvRNN), SSM/ResNet encoders, and GPT/Mamba/no attenders, trained supervised or with tactile-specific SimCLR/SimSiam/AE on biomechanically simulated whisker force-torque data, then compared to barrel cortex recordings via RSA.
Key results
- ConvRNN (IntersectionRNN) encoders outperform ResNet18 and S4 encoders on supervised tactile categorization, with the top model being Zhuang+GPT+Supervised.
- Best EAD models reach neural fit saturating the explainable variance and exceed mean inter-animal consistency (0.18 vs. best model ~0.96 for Inter+SimCLR per the appendix table), with a linear trend between categorization and neural fit (r=0.59).
- SimCLR-trained IntersectionRNN EAD achieves the best absolute neural alignment, comparable to supervised models, using tactile-specific augmentations; image augmentations failed to train.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal masking to Inter+SimCLR (ablation) | Inter+SimCLR · ShapeNet tactile categorization | top-5 accuracy: 0.15 -> 0.18 | Helped |
| Add temporal masking to Inter+SimCLR (ablation) | Inter+SimCLR · Rodent barrel cortex neural evaluation | neural fit: 0.96 -> 0.42 | Hurt |
| Add temporal masking to ResNet+Mamba+SimCLR (ablation) | ResNet+Mamba+SimCLR · ShapeNet tactile categorization | top-5 accuracy: 0.23 -> 0.28 | Helped |
| Add temporal masking to ResNet+Mamba+SimCLR (ablation) | ResNet+Mamba+SimCLR · Rodent barrel cortex neural evaluation | neural fit: 0.70 -> 0.67 | Hurt |
| Use tactile-specific force-and-torque augmentations instead of standard image-based augmentations | SimCLR-trained tactile models · ShapeNet tactile categorization | task performance | Helped |
| Use tactile-specific augmentations instead of standard image-based augmentations | SimCLR-trained tactile models · Rodent barrel cortex neural evaluation | neural fit | Helped |
| Use a ConvRNN encoder rather than a feedforward ResNet or S4 encoder | Supervised EAD models · ShapeNet tactile categorization | top-5 accuracy | Helped |
| Use a ConvRNN encoder rather than a feedforward ResNet or S4 encoder | Trained EAD models · Rodent barrel cortex neural evaluation | neural fit | Helped |
| Add a GPT-based attender rather than Mamba or no attention | EAD models · Tactile categorization | task performance | Helped |
| Add a GPT-based attender rather than Mamba or no attention | EAD models · Rodent barrel cortex neural evaluation | neural alignment | Helped |
| Train with contrastive SimCLR rather than supervised categorization | IntersectionRNN EAD · Rodent barrel cortex neural evaluation | neural alignment | No clear effect |
| Use contrastive self-supervision rather than autoencoding | Tactile EAD models · Rodent barrel cortex neural evaluation | neural alignment | Helped |
Surprising findings
- Models trained with standard image augmentations (color jitter, Gaussian blur) failed to train on tactile data.
- The SSL models with best neural fit achieve only moderate categorization performance, suggesting somatosensory representations prioritize general-purpose encoding over categorization.
Limitations
- Neural dataset has only two stimulus classes (convex/concave), limiting captured neural variability and the reliability of the noise ceiling.
- Training uses passive sweeps in simulation while neural evaluation uses active whisking; tactile augmentations are only approximately physically accurate.
- Several SSL/AE and S4 models were unstable or failed to train, limiting the search.
Experimental setup
Datasets: ShapeNet whisking dataset (simulated with WHISKiT Physics) mouse barrel cortex neural dataset (convex/concave discrimination, 999 units, 11 mice)
Benchmarks: ShapeNet whisking top-5 categorization accuracy mouse barrel cortex neural data (RSA)
Metrics: top-5 accuracy noise-corrected RSA Pearson r SVM decoding accuracy Spearman-Brown corrected reliability
Baselines: ResNet18 S4 Mamba SimSiam autoencoding raw sensor input randomly initialized models
Builds on: Zhuang's ConvRNN model WHISKiT S4 Mamba SimCLR
Closest papers by meaning
- AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors (ICLR 2025 · similarity 0.86)
- Toward Artificial Palpation: Representation Learning of Touch on Soft Bodies (NeurIPS 2025 · similarity 0.85)
- Tactile MNIST: Benchmarking Active Tactile Perception (NeurIPS 2026 · similarity 0.85)
- Jointly Modeling Spatio-Temporal Features of Tactile Signals for Action Classification (AAAI 2024 · similarity 0.85)
- Sensor-Invariant Tactile Representation (ICLR 2025 · similarity 0.85)
- EgoTac: In-the-wild Tactile Prediction from Egocentric Vision (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.