Research map / Architectures and efficiency / RNNs and normalization
scDataset: Scalable Data Loading for Deep Learning on Large-Scale Single-Cell Omics
Davide D'Ascenzo, Sebastiano Cultrera di Montesano
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add block sampling with batched fetching to replace per-cell random sampling | AnnData loading on Tahoe-100M · Tahoe-100M | throughput: ~20 samples/sec -> over 204x higher | Improved |
| increase fetch factor from 1 to 1024 (ablation) | scDataset streaming · Tahoe-100M streaming | throughput: 345 -> 5263 samples/sec (over 15x) | Improved |
| add batched fetching (f=256) with block size 16 (ablation) | scDataset block sampling · Tahoe-100M minibatch entropy | plate label entropy: 1.76 -> 3.61 | Improved |
| increase block size to 64 or more without batched fetching (ablation) | scDataset · Tahoe-100M minibatch entropy | plate label entropy: entropy near zero at b>=64, f=1 | Hurt |
| use sequential streaming instead of random or block shuffling (ablation) | drug classification · Tahoe-100M drug classification, linear | macro F1: 0.030 -> 0.002 | Hurt |
| use block shuffling b=16 f=256 instead of random sampling (ablation) | linear and MLP classifiers · Tahoe-100M classification tasks | macro F1: MoA broad linear 0.270 -> 0.264 | No clear effect |
| add a 16,384-cell shuffle buffer to streaming (ablation) | streaming loader · Tahoe-100M classification tasks | macro F1: MoA broad MLP 0.185 -> 0.201 | No clear effect |
| change sampling strategy on cell line classification (ablation) | linear and MLP classifiers · Tahoe-100M cell line classification | macro F1: linear 0.935-0.937; MLP 0.942 | No clear effect |
| use streaming instead of block shuffling (ablation) | training dynamics · Tahoe-100M training loss | training loss | Hurt |
Closest papers by meaning
- scMBERT: A Pre-Trained Deep Learning Model for Single-Cell Multiomic Data Representation and Prediction (Student Abstract) (AAAI 2025 · similarity 0.84)
- scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data (ICML 2025 · similarity 0.83)
- ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy (ICML 2025 · similarity 0.82)
- A Simple and Comprehensive Benchmark for Single-Cell Transcriptomics (AAAI 2025 · similarity 0.82)
- Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data (NeurIPS 2024 · similarity 0.81)
- scMRDR: A scalable and flexible framework for unpaired single-cell multi-omics data integration (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
