Research map / Learning theory and algorithms / Matrix and tensor methods
Robust and Fast Measure of Information via Low-Rank Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace matrix-based Rényi entropy with low-rank Rényi entropy in the information bottleneck objective | VGG16 on CIFAR10 · CIFAR10 | Accuracy (%): 94.13 $\pm$ 0.04 -> 94.16 $\pm$ 0.09 | Helped |
| Replace matrix-based Rényi entropy with low-rank Rényi entropy in the information bottleneck objective | VGG16 on CIFAR10 · CIFAR10 | IB calculation time (minutes): 46 -> 15 | Helped |
| Replace matrix-based Rényi entropy with low-rank Rényi entropy in the information bottleneck objective | VGG16 on CIFAR10 · CIFAR10 | Total training time (minutes): 126 -> 95 | Helped |
| Use low-rank instead of matrix-based Rényi mutual information for feature selection | LRMI versus MRMI · Feature selection datasets (average) | Average rank: 1.74 -> 1.64 | Helped |
| Use low-rank instead of matrix-based Rényi mutual information for feature selection | LRMI versus MRMI · Breast | Average rank: 2.6 -> 2.6 | No clear effect |
| Use low-rank instead of matrix-based Rényi mutual information for feature selection | LRMI versus MRMI · Statlog | Average rank: 2.0 -> 2.1 | Hurt |
| Use low-rank instead of matrix-based Rényi mutual information for feature selection | LRMI versus MRMI · Semeion, Madelon, Krvskp, Spambase, Waveform, Optdigits | Average rank: Semeion: 1.8 -> 1.4; Madelon: 1.2 -> 1.1; Krvskp: 1.7 -> 1.6; Spambase: 1.5 -> 1.5; Waveform: 1.8 -> 1.5; Optdigits: 1.3 | Mixed |
| Use low-rank instead of matrix-based Rényi mutual information for feature selection | LRMI versus MRMI · Feature selection datasets | Running time: 6 to 27 times speedup; 15 times on average | Helped |
| Use low-rank Rényi entropy rather than matrix-based Rényi entropy under random data-sample perturbations | Information potential under random noise · Synthetic noise perturbation simulation | Variance | Helped |
| Use low-rank Rényi entropy in place of the original matrix-based measure | Low-rank entropy approximation · 8192 × 8192 kernel matrix | Speedup: more than 25 times speedup | Helped |
Closest papers by meaning
- Measuring Dependence with Matrix-based Entropy Functional (AAAI 2021 · similarity 0.88)
- REMEDI: Corrective Transformations for Improved Neural Entropy Estimation (ICML 2024 · similarity 0.85)
- Structural-Entropy-Based Sample Selection for Efficient and Effective Learning (ICLR 2025 · similarity 0.84)
- The Underlying Universal Statistical Structure of Natural Datasets (ICML 2025 · similarity 0.84)
- Cross-Entropy Is All You Need To Invert the Data Generating Process (ICLR 2025 · similarity 0.84)
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.