Research map / Architectures and efficiency / RNNs and normalization
Lifting Layers: Analysis and Applications
Peter Ochs, Tim Meinhardt, Laura Leal-Taixe, Michael Moeller
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ReLU with lifting activation | one-hidden-layer FC network · MNIST | best test error (%): 3.07% -> 2.61% | Improved |
| replace maxout with lifting activation at similar parameter count | one-hidden-layer FC network · MNIST | best test error (%): 2.91% -> 2.61% | Improved |
| replace ReLU with lifting layers (L=3) in ME-model+BN | ME-model+BN · CIFAR-10 and CIFAR-100 | test error | Improved |
| add lifting layers instead of enlarging the ReLU model (Large ME-model+BN, more parameters) (ablation) | ME-model+BN · CIFAR-10 and CIFAR-100 | test error | Improved |
| add batch normalization to ME-model (ablation) | ME-model · CIFAR-10 and CIFAR-100 | test error | Improved |
| increase model size (Large ME-model+BN) (ablation) | ME-model+BN · CIFAR-10 and CIFAR-100 | test error | Improved |
| replace ReLU with lifting layers (L=3) in a DnCNN-style denoiser (Lift-46) | DnCNN-S · BSD68 sigma=15 | PSNR (dB): 31.72 -> 31.72 | No clear effect |
| replace ReLU with lifting layers (L=3) in a DnCNN-style denoiser (Lift-46) | DnCNN-S · BSD68 sigma=25 | PSNR (dB): 29.21 -> 29.21 | No clear effect |
| replace ReLU with lifting layers (L=3) in a DnCNN-style denoiser (Lift-46) | DnCNN-S · BSD68 sigma=50 | PSNR (dB): 26.21 -> 26.23 | Improved |
| replace parameterized filtering with lifting of the input for dimensionality increase | Std-Net · 2-D function fitting cos(x2 sin(x1)) | convergence and approximation RMSE | Improved |
| replace ReLU network with lifting-based linear model on 1-D sine fitting | Std-Net · 1-D sine fitting (50 training examples) | convergence | Improved |
| lift a truncated linear loss into a convex problem | regression with 40% outliers · synthetic robust regression | fit quality | Improved |
Closest papers by meaning
- Splat Feature Solver (ICLR 2026 · similarity 0.86)
- Sublabel-Accurate Relaxation of Nonconvex Energies (CVPR 2016 · similarity 0.86)
- Convex Regularization behind Neural Reconstruction (ICLR 2021 · similarity 0.85)
- Preserving Deep Representations in One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework (ICLR 2025 · similarity 0.85)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.85)
- 3D-LFM: Lifting Foundation Model (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
