Research map / Time series / Foundation models for time series
UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniRepLKNet, a large-kernel ConvNet built with four architectural guidelines and a Dilated Reparam Block, and applies it unchanged (after modality-specific embedding) to image, audio, video, point cloud, and time-series recognition.
Problem
Architectural design of large-kernel ConvNets has been under-explored (they mostly copy ConvNeXt or Swin designs), and it is unclear whether ConvNets have universal perception ability across modalities beyond vision.
Approach
Decouple the roles of convolution layers: use large kernels only in middle/high stages to get a large effective receptive field ("see wide without going deep"), add SE blocks and 3x3 convs for depth, and re-parameterize large kernels with parallel dilated small-kernel branches (Dilated Reparam Block) that merge into a single conv at inference. Non-image modalities are converted to 3D embedding map
Key results
- UniRepLKNet-XL (ImageNet-22K pretrained) reaches 88.0% ImageNet-1K top-1, 55.6 ADE20K MS mIoU, and 56.4 COCO box AP.
- On global temperature and wind speed forecasting, UniRepLKNet-S achieves MSE 7.602 and MAE 1.832 for temperature, beating Corrformer (7.709 / 1.888).
- UniRepLKNet-S reaches 98.5% on Speech Commands V2 without pretraining, above AST (96.2%) and Audio-MAE (98.3%); it reaches 54.8% on Kinetics-400 and 93.2% OA on ModelNet-40.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add an SE Block with 1/4 channel reduction (ablation) | UniRepLKNet block design · ImageNet-1K and ADE20K | ADE20K mIoU: 45.1 -> 46.5 | Helped |
| Replace the SE Block with a bottleneck (ablation) | UniRepLKNet block design · ADE20K | mIoU: 46.5 -> 46.3 | Hurt |
| Replace the SE Block with two 1×1 layers (ablation) | UniRepLKNet block design · ADE20K | mIoU: 46.5 -> 46.2 | Hurt |
| Replace the SE Block with two DW 3×3 layers (ablation) | UniRepLKNet block design · ADE20K | mIoU: 46.5 -> 45.4 | Hurt |
| Use parallel non-dilated kernels with the same kernel sizes (ablation) | 13×13 Dilated Reparam Block · ImageNet-1K | accuracy: 81.63±0.02 -> 81.55±0.01 | Hurt |
| Use parallel non-dilated kernels with the same equivalent kernel sizes (ablation) | 13×13 Dilated Reparam Block · ImageNet-1K | accuracy: 81.63±0.02 -> 81.59±0.02 | Hurt |
| Reduce large-kernel size from 13 to 11 in Stages 2–4 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 45.5 | Hurt |
| Use 3×3 kernels instead of 13×13 in Stage 2 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 46.1 | Hurt |
| Use 3×3 kernels instead of 13×13 in Stage 3 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 45.9 | Hurt |
| Use 3×3 kernels instead of 13×13 in Stage 4 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 45.8 | Hurt |
| Increase kernel size from 13 to 15 in Stages 2–4 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 45.9 | Hurt |
| Use 13×13 kernels in Stage 1 (ablation) | UniRepLKNet vanilla model · ADE20K | mIoU: 46.5 -> 44.9 | Hurt |
Surprising findings
- Using large kernels in early stages (Stage 1) or enlarging K from 13 to 15 barely changes ImageNet accuracy but lowers ADE20K mIoU under the full UPerNet, which the authors attribute to UPerNet's structural priors rather than poorer features.
- Scaling depth with additional large-kernel blocks gives little benefit over adding small-kernel blocks, so the added blocks should use 3x3 kernels.
Limitations
- Dilated reparam branches require more training resources.
- Vision-language, cross-attention, and generative applications are left under-explored.
- Video result (54.8% on Kinetics-400) trails specialist SOTA models such as MViT, and no pretraining was used there.
Experimental setup
Datasets: ImageNet-1K ImageNet-22K ADE20K COCO Speech Commands V2 Kinetics-400 ModelNet-40 NCEI Global Temperature and Wind Speed
Benchmarks: ImageNet-1K ADE20K COCO Global Temperature and Wind Speed Forecasting Speech Commands V2 Kinetics-400 ModelNet-40
Metrics: top-1 accuracy mIoU box AP mask AP MSE MAE throughput (img/s) overall accuracy mean accuracy
Baselines: RepLKNet-31B SLaK-T ConvNeXt V2 InternImage Swin Transformer FastViT Corrformer Pyraformer AST Audio-MAE Meta-Transformer ImageBind
Builds on: RepLKNet ConvNeXt Structural Re-parameterization SE Block Corrformer
Closest papers by meaning
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.87)
- Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness (ICML 2024 · similarity 0.87)
- LSNet: See Large, Focus Small (CVPR 2025 · similarity 0.87)
- PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution (CVPR 2024 · similarity 0.86)
- CAM Back Again: Large Kernel CNNs from a Weakly Supervised Object Localization Perspective (CVPR 2024 · similarity 0.86)
- Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.