AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

KAKURENBO: Adaptively Hiding Samples in Deep Neural Network Training

Thao Nguyen Truong, Balazs Gerofi, Edgar Josafat Martinez-Noriega, François Trahay, Mohamed Wahib

NeurIPS 2023 · Label-efficient and robust learning › Recognition and long-tail · approach: Loss design and classifier theory · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase max hidden fraction from 0.2 to 0.4 (ablation)KAKURENBO · ImageNet-1K ResNet-50 (B)top-1 accuracy: 76.11 -> 75.62Hurt
increase max hidden fraction from 0.2 to 0.4 (ablation)KAKURENBO · ImageNet-1K ResNet-50 (B)training time (sec): 61723 -> 57582Improved
remove move-back (v1110 -> v1010) (ablation)KAKURENBO · ImageNet-1K ResNet-50 (A)top-1 accuracy: 72.81 -> 72.96Improved
add reducing fraction (RF) (v1100 -> v1110) (ablation)KAKURENBO · ImageNet-1K ResNet-50 (A)top-1 accuracy: +0.59%Improved
add learning rate adjustment (ablation)KAKURENBO · ImageNet-1K ResNet-50 (A)top-1 accuracy: 0.46% to 0.83%Improved
use all strategies vs hiding examples only (ablation)KAKURENBO · ImageNet-1K ResNet-50 (A)top-1 accuracy: 73.6 vs baseline 73.68Improved
increase prediction confidence threshold tau (ablation)KAKURENBO · ImageNet-1Kaccuracy / speedupMixed
hide samples randomly instead of by loss (ablation)Random hiding · CIFAR-100 WRN-28-10top-1 accuracy: 77.20 -> 76.82Hurt
increase global batch size from 1024 to 8192 (ablation)KAKURENBO-0.4 · ImageNet-1K ResNet-50 (A)top-1 accuracy: 73.60 -> 72.84Hurt
use KAKURENBO instead of Selective BackpropKAKURENBO · ImageNet-1Ktop-1 accuracy: 3.5% degradation for SB vs +0.2 for KAKURENBOImproved
use ISWR instead of baselineISWR · ImageNet-1Ktraining time: 41% overheadHurt
use FORGET on WideResNetFORGET · CIFAR-100training time: 46.1% increaseHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy