AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Knowledge Matters: Importance of Prior Information for Optimization

Çağlar Gülçehre, Yoshua Bengio

JMLR 2016 · Label-efficient and robust learning › Recognition and long-tail · approach: Self-explaining and visual explanation networks

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add intermediate-level hints (two-phase training with patch-wise targets)SMLP · Pentomino (80k train)test error (%): 12.4 -> 0.01Improved
add intermediate-level hintsSMLP · Pentomino (40k train)test error (%): 36.0 -> 3.1Improved
add intermediate-level hintsSMLP · Pentomino (20k train)test error (%): 48.6 -> 30.7Improved
add standardization layer at intermediate level (online SGD) (ablation)SMLP-nohints · Pentomino onlinetest error (%): 50.1 -> 27.5Improved
increase training set size 20k -> 80kSMLP-nohints · Pentominotest error (%): 48.6 -> 12.4Improved
increase training set size 40k to 320kMLP with 3 hidden layers · Pentominotest error (%)No clear effect
pretrain with hints for 1 epoch, then continue without hintsSMLP · Pentomino (40k train)test error (%): 39 -> 0Improved
add unsupervised pretraining (CAE/DAE) before supervised finetuningSMLP-nohints · Pentomino (80k train)test error (%): 12.4 -> 49.7Hurt
add unsupervised RBM pretraining of first layerSMLP-nohints · Pentominotest error (%)No clear effect
use LBFGS optimizerMLP with 2 hidden layers · Pentomino (40k train)test error (%)No clear effect
use larger hidden layer (2050 hidden units in P1NN) with extensive hyperparameter searchSMLP-nohints · Pentomino (80k train)test error (%): 12.4 -> 6.7Improved
use maxout MLP instead of tanh MLPMLP · Pentomino Exp3 one-hot 80 inputstest error (%): 22.475 -> 0.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy