AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation

Michael Chang, Thomas L. Griffiths, Sergey Levine

NeurIPS 2022 · Learning theory and algorithms › Prediction and decision losses · approach: Theory of neural network sensitivity · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train slot attention with implicit differentiation rather than backpropagating through unrolled iterationsSLATE on CLEVR-Mirror 64x64 · CLEVR-Mirror 64x64cross entropy: around 130 -> 10x lower cross entropy lossImproved
Use implicit differentiation for slot attentionSLATE · CLEVR testFID: 25.89 -> 22.19Improved
Use implicit differentiation for slot attentionSLATE · CLEVR testMSE: 67.04 -> 10.66Improved
Use implicit differentiation for slot attentionSLATE · COCO testFID: 147.48 -> 127.79Improved
Use implicit differentiation for slot attentionSLATE · COCO testMSE: 1821.75 -> 1659.15Improved
Use implicit differentiation for slot attentionSLATE · ShapeStacks testFID: 34.76 -> 34.2Improved
Use implicit differentiation for slot attentionSLATE · ShapeStacks testMSE: 312.14 -> 108.67Improved
Use the IB configuration (iteration forward solver with Broyden gradient estimate) (ablation)Slot attention · CLEVR-Mirror 64x64optimization stabilityHurt
Replace the seven-iteration vanilla model with one iteration (ablation)Vanilla slot attention · CLEVRreconstruction MSE: 2x worse reconstruction MSE than implicit slot attention with seven iterationsHurt
Remove gradient clipping (ablation)Vanilla SLATE · Trainingtraining stabilityHurt
Remove gradient clipping (ablation)Implicit SLATE · Trainingtraining stabilityNo clear effect
Remove learning-rate warmup (ablation)Vanilla SLATE · Trainingtraining stabilityHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy