AI Paper Atlas

Research map / Video / Video-language understanding

SODA: Story Oriented Dense Video Captioning Evaluation Framework

Soichiro Fujita, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, Masaaki Nagata

ECCV 2020 · Video › Video-language understanding · approach: Dense video captioning · 58 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase number of evaluated captions from m=1.0 to allCurrent ActivityNet scorer (E2E Transformer) · ActivityNet Captions valscore (%): 4.10 -> 4.19No clear effect
increase number of evaluated captions from m=1.0 to allCurrent ActivityNet scorer (LSTM) · ActivityNet Captions valscore (%): 4.92 -> 4.97No clear effect
increase number of evaluated captions from m=1.0 to allSODA (c) (E2E Transformer) · ActivityNet Captions valF1 (%): 4.02 -> 0.63Hurt
increase number of evaluated captions from m=1.0 to allSODA (c) (LSTM) · ActivityNet Captions valF1 (%): 3.15 -> 0.71Hurt
reduce number of captions from m=1.0 to m=0.1SODA (c) (E2E Transformer) · ActivityNet Captions valF1 (%): 4.02 -> 1.47Hurt
use F-measure with tau 0 (b) instead of averaging over tau (a) (ablation)SODA E2E Transformer · ActivityNet Captions val, m=1.0F1 (%): 3.86 -> 7.55Improved
add IoU-weighted METEOR cost (SODA c) instead of METEOR-only cost (SODA b) (ablation)SODA LSTM · ActivityNet Captions val, m=1.0F1 (%): 6.07 -> 3.15No clear effect
shuffle caption order instead of correct orderSODA (c) (E2E Transformer) · oracle captionsscore (%): 17.8 -> 7.66Hurt
shuffle caption order instead of correct orderCurrent ActivityNet scorer (E2E Transformer) · oracle captionsscore (%): 16.1 -> 10.8Hurt
swap two adjacent captions instead of correct orderSODA (c) (LSTM) · oracle captionsscore (%): 10.7 -> 8.89Hurt
swap two adjacent captions instead of correct orderCurrent ActivityNet scorer (LSTM) · oracle captionsscore (%): 10.5 -> 9.92Hurt
use SODA (c) instead of current scorer for agreement with human ordering judgmentSODA (c) · manual evaluation, Swap vs Shuffleaccuracy: 0.72 -> 0.94Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy