AI Paper Atlas

Research map / Image generation / Autoregressive and latent generation

Quality Assured: Rethinking Annotation Strategies in Imaging AI

Tim Rädsch, Annika Reinke, Vivienn Weru, Minu D. Tizabi, Nicholas Heller, Fabian Isensee, Annette Kopp-Schneider, Lena Maier-Hein

ECCV 2024 · Image generation › Autoregressive and latent generation · approach: Learning-based image quality assessment · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use annotation companies instead of MTurk crowdworkersannotation pipeline · HeiCo instance segmentationannotation cost relative to MTurk: 61.1% cheaper in the median; 24.5% to 81.5% of MTurk costsImproved
use annotation companies instead of MTurk crowdworkersannotation pipeline · HeiCo instance segmentationspam annotations: no spam vs MTurk ~20%Improved
use annotation companies instead of MTurk crowdworkersannotation pipeline · HeiCo instance segmentationDSC, NSD and severe errorsImproved
add internal QA by annotation company workers (ablation)annotation companies, minimal text instructions · HeiCo instance segmentationmedian DSCNo clear effect
add internal QA by annotation company workers (ablation)annotation companies, extended text instructions · HeiCo instance segmentationmedian DSC: two companies no change, two below 2% relative improvementNo clear effect
add internal QA by annotation company workers (ablation)Company 2, minimal text instructions · HeiCo instance segmentationfraction of images with severe errors: 0.22 -> 0.28Hurt
add internal QA by annotation company workers (ablation)Company 3, extended text instructions · HeiCo instance segmentationfraction of images with severe errors: 0.25 -> 0.17Improved
add internal QA by annotation company workers (ablation)Company 4, all instruction types · HeiCo instance segmentationfraction of images with severe errors: 0.09 -> 0.09 with extended text + picturesNo clear effect
improve labeling instructions (add extended text and pictures) instead of adding QA (ablation)annotation companies · HeiCo instance segmentationDSC and severe errorsImproved
apply QA to images with underexposure, intersecting or occluded objects (ablation)internal QA · HeiCo instance segmentation, generalizable image characteristicsodds of improvementImproved
apply QA to images with domain-specific characteristics (ablation)internal QA · HeiCo instance segmentation, domain-specific image characteristicsodds of improvementNo clear effect
choose a more expensive or different-region annotation companyinternal QA · HeiCo instance segmentationQA quality improvementNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy