AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

UNISON: Unpaired Cross-Lingual Image Captioning

Jiahui Gao, Yi Zhou, Philip L. H. Yu, Shafiq Joty, Jiuxiang Gu

AAAI 2022 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use UNISON instead of Graph-Align followed by Google translationChinese image captioning without a caption corpus · COCO-CN testBLEU-1: 39.2 -> 44.9Improved
Use UNISON instead of FC-2k trained on pseudo Chinese captionsChinese image captioning with a caption corpus · COCO-CN testBLEU-1: 60.4 -> 63.4Improved
Map scene-graph nodes in encoding space rather than symbolically translate nodes (ablation)Cross-lingual graph mapping · MT corpus testBLEU-1: 26.6 -> 28.1Improved
Add subgraph-level contextual mapping to word-level mapping (ablation)GM_word · MT corpus testBLEU-1: 28.1 -> 29.2Improved
Add full-graph mapping to word- and subgraph-level mapping (ablation)GM_word+sub · MT corpus testBLEU-1: 29.2 -> 29.6Improved
Replace concatenation and an FC layer with self-gated adaptive fusion (ablation)HGM_base · MT corpus testBLEU-1: 29.6 -> 30.4Improved
Add KL joint training to align source- and target-language scene-graph features (ablation)HGM · MT corpus testBLEU-1: -0.5Improved
Add cross-modal feature mapping (ablation)HGM · COCO-CN testBLEU-1: 41.1 -> 44.9Improved
Add cross-modal feature mapping (ablation)GM_word · COCO-CN testBLEU-1: 40.1 -> 43.1Improved
Add cross-modal feature mapping (ablation)GM_word+sub · COCO-CN testBLEU-1: 37.3 -> 40.6Improved
Add cross-modal feature mapping (ablation)HGM_base · COCO-CN testBLEU-1: 38.0 -> 39.8Improved
Use contextual graph-level mapping without adaptive self-gating on noisy image scene graphs (ablation)GM_word+sub and HGM_base · COCO-CN testBLEU-1: GM_word 40.1; GM_word+sub 37.3; HGM_base 38.0Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy