Research map / Vision-language and multimodal / Multimodal LLMs

MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MORALISE, a benchmark of 2,481 expert-verified real-world image-text pairs spanning 13 moral topics, and evaluates 19 open and proprietary VLMs on moral judgment and moral norm attribution.

Problem

Moral alignment of vision-language models is underexplored; existing moral benchmarks are text-only or rely on AI-generated images, lacking real-image, morally diverse multimodal evaluation.

Approach

A taxonomy of 13 moral topics grounded in Turiel's Domain Theory, real-image data annotated with topic and modality (image- vs text-centric) labels, and two tasks (moral judgment and norm attribution at single- and multi-label levels).

Key results

• Proprietary models average 88.28 moral judgment accuracy versus 83.55 for open-source models, but only 66.60 and 42.63 hit rate on single-norm attribution.
• GPT-4o reaches only 42.32 multi-norm attribution F1 on respect.
• Textual cues yield higher accuracy and lower variance than image-only cues across all three subtasks.

Surprising findings

• Scaling beyond roughly 10B parameters yields little further moral-alignment gain.
• Models from the same family trained on different corpora (Qwen2 vs Qwen2.5) show weak output correlation.

Limitations

• Evaluation is limited to zero-shot inference with greedy decoding and a single prompt template per task.
• Image collection from web sources with graduate-student annotators; no inter-annotator agreement reported in the text.
• Moral taxonomy is Western-centric and fixed to 13 topics.

Experimental setup

Datasets: MORALISE

Benchmarks: MORALISE

Metrics: accuracy hit rate F1

Baselines: M3oralBench MoralBench ETHICS VIVA

Builds on: Turiel's Domain Theory ETHICS M3oralBench

Closest papers by meaning

• Tracing Moral Foundations in Large Language Models (NeurIPS 2026 · similarity 0.89)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.86)
• Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models (NeurIPS 2025 · similarity 0.86)
• Language Model Alignment in Multilingual Trolley Problems (ICLR 2025 · similarity 0.86)
• ProgressGym: Alignment with a Millennium of Moral Progress (NeurIPS 2024 · similarity 0.85)
• Words or Vision: Do Vision-Language Models Have Blind Faith in Text? (CVPR 2025 · similarity 0.85)