Research map / Large language models / Alignment and preferences

SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine

AAAI 2025 · Large language models › Alignment and preferences · approach: Reward model design and preference models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

SYNAPSE learns user-specific visual preference programs from a few robot demonstrations plus natural-language explanations, combining a neuro-symbolic DSL, GPT-4 program sketch synthesis, and MaxSMT parameter fitting.

Problem

Learning subjective, user-specific visual preferences (e.g., good pull-over or drop-off locations) from very few person-specific demonstrations, where factual concept learners and RL preference models are data-hungry or population-level.

Approach

Represent preferences as neuro-symbolic programs in a DSL over visual concepts; use an LLM to ground NL explanations into predicates and CNF formulas, recursively learn auxiliary predicates from user demos, and fill numeric parameters of an LLM-generated sketch by MaxSMT over demonstrations.

Key results

• On out-of-distribution test, SYNAPSE reaches 74.07 mIoU vs 57.42 for the closest baseline on CONTINGENCY, 80.72 vs 63.99 on DROPOFF, and 62.76 vs 52.91 on PARKING.
• SYNAPSE uses 29 demonstrations for CONTINGENCY, versus 224 images for neural baselines.
• On tabletop rearrangement with 10 users, SYNAPSE outperforms the closest LLM-GROP baseline by an average of 2.7 points on the 1-10 user rating scale.

Measured effects reported in the paper

Surprising findings

• Neural baselines fine-tuned on the same 29 samples generalize poorly, and GPT-4V performs near chance on these preference tasks.
• Sketch synthesis via CNF extraction and then sketch repair outperforms direct one-step LLM synthesis.

Limitations

• Relies heavily on quality of underlying neural perception modules and prompts; sensitive to parameter choices.
• Assumes demonstrations are reasonably clean; noise handled only by MaxSMT, not a probabilistic model.
• Does not model preferences that change over time.

Experimental setup

Datasets: custom 815-image UT Austin mobility preference dataset LLM-GROP tabletop arrangement tasks

Benchmarks: custom 815-image mobility preference dataset (CONTINGENCY, DROPOFF, PARKING) LLM-GROP tabletop object rearrangement

Metrics: mean IoU user ratings (1-10)

Baselines: SegFormer (SF-RGB-b0/b5, SF-RGBD) DinoV2-b DinoV2-g NS-CL VisProg GPT-4V LLM-GROP LATP GROP TPRA

Builds on: Code-as-Policies NS-CL VisProg LLM-GROP

Closest papers by meaning

• What Matters to You? Towards Visual Representation Alignment for Robot Learning (ICLR 2024 · similarity 0.89)
• LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation (NeurIPS 2026 · similarity 0.88)
• Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Model Using Implicit Feedback from Pre-training Demonstrations (ICLR 2025 · similarity 0.87)
• Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences (CVPR 2024 · similarity 0.86)
• Preference Learning for AI Alignment: a Causal Perspective (ICML 2025 · similarity 0.86)
• AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model (ICLR 2024 · similarity 0.86)