Research map / Large language models / Agents, code and math
ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ALI-Agent, a GPT-4-driven agent framework with memory, tool use, and iterative refinement that automatically generates realistic test scenarios to expose long-tail value misalignment in LLMs across stereotypes, morality, and legality.
Problem
Expert-designed static benchmarks for LLM alignment with human values have limited scope, cannot cover long-tail open-world risks, and cannot keep pace with rapidly evolving LLMs.
Approach
An LLM agent runs two stages: an Emulation stage that generates realistic test scenarios from misconduct seeds using retrieved past evaluation records as in-context demonstrations (evaluation memory), and a Refinement stage that iteratively rewrites scenarios to conceal malice and probe long-tail risks until the target model fails or the iteration budget is reached.
Key results
- ALI-Agent achieves the highest average model agreeability on DecodingTrust (30.00%), CrowS-Pairs (29.80%), ETHICS (36.95%), Social Chemistry 101 (59.20%) and Singapore regulations (42.83%) among the compared evaluation settings.
- On AdvBench, ALI-Agent combined with GPTFuzzer reaches 49.75% average ASR versus 28.77% for GPTFuzzer alone.
- Human evaluators judged over 85% of 200 sampled generated scenarios as realistic and properly encapsulating the misconduct.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · Social Chemistry 101 | model agreeability: 50.82 -> 59.20 | Helped |
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · Singapore Rapid Transit Systems Regulations | model agreeability: 17.32 -> 42.83 | Helped |
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · DecodingTrust | model agreeability: 35.00 -> 30.00 | Hurt |
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · CrowS-Pairs | model agreeability: 39.05 -> 29.80 | Hurt |
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · ETHICS | model agreeability: +3.42 | Helped |
| Use ALI-Agent instead of prevailing evaluation prompts | target LLMs · AdvBench | ASR: 28.77 -> 29.70 | Helped |
| Add a GPTFuzzer universal jailbreak prefix to ALI-Agent test scenarios | ALI-Agent on AdvBench · AdvBench | ASR: 29.70 -> 49.75 | Helped |
| Add evaluation memory (ablation) | ALI-Agent · ETHICS | model agreeability | Helped |
| Add an iterative refiner (ablation) | ALI-Agent · ETHICS | model agreeability | Helped |
| Increase the number of refinement iterations (ablation) | ALI-Agent · AdvBench | ASR | Helped |
Surprising findings
- Within model families, larger models (Vicuna-33B, Llama 2-70B) are not uniformly safer; several show higher misalignment than smaller siblings.
- Vicuna models, fine-tuned from Llama 2, show much higher misalignment rates than their Llama 2 bases.
Limitations
- Depends heavily on the closed-source core LLM (GPT-4-1106-preview), so performance is not controllable.
- Generating scenarios that bypass target guardrails is itself a form of jailbreaking and the core LLM may refuse such requests.
- Evaluation uses small test sets (for example 200 test samples per dataset), and the paper does not report variance.
Experimental setup
Datasets: DecodingTrust CrowS-Pairs ETHICS Social Chemistry 101 Singapore Rapid Transit Systems Regulations AdvBench
Benchmarks: DecodingTrust CrowS-Pairs ETHICS Social Chemistry 101 Singapore Rapid Transit Systems Regulations AdvBench
Metrics: model agreeability attack success rate OpenAI Moderation harmfulness score
Baselines: Zero-shot system prompt Untargeted system prompt Targeted system prompt Evasive Sentences In-Context Attack Jailbreak Prompts GPTFuzzer
Builds on: chain-of-thought SELF-REFINE ReAct GPTFuzzer
Closest papers by meaning
- On Evaluating LLM Alignment by Evaluating LLMs as Judges (NeurIPS 2025 · similarity 0.90)
- Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing (ICML 2025 · similarity 0.90)
- LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.89)
- CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses (NeurIPS 2024 · similarity 0.89)
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · similarity 0.89)
- Aligning LLMs by Predicting Preferences from User Writing Samples (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.