AI Paper Atlas

Research map / Large language models / Agents, code and math

Evaluation beyond Task Performance: Analyzing Concepts in AlphaZero in Hex

Charles Lovering, Jessica Zosa Forde, George Konidaris, Ellie Pavlick, Michael Littman

NeurIPS 2022 · Large language models › Agents, code and math · approach: Inductive and abstract reasoning evaluation · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use MCTS to select moves before the policy network has learned the conceptsAlphaZero in Hex · Hex positive-concept behavioral testsbehavioral test pass rateImproved
Train the policy network to match MCTS logitsAlphaZero in Hex · Hex concept representationconcept probing accuracyImproved
Use upper network layers for short-term concepts rather than long-term conceptsAlphaZero in Hex · Hex short-term conceptsprobe accuracyImproved
Use middle network layers for long-term concepts rather than short-term conceptsAlphaZero in Hex · Hex long-term conceptsprobe accuracyImproved
Train AlphaZero to recognize and use positive conceptsAlphaZero in Hex · Hex positive-concept behavioral testspass rateImproved
Train AlphaZero to recognize negative concepts such as dead and captured cellsAlphaZero in Hex · Hex negative-concept behavioral testspass rateMixed
Allow AlphaZero to choose moves without a preference for faster winsAlphaZero in Hex · Hex end-game playmoves that do not impact the outcome: 25%Hurt
Connect the defender in behavioral-test boards to force the conceptually correct moveAlphaZero in Hex · Hex positive-concept behavioral testspass rateImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy