Research map / Large language models / Agents, code and math
Measuring the Unmeasurable: Unveiling Latent Cognitive Capabilities of LLM
Cui Danxin, Sihang Jiang, Keyi Wang, Zhiyi Duan, Yanghua Xiao, Bi Yude, Jiaqing Liang, Minggui He, Shimin Tao, Yilun Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable think mode (ablation) | Qwen3-32B · CogEval Rule Execution (English) | accuracy: 0.16 -> 0.40 | Improved |
| enable think mode (ablation) | Qwen3-Plus · CogEval Inductive Reasoning (English) | accuracy: 0.268 -> 0.463 | Improved |
| enable think mode (ablation) | Qwen3-Plus · CogEval Inductive Reasoning (English) | average queries: 5.36 -> 0.32 | Hurt |
| add post-reflection step (ablation) | Qwen3-32B (On) · CogEval Rule Execution (English) | accuracy change: 0.400 (-0.270) | Hurt |
| add post-reflection step (ablation) | Qwen3-Plus (On) · CogEval Inductive Reasoning (English) | accuracy change: 0.463 (-0.169) | Hurt |
| add post-reflection step (ablation) | GPT-4.1 · CogEval operations (English) | accuracy change: Task Identification +0.027; Factual Recall -0.020 | Mixed |
| add post-reflection step (ablation) | Llama3-8B · CogEval Causal Reasoning (English) | accuracy change: 0.746 (-0.390) | Hurt |
| enable think mode (ablation) | Qwen3-Plus · CogEval Factual Recall (English) | accuracy: 0.570 -> 0.834 | Improved |
| enable think mode (ablation) | Qwen3-32B · CogEval Math Computation (English) | accuracy: 0.954 -> 0.954 | No clear effect |
| use Chinese for associative-thinking elicitation instead of English (ablation) | GPT-4.1 · Associative thinking network | clustering/efficiency balance: C 0.107 -> 0.516 at temp 0.8 (E 0.426 -> 0.387) | Mixed |
Closest papers by meaning
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.90)
- Truth Behind the Scene: Designing Evaluations Benchmarks to Assess LLMs’ Task-Specific Understanding over Test-Taking Strategies (AAAI 2025 · similarity 0.90)
- KoLA: Carefully Benchmarking World Knowledge of Large Language Models (ICLR 2024 · similarity 0.89)
- SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines (NeurIPS 2025 · similarity 0.89)
- Benchmarking LLMs via Uncertainty Quantification (NeurIPS 2024 · similarity 0.89)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
