Research map / Large language models / AI and society

Computer Science Conferences Should Require Nonrepudiable Experimental Results

NeurIPS 2026 · Large language models › AI and society · approach: Data commons and human baselines · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing conferences should require tamper-evident, nonrepudiable attestations binding reported experimental numbers to executed computations, illustrated with a Go testbed (K-Veritas) that signs session digests.

Problem

Reported experimental results in CS papers cannot be verified: checklists, artifact evaluation, author-controlled logs, and pre-registration do not bind the reported numbers to an actual executed computation.

Approach

Defines "experiment nonrepudiation" and a "proof-of-compute layer" as a general requirement that results be admitted only with a verifiable, author-inaccessible signed record of the computation; demonstrates feasibility with K-Veritas, which wraps training commands, hashes session data, and gets RSA-PSS signatures from an independent attestation service.

Key results

• K-Veritas captured signed fields from two runs (Keras LSTM and RoBERTa-base on SST-2, RTX 5060 Ti) with HMC scores of 0.80 (flagged ZERO_COST_METRIC) and 0.96 (PASS).
• Per the Olszewski et al. study cited in the paper, only about 18% of ~750 security-conference ML papers produced working, available code.
• The authors state that a reader cannot distinguish fabricated from real result tables by inspection, illustrated with two example tables.

Surprising findings

• Two example tables, one real and one generated, were designed so that a reader cannot tell which is real from inspection alone.
• The paper reports that the introduction of Artifact Evaluation Committees at four top security venues produced no statistically significant improvement in artifact availability or functionality.

Limitations

• Software-only observer cannot defeat OS-level or hardware-level adversaries.
• Security depends on the attestation service and its signing key; compromise enables forgery.
• Protocol only works if conferences mandate or strongly encourage compliance; verification does not check experimental design quality.

Experimental setup

Datasets: SST-2

Metrics: HMC score

Baselines: NeurIPS reproducibility checklist artifact evaluation Weights & Biases MLflow Neptune pre-registration REFORMS

Builds on: in-toto Sigstore OpenReview

Closest papers by meaning

• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents (NeurIPS 2026 · similarity 0.83)
• AI Control: Improving Safety Despite Intentional Subversion (ICML 2024 · similarity 0.81)
• The Unreasonable Effectiveness of Open Science in AI: A Replication Study (AAAI 2025 · similarity 0.81)
• Position: Let’s Strengthen Verifiability if We Can’t Enforce Reproducibility (NeurIPS 2026 · similarity 0.81)
• Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats (ICLR 2025 · similarity 0.80)
• Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming (NeurIPS 2025 · similarity 0.80)