Research map / Multi-agent systems and games / LLM agents

Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that passive JavaScript-collected UI action and timing traces from browser agents can identify the underlying LLM (14 models, up to 96% F1), studies open-set detection, and tests timing-delay defences.

Problem

Whether a website can passively identify which underlying LLM powers a browsing agent from in-page UI interaction traces, which would enable model-targeted attacks.

Approach

Treats model attribution of LLM browser agents as a supervised classification problem over action and inter-event timing features collected by an injected JavaScript tracker, with closed-set and leave-one-agent-out open-set evaluation and delay-based defence analysis.

Key results

• XGBoost closed-set classifier reaches up to 96.1% F1 for Seed-2.0-Lite on 2WikiMultiHopQA; macro F1 79.36% on 2Wiki across 14 models.
• Open-set leave-one-agent-out AUROC is mostly above 0.60, but Seed-2.0-Lite scores below chance (0.47 on 2Wiki) despite being best closed-set.
• Delay injection substantially degrades unadapted classifiers, but retraining on 5-second-delayed traces largely recovers performance.

Surprising findings

• The best closed-set identifiable model (Seed-2.0-Lite) is the worst on open-set detection, showing closed-set identifiability and open-set identifiability are dissociated.
• Random delay defences do not protect against an adversary who retrains on delayed traces.

Limitations

• Single agent harness (Midscene.js) in pure-vision mode, limiting generality.
• Closed set of 14 models; open-set detection remains imperfect.
• Single-task transfer is weak; pooling tasks on the same site is needed for strong attribution.

Experimental setup

Datasets: 2WikiMultiHopQA FRAMES WebShop DeepShop

Benchmarks: 2WikiMultiHopQA FRAMES WebShop DeepShop

Metrics: F1 macro F1 AUROC

Baselines: Random Forest Logistic Regression Lasso Regression LSTM

Builds on: Midscene.js WebArena Mind2Web XGBoost

Closest papers by meaning

• Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents (NeurIPS 2024 · similarity 0.86)
• Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents (ICLR 2026 · similarity 0.86)
• It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents (ICML 2026 · similarity 0.86)
• From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents (ICML 2026 · similarity 0.85)
• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (NeurIPS 2024 · similarity 0.85)
• How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.85)