Research map / Architectures and efficiency / RNNs and normalization
Hardware and Software Platform Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes hardware and software platform inference (HSPI), which identifies the GPU type and software stack serving a black-box model from its input-output behavior, via border inputs (HSPI-BI) and logit-distribution classifiers (HSPI-LD).
Problem
Buyers of LLM inference cannot verify which hardware (e.g., H100) or software stack serves their requests, and providers may silently substitute cheaper models or hardware.
Approach
Exploit numerical differences across GPU architectures, arithmetic formats, and compilers/kernels: adversarially crafted border inputs (PGD variant) that flip predictions differently across platforms, and SVM classifiers over the bit-level representation of output logits from random-word prompts.
Key results
- White-box HSPI-LD distinguishes real GPUs (A100, H100, L40, A40, MI300X, Inferentia, etc.) with 83.9% to 100% accuracy, versus 5.6% random guess on an 18-class LLM task.
- Quantized LLMs (FP16, BF16, INT8-FD, HQQ-4bit) are distinguished with average accuracy of 0.995 in the white-box setting.
- Black-box transfer of HSPI-LD across LLMs reaches 22.7% to 60.3% accuracy versus 25% random, and black-box vision transfer F1 reaches up to 0.677 vs 0.144 random.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HSPI-LD to classify quantization levels on white-box LLMs | LLaMA-3.1-8B, QWen-2.5-3B, and Phi-3.5-mini · Quantized LLMs | accuracy: 0.999; 1; 0.995 | Helped |
| Use HSPI-LD to infer the GPU, dtype, and serving configuration | SGLang cloud GPU service · SGLang cloud GPUs, white-box | accuracy: 0.991 | Helped |
| Use HSPI-LD on real GPU systems | Llama-3.1-70B · Real GPUs, white-box | accuracy: 0.973 | Helped |
| Use HSPI-LD to identify hardware from black-box logits of unseen LLMs | Quantized LLMs · Black-box transfer | average accuracy: between 22.7% and 60.3% (random guess accuracy is 25%) | Mixed |
| Use HSPI-BI to create border inputs for LLMs | HSPI-BI · LLMs | stability | Hurt |
| Use HSPI-LD instead of HSPI-BI for LLM inference | LLMs · White-box LLMs | memory use | Helped |
| Train HSPI-BI border inputs on MobileNet-v2 rather than ResNet18 or VGG16 | HSPI-BI · CIFAR10, cross-model transfer | average F1-score: 0.345 | Helped |
| Increase the batch size used to train HSPI-BI border inputs (ablation) | HSPI-BI · Vision models, cross-model transfer | transferability | Helped |
| Train HSPI-BI border inputs for too many iterations (ablation) | HSPI-BI · Vision models, cross-model transfer | transferability: 800 and 1200 iterations | Hurt |
| Train HSPI-LD using ResNet18 rather than other vision models | HSPI-LD · Quantized vision models, cross-model transfer | average F1-score: 0.677 | Helped |
| Merge labels for the same GPU across software configurations (ablation) | HSPI-LD · White-box GPU classification | accuracy: 0.985 | Helped |
| Change CUDA versions while keeping hardware and the rest of the software stack fixed (ablation) | HSPI-BI on ResNet18 · CIFAR10, RTX8000 | distinguishability | No clear effect |
Surprising findings
- HSPI-BI border inputs fail to distinguish RTX8000 from RTX2080Ti, and HSPI-LD likewise fails between these two GPUs, which share compute capability 7.5.
- Black-box transferability for LLMs is poor for some configurations (e.g., f3 test set accuracy 0.227 below random 25%).
Limitations
- HSPI-BI is unstable and memory-hungry for LLMs, so it is unsuitable for them.
- Some configurations are indistinguishable (same EQC), e.g., different CUDA versions and RTX8000 vs RTX2080Ti.
- Batch size strongly affects logits, requiring separate analysis per batch size; black-box transfer is limited.
Experimental setup
Datasets: CIFAR10 NLTK popular word corpora
Benchmarks: CIFAR10
Metrics: accuracy F1-score
Baselines: Schlögl et al. boundary samples
Builds on: Schlögl et al. border inputs / EQC analysis projected gradient descent
Closest papers by meaning
- 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11) (NeurIPS 2025 · similarity 0.85)
- Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs (ICML 2025 · similarity 0.84)
- Practical Offloading for Fine-Tuning LLM on Commodity GPU via Learned Sparse Projectors (AAAI 2025 · similarity 0.84)
- A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints (ICML 2026 · similarity 0.83)
- Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding (NeurIPS 2025 · similarity 0.83)
- Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.