Research map / 3D vision / Human pose estimation
LPSNet: End-to-End Human Pose and Shape Estimation with Lensless Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LPSNet, an end-to-end framework that estimates 3D human pose and shape directly from lensless measurements using a multi-scale lensless feature decoder and double-head auxiliary supervision.
Problem
Estimating 3D human pose and shape from lensless imaging measurements, which are ambiguous and lack effective direct estimation methods; the prior route of reconstructing RGB images first yields poor-quality images and inaccurate poses.
Approach
An HRNet-inspired multi-scale lensless feature decoder (MSFDecoder) decodes optically encoded measurements into multi-scale features, which feed a PyMAF-style iterative SMPL regressor; a double-head auxiliary supervision (SimCC keypoint heatmaps plus DensePose IUV prediction) is applied only during training to improve limb accuracy.
Key results
- LPSNet achieves MPJPE 119.20 mm, PA-MPJPE 81.52 mm and PVE 134.74 mm on LenslessHuman3.6M, versus 126.28 / 81.37 / 151.60 for the fine-tuned two-stage baseline PyMAF†.
- Removing both MSFDecoder and DHAS raises MPJPE from 119.20 to 142.13 mm in the ablation.
- Double-head auxiliary supervision lowers MPJPE from 129.70 (IUV only) and 123.30 (2D keypoint only) to 119.20 mm when both heads are used.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LPSNet instead of fine-tuned PyMAF | PyMAF on lensless measurements · LenslessHuman3.6M | MPJPE: 126.28 -> 119.20 | Helped |
| use LPSNet instead of fine-tuned PyMAF | PyMAF on lensless measurements · LenslessHuman3.6M | PVE: 151.60 -> 134.74 | Helped |
| use LPSNet instead of fine-tuned PyMAF | PyMAF on lensless measurements · LenslessHuman3.6M | PA-MPJPE: 81.37 -> 81.52 | No clear effect |
| add MSFDecoder instead of convolutional and deconvolutional layers (ablation) | LPSNet without DHAS · LenslessHuman3.6M | MPJPE: 142.13 -> 139.34 | Helped |
| add Double-Head Auxiliary Supervision (ablation) | LPSNet · LenslessHuman3.6M | MPJPE: 139.34 -> 119.20 | Helped |
| add the 2D keypoint auxiliary supervision head (ablation) | LPSNet · LenslessHuman3.6M | MPJPE: 123.30 -> 119.20 | Helped |
| add the IUV auxiliary supervision head (ablation) | LPSNet · LenslessHuman3.6M | MPJPE: 129.70 -> 119.20 | Helped |
Surprising findings
- The PA-MPJPE of LPSNet is slightly worse than the fine-tuned baseline (81.52 vs 81.37 mm), which the authors attribute to PA-MPJPE not fully reflecting mesh-image alignment.
Limitations
- Results are not as robust as RGB-based HPS methods.
- Scarcity of lensless datasets prevents pre-training MSFDecoder on large corpora such as ImageNet.
- Most training data are lensless measurements simulated or captured by screen display rather than real in-the-wild captures.
Experimental setup
Datasets: Human3.6M MPII COCO 3DPW MIP-INF-3DHP LSP
Benchmarks: LenslessHuman3.6M
Metrics: MPJPE PA-MPJPE PVE
Baselines: PyMAF Rego et al. lensless reconstruction followed by PyMAF
Builds on: PyMAF HRNet SimCC DensePose HMR
Closest papers by meaning
- Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms (NeurIPS 2022 · similarity 0.84)
- Capturing the Motion of Every Joint: 3D Human Pose and Shape Estimation with Independent Tokens (ICLR 2023 · similarity 0.84)
- End-to-End Recovery of Human Shape and Pose (CVPR 2018 · similarity 0.83)
- Optimizing Human Pose Estimation Through Focused Human and Joint Regions (AAAI 2025 · similarity 0.82)
- BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation (CVPR 2025 · similarity 0.82)
- FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.