Research map / 3D vision / Face and shape
GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GPSToken, a tokenizer that represents an image as entropy-guided 2D Gaussian parameterized tokens rendered via splatting, and a two-stage generator that first synthesizes Gaussian layout then texture.
Problem
Grid-based 2D/1D image tokenizers are inflexible in representing regions of varying shape, texture and location, limiting representation efficiency and generation quality.
Approach
Parameterize each token as a 2D Gaussian (mean for position, covariance for shape) plus texture features, initialize Gaussians from an entropy-driven recursive region partition, refine them with a transformer, render them to 2D feature maps by splatting for standard decoders, and factor generation into layout (Gaussian parameters) then texture.
Key results
- GPSToken-M128 (128 tokens) reaches rec. FID 0.65 and PSNR 24.06 on ImageNet 256 reconstruction; GPSToken-L256 reaches rec. FID 0.22 and PSNR 28.81.
- Two-stage GPSToken generator achieves FID 1.50 on ImageNet 256 class-conditional generation with classifier-free guidance, vs. 1.67 for MAETok.
- Two-stage generator reaches FID-10K 25.48 in 100K iterations, about 3x faster than SiT-XL/2 per the paper's comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GPSToken-L256 rather than SDXL-VAE | ImageNet reconstruction at 256×256 · ImageNet val 256×256 reconstruction | PSNR: 25.55 -> 28.81 | Helped |
| use two-stage layout synthesis followed by texture generation rather than one-stage generation | GPSToken generator · ImageNet val 256×256 class-conditional generation | FID: 2.13 -> 1.50 | Helped |
| add spatially-adaptive refinement (ablation) | Gaussian-parameterized GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 1.02 -> 0.81 | Helped |
| add spatially-adaptive initialization to spatially-adaptive refinement (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.81 -> 0.65 | Helped |
| set entropy threshold λ to 0 instead of λ=2.5 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 1.02 | Hurt |
| set entropy threshold λ to 5 instead of λ=2.5 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 0.68 | Hurt |
| reduce Gaussian support factor s from 5 to 1 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 160 | Hurt |
| set Gaussian support factor s to 3 rather than 5 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 0.66 | No clear effect |
| set Gaussian support factor s to 7 rather than 5 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 0.66 | No clear effect |
| increase minimum region size s_min from 4 to 16 (ablation) | GPSToken-M128 · ImageNet val 256×256 reconstruction | rec. FID: 0.65 -> 0.66 | No clear effect |
Surprising findings
- The one-stage generator (127.8M-param tokenizer with joint Gaussian and texture tokens) slightly underperforms the SiT-XL/2 baseline (FID 2.13 vs. 2.06), while the two-stage design gives 1.50.
Limitations
- Heuristic initialization of Gaussian parameters may not give optimal configurations.
- Layout synthesis relies on post-processing calibration rather than a dedicated architecture.
- Decoder FLOPs (2689G) are much higher than some baselines.
Experimental setup
Datasets: ImageNet COCO2017 FFHQ STARE WHU_RS19
Benchmarks: ImageNet 256x256 reconstruction ImageNet 256x256 class-conditional generation COCO FFHQ STARE WHU_RS19
Metrics: PSNR SSIM LPIPS rFID FID sFID
Baselines: SDXL-VAE GaussianToken VQVAE-f16 MaskGIT-VAE VAVAE DCAE SoftVQ TiTok-B64 TiTok-S128 MAETok FlexTok One-D-Piece
Builds on: Gaussian Splatting VQGAN SiT SDXL-VAE RoIAlign
Closest papers by meaning
- An Image is Worth 32 Tokens for Reconstruction and Generation (NeurIPS 2024 · similarity 0.87)
- Image Understanding Makes for A Good Tokenizer for Image Generation (NeurIPS 2024 · similarity 0.84)
- PredToken: Predicting Unknown Tokens and Beyond with Coarse-to-Fine Iterative Decoding (CVPR 2024 · similarity 0.84)
- FlexTok: Resampling Images into 1D Token Sequences of Flexible Length (ICML 2025 · similarity 0.84)
- Scaling Mesh Generation via Compressive Tokenization (CVPR 2025 · similarity 0.83)
- UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.