Research map / Detection and segmentation / Open-set and few-shot detection
A Hybrid Approach for Visual Recognition of Object States
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A doctoral-consortium style summary of the author's PhD research on zero-shot object-agnostic state classification (ZS-OASC) via a hybrid of CNN/GNN neural modules and knowledge graphs, plus a new Object States Dataset and state change anticipation in videos.
Problem
Recognizing object states in images and videos, especially in the zero-shot setting where no state-specific visual training data and no object-class cues are available.
Approach
Combine neural modules (CNNs and GNNs) with symbolic knowledge graphs to project semantic knowledge into the visual space, generating visual embeddings for unseen state classes; later extended with LLM-generated semantic features and LLM-assisted KG construction.
Key results
- Hybrid CNN/GNN plus knowledge graph method reportedly achieves state-of-the-art performance across all available object states datasets (no numbers given in text).
- Object States Dataset publicly released as a new resource for object state classification.
- LLM-based semantic features and LLM-assisted KG refinement reportedly further improve ZS-OASC performance (no numbers given).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate CNNs and GNNs with knowledge graphs to project semantic knowledge into visual space | Hybrid ZS-OASC method · ZS-OASC, available object state datasets | performance | Helped |
| Combine a domain-specific corpus generated with tailored LLM queries with general-purpose web corpora | Hybrid ZS-OASC method · ZS-OASC | performance | Helped |
| Use LLMs to construct and refine knowledge graphs | Hybrid ZS-OASC method · ZS-OASC | performance | Helped |
| Train GNNs with noisy domain-agnostic knowledge graphs | Hybrid ZS-OASC method · ZS-OASC | performance | Hurt |
Surprising findings
- Object state classification cannot be effectively addressed with standard methods used for object classification, according to the author's analysis.
Limitations
- Noise in domain-agnostic knowledge graphs used to train GNNs adversely affects performance.
- Results and experimental details are deferred to cited prior works; this summary gives no quantitative results.
Experimental setup
Datasets: Object States Dataset
Benchmarks: Object States Dataset
Closest papers by meaning
- General Object Foundation Model for Images and Videos at Scale (CVPR 2024 · similarity 0.84)
- Representation-Level Counterfactual Calibration for Debiased Zero-Shot Recognition (NeurIPS 2025 · similarity 0.83)
- Boosting Segment Anything Model Towards Open-Vocabulary Learning (AAAI 2025 · similarity 0.82)
- Learning Object State Changes in Videos: An Open-World Perspective (CVPR 2024 · similarity 0.82)
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.82)
- The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.