Bidirectional Semantic-Vision Guidance for Zero-Shot Remote Sensing Image Scene Classification

Zero-shot remote sensing image scene classification aims to recognize unseen categories by leveraging the mapping between vision and semantic information. The robustness of scene attributes is crucial to the semantic transfer of zero-shot learning (ZSL). However, different acquisition conditions cause instance attributes to deviate from category attributes, thereby leading to negative transfer of semantic knowledge from seen to unseen categories. To address this issue, we propose a bidirectional semantic-vision guidance (BSVG) method. In order to enhance the vision related to category attributes, we design the semantic guided frequency decoupling (SGFD) module, which uses image structure and semantic correlation to suppress vision features that are weakly correlated with semantics. Then, we design the vision guided semantics (VGS) module to bridge the gap between instance and category attributes, which employ vision to dynamically reconstruct semantics per instance. This module align category attributes with single instance attributes and enhance the transferability of semantic knowledge. Abundant experiments demonstrate that the proposed method suppresses semantic negative transfer by mutual calibration of vision and semantic, and achieves a classification balance between seen and unseen categories on three public datasets.