论文平台收录多模态多模态融合高光谱影像激光雷达傅里叶变换曼巴状态空间模型注意力机制

Multimodal Remote Sensing Image Classification via Hierarchical Frequency-Enhanced Mamba Fusion Network

基于层次频率增强曼巴融合网络的多模态遥感影像分类

Ping Li; Qiya Song; Jianle Peng; Linhui Ma; Renwei Dian; Tao Yang
Journal / Publication
IEEE Transactions on Geoscience and Remote Sensing
Published
时间暂未填写

论文摘要

Hyperspectral image (HSI) and light detection and ranging (LiDAR) data provide complementary characteristics that improve the performance of remote sensing image classification. However, existing multimodal fusion methods still struggle against two critical challenges. First, most cross-modal fusion strategies adopt overly simplistic feature interaction designs, which prevents them from fully exploiting the complementary information between the two modalities. Second, although the Mamba architecture efficiently models long-range dependencies, its inherent tendency toward excessive smoothing limits its ability to capture high-frequency details. To address these challenges, we propose a Hierarchical Frequency-Enhanced Mamba Fusion Network (HFMNet) for multimodal remote sensing image classification, which enhances global representation capability by integrating state-space modeling with frequency-enhanced analysis. Specifically, the Fourier transform is embedded into the Spatial Mamba modules to compensate for their limited sensitivity to high-frequency variations, thereby enhancing the representation of fine-grained spatial details while maintaining efficient long-range dependency modeling. Furthermore, a multi-level dual-attention-guided fusion module is developed to explicitly integrate multimodal features along both spatial and channel dimensions. This design facilitates the alignment and complementary exploitation of heterogeneous information. The level-specific fused representations are then progressively aggregated in a multi-level manner to strengthen multi-level feature consistency and improve classification reliability. Under a challenging few-shot learning scenario (only 20 training samples per class), comprehensive experiments on Houston2013, MUUFL, and Trento show that our method obtains OA values of 95.45%, 86.08%, and 99.35%, respectively. This demonstrates its robust feature extraction capabilities under sparse supervision, and it performs favorably against state-of-the-art approaches.

论文标识

成果类型
论文