An End-to-End Decoupled Cross-Stream Attention and Hierarchical Fusion Network with Contrastive Loss for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is inherently constrained by high spectral dimensionality, strong inter-band correlation, and the entanglement of spectral and spatial cues that arises when existing deep architectures fuse these two heterogeneous modalities at an early processing stage. This coupling introduces mutual interference, amplifies sensitivity to noisy or redundant bands, and limits generalization across sensors with differing spectral configurations. Furthermore, prior work's reliance on fixed or manually defined band-partitioning schemes limits adaptability across datasets with differing spectral characteristics. To address these limitations, this article proposes the Spectral-Spatial Decoupled Attention with Adaptive Band Grouping Network (SSDAG-Net), an end-to-end framework composed of five jointly optimized modules. A Learnable Spectral Band Grouping module performs data-driven soft assignment of spectral bands into semantic groups, replacing hand-crafted band selection. Grouped features are then processed through a Decoupled Dual-Stream architecture, in which one-dimensional convolutions model spectral dependencies. In contrast, two dimensional convolutions independently capture spatial context, preventing premature cross-modal interference. A Cross-Stream Attention module then performs bidirectional gating to exchange complementary information between streams selectively. In contrast, a Hierarchical Spectral-Spatial Fusion module progressively compresses the fused representation across three stages rather than through single-step concatenation. Training is further regularized by a Contrastive Spectral Loss that enforces intra-class compactness and inter-class separability in the embedding space, mitigating confusion among spectrally similar classes. SSDAG-Net is evaluated on six benchmark datasets acquired from distinct sensors and geographic settings—Indian Pines, Kennedy Space Center, Pavia University, Salinas, Botswana, and Houston 2013—achieving Overall Accuracies of 99.61%, 99.81%, 100%, 99.89%, 99.98%, and 100%, respectively, with correspondingly high Kappa coefficients and low expected calibration error. These results are obtained with only 1.0666 M parameters, 0.0460 GFLOPs, and 2.69 ms inference latency, substantially lower than representative CNN-, transformer-, and state-space-based baselines. The proposed framework thus offers a favorable accuracy-efficiency trade-off suitable for deployment in resource-constrained remote sensing scenarios, with promising potential for adaptation across sensors given lightweight fine tuning.