• Unmixing before Fusion: A Generalized Paradigm for Multi-Source-based Hyperspectral Image Synthesis

    A new paradigm for hyperspectral image synthesis

    Yang Yu*, Erting Pan*, Xinya Wang, Yuheng Wu, Xiaoguang Mei, Jiayi Ma

    * Equal contribution Corresponding author

    Wuhan University

  • Abstract

    In the realm of artificial intelligence, data serves as a pivotal resource. Among these, real-world hyperspectral images (HSIs), bearing a broad spectrum of characteristics and diversity, are particularly valuable. However, the acquisition of HSIs can often be both financially burdensome and time-intensive. This results in a significant scarcity of HSI data, thereby hampering the progress and potential of deep learning applications that rely on HSIs. Current solutions aimed at remedying this shortage are largely inadequate, failing in generating a sufficient volume of diverse and reliable synthetic HSIs. In response to this, our study formulate a novel, generalized paradigm for HSI synthesis that initiates with unmixing, and follows by multi-source data fusion. Our approach pioneers the synthesis of hyperspectral data in the abundance domain, rather than the original, high-dimensional HSI space. We also incorporate multi-source data to augment the diversity of spatial distribution that the model can perceive and employ the unmixing concept to ensure the reliability of the spectral profiles of the synthesized HSIs. As a result, our approach is capable of generating a vast quantity of HSIs that cover a wide range of categories and scenes, closely mirroring realistic data. Our proposed generalized paradigm is instrumental in enhancing the reliability of the generative model's output and its applicability to real-world scenarios.

  • Brief Review

    broken image

    Comparisons of existing techniques for data augmentation. (a)The physical-modeling method induces the HSI formed by the bidirectional reflectance distribution function (BRDF). (b) Affine transformation applies the corresponding transformation to the original HSI. (c) Spectral super-resolution performs spectral expansion to obtain the HSI of the same scene as RGB. (d) Our proposed method can generate new HSI samples by multi-source fusion.

  • Framework

    broken image
  • Synthesis Results

    broken image
  • Code

    To be done.

  • Citation