Single-pixel imaging (SPI) is a powerful technique for capturing images under challenging conditions, such as low-light environments or spectral bands where traditional multi-pixel sensors are not readily available. This is particularly crucial in near-infrared (NIR) imaging, covering wavelengths from 850 to 1550 nm, where conventional imaging systems often struggle. In this blog post, we introduce a hybrid approach that leverages Deep Image Prior (DIP) and Generative Adversarial Networks (GANs) to enhance the resolution of SPI-based images.
The Challenge of SPI Resolution
SPI reconstructs images from a series of intensity measurements using a single photodetector. While this method offers advantages in low-light and specialized spectral ranges, it suffers from resolution limitations due to the inherent under-sampling of spatial information. Traditional deep learning-based super-resolution techniques require extensive labeled datasets, which are difficult to acquire for SPI in NIR bands. Our proposed approach mitigates this limitation by utilizing an unsupervised learning framework.
Hybrid Approach: DIP Meets GAN
Deep Image Prior (DIP) is a compelling technique that reconstructs high-quality images without requiring a large training dataset. By coupling DIP with a Generative Adversarial Network (GAN), we improve SPI resolution through an unsupervised learning paradigm. This approach offers several advantages:
Reduced Data Dependency: Unlike supervised methods, DIP leverages image priors, reducing the need for extensive SPI datasets.
Enhanced Super-Resolution: The GAN component learns to refine the image quality, making it more detailed and perceptually accurate.
Optimized Neural Architectures: We enhance the performance by leveraging variations of UNet and GAN architectures across four different neural network configurations.
Implementation and Results
We conducted both numerical simulations and experimental validations to assess the performance of our hybrid model. Key findings include:
Improved Image Quality: Our model consistently enhances SPI image resolution, particularly in the NIR range.
Robustness to Noise: The DIP-GAN approach exhibits strong resilience to noisy measurements, a common challenge in SPI applications.
Architectural Refinements: By optimizing UNet and GAN structures, we achieve significant improvements in feature extraction and detail preservation.
Future Perspectives
Our results demonstrate that combining DIP with GANs is a promising direction for SPI super-resolution, particularly for niche applications in biomedical imaging, remote sensing, and defense technology. Future research could explore:
Real-time implementations for SPI-based imaging systems.
Adaptations to other spectral bands beyond NIR.
Hybrid models incorporating physics-informed neural networks (PINNs) for further refinement.
Conclusion
By integrating DIP and GANs, we propose an innovative, unsupervised approach to improving SPI resolution. This hybrid model significantly reduces the need for large SPI datasets while maintaining high-quality reconstructions, making it a valuable advancement in computational imaging for the NIR spectrum. Our experimental and numerical results validate its effectiveness, paving the way for broader applications in optical imaging and beyond.
BibTeX
@article{OsorioQuero:25,
author = {Carlos Osorio Quero and Irving Rondon and Jose Martinez-Carranza},
journal = {J. Opt. Soc. Am. A},
keywords = {Ghost imaging; Neural networks; Single pixel imaging; Spatial light modulators; Three dimensional imaging; Underwater imaging},
number = {2},
pages = {201--210},
publisher = {Optica Publishing Group},
title = {Improving NIR single-pixel imaging: using deep image prior and GANs},
volume = {42},
month = {Feb},
year = {2025},
url = {https://opg.optica.org/josaa/abstract.cfm?URI=josaa-42-2-201},
doi = {10.1364/JOSAA.541763}
}
Comments