Single-Pixel Near-Infrared 3D Image Reconstruction in Outdoor Conditions

In the last decade, vision systems have improved their capabilities to capture 3D images in bad weather scenarios. Several techniques exist for image acquisition in foggy or rainy scenarios that use infrared (IR) sensors. Due to the reduced light scattering at the IR spectra, it is possible to discriminate the objects in a scene compared with the images obtained in the visible spectrum. Therefore, in this work, we proposed 3D image generation in foggy conditions using the single-pixel imaging (SPI) active illumination approach in combination with the Time-of-Flight technique (ToF) at 1550 nm wavelength. For the generation of 3D images, we use space-filling projection with compressed sensing (CS-SRCNN) and depth information based on ToF. To evaluate the performance, the vision system included a designed test chamber to simulate different fog and background illumination environments and calculate the parameters related to image quality.

NIR-SPI System Test~Architecture

Fig.1. Sequence algorithm used to generate 2D/3D images.

In this work, we propose an NIR-SPI vision system based on the structured illumination scheme depicted in Figure 1. Still, instead of using an SLM or a DMD to generate the structured illumination patterns, an array of 8 × 8 NIR LEDs is used, emitting radiation with the wavelength λ = 1550 nm. The NIR-SPI system architecture is divided into two stages: the first one controls the elements used to generate images by applying the already explained Fig.2. 3D NIR-SPI camera system developed

single-pixel imaging principle: an InGaAs photodetector (diode FGA015 @ 1550 nm), accompanied by an array of 8 × 8 NIR LEDs. Nevertheless, the spatial resolution of the objects in the scene is achieved by applying the Shape-From-Shading (SFS) method and the unified reflectance model, additionally applying mesh enhancement algorithms, is still very much away from the aimed resolution goal of below 10 mm at a distance of 3 m. Thus, four control spots were incorporated into the system illumination array, consisting of NIR lasers with controlled variable light intensity emulating an illumination sinusoidal signal modulated in time and four additional InGaAs photodiode pairs to measure the distance to the objects in the depicted scene with much higher precision, using the indirect Time-of-Flight (iTOF) ranging method (see Figure 3a). The second stage of the system is responsible for processing the captured signals by the photodiode module through the use of an analog-to-digital converter (ADC), which is controlled by a Graphics Processing Unit (GPU) (see Figure 2). The GPU unit (Jetson–Nano) generates the Hadamard patterns and processes the converted data by the ADC. The 2D/3D image reconstruction is performed using the OMP-GPU algorithm.

---

https://www.mdpi.com/2072-666X/13/5/795/htm

Araguaney Solar vehicle telemetry system

In an automotive prototype, having a telemetry system is of great importance since it allows measuring and recording its behavior under real operating conditions. For this reason, a telemetry system was developed, composed of hardware and software stages. For the system's development process, first, the variables to be measured were defined, and the data transmission networks and communication stages were chosen. Both the RF communication cards and the 915 MHz band were chosen.

For this application, we use data packet formats, communication protocols, encryption algorithms, and Red Solomon coding to increase the transmitted data's reliability and efficiency. The radio system developed comprises a transmitter chipset (CC1101), coupling networks, amplifiers, duplexers, and an antenna (see Fig. 1). The transfer of information to the transmitter and receiver card is done through the CAN 2.0 protocol and the telemetry card, as shown in Fig.1.Transmitter and receiver card Figure 2.

using chip CC1101.

Communication protocol CC1101

The information packets coming from the CAN bus are transferred to the RF module via SPI protocol. The captured information is stored in registers depending on the packet type. These packets are assigned an identification command,

Fig.2. Control card of transmitter and length, and CRC16 polymode, encrypted using AES. After

the receiver based on chip CC1101 passing through this process, the packets are separated into

128-byte blocks. Each block is encoded using Reed Solomon, forming three blocks of 160 bytes to which an identifier is assigned. The new block encoding is transferred via SPI to the chipset in 256-byte blocks. The maximum data buffer size of the CC1101 is 256 bytes.

At the hardware level, the telemetry system has the following characteristics:

- Modulation type: 2-FSK.

Communication technique: FHSS 510 hops/s.

- ∆f = 80 KHz.

- Fc = 915 MHz.

- Rb = 125 kbps.

- Modulation index: 0.64.

- Channel frequency: 200 kHz.

- Number of channels 16.

- Minimum output power: +0dBm.

- Maximum output power: +27dBm.

- Sensitivity: -86.5 dBm.

- Network losses: 2dB.

- Free space losses 97.748dB.

- Maximum range: 1Km

---

Three-dimensional hyperspetral camera based on near-infrared single-pixel imaging

This work presents the development of a multi-spectral vision system that combines different technologies detecting radiation in the visible spectrum (380-750 nm wavelengths), in the near-infrared NIR (1550 nm wavelengths), in the thermal spectrum band 1.4µm, and 77 GHz millimeter band. The main motivation of the this work is to propose a new hyperspectral approach to overcome the limitations present in commercial vision systems, such as the high amount of memory needed to generate images, the low quantum efficiency of silicon sensors, noise due to disturbance of the visible spectrum, limitations caused by foggy or low visibility conditions, and the difficulty to determine the depth information for objects in the image. Therefore, we propose a hybrid vision system that combines the capabilities offered by RADAR technology at the a range of tens or hundreds of meters, the ability to trace surfaces and operate under low visibility conditions, using a vision system based on single-pixel imaging (SPI) operating in the near-infrared NIR (@ 1550 nm) part of the spectra and Time-of-flight (ToF) based system using an InGaAs high quantum efficiency detector (QE(λ)>80 @ 1550 nm wavelength). Furthermore, SPI systems have a high intrinsic compression capacity at the hardware level, so less memory is needed for image generation, reaching PSNR values > 20 db (the PSNR level acceptable image quality in the industry is between 20 and 50 dB). Developing a system with these characteristics offers a high advantage for applications such as drones, autonomous vehicles, and other techniques that operate with 2D/3D vision or video systems. For the implementation of the system, a structured lighting SPI architecture was developed using an array of 32 x 32 NIR LEDs emitting radiation in the wavelength of 1550 nm that operates in combination with other technologies such as an 80 GHz millimeter band radar, a Time-of-flight system based on four pulsed lasers, a 32 x 24 thermal camera sensor, and a camera with northern vision capability. The developed vision system has an operating range of 37.5 mm to 5 m with a maximum spatial resolution of 10 mm in the distance range below 1 m. As a single-pixel image processing strategy, the Orthogonal Matching Pursuit (OMP) compressed sensing algorithm supported on a GPU architecture is used to achieve a processing times of below 30 ms, which corresponds to a 24 frame/sec video rate.

Future applications NIR-SPI hyperspectral system

The last decade has promoted different detection technologies adapted to different conditions in rainy, fog, or low-visible scenarios. The object detection under scattering conditions has converter in a challenge to which they have proposed some solution as use of the RADAR or LIDAR . A point important to considerer in a scattering condition is the elastic Mie scattering effects which in NIR spectrum decreases, for which a NIR-SPI system is a solution efficiency with low-cost and integration which in recent years have improved, reaching processing times close to real-time and capacity of 3D images. Therefore, these offer an advantage for its integration in systems perception using today in different vehicles, where it doesn't need to have a high-quality image for application as vision sensors detect and avoid obstacles, 2D/3D pre-mapping, or navigation applications.

Our NIR-SPI system developed if adopted in UAVs (see Fig. 1) offers a solution to perform better visualization of the scene in outdoor conditions. If compared to conventional vision systems that usually use RGB sensors operating in the visible spectrum. As explained, conventional systems operating in the VIS range present much higher image degradation in scattering conditions than those operating in the NIR part of the spectra. The integration of our NIR-SPI system into UAVs increases spectral capacity detection. In addition, it provides fast deployment at remote or challenging locations, being essential for many civil and military applications such as post-disaster relief assistance. Therefore, we believe that the NIR-SPI system presented has a great potential to be used in similar applications. Furthermore, SPI could be an alternative to the use of LIDAR technology.

Fig. 1. Future applications NIR-SPI hyperspectral system recues Drone

SPI applications are not limited to vision sensors, but new applications have also been developed. For example, in VR/AR systems, we can cite MEMSEye system that works using SPI and MEMS mirror to a generation of 3D images (see Fig .2b), and applications in holography . Others applications that are studied is adaption as vision system for ROVs or underwater, SPI-RADAR (see Fig.2c), and hyper-spectral . This later field of application has great importance in the area medical [241] (see Fig. 2d), the space exploration , and CubeSat.

Fig.2. Overview of new fields for applications system vision SPI: (a) SPI-LIDAR , (b) VR/AR , (c) SPI-RADAR , (d) SPI-medical.