The Californer - Intelligent photodetectors recognize materials directly from light spectra

Trending...

LOS ANGELES - Californer -- Researchers at the University of California, Los Angeles (UCLA), in collaboration with UC Berkeley, have developed a new type of intelligent image sensor that can perform machine-learning inference during the act of photodetection itself. Reported in Science, the breakthrough redefines how spectral imaging, machine vision and AI can be integrated within a single semiconductor device.

Traditionally, spectral cameras capture a dense stack of images, each image corresponding to a different wavelength, and then transfer this large dataset to digital processors for computation and scene analysis. This workflow, while powerful, creates a severe bottleneck: the hardware must move and process massive amounts of data, which limits speed, power efficiency, and the achievable spatial–spectral resolution. The new device platform, called spectral kernel machines (SKMs), completely bypasses this bottleneck. Instead of recording large data cubes, SKMs directly encode the spectral and spatial information into the output photocurrent, allowing the sensor itself to perform the task of identifying materials, chemicals, and objects within a complex scene.

More on The Californer

The work was conducted in close collaboration with Professor Ali Javey's research group and Dr. Dehui Zhang at Lawrence Berkeley National Laboratory and UC Berkeley.
Each SKM device can be electrically tuned to enhance or suppress specific spectral signatures. During training, the researchers displayed sensor images, such as colorful birds in forest scenes, allowing the SKM device to randomly sample a subset of the pixels while receiving simple external commands like "identify bird" or "identify background". From these examples, the device learned the optimal electrical control sequence to highlight bird pixels and suppress background regions. When later presented with new images never seen before, the sensor produced a positive photocurrent only for pixels belonging to the target object, demonstrating that it had learned from prior examples and could "sniff and seek" desired features, much like a retriever dog.

The team demonstrated that SKM devices can intelligently sense and analyze complex scenes across the visible to mid-infrared spectrum without relying on conventional hyperspectral image stacks. In the visible band, silicon-based photoconductors performed semiconductor wafer metrology tasks and feature identification, offering speed and power advantages over traditional digital hyperspectral machine vision pipelines. In the mid-infrared, a room-temperature, electrically tunable photodiode enabled chemical identification and the analysis of mixtures. The researchers further showcased applications such as plant-leaf hydration sensing and object segmentation, all derived directly from the sensor's photocurrent without the need to capture or process a hyperspectral data cube. By embedding intelligence and machine learning directly into the physics of photodetection, SKM devices eliminate data-movement bottlenecks and significantly reduce energy consumption, enabling ultrafast spectral analysis in a compact, low-power form. These capabilities make SKMs ideal for mobile devices, autonomous robots, environmental monitoring, industrial inspection, and satellite imaging, among other applications.

More on The Californer