Abstract: A diffractive optical network-based multispectral imaging system is trained using deep learning to create a virtual spectral filter array at the output image field-of-view. The diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states.

Abstract: A computer-free system and method is disclosed that uses an all-optical image reconstruction method to see through random diffusers at the speed of light. Using deep learning, a set of transmissive layers are trained to all-optically reconstruct images of arbitrary objects that are distorted by random phase diffusers. After the training stage, the resulting diffractive layers are fabricated and form a diffractive optical network that is physically positioned between the unknown object and the image plane to all-optically reconstruct the object pattern through an unknown, new phase diffuser. Unlike digital methods, all-optical diffractive reconstructions do not require power except for the illumination light. This diffractive solution to see through diffusive and/or scattering media can be extended to other wavelengths, and can fuel various applications in biomedical imaging, astronomy, atmospheric sciences, oceanography, security, robotics, autonomous vehicles, among many others.

Abstract: A machine vision task, machine learning task, and/or classification of objects is performed using a diffractive optical neural network device. Light from objects passes through or reflects off the diffractive optical neural network device formed by multiple substrate layers. The diffractive optical neural network device defines a trained function between an input optical signal from the object light illuminated at a plurality or a continuum of wavelengths and an output optical signal corresponding to one or more unique wavelengths or sets of wavelengths assigned to represent distinct data classes or object types/classes created by optical diffraction and/or reflection through/off the substrate layers. Output light is captured with detector(s) that generate a signal or data that comprise the one or more unique wavelengths or sets of wavelengths assigned to represent distinct data classes or object types or object classes which are used to perform the task or classification.

Abstract: A broadband diffractive optical neural network simultaneously processes a continuum of wavelengths generated by a temporally-incoherent broadband source to all-optically perform a specific task learned using network learning. The optical neural network design was verified by designing, fabricating and testing seven different multi-layer, diffractive optical systems that transform the optical wavefront generated by a broadband THz pulse to realize (1) a series of tunable, single passband as well as dual passband spectral filters, and (2) spatially-controlled wavelength de-multiplexing. Merging the native or engineered dispersion of various material systems with a deep learning-based design, broadband diffractive optical neural networks help engineer light-matter interaction in 3D, diverging from intuitive and analytical design methods to create task-specific optical components that can all-optically perform deterministic tasks or statistical inference for optical machine learning.

Abstract: A particulate matter detection device takes holographic images of flowing particulate matter concentrated by a virtual impactor, which selectively slows down and guides larger particles to fly through an imaging window. The flowing particles are illuminated by a pulsed laser diode, casting their inline holograms on a CMOS image sensor in a lens-free mobile imaging device. The illumination contains three short pulses with a negligible shift of the flowing particle within one pulse and triplicate holograms of the same particle are recorded at a single frame revealing different perspectives of each particle. A deep neural network classifies the particles based on the acquired holographic images. The device was tested using different types of pollen and achieved a blind classification accuracy of 92.91%. This mobile and cost-effective device weighs ˜700 g and can be used for label-free sensing and quantification of various bio-aerosols over extended periods.

Abstract: A particulate matter detection device takes holographic images of flowing particulate matter concentrated by a virtual impactor, which selectively slows down and guides larger particles to fly through an imaging window. The flowing particles are illuminated by a pulsed laser diode, casting their inline holograms on a CMOS image sensor in a lens-free mobile imaging device. The illumination contains three short pulses with a negligible shift of the flowing particle within one pulse and triplicate holograms of the same particle are recorded at a single frame revealing different perspectives of each particle. A deep neural network classifies the particles based on the acquired holographic images. The device was tested using different types of pollen and achieved a blind classification accuracy of 92.91%. This mobile and cost-effective device weighs ˜700 g and can be used for label-free sensing and quantification of various bio-aerosols over extended periods.

Abstract: An all-optical hologram reconstruction system and method is disclosed that can instantly retrieve the image of an unknown object from its in-line hologram and eliminate twin-image artifacts without using a digital processor or a computer. Multiple transmissive diffractive layers are trained using deep learning so that the diffracted light from an arbitrary input hologram is processed all-optically to reconstruct the image of an unknown object at the speed of light propagation and without the need for any external power. This passive diffractive optical network, which successfully generalizes to reconstruct in-line holograms of unknown, new objects and exhibits improved diffraction efficiency as well as extended depth-of-field at the hologram recording distance. The system and method can find numerous applications in coherent imaging and holographic display-related applications owing to its major advantages in terms of image reconstruction speed and computer-free operation.

Abstract: A time-lapse image classification device and method is disclosed that uses a diffractive optical network to classify an optical input, significantly advancing classification accuracy and generalization performance on complex input objects by using the lateral movements of the input objects and/or the diffractive optical network relative to each other. The design space and performance limits of time-lapse diffractive optical networks were numerically tested, revealing a blind testing accuracy of 62.03% on the optical classification of objects from the CIFAR-10 dataset. This constitutes the highest inference accuracy achieved so far using a single diffractive optical network on the CIFAR-10 dataset. Time-lapse diffractive optical networks will be broadly useful for the spatio-temporal analysis of input signals using all-optical processors.

Abstract: A stain-free quantitative viral plaque assay device uses lens-free holographic imaging and deep learning to quickly detect the plaque formations. The device captures phase and/or amplitude information of the plaque formations in contained within wells or sample-holding regions of sample holder in a label-free manner. The device uses a trained neural network to automatically detect the cell lysing events due to viral replication as early as 5 hours or earlier after the incubation, and achieved >90% detection rate for the plaque-forming units (PFUs) with 100% specificity in <20 hours, providing major time savings compared to the traditional plaque assays that take ?48 hours. This data-driven plaque assay also offers the capability of quantifying the infected area of the cell monolayer, performing automated counting and quantification of PFUs and virus-infected areas over a 10-fold larger dynamic range of virus concentration than standard viral plaque assays.

Abstract: A deep learning-enabled system for the display or projection of high-resolution images is disclosed that is based on a jointly-trained pair of an electronic encoder network and an all-optical decoder network to synthesize/project super-resolved images using low-resolution wavefront modulators. The electronic encoder network rapidly pre-processes the high-resolution images of interest so that their spatial information is encoded into low-resolution (LR) modulation patterns, projected via a low SBP wavefront modulator. The all-optical decoder network processes this LR encoded information using thin transmissive layers that are structured using deep learning to all-optically synthesize and project super-resolved images at its output FOV. Results indicate that this diffractive image display system can achieve a super-resolution factor of ˜4, demonstrating a ˜16-fold increase in SBP.

Abstract: A deep learning framework, termed Fourier Imager Network (FIN) is disclosed that can perform end-to-end phase recovery and image reconstruction from raw holograms of new types of samples, exhibiting success in external generalization. The FIN architecture is based on spatial Fourier transform modules with the deep neural network that process the spatial frequencies of its inputs using learnable filters and a global receptive field. FIN exhibits superior generalization to new types of samples, while also being much faster in its image inference speed, completing the hologram reconstruction task in ˜0.04 s per 1 mm2 of the sample area. Beyond holographic microscopy and quantitative phase imaging applications, FIN and the underlying neural network architecture may open up various new opportunities to design broadly generalizable deep learning models in computational imaging and machine vision fields.

Abstract: A diffractive network is disclosed that utilizes, in some embodiments, diffractive elements, which are used to shape an arbitrary broadband pulse into a desired optical waveform, forming a compact and passive pulse engineering system. The diffractive network was experimentally shown to generate various different pulses by designing passive diffractive layers that collectively engineer the temporal waveform of an input terahertz pulse. The results constitute the first demonstration of direct pulse shaping in terahertz spectrum, where the amplitude and phase of the input wavelengths are independently controlled through a passive diffractive device, without the need for an external pump. Furthermore, a modular physical transfer learning approach is presented to illustrate pulse-width tunability by replacing part of an existing diffractive network with newly trained diffractive layers, demonstrating its modularity. This learning-based diffractive pulse engineering framework can find broad applications in e.g.

Abstract: A diffractive optical neural network device includes a plurality of diffractive substrate layers arranged in an optical path. The substrate layers are formed with physical features across surfaces thereof that collectively define a trained mapping function between an optical input and an optical output. A plurality of groups of optical sensors are configured to sense and detect the optical output, wherein each group of optical sensors has at least one optical sensor configured to capture a positive signal from the optical output and at least one optical sensor configured to capture a negative signal from the optical output. Circuitry and/or computer software receives signals or data from the optical sensors and identifies a group of optical sensors in which a normalized differential signal calculated from the positive and negative optical sensors within each group is the largest or the smallest of among all the groups.

Abstract: A diffractive camera performs class-specific imaging of target objects with instantaneous all-optical erasure of other classes of objects. This diffractive camera includes transmissive surfaces structured using deep learning to perform selective imaging of target classes of objects positioned at its input field-of-view. After fabrication, the substrate layers collectively perform optical mode filtering to accurately form images of the objects that belong to a target data class or group of classes, while instantaneously erasing objects of the other data classes at the output field-of-view. In another embodiment, a class-specific permutation camera is disclosed where objects of a target data class are pixel-wise permuted for all-optical class-specific encryption, while the other objects are irreversibly erased from the output image.

Abstract: A deep learning-based spectral analysis device and method are disclosed that employs a spectral encoder chip containing a plurality of nanohole array tiles, each with a unique geometry and, thus, a unique optical transmission spectrum. Illumination impinges upon the encoder chip and a CMOS image sensor captures the transmitted light, without any lenses, gratings, or other optical components. A spectral reconstruction neural network uses the transmitted intensities from the image to faithfully reconstruct the input spectrum. In one embodiment that used a spectral encoder chip with 252 nanohole array tiles, the network was trained on 50,352 spectra randomly generated by a supercontinuum laser and blindly tested on 14,648 unseen spectra. The system identified 96.86% of spectral peaks, with a peak localization error of 0.19 nm, peak height error of 7.60%, and peak bandwidth error of 0.18 nm.

Abstract: A bacterial colony-forming-unit (CFU) detection system is disclosed that exploits a thin-film-transistor (TFT)-based image sensor array that saves ˜12 hours compared to the Environmental Protection Agency (EPA)-approved methods. A lensfree imaging modality was built using the TFT image sensor with a sample field-of-view of ˜10 cm2. Time-lapse images of bacterial colonies cultured on chromogenic agar plates were automatically collected at 5-minute intervals. Two deep neural networks were used to detect and count the growing colonies and identify their species. When blindly tested with 265 colonies of E. coli and other coliform bacteria (i.e., Citrobacter and Klebsiella pneumoniae), the system reached an average CFU detection rate of 97.3% at 9 hours of incubation and an average recovery rate of 91.6% at ˜12 hours. This TFT-based sensor can be applied to various microbiological detection methods. The imaging field-of-view of this platform can be cost-effectively increased to >100 cm2.

Abstract: A deep learning-based digital staining method and system are disclosed that enables the creation of digitally/virtually-stained microscopic images from label or stain-free samples based on autofluorescence images acquired using a fluorescent microscope. The system and method have particular applicability for the creation of digitally/virtually-stained whole slide images (WSIs) of unlabeled/unstained tissue samples that are analyzes by a histopathologist. The methods bypass the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses, in one embodiment, a convolutional neural network trained using a generative adversarial network model to transform fluorescence images of an unlabeled sample into an image that is equivalent to the brightfield image of the chemically stained-version of the same sample.

Abstract: A deep learning-based digital/virtual staining method and system enables the creation of digitally/virtually-stained microscopic images from label or stain-free samples. In one embodiment, the method of generates digitally/virtually-stained microscope images of label-free or unstained samples using fluorescence lifetime (FLIM) image(s) of the sample(s) using a fluorescence microscope. In another embodiment, a digital/virtual autofocusing method is provided that uses machine learning to generate a microscope image with improved focus using a trained, deep neural network. In another embodiment, a trained deep neural network generates digitally/virtually stained microscopic images of a label-free or unstained sample obtained with a microscope having multiple different stains. The multiple stains in the output image or sub-regions thereof are substantially equivalent to the corresponding microscopic images or image sub-regions of the same sample that has been histochemically stained.

Abstract: A polarization multiplexed diffractive processor is disclosed that all-optically performs multiple, arbitrarily-selected transformations (e.g., linear) through a single diffractive network trained using deep learning. In this framework, an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic, and different target linear transformations (complex-valued) are uniquely assigned to different combinations of input/output polarization states. The transmission layers of this polarization multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to different input/output polarization combinations. This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.

Abstract: A deep learning-based digital staining method and system are disclosed that enables the creation of digitally/virtually-stained microscopic images from label or stain-free samples based on autofluorescence images acquired using a fluorescent microscope. The system and method have particular applicability for the creation of digitally/virtually-stained whole slide images (WSIs) of unlabeled/unstained tissue samples that are analyzes by a histopathologist. The methods bypass the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses, in one embodiment, a convolutional neural network trained using a generative adversarial network model to transform fluorescence images of an unlabeled sample into an image that is equivalent to the brightfield image of the chemically stained-version of the same sample.