Abstract: A computer-vision pipeline is organized as a closed loop of a sensor-processing phase, an image-processing phase, and an object-detection phase, each comprising a respective phase processor coupled to a master processor. The sensor-processing phase creates multiple exposure images, and derives multi-exposure multi-scale zonal illumination-distributions, to be processed independently in the image-processing phase. In a first implementation of the object-detection phase, extracted exposure-specific features are pooled prior to overall object detection. In a second implementation, exposure-specific objects, detected from the exposure-specific features, are fused to produce the sought objects of a scene under consideration. The two implementations enable detecting fine details of a scene under diverse illumination conditions. The master processor performs loss-function computations to derive updated training parameters of the processing phases.

Abstract: System and method for end-to-end differentiable joint image refinement and perception are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: A method for obstacle avoidance in degraded environments of robots based on intrinsic plasticity of an SNN is disclosed. A decision network in a synaptic autonomous learning module takes lidar data, distance from a target point and velocity at a previous moment as state input, and outputs the velocity of left and right wheels of the robot through the autonomous adjustment of the dynamic energy-time threshold, so as to carry out autonomous perception and decision making. The method solves the difficulty of the lack of intrinsic plasticity in the SNN, which leads to the difficulty of adapting to degraded environments due to the homeostasis imbalance of the model, is successfully deployed in mobile robots to maintain a stable trigger rate for autonomous navigation and obstacle avoidance in degraded, disturbed and noisy environments, and has validity and applicability on different degraded scenes.

Abstract: System and method for end-to-end differentiable joint image refinement and perception are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: System and method for end-to-end differentiable joint image refinement and perception are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: System and method for joint refinement and perception of images are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: A method for learned hardware-in-the-loop phase retrieval for holographic near-eye displays includes generating simulated ideal output images of a holographic display. The method further includes capturing real output images of the holographic display. The method further includes learning a mapping between the simulated ideal output images and the real output images. The method further includes using the learned mapping to solve for an aberration compensating hologram phase and using the aberration compensating hologram phase to adjust a phase pattern of a spatial light modulator of the holographic display.

Abstract: A method for obstacle avoidance in degraded environments of robots based on intrinsic plasticity of an SNN is disclosed. A decision network in a synaptic autonomous learning module takes lidar data, distance from a target point and velocity at a previous moment as state input, and outputs the velocity of left and right wheels of the robot through the autonomous adjustment of the dynamic energy-time threshold, so as to carry out autonomous perception and decision making. The method solves the difficulty of the lack of intrinsic plasticity in the SNN, which leads to the difficulty of adapting to degraded environments due to the homeostasis imbalance of the model, is successfully deployed in mobile robots to maintain a stable trigger rate for autonomous navigation and obstacle avoidance in degraded, disturbed and noisy environments, and has validity and applicability on different degraded scenes.

Abstract: The microlens amplitude masks for flying pixel removal in time-of-flight imaging includes systems, devices, methods, and instructions for image depth determination, including receiving an image, adding noise to the image, determining a set of correlation images, each correlation image having a varying phase offset, for each pixel of the image, generating a masked pixel by applying a mask array, and for each masked pixel, determining the depth of the masked pixel to generate a depth map for the image on a per pixel basis.

Abstract: The microlens amplitude masks for flying pixel removal in time-of-flight imaging includes systems, devices, methods, and instructions for image depth determination, including receiving an image, adding noise to the image, determining a set of correlation images, each correlation image having a varying phase offset, for each pixel of the image, generating a masked pixel by applying a mask array, and for each masked pixel, determining the depth of the masked pixel to generate a depth map for the image on a per pixel basis.

Abstract: A system for testing control units via simulation includes: a simulator; a host computer; and at least one connection for a communication system. At least one communication tool is stored on the system. Real control units are connectable to the system via the communication system. At least one controller is provided on the system for the connection to the communication system. Driver software for the at least one controller is stored on the system. The at least one communication tool is configured to generate communication code for communication between simulated control units and/or the real control units, wherein the communication code is configured to interact with the driver software and to relay signals and/or messages from the real and simulated control units to the driver software and to receive the signals and/or messages from the driver software. A loop mode is provided for the driver software.

Abstract: The present disclosure relates generally to image processing, and more particularly, toward techniques for structured illumination and reconstruction of three-dimensional (3D) images. Disclosed herein is a method to jointly learn structured illumination and reconstruction, parameterized by a diffractive optical element and a neural network in an end-to-end fashion. The disclosed approach has a differentiable image formation model for active stereo, relying on both wave and geometric optics, and a trinocular reconstruction network. The jointly optimized pattern, dubbed “Polka Lines,” together with the reconstruction network, makes accurate active-stereo depth estimates across imaging conditions. The disclosed method is validated in simulation and used with an experimental prototype, and several variants of the Polka Lines patterns specialized to the illumination conditions are demonstrated.

Abstract: System and method for end-to-end differentiable joint image refinement and perception are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: System and method for end-to-end differentiable joint image refinement and perception are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: An auto-exposure control is proposed for high dynamic range images, along with a neural network for exposure selection that is trained jointly, end-to-end with an object detector and an image signal processing (ISP) pipeline. Corresponding method and system for high dynamic range object detection are also provided.

Abstract: System and method for joint refinement and perception of images are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: System and method for joint refinement and perception of images are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.

Abstract: A method for digital holography includes modeling a hologram using a forward propagation model that models propagation of a light field from a hologram plane to an image plane. The method further includes computing the hologram as a solution to an optimization problem that is based on the model. The method further includes configuring at least one spatial light modulator using the hologram. The method further includes illuminating the spatial light modulator using a light source to create a target image.

Abstract: Systems and methods for imaging object velocity are provided. In an embodiment, at least one Time-of-Flight camera is used to capture a signal representative of an object in motion over an exposure time. Illumination and modulation frequency of the captured motion are coded within the exposure time. A change of illumination frequency is mapped to measured pixel intensities of the captured motion within the exposure time, and information about a Doppler shift in the illumination frequency is extracted to obtain a measurement of instantaneous per pixel velocity of the object in motion. The radial velocity information of the object in motion can be simultaneously captured for each pixel captured within the exposure time. In one or more aspects, the illumination frequency can be coded orthogonal to the modulation frequency of the captured motion. The change of illumination frequency can correspond to radial object velocity.

Abstract: System and method for joint refinement and perception of images are provided. A learning machine employs an image acquisition device for acquiring a set of training raw images. A processor determines a representation of a raw image, initializes a set of image representation parameters, defines a set of analysis parameters of an image analysis network configured to process the image's representation, and jointly trains the set of representation parameters and the set of analysis parameters to optimize a combined objective function. A module for transforming pixel-values of the raw image to produce a transformed image comprising pixels of variance-stabilized values, a module for successively performing processes of soft camera projection and image projection, and a module for inverse transforming the transformed pixels are disclosed. The image projection performs multi-level spatial convolution, pooling, subsampling, and interpolation.