Abstract: A image sensor includes a first integrated circuit layer including pixel sensors that are grouped based on position into pixel sensor groups, a second integrated circuit layer in electrical communication with the first integrated circuit layer, the second integrated circuit layer including image processing circuitry groups that are configured to each receive pixel information from a corresponding pixel sensor group, the image processing circuitry groups further configured to perform image processing operations on the pixel information to provide processed pixel information during operation of the image sensor, a third integrated circuit layer in electrical communication with the second integrated circuit layer, and the third integrated circuit layer including neural network circuitry groups that are configured to each receive the processed pixel information from a corresponding image processing circuitry group and perform analysis for object detection on the processed pixel information during operation of the

Abstract: Methods, systems, and apparatus, for an autonomous vehicle navigation maintenance system. In one aspect, an autonomous vehicle monitoring system includes a mounting surface located on an autonomous vehicle, and a thermal imaging system affixed to the mounting surface, the thermal imaging system including a thermal imaging camera, a replaceable filter, and a filter fixture configured to affix the replaceable filter to the thermal imaging camera and aligned with an optical axis of the thermal imaging camera, and where maintenance of the replaceable filter does not include re-calibrating the thermal imaging camera.

Abstract: Systems, apparatus, and methods are presented for rendering color images. One method may include receiving first color values of a first image from a first image capture device. The first color values may represent colors in a first color space. The method also may include applying first color transformation information to the first color values to generate first transformed color values of the first image in a particular color space and applying desaturation information to the first transformed color values to generate first desaturated color values of the first image. Further, the method may include providing the first desaturated color values of the first image for rendering on a display device.

Abstract: Methods, systems, and apparatus, for an autonomous vehicle navigation maintenance system. In one aspect, an autonomous vehicle monitoring system includes a mounting surface located on an autonomous vehicle, and a thermal imaging system affixed to the mounting surface, the thermal imaging system including a thermal imaging camera, a replaceable filter, and a filter fixture configured to affix the replaceable filter to the thermal imaging camera and aligned with an optical axis of the thermal imaging camera, and where maintenance of the replaceable filter does not include re-calibrating the thermal imaging camera.

Abstract: A method includes obtaining multiple images captured by pixel sensors of an image sensor, analyzing, using neural network circuitry integrated in the image sensor, the multiple images for object detection, generating, for each of the multiple images using the neural network circuitry integrated in the image sensor, neural network output data related to results of the analysis of the multiple images for object detection, and transmitting, from the image sensor, the neural network output data for each of the multiple images and image data for a subset of the multiple images instead of image data of each of the multiple images.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a computing system that receives images that each have a predefined exposure attribute. For each image, a first set of features of the image are extracted. The first set of features are associated with a label indicating no modification of the image is required. A luminosity characteristic of the image is adjusted to form an adjusted image. A second set of features of the adjusted image are extracted. A neural network is trained to adjust luminosity characteristics of images using the first set of features and the second set of features of the adjusted image. An exposure adjustment model adjusts luminosity characteristics of images based on correction values determined using the trained neural network.

Abstract: A method includes obtaining multiple images captured by pixel sensors of an image sensor, analyzing, using neural network circuitry integrated in the image sensor, the multiple images for object detection, generating, for each of the multiple images using the neural network circuitry integrated in the image sensor, neural network output data related to results of the analysis of the multiple images for object detection, and transmitting, from the image sensor, the neural network output data for each of the multiple images and image data for a subset of the multiple images instead of image data of each of the multiple images.

Abstract: A image sensor includes a first integrated circuit layer including pixel sensors that are grouped based on position into pixel sensor groups, a second integrated circuit layer in electrical communication with the first integrated circuit layer, the second integrated circuit layer including image processing circuitry groups that are configured to each receive pixel information from a corresponding pixel sensor group, the image processing circuitry groups further configured to perform image processing operations on the pixel information to provide processed pixel information during operation of the image sensor, a third integrated circuit layer in electrical communication with the second integrated circuit layer, and the third integrated circuit layer including neural network circuitry groups that are configured to each receive the processed pixel information from a corresponding image processing circuitry group and perform analysis for object detection on the processed pixel information during operation of the

Abstract: A image sensor includes a first integrated circuit layer including pixel sensors that are grouped based on position into pixel sensor groups, a second integrated circuit layer in electrical communication with the first integrated circuit layer, the second integrated circuit layer including image processing circuitry groups that are configured to each receive pixel information from a corresponding pixel sensor group, the image processing circuitry groups further configured to perform image processing operations on the pixel information to provide processed pixel information during operation of the image sensor, a third integrated circuit layer in electrical communication with the second integrated circuit layer, and the third integrated circuit layer including neural network circuitry groups that are configured to each receive the processed pixel information from a corresponding image processing circuitry group and perform analysis for object detection on the processed pixel information during operation of the

Abstract: Methods, systems, and apparatus, for an autonomous vehicle navigation maintenance system. In one aspect, an autonomous vehicle monitoring system includes a mounting surface located on an autonomous vehicle, and a thermal imaging system affixed to the mounting surface, the thermal imaging system including a thermal imaging camera, a replaceable filter, and a filter fixture configured to affix the replaceable filter to the thermal imaging camera and aligned with an optical axis of the thermal imaging camera, and where maintenance of the replaceable filter does not include re-calibrating the thermal imaging camera.

Abstract: The technology relates to determining whether a camera is occluded. For instance, an image may be captured using a camera having red, green, and blue pixels each including a photosensor. Output values for the photosensors of each of the red pixels, green pixels, and blue pixels may be determined for the images. The output values of the green pixels may be compared to one or more of the output values of the red pixels or the output values of the blue pixels. That the camera is occluded is determined based on the comparison.

Abstract: A method includes obtaining multiple images captured by pixel sensors of an image sensor, analyzing, using neural network circuitry integrated in the image sensor, the multiple images for object detection, generating, for each of the multiple images using the neural network circuitry integrated in the image sensor, neural network output data related to results of the analysis of the multiple images for object detection, and transmitting, from the image sensor, the neural network output data for each of the multiple images and image data for a subset of the multiple images instead of image data of each of the multiple images.

Abstract: A image sensor includes a first integrated circuit layer including pixel sensors that are grouped based on position into pixel sensor groups, a second integrated circuit layer in electrical communication with the first integrated circuit layer, the second integrated circuit layer including image processing circuitry groups that are configured to each receive pixel information from a corresponding pixel sensor group, the image processing circuitry groups further configured to perform image processing operations on the pixel information to provide processed pixel information during operation of the image sensor, a third integrated circuit layer in electrical communication with the second integrated circuit layer, and the third integrated circuit layer including neural network circuitry groups that are configured to each receive the processed pixel information from a corresponding image processing circuitry group and perform analysis for object detection on the processed pixel information during operation of the

Abstract: The technology relates to determining whether a camera is occluded. For instance, an image may be captured using a camera having red, green, and blue pixels each including a photosensor. Output values for the photosensors of each of the red pixels, green pixels, and blue pixels may be determined for the images. The output values of the green pixels may be compared to one or more of the output values of the red pixels or the output values of the blue pixels. That the camera is occluded is determined based on the comparison.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a computing system that receives images that each have a predefined exposure attribute. For each image, a first set of features of the image are extracted. The first set of features are associated with a label indicating no modification of the image is required. A luminosity characteristic of the image is adjusted to form an adjusted image. A second set of features of the adjusted image are extracted. A neural network is trained to adjust luminosity characteristics of images using the first set of features and the second set of features of the adjusted image. An exposure adjustment model adjusts luminosity characteristics of images based on correction values determined using the trained neural network.

Abstract: A method of determining a modulation transfer function (MTF) for an image includes receiving an image captured through the optical system, performing edge detection on columns or rows in the image to calculate a plurality of edge points, calculating a plurality polynomials to fit to the calculated edge points, each of the plurality of polynomials varying in degree, selecting a polynomial from the plurality of polynomials to represent the detected edge, and estimating the MTF based on the selected polynomial.

Abstract: An apparatus is described having an image signal processor. The image signal processor has a plurality of depth calculation units to calculate a respective time of flight depth value for different pixel array locations. Each of the plurality of depth calculation units is to receive a response signal from a same pixel in a pixel array so that the plurality of depth calculation units are able to calculate multiple depth values for the different locations of the pixel array from respective response signals from different groups of pixels in the pixel array of which the pixel is a member. Each of the groups include pixels of different receive clock phases sufficient to calculate a respective depth value. Another apparatus is also described where multiple depth values are similarly calculated from different groups of pixels that each include a same pixel but where a depth calculation unit calculates each of the multiple depth values.

Abstract: An apparatus is described having an image signal processor. The image signal processor has a plurality of depth calculation units to calculate a respective time of flight depth value for different pixel array locations. Each of the plurality of depth calculation units is to receive a response signal from a same pixel in a pixel array so that the plurality of depth calculation units are able to calculate multiple depth values for the different locations of the pixel array from respective response signals from different groups of pixels in the pixel array of which the pixel is a member. Each of the groups include pixels of different receive clock phases sufficient to calculate a respective depth value. Another apparatus is also described where multiple depth values are similarly calculated from different groups of pixels that each include a same pixel but where a depth calculation unit calculates each of the multiple depth values.

Abstract: An apparatus is described having an image signal processor. The image signal processor has a plurality of depth calculation units to calculate a respective time of flight depth value for different pixel array locations. Each of the plurality of depth calculation units is to receive a response signal from a same pixel in a pixel array so that the plurality of depth calculation units are able to calculate multiple depth values for the different locations of the pixel array from respective response signals from different groups of pixels in the pixel array of which the pixel is a member. Each of the groups include pixels of different receive clock phases sufficient to calculate a respective depth value. Another apparatus is also described where multiple depth values are similarly calculated from different groups of pixels that each include a same pixel but where a depth calculation unit calculates each of the multiple depth values.

Abstract: A method of determining a modulation transfer function (MTF) for an image includes receiving an image captured through the optical system, performing edge detection on columns or rows in the image to calculate a plurality of edge points, calculating a plurality polynomials to fit to the calculated edge points, each of the plurality of polynomials varying in degree, selecting a polynomial from the plurality of polynomials to represent the detected edge, and estimating the MTF based on the selected polynomial.