Abstract: Disclosed are systems and methods for a sensor synchronization system. In some aspects, a method includes synchronizing sensors of an autonomous vehicle (AV) to cause sampling of a first sensor of the sensors to occur in alignment with sampling of a second sensor of the sensors; determining a sampling time point for the first sensor based on the synchronizing the sensors, the sampling time point comprising a time when the first sensor is in alignment with the second sensor; providing the sampling time point to the first sensor; determining, at a controller circuit of the first sensor, an offset of the first sensor to apply to a local system time of the first sensor to cause a data acquisition of the first sensor to occur at the sampling time point; and causing the data acquisition to be performed at the first sensor using the sampling time point and the offset.

Abstract: Methods and systems are provided for preserving dynamic range in images. In some aspects, a process can include steps for receiving, at an autonomous vehicle system, image data from an image sensor, generating, by the autonomous vehicle system, a high dynamic range (HDR) output by performing HDR processing on the image data from the image sensor, generating, by the autonomous vehicle system, a least significant bit (LSB) output by performing LSB processing on the image data from the image sensor, and generating, by the autonomous vehicle system, an 8-bit output by performing a buffer process on the HDR output and the LSB output of the image sensor.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ling-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Color filter array patterns are provided for enhancing an image sensor’s light sensitivity while preserving a color accuracy for image signal processing applications. In one example, an image sensor can include a substrate layer containing a first set of photodiodes and a second set of photodiodes, wherein each of the first set of photodiodes is larger than each of the second set of photodiodes; a first color filter array (CFA) covering the first set of photodiodes, wherein the first CFA includes a first set of color filters and a portion of the first set of color filters includes one or more clear filters; a second CFA covering the second set of photodiodes, wherein the second CFA includes a second set of color filters that is different than the first set of color filters; and one or more lenses covering the first CFA and the second CFA.

Abstract: Methods and systems are provided for preserving dynamic range in images. In some aspects, a process can include steps for receiving, at an autonomous vehicle system, image data from an image sensor, generating, by the autonomous vehicle system, a high dynamic range (HDR) output by performing HDR processing on the image data from the image sensor, generating, by the autonomous vehicle system, a least significant bit (LSB) output by performing LSB processing on the image data from the image sensor, and generating, by the autonomous vehicle system, an 8-bit output by performing a buffer process on the HDR output and the LSB output of the image sensor.

Abstract: Color filter array patterns are provided for enhancing an image sensor's light sensitivity while preserving a color accuracy for image signal processing applications. In one example, an image sensor can include a substrate layer containing a first set of photodiodes and a second set of photodiodes, wherein each of the first set of photodiodes is larger than each of the second set of photodiodes; a first color filter array (CFA) covering the first set of photodiodes, wherein the first CFA includes a first set of color filters and a portion of the first set of color filters includes one or more clear filters; a second CFA covering the second set of photodiodes, wherein the second CFA includes a second set of color filters that is different than the first set of color filters; and one or more lenses covering the first CFA and the second CFA.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ling-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Various technologies described herein pertain to a SPAD lidar system that includes a transmitter and a receiver. The transmitter is configured to transmit a transmitted electromagnetic signal into an environment nearby the SPAD lidar system, and the receiver is configured to receive a received electromagnetic signal from the environment nearby the SPAD lidar system. The receiver includes a SPAD sensor array (which includes SPAD pixels) and a binning system. The received electromagnetic signal is inputted to the SPAD pixels of the SPAD sensor array. The binning system is configured to combine outputs of groups of the SPAD pixels to generate combined outputs for the groups and assign the combined outputs to SPAD pixels in the groups, where the groups are overlapping. A processing system of the SPAD lidar system can generate lidar data based on the combined outputs.

Abstract: Described herein are various technologies pertaining to a sensor system for an autonomous vehicle (AV) that includes a single photon accelerated diode (SPAD) array. The SPAD array includes several pixels. A filter layer is arranged proximate the several pixels, where the filter layer includes color filter portions and unfiltered portions that are respectively aligned with first pixels of the SPAD array and second pixels of the SPAD array. A computing system determines both range to an object and information pertaining to color of the object based upon values read out from the pixels of the SPAD array.

Abstract: Methods and systems are provided for preserving dynamic range in images. In some aspects, a process can include steps for receiving, at an autonomous vehicle system, image data from an image sensor, generating, by the autonomous vehicle system, a high dynamic range (HDR) output by performing HDR processing on the image data from the image sensor, generating, by the autonomous vehicle system, a least significant bit (LSB) output by performing LSB processing on the image data from the image sensor, and generating, by the autonomous vehicle system, an 8-bit output by performing a buffer process on the HDR output and the LSB output of the image sensor.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Various technologies described herein pertain to a fused camera and lidar system for an autonomous vehicle. The fused camera and lidar system includes a fused receiver. The fused receiver includes optics configured to receive a received electromagnetic signal from an environment nearby the fused camera and lidar system. The fused receiver further includes a beam splitter configured to split the received electromagnetic signal into a first split electromagnetic signal (including wavelengths in a visible spectrum) and a second split electromagnetic signal (including wavelengths in an infrared spectrum). The fused receiver also includes a camera pipeline and a lidar pipeline. The camera pipeline can generate image data based on the first split electromagnetic signal, and the lidar pipeline can generate lidar data based on the second split electromagnetic signal.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The present disclosure relates to optical systems and methods of their manufacture. An example optical system includes a lens assembly having at least one lens. The lens assembly defines an optical axis, a focal distance, and a corresponding focal plane. The optical system includes a substrate having a first surface and an image sensor attached to the first surface of the substrate. The optical system also includes a sensor holder attached to the first surface of the substrate. The sensor holder positions the image sensor along the optical axis and substantially at the focal plane. The optical system includes a registration body having a first registration surface and a second registration surface. The lens assembly is coupled to the first registration surface and the sensor holder is coupled to the second registration surface.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Various aspects of the subject technology relate to an automotive camera unit. The automotive camera unit comprises a housing comprising an aperture, a lens positioned to receive an optical image through the aperture of the housing, and an image sensor board mounted within the housing, the image sensor board comprising an image sensor configured to convert the optical image into sensor data. The automotive camera unit further includes an image signal processor (ISP) board mounted within the housing and above the image sensor board, the image signal processor board comprising an image signal processor configured to covert the sensor data into image data for use by an automotive system.

Abstract: Technologies for steering sensors in a payload carrier structure on an autonomous vehicle (AV) are described herein. An example method can include receiving, by a motor control system on the AV, instructions for controlling a motor on the motor control system to reposition the payload carrier structure from a first position to a second position; based on the instructions, sending, by a controller on the motor control system to a motor driver on the motor control system, a command instructing the motor driver to reposition the payload carrier structure from the first position to the second position; sending, by the motor driver to the motor, a control signal generated by the motor driver based on the command, the control signal controlling the motor to reposition the payload carrier structure to the second position; and moving, by the motor, the payload carrier structure and sensors to the second position.

Abstract: Various aspects of the subject technology relate to an automotive camera unit. The automotive camera unit comprises a housing comprising an aperture, a lens positioned to receive an optical image through the aperture of the housing, and an image sensor board mounted within the housing, the image sensor board comprising an image sensor configured to convert the optical image into sensor data. The automotive camera unit further includes an image signal processor (ISP) board mounted within the housing and above the image sensor board, the image signal processor board comprising an image signal processor configured to covert the sensor data into image data for use by an automotive system.

Abstract: Color filter array patterns are provided for enhancing an image sensor's light sensitivity while preserving a color accuracy for image signal processing applications. In one example, an image sensor can include a substrate layer containing a first set of photodiodes and a second set of photodiodes, wherein each of the first set of photodiodes is larger than each of the second set of photodiodes; a first color filter array (CFA) covering the first set of photodiodes, wherein the first CFA includes a first set of color filters and a portion of the first set of color filters includes one or more clear filters; a second CFA covering the second set of photodiodes, wherein the second CFA includes a second set of color filters that is different than the first set of color filters; and one or more lenses covering the first CFA and the second CFA.