Abstract: The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

Abstract: Systems and methods of capturing imagery are provided. In particular, an imaging platform can be configured to capture imagery using a dual-axis steering mirror and one or more image capture devices. The line of sight angle of the imaging platform can be controlled by controlling the motion of the steering mirror in accordance with a motion profile. In example embodiments, the motion profile can correspond to a sawtooth wave. The imaging platform can further include one or more position sensors used to determine a position and/or orientation of the imaging platform along a path on which the imaging platform travels. The motion of the steering mirror can then be controlled to rotate about a first axis and a second axis to compensate for line of sight errors based at least in part on the determined position and/or orientation.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The subject disclosure relates to features for improving the modularity and interoperability of sensors and in particular, sensors deployed for use in vehicle related sensing applications. In some aspects, the disclosed technology encompasses a sensor aggregation module (e.g., sensor pod) that includes a sensor enclosure having an interior volume configured for housing one or more sensors, a data port integrated into an exterior surface of the sensor enclosure, wherein the data port is configured to be electrically coupled to a vehicle data port, and a plurality of sensor ports disposed within an interior volume of the sensor enclosure, wherein the sensor ports are configured for modular connectivity to the one or more sensors.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: An optical system for a vehicle may be configured with a plurality of camera sensors. Each camera sensor may be configured to create respective image data of a respective field of view. The optical system is further configured with a plurality of image processing units coupled to the plurality of camera sensors. The image processing units are configured to compress the image data captured by the camera sensors. A computing system is configured to store the compressed image data in a memory. The computing system is further configured with a vehicle-control processor configured to control the vehicle based on the compressed image data. The optical system and the computing system can be communicatively coupled by a data bus.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Technologies for steering sensors in a payload carrier structure on an autonomous vehicle (AV) are described herein. An example method can include receiving, by a motor control system on the AV, instructions for controlling a motor on the motor control system to reposition the payload carrier structure from a first position to a second position; based on the instructions, sending, by a controller on the motor control system to a motor driver on the motor control system, a command instructing the motor driver to reposition the payload carrier structure from the first position to the second position; sending, by the motor driver to the motor, a control signal generated by the motor driver based on the command, the control signal controlling the motor to reposition the payload carrier structure to the second position; and moving, by the motor, the payload carrier structure and sensors to the second position.

Abstract: The present disclosure relates to optical systems. An example optical system includes at least one primary optical element configured to receive incident light from a scene and a plurality of relay mirrors optically coupled to the at least one primary optical element. The optical system also includes a lens optically coupled to the plurality of relay mirrors, and an image sensor configured to receive focused light from the lens. The image sensor includes a first light-sensitive area and a second light-sensitive area. The primary optical element, the plurality of relay mirrors, and the lens interact with the incident light to form a first focused light portion and a second focused light portion. The first focused light portion forms a first image portion of the scene on the first light-sensitive area and the second focused light portion forms a second image portion of the scene on the second light-sensitive area.

Abstract: Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.

Abstract: The present disclosure relates to multiple view optical systems. An example optical system includes at least one primary optical element configured to receive incident light from a scene and a plurality of relay mirrors optically coupled to the at least one primary optical element. The optical system also includes a lens optically coupled to the plurality of relay mirrors, and an image sensor configured to receive focused light from the lens. The image sensor includes a first light-sensitive area and a second light-sensitive area. The primary optical element, the plurality of relay mirrors, and the lens interact with the incident light to form a first focused light portion and a second focused light portion. The first focused light portion forms a first image portion of the scene on the first light-sensitive area and the second focused light portion forms a second image portion of the scene on the second light-sensitive area.

Abstract: Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.

Abstract: Technologies for steering sensors in a payload carrier structure on an autonomous vehicle (AV) are described herein. An example method can include receiving, by a motor control system on the AV, instructions for controlling a motor on the motor control system to reposition the payload carrier structure from a first position to a second position; based on the instructions, sending, by a controller on the motor control system to a motor driver on the motor control system, a command instructing the motor driver to reposition the payload carrier structure from the first position to the second position; sending, by the motor driver to the motor, a control signal generated by the motor driver based on the command, the control signal controlling the motor to reposition the payload carrier structure to the second position; and moving, by the motor, the payload carrier structure and sensors to the second position.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.

Abstract: Example implementations may relate to use of a light-control feature to control extent of light encountered by an image capture device of a self-driving vehicle. In particular, a computing system of the vehicle may make a determination that quality of image data generated by an image capture device is or is expected to be lower than a threshold quality due to external light encountered or expected to be encountered by the image capture device. In response to the determination, the computing system may make an adjustment to the light-control feature to control the extent of external light encountered or expected to be encountered by the image capture device. This adjustment may ultimately help improve quality of image data generated by the image capture device. As such, the computing system may operate the vehicle based at least on image data generated by the image capture device.

Abstract: The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

Abstract: The present disclosure relates to multiple view optical systems. An example optical system includes at least one primary optical element configured to receive incident light from a scene and a plurality of relay mirrors optically coupled to the at least one primary optical element. The optical system also includes a lens optically coupled to the plurality of relay mirrors, and an image sensor configured to receive focused light from the lens. The image sensor includes a first light-sensitive area and a second light-sensitive area. The primary optical element, the plurality of relay mirrors, and the lens interact with the incident light to form a first focused light portion and a second focused light portion. The first focused light portion forms a first image portion of the scene on the first light-sensitive area and the second focused light portion forms a second image portion of the scene on the second light-sensitive area.