Abstract: One example system includes a first light detection and ranging (LIDAR) device that scans a first field-of-view defined by a first range of pointing directions associated with the first LIDAR device. The system also includes a second LIDAR device that scans a second FOV defined by a second range of pointing directions associated with the second LIDAR device. The second FOV at least partially overlaps the first FOV. The system also includes a first controller that adjusts a first pointing direction of the first LIDAR device. The system also includes a second controller that adjusts a second pointing direction of the second LIDAR device synchronously with the adjustment of the first pointing direction of the first LIDAR device.

Abstract: The technology relates to partially redundant equipment architectures for vehicles able to operate in an autonomous driving mode. Aspects of the technology employ fallback configurations, such as two or more fallback sensor configurations that provide some minimum amount of field of view (FOV) around the vehicle. For instance, different sensor arrangements are logically associated with different operating domains of the vehicle. Fallback configurations for computing resources and/or power resources are also provided. Each fallback configuration may have different reasons for being triggered, and may result in different types of fallback modes of operation. Triggering conditions may relate, e.g., to a type of failure, fault or other reduction in component capability, the current driving mode, environmental conditions in the vicinity of vehicle or along a planned route, or other factors.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The present disclosure is directed to an autonomous vehicle having a vehicle control system. The vehicle control system includes an image processing system. The image processing system receives an image that includes a plurality of image portions. The image processing system also calculates a score for each image portion. The score indicates a level of confidence that a given image portion represents an illuminated component of a traffic light. The image processing system further identifies one or more candidate portions from among the plurality of image portions. Additionally, the image processing system determines that a particular candidate portion represents an illuminated component of a traffic light using a classifier. Further, the image processing system provides instructions to control the autonomous vehicle based on the particular candidate portion representing an illuminated component of a traffic light.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ling-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The disclosure relates to assessing operation of two or more cameras. These cameras may be a group of cameras of a perception system of a vehicle having an autonomous driving mode. A first image captured by a first camera and a second image captured by a second camera may be received. A first feature vector for the first image and a second feature vector for the second image may be generated. A similarity score may be determined using the first feature vector and the second feature vector. This similarity score may be used to assess the operation of the two cameras and an appropriate action may be taken.

Abstract: Aspects of the disclosure relate to determining and responding to an internal state of a self-driving vehicle. For instance, an image of an interior of the vehicle captured by a camera mounted in the vehicle is received. The image is processed in order to identify one or more visible markers at predetermined locations within the vehicle. The internal state of the vehicle is determined based on the identified one or more visible markers. A responsive action is identified action using the determined internal state, and the vehicle is controlled in order to perform the responsive action.

Abstract: One example system includes a first light detection and ranging (LIDAR) device that scans a first field-of-view defined by a first range of pointing directions associated with the first LIDAR device. The system also includes a second LIDAR device that scans a second FOV defined by a second range of pointing directions associated with the second LIDAR device. The second FOV at least partially overlaps the first FOV. The system also includes a first controller that adjusts a first pointing direction of the first LIDAR device. The system also includes a second controller that adjusts a second pointing direction of the second LIDAR device synchronously with the adjustment of the first pointing direction of the first LIDAR device.

Abstract: The present disclosure relates to a target, a method, and a system for calibrating a camera. One example embodiment includes a target. The target includes a first pattern of fiducial markers. The target also includes a second pattern of fiducial markers. The first pattern of fiducial markers is a scaled version of the second pattern of fiducial markers, such that a calibration image captured of the target simulates multiple images of a single pattern captured at multiple calibration perspectives.

Abstract: Example implementations may relate to sun-aware vehicle routing. In particular, a computing system of a vehicle may determine an expected position of the sun relative to a geographic area. Based on the expected position, the computing system may make a determination that travel of the vehicle through certain location(s) within the geographic area is expected to result in the sun being proximate to an object within a field of view of the vehicle's image capture device. Responsively, the computing system may generate a route for the vehicle in the geographic area based at least on the route avoiding travel of the vehicle through these certain location(s), and may then operate the vehicle to travel in accordance with the generated route. Ultimately, this may help reduce or prevent situations where quality of image(s) degrades due to sunlight, which may allow for use of these image(s) as basis for operating the vehicle.

Abstract: Examples described may related to an imaging sensor used by a vehicle, including a light sensor. The light sensor comprises a plurality of cells aligned in a plurality of horizontal rows and a plurality of vertical columns. The apparatus further includes an optical system configured to provide the light sensor with a field of view of an external environment of the apparatus. Additionally, the system includes a processing unit configured to: divide the plurality of horizontal rows of the light sensor into one or more enabled rows and one or more disabled rows; obtain image data from the light sensor by sampling one or more cells in the one or more enabled rows; and store the received image data in a memory.

Abstract: One example system comprises a LIDAR sensor that rotates about an axis to scan an environment of the LIDAR sensor. The system also comprises one or more cameras that detect external light originating from one or more external light sources. The one or more cameras together provide a plurality of rows of sensing elements. The rows of sensing elements are aligned with the axis of rotation of the LIDAR sensor. The system also comprises a controller that operates the one or more cameras to obtain a sequence of image pixel rows. A first image pixel row in the sequence is indicative of external light detected by a first row of sensing elements during a first exposure time period. A second image pixel row in the sequence is indicative of external light detected by a second row of sensing elements during a second exposure time period.

Abstract: Methods and system are provided for training and using a model to determine states of lanes of interest. For instance, image data including an image and an associated label identifying at least one traffic light, a state of the at least one traffic light, and a lane controlled by the at least one traffic light are received and used to train the model such that the model is configured to, in response to receiving an image and a lane of interest included in the image, output a lane state for the lane of interest. This model is then used by a vehicle in order to determine a state of a lane of interest. This state is then used to control the vehicle in an autonomous driving mode based on the state of the lane of interest.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ling-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: Aspects of the disclosure relate to determining and responding to an internal state of a self-driving vehicle. For instance, an image of an interior of the vehicle captured by a camera mounted in the vehicle is received. The image is processed in order to identify one or more visible markers at predetermined locations within the vehicle. The internal state of the vehicle is determined based on the identified one or more visible markers. A responsive action is identified action using the determined internal state, and the vehicle is controlled in order to perform the responsive action.

Abstract: A sensor assembly for autonomous vehicles includes a side mirror assembly configured to mount to a vehicle. The side mirror assembly includes a first camera having a field of view in a direction opposite a direction of forward travel of the vehicle; a second camera having a field of view in the direction of forward travel of the vehicle; and a third camera having a field of view in a direction substantially perpendicular to the direction of forward travel of the vehicle. The first camera, the second camera, and the third camera are oriented to provide, in combination with a fourth camera configured to be mounted on a roof of the vehicle, an uninterrupted camera field of view from the direction of forward travel of the vehicle to a direction opposite the direction of forward travel of the vehicle.

Abstract: A sensor assembly for autonomous vehicles includes a side minor assembly configured to mount to a vehicle. The side mirror assembly includes a first camera having a field of view in a direction opposite a direction of forward travel of the vehicle; a second camera having a field of view in the direction of forward travel of the vehicle; and a third camera having a field of view in a direction substantially perpendicular to the direction of forward travel of the vehicle. The first camera, the second camera, and the third camera are oriented to provide, in combination with a fourth camera configured to be mounted on a roof of the vehicle, an uninterrupted camera field of view from the direction of forward travel of the vehicle to a direction opposite the direction of forward travel of the vehicle.

Abstract: A sensor assembly for autonomous vehicles includes a side minor assembly configured to mount to a vehicle. The side mirror assembly includes a first camera having a field of view in a direction opposite a direction of forward travel of the vehicle; a second camera having a field of view in the direction of forward travel of the vehicle; and a third camera having a field of view in a direction substantially perpendicular to the direction of forward travel of the vehicle. The first camera, the second camera, and the third camera are oriented to provide, in combination with a fourth camera configured to be mounted on a roof of the vehicle, an uninterrupted camera field of view from the direction of forward travel of the vehicle to a direction opposite the direction of forward travel of the vehicle.

Abstract: A sensor assembly for autonomous vehicles includes a side mirror assembly configured to mount to a vehicle. The side mirror assembly includes a first camera having a field of view in a direction opposite a direction of forward travel of the vehicle; a second camera having a field of view in the direction of forward travel of the vehicle; and a third camera having a field of view in a direction substantially perpendicular to the direction of forward travel of the vehicle. The first camera, the second camera, and the third camera are oriented to provide, in combination with a fourth camera configured to be mounted on a roof of the vehicle, an uninterrupted camera field of view from the direction of forward travel of the vehicle to a direction opposite the direction of forward travel of the vehicle.