Abstract: Example implementations may relate to sun-aware vehicle routing. In particular, a computing system of a vehicle may determine an expected position of the sun relative to a geographic area. Based on the expected position, the computing system may make a determination that travel of the vehicle through certain location(s) within the geographic area is expected to result in the sun being proximate to an object within a field of view of the vehicle's image capture device. Responsively, the computing system may generate a route for the vehicle in the geographic area based at least on the route avoiding travel of the vehicle through these certain location(s), and may then operate the vehicle to travel in accordance with the generated route. Ultimately, this may help reduce or prevent situations where quality of image(s) degrades due to sunlight, which may allow for use of these image(s) as basis for operating the vehicle.

Abstract: Aspects of the disclosure relate to determining and responding to an internal state of a self-driving vehicle. For instance, an image of an interior of the vehicle captured by a camera mounted in the vehicle is received. The image is processed in order to identify one or more visible markers at predetermined locations within the vehicle. The internal state of the vehicle is determined based on the identified one or more visible markers. A responsive action is identified action using the determined internal state, and the vehicle is controlled in order to perform the responsive action.

Abstract: The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle's environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle's environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.

Abstract: One example system comprises a LIDAR sensor that rotates about an axis to scan an environment of the LIDAR sensor. The system also comprises one or more cameras that detect external light originating from one or more external light sources. The one or more cameras together provide a plurality of rows of sensing elements. The rows of sensing elements are aligned with the axis of rotation of the LIDAR sensor. The system also comprises a controller that operates the one or more cameras to obtain a sequence of image pixel rows. A first image pixel row in the sequence is indicative of external light detected by a first row of sensing elements during a first exposure time period. A second image pixel row in the sequence is indicative of external light detected by a second row of sensing elements during a second exposure time period.

Abstract: Aspects of the disclosure relate to determining and responding to an internal state of a self-driving vehicle. For instance, an image of an interior of the vehicle captured by a camera mounted in the vehicle is received. The image is processed in order to identify one or more visible markers at predetermined locations within the vehicle. The internal state of the vehicle is determined based on the identified one or more visible markers. A responsive action is identified action using the determined internal state, and the vehicle is controlled in order to perform the responsive action.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle's environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle's environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: The technology relates to partially redundant equipment architectures for vehicles able to operate in an autonomous driving mode. Aspects of the technology employ fallback configurations, such as two or more fallback sensor configurations that provide some minimum amount of field of view (FOV) around the vehicle. For instance, different sensor arrangements are logically associated with different operating domains of the vehicle. Fallback configurations for computing resources and/or power resources are also provided. Each fallback configuration may have different reasons for being triggered, and may result in different types of fallback modes of operation. Triggering conditions may relate, e.g., to a type of failure, fault or other reduction in component capability, the current driving mode, environmental conditions in the vicinity of vehicle or along a planned route, or other factors.

Abstract: Example implementations may relate to use of a light-control feature to control extent of light encountered by an image capture device of a self-driving vehicle. In particular, a computing system of the vehicle may make a determination that quality of image data generated by an image capture device is or is expected to be lower than a threshold quality due to external light encountered or expected to be encountered by the image capture device. In response to the determination, the computing system may make an adjustment to the light-control feature to control the extent of external light encountered or expected to be encountered by the image capture device. This adjustment may ultimately help improve quality of image data generated by the image capture device. As such, the computing system may operate the vehicle based at least on image data generated by the image capture device.

Abstract: One example system includes a first light detection and ranging (LIDAR) device that scans a first field-of-view defined by a first range of pointing directions associated with the first LIDAR device. The system also includes a second LIDAR device that scans a second FOV defined by a second range of pointing directions associated with the second LIDAR device. The second FOV at least partially overlaps the first FOV. The system also includes a first controller that adjusts a first pointing direction of the first LIDAR device. The system also includes a second controller that adjusts a second pointing direction of the second LIDAR device synchronously with the adjustment of the first pointing direction of the first LIDAR device.

Abstract: Aspects of the disclosure relate to training and using a model for determine states of lanes of interest. For instance, image data including an image and an associated label identifying at least one traffic light, a state of the at least one traffic light, and a lane controlled by the at least one traffic light may be received and used to train the mode such that the model is configured to, in response to receiving an image and a lane of interest included in the image, output a lane state for the lane of interest. This model may then be used by a vehicle in order to determine a state of a lane of interest. This state may then be used to control the vehicle in an autonomous driving mode based on the state of the lane of interest.

Abstract: The disclosure relates to assessing operation of two or more cameras. These cameras may be a group of cameras of a perception system of a vehicle having an autonomous driving mode. A first image captured by a first camera and a second image captured by a second camera may be received. A first feature vector for the first image and a second feature vector for the second image may be generated. A similarity score may be determined using the first feature vector and the second feature vector. This similarity score may be used to assess the operation of the two cameras and an appropriate action may be taken.

Abstract: Example implementations may relate to use of a light-control feature to control extent of light encountered by an image capture device of a self-driving vehicle. In particular, a computing system of the vehicle may make a determination that quality of image data generated by an image capture device is or is expected to be lower than a threshold quality due to external light encountered or expected to be encountered by the image capture device. In response to the determination, the computing system may make an adjustment to the light-control feature to control the extent of external light encountered or expected to be encountered by the image capture device. This adjustment may ultimately help improve quality of image data generated by the image capture device. As such, the computing system may operate the vehicle based at least on image data generated by the image capture device.

Abstract: The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle's environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle's environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.

Abstract: One example system comprises a LIDAR sensor that rotates about an axis to scan an environment of the LIDAR sensor. The system also comprises one or more cameras that detect external light originating from one or more external light sources. The one or more cameras together provide a plurality of rows of sensing elements. The rows of sensing elements are aligned with the axis of rotation of the LIDAR sensor. The system also comprises a controller that operates the one or more cameras to obtain a sequence of image pixel rows. A first image pixel row in the sequence is indicative of external light detected by a first row of sensing elements during a first exposure time period. A second image pixel row in the sequence is indicative of external light detected by a second row of sensing elements during a second exposure time period.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: The present disclosure relates to a target, a method, and a system for calibrating a camera. One example embodiment includes a target. The target includes a first pattern of fiducial markers. The target also includes a second pattern of fiducial markers. The first pattern of fiducial markers is a scaled version of the second pattern of fiducial markers, such that a calibration image captured of the target simulates multiple images of a single pattern captured at multiple calibration perspectives.

Abstract: Aspects of the disclosure relate to determining whether a vehicle should continue through an intersection. For example, the one or more of the vehicle's computers may identify a time when the traffic signal light will turn from yellow to red. The one or more computers may also estimate a location of a vehicle at the time when the traffic signal light will turn from yellow to red. A starting point of the intersection may be identified. Based on whether the estimated location of the vehicle is at least a threshold distance past the starting point at the time when the traffic signal light will turn from yellow to red, the computers can determine whether the vehicle should continue through the intersection.

Abstract: The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle's environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle's environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.