Abstract: A method and apparatus are provided for determining one or more behavior models used by an autonomous vehicle to predict the behavior of detected objects. The autonomous vehicle may collect and record object behavior using one or more sensors. The autonomous vehicle may then communicate the recorded object behavior to a server operative to determine the behavior models. The server may determine the behavior models according to a given object classification, actions of interest performed by the object, and the object's perceived surroundings.

Abstract: A first driving solution for a vehicle along a portion of a route is determined based on an agent detected in the vehicle's environment following a first trajectory of a plurality of possible trajectories. A switching time is determined for the vehicle to deviate from the first driving solution for a situation in which the agent is following a second trajectory of the plurality of possible trajectories. The first driving solution is revised such that the vehicle will be able to switch from the revised first driving solution to another driving solution at the switching time in case if the detected agent is following the second trajectory.

Abstract: Aspects of the present disclosure relate to using audible cues to guide a passenger to a vehicle having an autonomous driving mode. For instance, one or more processors of the vehicle may receive, from a server computing device, instructions to pick up the passenger at a pickup location. The one or more processors may maneuver the vehicle towards the pickup location in the autonomous driving mode. The one or more processors may receive a signal indicating that the passenger requests assistance locating the vehicle. The one or more processors may use the signal to generate the audible cues. The audible cues may be played by the one or more processors through a speaker of the vehicle in order to guide the passenger towards the vehicle.

Abstract: Example embodiments relate to multiple operating modes to expand dynamic range. An example embodiment includes a camera system. The camera system may include a first image sensor having a first dynamic range corresponding to a first range of luminance levels in a scene. The system may also include a second image sensor having a second dynamic range corresponding to a second range of luminance levels in the scene. The camera system may further include a processor coupled to the first image sensor and the second image sensor. The processor may be configured to execute instructions to identify objects of a first type in a first image of the scene captured by the first image sensor and identify objects of a second object type in a second image of the scene captured by the second image sensor.

Abstract: A light detection and ranging (LIDAR) device scans through a scanning zone while emitting light pulses and receives reflected signals corresponding to the light pulses. The LIDAR device scans the emitted light pulses through the scanning zone by reflecting the light pulses from an array of oscillating mirrors. The mirrors are operated by a set of electromagnets arranged to apply torque on the mirrors, and an orientation feedback system senses the orientations of the mirrors. Driving parameters for each mirror are determined based on information from the orientation feedback system. The driving parameters can be used to drive the mirrors in phase at an operating frequency despite variations in moments of inertia and resonant frequencies among the mirrors.

Abstract: Aspects of the disclosure relate to providing an end of trip sequence when a vehicle is nearing its destination. One or more computing devices may generate and display a video indicating a projected trajectory of the vehicle and objects detected by sensors on the vehicle, on a map corresponding to a route the vehicle is currently following, where the video is generated from a perspective of a virtual camera at a default position and default pitch A determination that the vehicle has reached a threshold relative to the route of the vehicle may be made and the position and pitch of the virtual camera may be adjusted to an updated position above the vehicle and a perspective which looks downwards towards a roof of the vehicle. The video may then be generated and displayed from the perspective of the virtual camera at the updated position.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining an alignment between cross-modal sensor data. In one aspect, a method comprises: obtaining (i) an image that characterizes a visual appearance of an environment, and (ii) a point cloud comprising a collection of data points that characterizes a three-dimensional geometry of the environment; processing each of a plurality of regions of the image using a visual embedding neural network to generate a respective embedding of each of the image regions; processing each of a plurality of regions of the point cloud using a shape embedding neural network to generate a respective embedding of each of the point cloud regions; and identifying a plurality of region pairs using the embeddings of the image regions and the embeddings of the point cloud regions.

Abstract: Aspects of the disclosure relate providing a lane change notification when a vehicle is to perform a lane change. One or more computing devices may generate and display a video, where the video is generated from a perspective of a virtual camera at a default position and default pitch. The one or more computing devices may receive an indication that the vehicle is to perform a lane change from a first lane to a second lane and adjust, after the vehicle receives the indication, the default position and default pitch of the virtual camera to an updated position further above the vehicle relative to ground than the default position and an updated pitch directed more towards the ground than the default pitch. The video may be generated and displayed from the perspective of the virtual camera at the updated position and updated pitch.

Abstract: Models can be generated of a vehicle's view of its environment and used to maneuver the vehicle. This view need not include what objects or features the vehicle is actually seeing, but rather those areas that the vehicle is able to observe using its sensors if the sensors were completely un-occluded. For example, for each of a plurality of sensors of the object detection component, a computer may generate an individual 3D model of that sensor's field of view. Weather information is received and used to adjust one or more of the models. After this adjusting, the models may be aggregated into a comprehensive 3D model. The comprehensive model may be combined with detailed map information indicating the probability of detecting objects at different locations. The model of the vehicle's environment may be computed based on the combined comprehensive 3D model and detailed map information.

Abstract: Aspects of the technology relate to assisting a passenger in an autonomous vehicle without a driver. For instance, after a door of the vehicle is opened, a predetermined period of time may be waited by processors of computing devices of the vehicle. After waiting the predetermined period of time and when the vehicle's door remains open, a set of instructions for closing the vehicle's door may be played by the processors through a speaker of the vehicle. Once the door of the vehicle is closed, an announcement may be played by the processors through the speaker requesting that the passenger press a first button to initiate a ride to a destination. In response to the first button being pressed, the ride to the destination may be initiated by the processors by maneuvering the vehicle autonomously to the destination.

Abstract: A rotatable LIDAR device including contactless electrical couplings is disclosed. An example rotatable LIDAR device includes a vehicle electrical coupling including (i) a first conductive ring, (ii) a second conductive ring, and (iii) a first coil. The example rotatable LIDAR device further includes a LIDAR electrical coupling including (i) a third conductive ring, (ii) a fourth conductive ring, and (iii) a second coil. The example rotatable LIDAR device still further includes a rotatable LIDAR electrically coupled to the LIDAR electrical coupling. The first conductive ring and the third conductive ring form a first capacitor configured to transmit communications to the rotatable LIDAR, the second conductive ring and the fourth conductive ring form a second capacitor configured to transmit communications from the rotatable LIDAR, and the first coil and the second coil form a transformer configured to provide power to the rotatable LIDAR.

Abstract: Methods and systems for the use of detected objects for image processing are described. A computing device autonomously controlling a vehicle may receive images of the environment surrounding the vehicle from an image-capture device coupled to the vehicle. In order to process the images, the computing device may receive information indicating characteristics of objects in the images from one or more sources coupled to the vehicle. Examples of sources may include RADAR, LIDAR, a map, sensors, a global positioning system (GPS), or other cameras. The computing device may use the information indicating characteristics of the objects to process received images, including determining the approximate locations of objects within the images. Further, while processing the image, the computing device may use information from sources to determine portions of the image to focus upon that may allow the computing device to determine a control strategy based on portions of the image.

Abstract: Aspects of the disclosure relate to validating map data using challenge questions. For instance, an attributes to be validated may be identified from the map data. At least one challenge question may be selected from a plurality of predetermined challenge questions based on the attribute. An image may be retrieved based on image information associated with the at least one challenge question. The image and the at least one challenge question may be provided for display. In response to the providing, operator input identifying an answer to the at least one challenge question may be received. This answer may be then used to validate the attribute.

Abstract: The technology relates to autonomous vehicles that use a perception system to detect objects and features in the vehicle's surroundings. A camera assembly having a ring-type structure is provided that gives the perception system an overall 360° field of view around the vehicle. Image sensors are arranged in camera modules around the assembly to provide a seamless panoramic field of view. One subsystem has multiple pairs of image sensors positioned to provide the overall 360° field of view, while another subsystem provides a set of image sensors generally facing toward the front of the vehicle to provide enhanced object identification. The camera assembly may be arranged in a housing located on top of the vehicle. The housing may include other sensors such as LIDAR and radar. The assembly includes a chassis and top and base plates, which may provide EMI protection from other sensors disposed in the housing.

Abstract: A communication system is disclosed. The system may include an antenna unit having a port. The system may also include a communication chip communicably coupled to the antenna unit and having an antenna configured to transmit electromagnetic signals into the port. In addition, the system may include a slotted structure configured for receiving the electromagnetic signals from the antenna and coupling the electromagnetic signals from the antenna into the port.

Abstract: The present disclosure relates to systems and methods that utilize machine learning techniques to improve object classification in thermal imaging systems. In an example embodiment, a method is provided. The method includes receiving, at a computing device, one or more infrared images of an environment. The method additionally includes, applying, using the computing device, a trained machine learning system on the one or more infrared images to determine an identified object type in the environment by at least: determining one or more prior thermal maps associated with the environment; using the one or more prior thermal maps and the one or more infrared images, determining a current thermal map associated with the environment; and determining the identified object type based on the current thermal map. The method also includes providing the identified object type using the computing device.