Abstract: Systems and methods for determining a passenger pick-up/drop-off (PUDO) state of a mass transit vehicle by an autonomous vehicle (AV) are disclosed. A system includes a memory one or more processing devices, coupled to the memory, configured to perform operations that include identifying a mass transit vehicle in an environment of an AV, responsive to determining the mass transit vehicle is in a PUDO state, providing movement instructions to a planning system of the AV, and autonomously modifying operation of the AV based on the movement instructions.

Abstract: The disclosed systems and techniques facilitate efficient detection and navigation of blocked lanes in driving environments. The disclosed techniques include obtaining sensing data associated with a driving environment and identifying obstruction marker(s) associated with the driving environment based on the sensing data. The techniques further include obtaining a first determination whether an object, represented in the sensing data, is obstructing traffic, the first determination based on the obstruction marker(s). The techniques further include obtaining a second determination whether the object is obstructing traffic by applying a machine learning model to an input that includes at least a portion of the sensing data. The techniques further include identifying blocked lane(s) using the obtained determinations and modifying, in view of the blocked lane(s), a driving path of the vehicle.

Abstract: The described aspects and implementations use artificial intelligence (AI) to detect passengers in a vehicle. A method of an implementation includes obtaining one or more images captured by one or more cameras of a vehicle. The method includes generating, using one or more artificial intelligence models and the one or more images, passenger data indicating locations of one or more passengers of the vehicle. The method includes generating, based on the passenger data, vehicle area data indicating one or more areas of the vehicle at which the one or more passengers are located. The method includes determining, based on the passenger data and the vehicle area data, whether at least one passenger seating configuration criterion is satisfied. The method includes, responsive to determining that such a criterion is satisfied, causing the vehicle to perform an action associated with a passenger seating configuration in the vehicle.

Abstract: Example embodiments relate to techniques for using visual language models (VLMs) to provide assistance to vehicles that encounter unexpected issues during navigation. A vehicle computing system may use sensor data to detect an unexpected issue impeding the vehicle from autonomously navigating a path and generate a question based on the unexpected issue. The computing system can then provide the question and a portion of the sensor data used to detect the unexpected issue to a remote computing device. The remote computing device inputs the question and the portion of the sensor data into a VLM trained to answer the question using the portion of the sensor data. The vehicle computing system is able to receive a response from the remote computing system and generate, based on the response, a control strategy for controlling the vehicle.

Abstract: Aspects of the disclosure provide for the identification of parkable areas. In one instance, observations of parked vehicles may be identified from logged data. The observations may be used to determine whether a sub-portion of an edge of a roadgraph corresponds to a parkable area. In some examples, the edge may define a drivable area in the roadgraph. In addition, map information is generated based on the determination of whether the sub-portion of the edge corresponds to the parkable area.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: Aspects of the disclosure provide for the identification of parkable areas. In one instance, observations of parked vehicles may be identified from logged data. The observations may be used to determine whether a sub-portion of an edge of a roadgraph corresponds to a parkable area. In some examples, the edge may define a drivable area in the roadgraph. In addition, map information is generated based on the determination of whether the sub-portion of the edge corresponds to the parkable area.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting the intent of an agent in an environment. One of the methods includes obtaining context data characterizing an environment, the context data comprising data characterizing a plurality of agents in the environment; and generating, based on processing the context data using a neural network having a plurality of network parameters, a respective predicted stationary state classification over a plurality of categories for each of one or more target agents of the plurality of agents in the environment, the categories including one or more categories that indicate that the target agent is a stationary non-participant of traffic in the environment.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: Aspects of the disclosure provide for the identification of parkable areas. In one instance, observations of parked vehicles may be identified from logged data. The observations may be used to determine whether a sub-portion of an edge of a roadgraph corresponds to a parkable area. In some examples, the edge may define a drivable area in the roadgraph. In addition, map information is generated based on the determination of whether the sub-portion of the edge corresponds to the parkable area.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: Aspects of the disclosure relate to determining relevant content in response to a request for information. One or more computing devices may receive a query requesting information regarding a place of interest and identify a first place which satisfies the query associated with a first database and a set of displayable content associated with the first place within a second database. The one or more computing devices may identify a subset of the displayable content for display in response to the query, the identification of the subset of displayable content comprising determining a ranking for each item of displayable content in the set of displayable content according to the query. The subset of displayable content and the first place may be provided for presentation on a client device.

Abstract: The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.

Abstract: Aspects of the disclosure relate to determining relevant content in response to a request for information. One or more computing devices may receive a query requesting information regarding a place of interest and identify a first place which satisfies the query associated with a first database and a set of displayable content associated with the first place within a second database. The one or more computing devices may identify a subset of the displayable content for display in response to the query, the identification of the subset of displayable content comprising determining a ranking for each item of displayable content in the set of displayable content according to the query. The subset of displayable content and the first place may be provided for presentation on a client device.