Abstract: Systems, methods, and non-transitory computer-readable media can receive transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location. A first route associated with the transportation request and a non-autonomous vehicle can be determined. A second route associated with the transportation request and an autonomous vehicle can be determined based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles. At least one performance metric associated with the second route can be determined. The second route can be selected based at least in part on the at least one performance metric and a comparison of the first route and the second route. An autonomous vehicle from the fleet of vehicles can be assigned to the transportation request based on selection of the second route.

Abstract: Systems, methods, and non-transitory computer-readable media can access a plurality of parameter-based encodings providing a structured representation of an environment captured by one or more sensors associated with a plurality of vehicles traveling through the environment. A given parameter-based encoding of the environment identifies one or more agents that were detected by a vehicle within the environment and respective location information for the one or more agents within the environment. The plurality of parameter-based encodings can be clustered into one or more clusters of parameter-based encodings. At least one scenario associated with the environment can be determined based at least in part on the one or more clusters of parameter-based encodings.

Abstract: Systems, methods, and non-transitory computer-readable media can access a plurality of parameter-based encodings providing a structured representation of an environment captured by one or more sensors associated with a plurality of vehicles traveling through the environment. A given parameter-based encoding of the environment identifies one or more agents that were detected by a vehicle within the environment and respective location information for the one or more agents within the environment. The plurality of parameter-based encodings can be clustered into one or more clusters of parameter-based encodings. At least one scenario associated with the environment can be determined based at least in part on the one or more clusters of parameter-based encodings.

Abstract: In particular embodiments, a computing system may determine a measured driving characteristic of a driving control system based on observations of vehicles driven by the driving control system. The system may determine a difference between the measured driving characteristic and a target driving characteristic, which is based on objectivations of one or more manually controlled vehicles. The system may determine an evaluation objective for the driving control system. The system may determine a weight function for the evaluation objective. The system may determine a score for the driving control system with respect to the evaluation objective by weighting the difference between the measured driving characteristic and the target driving characteristic using the weight function. The system may apply, based on the score, an adjustment to the driving control system to reduce a difference between a subsequently measured driving characteristic of the driving control system and the target driving characteristic.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for determining data collection values for enhanced sensor provider vehicles in a provider vehicle pool and utilizing the data collection values to select and dispatch a provider vehicle for a transportation request. In particular, in one or more embodiments, the disclosed systems determine data collection values based on various route features for a particular enhanced sensor provider vehicle. Additionally, in one or more embodiments, the disclosed systems can utilize the data collection values in conjunction with a device matching algorithm and transportation values to select and dispatch a provider vehicle. Further, in some embodiments, the disclosed systems utilize collected sensory data to train one or more computer implemented machine learning models, such as an autonomous vehicle driving model.

Abstract: In one embodiment, a method includes receiving parameter data associated with trajectories captured using at least one sensor associated with one or more vehicles, determining a parameter distribution associated with movement parameters for the parameter data based on the trajectories, adjusting the parameter distribution for the parameter data associated with the movement parameters up to a respective threshold constraint, wherein the respective threshold constraint is based on at least one of the parameter data, generating driving simulations that are based on the adjusted parameter distribution for the parameter data, and evaluating an autonomous-driving system using the driving simulations.

Abstract: Systems, methods, and non-transitory computer-readable media can determine mission data associated with a scenario encountered during operation of a vehicle. A first evaluation of the scenario can be determined by evaluating the mission data using a simulation behavior model based on simulated driving data. A second evaluation of the scenario can be determined by evaluating the mission data using an observed behavior model based on observed driving data. Vehicle performance of an autonomy system of the vehicle can be evaluated based on the first evaluation and the second evaluation.

Abstract: Systems, methods, and non-transitory computer-readable media can receive a search query including one or more high-level primitives. One or more low-level parameters describing behavior of at least one agent associated with at least one value that satisfies at least one annotation rule associated with the one or more high-level primitives can be determined. In response to determining that the at least one value satisfies the at least one annotation rule, one or more scenarios associated with the one or more low-level parameters that satisfy the at least one annotation rule can be identified by using the one or more high-level primitives included in the search query. Information describing the one or more identified scenarios in response to the search query can be provided.

Abstract: Systems, methods, and non-transitory computer-readable media can receive disengagement information associated with one or more autonomous vehicles, the disengagement information identifying a plurality of disengagements of an autonomy system during operation of the one or more autonomous vehicles. Each disengagement of the plurality of disengagements can be categorized based on a plurality of categories, wherein a first category of the plurality of categories is associated with disengagement that would not have led to a negative outcome. A performance metric associated with the one or more autonomous vehicles can be determined based on the categorizing each disengagement of the plurality of disengagements. Autonomous vehicle performance of the one or more autonomous vehicles can be evaluated based on the performance metric.

Abstract: Systems, methods, and non-transitory computer-readable media can determine sensor data captured by at least one sensor of a vehicle while navigating an environment over a period of time. Information describing one or more agents associated with the environment during the period of time can be determined based at least in part on the captured sensor data. A parameter-based encoding describing the one or more agents associated with the environment during the period of time can be generated based at least in part on the determined information and a scenario schema, wherein the parameter-based encoding provides a structured representation of the information describing the one or more agents associated with the environment. A scenario represented by the parameter-based encoding can be determined based at least in part on a cluster of parameter-based encodings to which the parameter-based encoding is assigned.

Abstract: Systems, methods, and non-transitory computer-readable media can access a plurality of parameter-based encodings providing a structured representation of an environment captured by one or more sensors associated with a plurality of vehicles traveling through the environment. A given parameter-based encoding of the environment identifies one or more agents that were detected by a vehicle within the environment and respective location information for the one or more agents within the environment. The plurality of parameter-based encodings can be clustered into one or more clusters of parameter-based encodings. At least one scenario associated with the environment can be determined based at least in part on the one or more clusters of parameter-based encodings.

Abstract: In particular embodiments, a computing system may determine a measured driving characteristic of a driving control system based on observations of vehicles driven by the driving control system. The system may determine a difference between the measured driving characteristic and a target driving characteristic, which is based on objectivations of one or more manually controlled vehicles. The system may determine an evaluation objective for the driving control system. The system may determine a weight function for the evaluation objective. The system may determine a score for the driving control system with respect to the evaluation objective by weighting the difference between the measured driving characteristic and the target driving characteristic using the weight function. The system may apply, based on the score, an adjustment to the driving control system to reduce a difference between a subsequently measured driving characteristic of the driving control system and the target driving characteristic.

Abstract: Systems, methods, and non-transitory computer-readable media can receive transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location. A first route associated with the transportation request and a non-autonomous vehicle can be determined. A second route associated with the transportation request and an autonomous vehicle can be determined based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles. At least one performance metric associated with the second route can be determined. The second route can be selected based at least in part on the at least one performance metric and a comparison of the first route and the second route. An autonomous vehicle from the fleet of vehicles can be assigned to the transportation request based on selection of the second route.

Abstract: Systems, methods, and non-transitory computer-readable media can determine a first utility metric associated with a region and first autonomous vehicle eligibility criteria, wherein the first utility metric is determined based on a first plurality of rides and a subset of the first plurality of rides that can be successfully executed within the region based on the first autonomous vehicle eligibility criteria. A second utility metric associated with the region and second autonomous vehicle eligibility criteria can be determined, wherein the second utility metric is determined based on a second plurality of rides and a subset of the second plurality of rides that can be successfully executed within the region based on the second autonomous vehicle eligibility criteria. An autonomous vehicle associated with the first autonomous vehicle eligibility criteria can be selected to drive in the region based on a comparison of the first utility metric and the second utility metric.

Abstract: Systems, methods, and non-transitory computer-readable media can receive disengagement information associated with one or more autonomous vehicles, the disengagement information identifying a plurality of disengagements of an autonomy system during operation of the one or more autonomous vehicles. Each disengagement of the plurality of disengagements can be categorized based on a plurality of categories, wherein a first category of the plurality of categories is associated with disengagement that would not have led to a negative outcome. A performance metric associated with the one or more autonomous vehicles can be determined based on the categorizing each disengagement of the plurality of disengagements. Autonomous vehicle performance of the one or more autonomous vehicles can be evaluated based on the performance metric.

Abstract: In one embodiment, a method includes accessing a first set of driving metrics of a vehicle navigating a segment using a driving control system; computing, based on the first set of driving metrics, a driving characteristic for the vehicle. The driving characteristic of the vehicle being represented by the first set of driving metrics. The method also includes comparing the driving characteristic of the vehicle to a driving characteristic of a number of manually controlled vehicles represented by a second set of driving metrics collected while navigating the segment; and computing a score indicating an elegance of the driving control system in navigating the segment based on the comparison of the driving characteristic of the vehicle to the driving characteristic of the number of manually controlled vehicles.

Abstract: The present disclosure describes system and methods for network planning. The systems and methods can incorporate network traffic demands, availability requirements, latency, physical infrastructure and networking device capability, and detailed cost structures to calculate a network design with minimum or reduced cost compared to conventional methods. In some implementations, the method include providing an initial, deterministic set of failures, and then successively performing a network optimization and a network availability simulation to determine which failures most impact the performance of the network model. The high impact failures can then be provided back into the system, which generates an improved network design while still maintaining minimum cost.

Abstract: The present disclosure provides a probabilistic framework that can calculate the probability of fulfilling demands for a given set of traffic flows. In some implementations, the probability of fulfilling demands can be based on the probability of infrastructure component failures, shared risk link groups derived from a cross-layer network topology, and traffic engineering (TE) considerations. The consideration of the cross-layer network topology enables the systems and methods described herein to account for the relationship between the physical and logical topologies.

Abstract: The present disclosure describes system and methods for network planning. The systems and methods can incorporate network traffic demands, availability requirements, latency, physical infrastructure and networking device capability, and detailed cost structures to calculate a network design with minimum or reduced cost compared to conventional methods. In some implementations, the method include providing an initial, deterministic set of failures, and then successively performing a network optimization and a network availability simulation to determine which failures most impact the performance of the network model. The high impact failures can then be provided back into the system, which generates an improved network design while still maintaining minimum cost.

Abstract: The present disclosure provides a probabilistic framework that can calculate the probability of fulfilling demands for a given set of traffic flows. In some implementations, the probability of fulfilling demands can be based on the probability of infrastructure component failures, shared risk link groups derived from a cross-layer network topology, and traffic engineering (TE) considerations. The consideration of the cross-layer network topology enables the systems and methods described herein to account for the relationship between the physical and logical topologies.