Abstract: Systems and techniques for determining and predicting the operational capacity of a fleet of vehicles are described. Data may be collected for various fleet parameters (e.g., based on vehicle and fleet operations) and distributions of such data may be used to determine rates of change for the parameters. Predicted distributions of the fleet parameters may be determined based on historical parameter distributions and/or predicted distributions and the corresponding rates of change. Fleet operational capacity determinations may be made based on these parameter distributions, including probability distributions of predicted fleet operational capacity measurements.

Abstract: Techniques for determining a metrics associated with a vehicle are disclosed. The techniques may comprise receiving a driving simulation scenario associated with a simulated vehicle. Based at least in part on the scenario and a dataset of recorded driving associated with operation of a test vehicle, a plurality of driving events present in the dataset may be determined. Based at least in part on the plurality of driving events, a first metric associated with a likelihood of occurrence of the driving simulation scenario may be determined. A simulation may be instantiated based at least in part on the driving simulation scenario. Based at least in part on the simulation, a safety metric indicative of the safe performance of the simulated vehicle in the simulation may be determined. An aggregate safety metric may be determined based at least in part on the safety metric and the first metric.

Abstract: Techniques for generating simulations to evaluate an update to a controller. The controller may be configured to control one or more functionalities of an autonomous and/or a semi-autonomous vehicle. A simulation computing system may receive a request to evaluate a first controller. The simulation computing system may generate a simulation based on data associated with a previous operation of the vehicle in an environment, the previous operation being controlled by a second controller (e.g., standard for evaluation, control version, etc.). The simulation computing device may cause the first controller to control a simulated vehicle in the simulation and may determine whether to validate the update to the controller based on a difference between first metrics associated with a control of the simulated vehicle by the first controller and second metrics associated with a control of the vehicle by the second controller.