Abstract: Herein, a “perception statistical performance model” (PSPM) for modelling a perception slice of a runtime stack for an autonomous vehicle or other robotic system may be used e.g. for safety/performance testing. A PSPM is configured to: receive a computed perception ground truth; determine from the perception ground truth, based on a set of learned parameters, a probabilistic perception uncertainty distribution, the parameters learned from a set of actual perception outputs generated using the perception slice to be modelled. The PSPM comprises a time-dependent model such that the perception output sampled at the current time instant depends on at least one of: an earlier one of the perception outputs sampled at a previous time instant, and an earlier one of the perception ground truths computed for a previous time instant.

Abstract: Herein, a “perception statistical performance model” (PSPM) for modelling a perception slice of a runtime stack for an autonomous vehicle or other robotic system may be used e.g. for safety/performance testing. A PSPM is configured to: receive a computed perception ground truth; determine from the perception ground truth, based on a set of learned parameters, a probabilistic perception uncertainty distribution, the parameters learned from a set of actual perception outputs generated using the perception slice to be modelled. The modelled perception slice includes an online error estimator, and the computer system is configured to use the PSPM to obtain a predicted online error estimate for the perception output in response to the perception ground truth. This recognizes that online perception error estimates may, themselves, be subject to error.

Abstract: One aspect herein pertains to a computer-implemented method of predicting agent motion comprises receiving a first observed agent state corresponding to a first time instant; determining a set of agent goals; for each agent goal, planning an agent trajectory based on the agent goal and the first observed agent state; receiving a second observed agent state corresponding to a second time instant later than the first time instant; for each goal, comparing the second observed agent state with the at least one agent trajectory planned for the goal, and thereby computing a likelihood of the goal and/or the planned agent trajectory for the goal. Another aspect pertains to trajectory generation, e.g., within a motion planner.

Abstract: A computer-implemented method of planning a path for a mobile robot in the presence of a set of obstacles comprises: computing for each obstacle a probabilistic obstacle position distribution; computing at least one path-independent function as a combination of the probabilistic obstacle position distributions; and for at least one candidate path, determining an upper bound on the total probability of obstacle collision along that path, by aggregating the path-independent function based on an area defined by the candidate path and a mobile robot shape, wherein the path-independent function is independent of the candidate path.

Abstract: The problem of domain shift error in computer vision models and other perception components is addressed. In a label approximation phase, an approximate label distribution is computed for each input of a target batch using a trained machine learning (ML) perception component. In an online label optimization phase, a modified label distribution is assigned to each input of the target batch, via optimization of an unsupervised loss function that (i) penalizes divergence between the approximate label distribution and the modified label distribution for each input of the target batch (ii) penalizes deviation between the modified label distributions assigned to input pairs of the target batch having similar features.

Abstract: Herein, a “perception statistical performance model” (PSPM) for modelling a perception slice of a runtime stack for an autonomous vehicle or other robotic system may be used e.g. for safety/performance testing. A first PSPM is configured to: receive a computed perception ground truth; determine from the perception ground truth, based on a set of learned parameters, a probabilistic perception uncertainty distribution, the parameters learned from a set of actual perception outputs generated using the perception slice to be modelled, in order to compute a first time series of perception outputs. A second time series of perception outputs is computed using a second PSPM for modelling a second perception slice of the runtime stack, the first PSPM learned from data of a first sensor modality of the perception slice and the time series, and the second PSPM learned independently thereof from data of a second sensor modality of the second perception slice and the second time series.

Abstract: For generating driving scenarios for testing an autonomous vehicle planner in a simulation environment, a scenario model comprises a scenario variable and a distribution associated with the scenario variable. The scenario variable is a road layout variable. Multiple sampled values of the scenario variable are computed based on the distribution associated the scenario variable. Based on the scenario model, multiple driving scenarios are generated for testing an autonomous vehicle planner in a simulation environment, each driving scenario comprising a road layout generated using a sampled value of said multiple sampled values of the scenario variable.

Abstract: A method of annotating known objects in road images captured from a sensor-equipped vehicle, the method implemented in an annotation system and comprising: receiving at the annotation system a road image containing a view of a known object; receiving ego localization data, as computed in a map frame of reference, via localization applied to sensor data captured by the sensor-equipped vehicle, the ego localization data indicating an image capture pose of the road image in the map frame of reference; determining, from a predetermined road map, an object location of the known object in the map frame of reference, the predetermined road map representing a road layout the map frame of reference, wherein the known object is one of: a piece of road structure, and an object on or adjacent a road; computing, in an image plane defined by the image capture pose, an object projection, by projecting an object model of the known object from the object location into the image plane; and storing, in an image database, image

Abstract: A computer system comprising: computer storage configured to store a static road layout; a topological indexing component configured to generate an in-memory topological index of the static road layout, the topological index in the form of a graph of nodes and edges, wherein each node corresponds to a road structure element of the static road layout, and the edges encode topological relationships between the road structure elements; a geometric indexing component configured to generate at least one in-memory geometric index of the static road layout for mapping geometric constraints to road structure elements of the static road layout; and a scenario query engine configured to receive a geometric query, search the geometric index to locate at least one static road element satisfying one or more geometric constraints of the geometric query, and return a descriptor of the at least one road structure element(s), wherein the scenario query engine is configured to receive a topological query comprising a descripto

Abstract: A computer implemented method of generating a scenario to be run in a simulation environment for testing the behaviour of an autonomous vehicle includes rendering, on a display of a computer device, an image of a static scene topology; and rendering on the display an object editing node comprising a set of input fields for receiving user input. The object editing node is for parameterizing an interaction of a challenger object relative to an ego object; and the method includes receiving into the input fields of the object editing node user input defining at least one temporal or relational constraint of the challenger object relative to the ego object. The at least one temporal or relational constraints define an interaction point of a defined interaction stage between the ego object and the challenger object.

Abstract: A computer-implemented method of predicting an external actor trajectory comprises receiving, at a computer, sensor inputs for detecting and tracking an external actor; applying object tracking to the sensor inputs, in order track the external actor, and thereby determine an observed trace of the external actor over a time interval; determining a set of available goals for the external actor; for each of the available goals, determining an expected trajectory model; and comparing the observed trace of the external actor with the expected trajectory model for each of the available goals, to determine a likelihood of that goal.

Abstract: A computer-implemented method of evaluating the performance of a trajectory planner for a mobile robot in a scenario, in which the trajectory planner is used to control the mobile robot responsive to at least one other agent of the scenario the method comprising: determining a scenario parameter set for the scenario and a likelihood of the scenario parameter set; computing an impact score for a failure event or near failure event between the mobile robot and the other agent occurring in the scenario instance, the impact score quantifying severity of the failure event or near failure event; and computing a risk score for the instance of the scenario based on the impact score and the likelihood of the scenario parameter set.

Abstract: Depth information is extracted from a stereoscopic image pait by an image processing system. For each pixel of a target image of the stereoscopic image pair, a final disparity cost vector is computed having cost components corresponding to different disparities. The final disparity cost vector is stored in association with that pixel. That pixel is assigned the disparity corresponding to the lowest cost component of the final disparity cost vector, wherein the extracted depth information comprises the disparities assigned to the pixels of the target image. For at least a subset of the pixels of the target image, the final disparity cost vector is computed for each of those pixels by computing, with respect to the reference image, a set of matching costs for that pixel and the different disparities, and combining the matching costs with the one or more final disparity cost vectors stored in association with one or more of the adjacent pixels of the target image.

Abstract: One aspect herein provides a method of analysing driving behaviour in a data processing computer system, the method comprising: receiving at the data processing computer system driving behaviour data to be analysed, wherein the driving behaviour data records vehicle movements within a monitored driving area; analysing the driving behaviour data to determine a normal driving behaviour model for the monitored driving area; using object tracking to determine driving trajectories of vehicles driving in the monitored driving area; comparing the driving trajectories with the normal driving behaviour model to identify at least one abnormal driving trajectory; and extracting a portion of the driving behaviour data corresponding to a time interval associated with the abnormal driving trajectory.

Abstract: A computer-implemented method of perceiving structure in an environment comprises steps of: receiving at least one structure observation input pertaining to the environment; processing the at least one structure observation input in a perception pipeline to compute a perception output; determining one or more uncertainty source inputs pertaining to the structure observation input; and determining for the perception output an associated uncertainty estimate by applying, to the one or more uncertainty source inputs, an uncertainty estimation function learned from statistical analysis of historical perception outputs.

Abstract: A method of evaluating the performance of a target planner for an ego robot in a scenario. Evaluation data is generated by applying the target planner from an initial scenario state to generate an actual ego trajectory taken by the ego robot. The actual ego trajectory is defined by a target trajectory parameter. Comparison data is generated by applying a comparison planner from the same initial scenario state to generate a comparison ego trajectory for a comparison ego robot, the comparison ego trajectory comprising at least one comparison trajectory parameter. A juncture point at which the comparison trajectory parameter differs from the actual trajectory parameter is determined and a difference between the actual trajectory parameter and the comparison trajectory parameter at the juncture point is determined. A comparison between the determined difference and a threshold value indicates whether the juncture point is significant.

Abstract: A computer-implemented method of generating black-box adversarial inputs to a perception component comprises computing an adversarial input by applying a perturbation to an original input, the adversarial input satisfying an attack objective when inputted to the perception component. The perturbation is determined by selectively combining component perturbations selected from a predetermined set of component perturbations. Inputs correspond to respective points in an input vector space, and the component perturbations encode principal attack directions in the input vector space for satisfying said attack objective, the principal attack directions having been determined by analyzing: (i) a set of sample attack directions, or (ii) a set of input samples.

Abstract: Performance of a substitute upstream processing component is tested, in order to determine whether that performance is sufficient to support a downstream processing component, within an autonomous driving system, in place of an existing upstream processing component. The existing upstream processing component and the substitute upstream processing component are mutually interchangeable in so far as they provide the same form of outputs interpretable by the downstream processing component, such that either upstream processing component may be used without modification to the downstream processing component. A direct or indirect metric-based comparison is formulated in terms of the resulting performance of the downstream processing component.

Abstract: A computer-implemented method of modelling a common structure component, the method comprising, in a modelling computer system: receiving a plurality of captured frames, each frame comprising a set of 3D structure points, in which at least a portion of a common structure component is captured; computing a first reference position within at least one first frame of the plurality of frames; selectively extracting first 3D structure points of the first frame based on the first reference position computed for the first frame; computing a second reference position within a second frame of the plurality of frames; selectively extracting second 3D structure points of the second frame based on the second reference position computed for the second frame; and aggregating the first 3D structure points and the second 3D structure points, thereby generating an aggregate 3D model of the common structure component based on the first and second reference positions.

Abstract: To locate and model a 3D object captured in multiple time-series of sensor data of multiple sensor modalities, a cost function applied to the multiple time-series of sensor data is optimized. The cost function aggregates over time and the multiple sensor modalities, and is defined over a set of variables comprising one or more shape parameters of a 3D object model and a time sequence of poses of the 3D object model. The cost function penalizes inconsistency between the multiple time-series of sensor data and the set of variables. The object belongs to a known object class, and the 3D object model or the cost function encodes expected 3D shape information associated with the known object class, whereby the 3D object is located at multiple time instants and modelled by tuning each pose and the shape parameters with the objective of optimizing the cost function.