Abstract: The disclosure relates to running simulations in order to test software used to control a vehicle in an autonomous driving mode. For instance, logged data may be identified for a simulation. The logged data may have been collected by a first vehicle and may identifying an agent that is a road user. The logged data may be analyzed to identify one or more signals of intent of the agent including a logged path of the agent. One or more characteristics may be identified based on the one or more signals. The simulation may be run using the logged data by replacing the agent with an interactive agent having the one or more characteristics. The interactive agent may be capable of responding to actions performed by a simulated vehicle in the simulation using software for controlling a vehicle in an autonomous driving mode.

Abstract: The disclosure relates to testing software for operating an autonomous vehicle. In one instance, a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The simulation may be run using the software to control a simulated vehicle and by modifying a characteristic of an agent identified in the log data. During the running of the simulation, that a first type of interaction between the first simulated vehicle and the modified agent will occur may be determined. In response to determining that the particular type of interaction will occur, the modified agent may be replaced by a interactive agent that simulates a road user corresponding to the modified agent that is capable of responding to actions performed by simulated vehicles. That the particular type of interaction between the simulated vehicle and the interactive agent has occurred in the simulation may be determined.

Abstract: The disclosure relates to running simulations in order to test software used to control a vehicle in an autonomous driving mode. For instance, logged data may be identified for a simulation. The logged data may have been collected by a first vehicle and may identifying an agent that is a road user. The logged data may be analyzed to identify one or more signals of intent of the agent including a logged path of the agent. One or more characteristics may be identified based on the one or more signals. The simulation may be run using the logged data by replacing the agent with an interactive agent having the one or more characteristics. The interactive agent may be capable of responding to actions performed by a simulated vehicle in the simulation using software for controlling a vehicle in an autonomous driving mode.

Abstract: The disclosure relates to testing software for operating an autonomous vehicle. In one instance, a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The simulation may be run using the software to control a simulated vehicle and by modifying a characteristic of an agent identified in the log data. During the running of the simulation, that a first type of interaction between the first simulated vehicle and the modified agent will occur may be determined. In response to determining that the particular type of interaction will occur, the modified agent may be replaced by a interactive agent that simulates a road user corresponding to the modified agent that is capable of responding to actions performed by simulated vehicles. That the particular type of interaction between the simulated vehicle and the interactive agent has occurred in the simulation may be determined.

Abstract: The disclosure relates to testing software for operating an autonomous vehicle. In one instance, a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The simulation may be run using the software to control a simulated vehicle and by modifying a characteristic of an agent identified in the log data. During the running of the simulation, that a first type of interaction between the first simulated vehicle and the modified agent will occur may be determined. In response to determining that the particular type of interaction will occur, the modified agent may be replaced by a interactive agent that simulates a road user corresponding to the modified agent that is capable of responding to actions performed by simulated vehicles. That the particular type of interaction between the simulated vehicle and the interactive agent has occurred in the simulation may be determined.

Abstract: A simulation for sensor data may be evaluated and used for future simulations for an autonomous vehicle software. The method includes receiving log data collected for an environment along a given run for a given vehicle, using a software for autonomous driving to perform a simulated run of the given run using logged sensor data from the log data and environment data constructed using the log data, and determining first details regarding detection of objects during the given run using logged sensor data. The method also includes using the software to run a simulation of one or more detection devices on a simulated vehicle driving along the given run to obtain simulated sensor data, and determining second details regarding detection of objects using the simulated sensor data. Metrics may then be extracted from the first details and the second details, and the simulation may be evaluated based on the metrics.

Abstract: Aspects of the disclosure relate to improving realism in simulations for testing software for operating a vehicle in an autonomous driving mode. In one instance, an initial observation of a road user object may be identified in a log data segment captured by a perception system of a vehicle. The perception system having one or more sensors. The initial observation includes a point in time and an initial location of the road user object. A distance traveled by the road user object from a start of the log data segment to the point in time may be determined. A starting location for the road user object may be determined using the distance traveled. A trajectory for the road user object may be determined between the starting location and the initial location of the road user object. The trajectory may be appended to the log data segment.

Abstract: The disclosure relates to running simulations in order to test software used to control a vehicle in an autonomous driving mode. For instance, logged data may be identified for a simulation. The logged data may have been collected by a first vehicle and may identifying an agent that is a road user. The logged data may be analyzed to identify one or more signals of intent of the agent including a logged path of the agent. One or more characteristics may be identified based on the one or more signals. The simulation may be run using the logged data by replacing the agent with an interactive agent having the one or more characteristics. The interactive agent may be capable of responding to actions performed by a simulated vehicle in the simulation using software for controlling a vehicle in an autonomous driving mode.

Abstract: The disclosure relate to determining a realism metric for testing software for operating a vehicle in an autonomous driving mode. For instance, a plurality of versions of a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The plurality of versions may be run using the software to control a simulated vehicle, and each of the plurality of versions may have a set of timing requirements different from the set of timing requirements of other of the plurality of versions. Results of the plurality of versions may be compared to the log data. A realism metric defining timing requirements for one or more future simulations may be generated based on the comparison.

Abstract: The disclosure relates to testing software for operating an autonomous vehicle. In one instance, a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The simulation may be run using the software to control a simulated vehicle and by modifying a characteristic of an agent identified in the log data. During the running of the simulation, that a first type of interaction between the first simulated vehicle and the modified agent will occur may be determined. In response to determining that the particular type of interaction will occur, the modified agent may be replaced by a interactive agent that simulates a road user corresponding to the modified agent that is capable of responding to actions performed by simulated vehicles. That the particular type of interaction between the simulated vehicle and the interactive agent has occurred in the simulation may be determined.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rewriting contextual queries. In one aspect, a method includes receiving a first query for a user session, determining contextual data associated with the first query, receiving a second query subsequent to the first query for the user session, determining that the second query is associated with the contextual data, determining that the second query corresponds to a particular template determined based on at least one of the contextual data or the second query, generating a third query based on the particular template, the contextual data, and the second query, and providing the third query as an input for an action that is responsive to the second query.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for displaying information about computer network resources identified as engaging in malicious activities. One of the systems includes one or more computers including one or more processors and one or more memory devices, the one or more computers configured to: identify resources associated with an attack, and provide an attach resource dashboard user interface that displays information related to attack resources, wherein the user interface presents resource information comparing behavior of a particular resource at a single online service with behavior of the resource at other online services, and comparing the behavior of that resource with behavior of other resources.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for ranking events. One of the methods includes receiving data identifying a physical location; obtaining data identifying a plurality of events occurring in the physical location during a particular time period; computing a respective plurality of signal scores for each of the events, wherein computing the respective plurality of signal scores for each of the events comprises computing a first signal score for each of the events based at least in part on a count of Internet sites that include at least one mention of the event; computing a respective popularity score for each of the plurality of events by combining the respective plurality of signal scores for the event; and generating a ranking of events occurring in the physical location during the particular time period based at least in part on the popularity scores.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for ranking events. One of the methods includes receiving data identifying a physical location; obtaining data identifying a plurality of events occurring in the physical location during a particular time period; computing a respective plurality of signal scores for each of the events, wherein computing the respective plurality of signal scores for each of the events comprises computing a first signal score for each of the events based at least in part on a count of Internet sites that include at least one mention of the event; computing a respective popularity score for each of the plurality of events by combining the respective plurality of signal scores for the event; and generating a ranking of events occurring in the physical location during the particular time period based at least in part on the popularity scores.