Abstract: Techniques for determining a speed for a vehicle as it traverses an environment with pedestrians are discussed herein. For example, a vehicle computing system may implement techniques to determine an action for a vehicle to take based on a detected pedestrian in an environment. The vehicle computing system may receive sensor data of an environment from a sensor associated with a vehicle, determine, based at least in part on the sensor data, an object in the environment and receive a predicted object trajectory associated with the object. The vehicle computing system may then determine, based on the predicted object trajectory, a distance between a simulated vehicle passing location and a predicted object location, determine, based on the distance, a speed, determine, based on the distance and the speed, a trajectory for the vehicle to follow, and control the vehicle based on the trajectory.