Abstract: Embodiments herein include an autonomy system of an automated vehicle performing operations for detecting and avoiding lazy or unpredictable vehicles on a roadway. The autonomy system generates bounding boxes for tracking traffic vehicles recognized in sensor data. For close vehicles, the autonomy system generates expanded bounding boxes. The autonomy system determines the close vehicle is an encroaching vehicle by detecting that the expanded bounding box of the close vehicle overlapped the middle lane line or enters into the automated vehicle's current lane of travel. The autonomy system may simulate the behavior of the close vehicle by forward-propagating predicted positions of the close vehicle and expanded bounding box over some future time. In response to detecting the encroaching vehicle, the autonomy system may determine whether to perform an avoidance action or select a particular avoidance action, such as biasing away from the encroaching vehicle.