Abstract: Environmental tracking systems and methods are disclosed. An environmental tracking system receives sensor data from the one or more sensors, such as camera(s) and Light Detection and Ranging (LIDAR) sensors. The system uses trained machine learning (ML) model(s) to detect, within the sensor data, representation(s) of at least a portion of a vehicle with a door that is at least partially open. Based on these representation(s), the system generates a boundary for the vehicle that includes the door and is sized based on the door being at least partially open. The system determines a route that avoids the boundary, for example by planning the route around the boundary or by planning to stop before intersecting with the boundary. In some examples, the sensors are sensors coupled to a second vehicle, and the second vehicle traverses the route.