Abstract: Autonomous vehicles utilize perception and understanding of vehicles to predict behaviors of the vehicles, and to plan a trajectory. Understanding of attributes of vehicles may be improved through sensor fusion. Sensor fusion can be computationally expensive and may be difficult to implement in a real-time vehicle understanding system. To limit computational complexity while benefiting from machine learning across modalities, sensor fusion may be selectively implemented for a subset of task groups of a multi-task machine learning model. In some cases, part-based understanding may be implemented before fusion to limit the features being fused together to part features that are most salient for the task group. In addition, sensor data and features that may be fused together can be limited to sensor data and features within a desired field of view. A model that implements sensor fusion may be disabled for objects that are beyond a threshold distance.

Abstract: Autonomous vehicles utilize perception and understanding of road users and road objects to predict behaviors of the road users and road objects, and to plan a trajectory for the vehicle. Understanding subtypes and attributes of vulnerable road users may help autonomous vehicles better predict behaviors of and react to vulnerable road users. To offer additional understanding capabilities, an additional understanding model is added to the perception and understanding pipeline to improve classification of vulnerable road users and extraction of attributes of the vulnerable road users. The exemplary architectures of the understanding model balance recall and precision performance metrics and computational complexity.

Abstract: Autonomous vehicles utilize perception and understanding of road users to predict behaviors of the road users, and to plan a trajectory for the vehicle. Understanding subtypes and attributes of vehicles may help autonomous vehicles better predict behaviors of and react to vehicles in a variety of road situations. To offer additional understanding capabilities, an additional understanding model is added to the perception and understanding pipeline to improve classification of vehicles and extraction of attributes of the vehicles. The exemplary architectures of the understanding model balance recall and precision performance metrics and computational complexity.