Abstract: In various examples, occupant assessment using multi-modal sensor fusion for monitoring systems and applications are provided. In some embodiments, an occupant monitoring system comprises an occupant evaluation function that may predict at least one characteristic representative of a size of the occupant. The occupant evaluation function may include a first processing path that generates a representation of features corresponding to the occupant based on optical image data, and a second processing path that performs operations to determine a depth corresponding to the one or more features based on depth data derived from the optical image data and the point cloud depth data. In some embodiments, a three-dimensional pose detection model generates a three-dimensional pose estimate of the occupant using the optical image data, and the three-dimensional pose estimate is scaled to an absolute pose based on the point cloud depth data.

Abstract: In various examples, occupant assessment using multi-modal sensor fusion for monitoring systems and applications are provided. In some embodiments, an occupant monitoring system comprises an occupant evaluation function that may predict at least one characteristic representative of a size of the occupant. The occupant evaluation function may include a first processing path that generates a representation of features corresponding to the occupant based on optical image data, and a second processing path that performs operations to determine a depth corresponding to the one or more features based on depth data derived from the optical image data and the point cloud depth data. In some embodiments, a three-dimensional pose detection model generates a three-dimensional pose estimate of the occupant using the optical image data, and the three-dimensional pose estimate is scaled to an absolute pose based on the point cloud depth data.

Abstract: In various examples, sensor data (e.g., image and/or RADAR data) may be used to detect occupants and classify them (e.g., as children or adults) using one or more predictions that represent estimated age (e.g., based on detected limb length, a detected face) and/or detected child presence (e.g., based on detecting an occupied child seat). In some embodiments, multiple predictions generated using multiple machine learning models (and optionally one or more corresponding confidence values) may be combined using a state machine and/or one or more machine learning models to generate a combined assessment of occupant presence and/or age for each occupant and/or supported occupant slot. As such, the techniques described herein may be utilized to detect child presence, detect unattended child presence, determine age or size of a particular occupant, and/or take some responsive action (e.g., trigger an alarm, control temperature, unlock door(s), permit or disable airbag deployment, etc.).

Abstract: In various examples, image-based three-dimensional occupant assessment for in-cabin monitoring systems and applications are provided. An evaluation function may determine a 3D representation of an occupant of a machine by evaluating sensor data comprising an image frame from an optical image sensor. The 3D representation may comprise at least one characteristic representative of a size of the occupant, (e.g., a 3D pose and/or 3D shape), which may be used to derive other characteristics such as, but not limited to weight, height, and/or age. A first processing path may generate a representation of one or more features corresponding to at least a portion of the occupant based on optical image data, and a second processing path may determine a depth corresponding to the one or more features based on depth data derived from the optical image data and ground truth depth data corresponding to the interior of the machine.

Abstract: In various examples, occupant assessment using multi-modal sensor fusion for monitoring systems and applications are provided. In some embodiments, an occupant monitoring system comprises an occupant evaluation function that may predict at least one characteristic representative of a size of the occupant. The occupant evaluation function may include a first processing path that generates a representation of features corresponding to the occupant based on optical image data, and a second processing path that performs operations to determine a depth corresponding to the one or more features based on depth data derived from the optical image data and the point cloud depth data. In some embodiments, a three-dimensional pose detection model generates a three-dimensional pose estimate of the occupant using the optical image data, and the three-dimensional pose estimate is scaled to an absolute pose based on the point cloud depth data.

Abstract: In various examples, systems and methods for pose detection model training for predicting three-dimensional pose estimates using two-dimensional image data are provided. The occupant pose detection model may be trained using multi-view image sensor training data that includes image frames that capture a pose of a training subject within a machine interior using multiple synchronized optical image sensors placed around the machine interior that produce a set of captured image frames of the training subject from different viewpoints. Based on the multi-view image sensor training data, the occupant pose detection model may generate a set of individual, predicted 3D pose estimates for the training subject from a captured image frame from each of the respective optical image sensors. To adjust the occupant pose detection model during training, a loss feedback may be generated that comprises a pose alignment loss, a pose depth loss, and/or a ground truth kinematic loss.