Abstract: A device may receive video data that includes video frames depicting monocular frontal views, and may select a reference video frame and a target video frame from the video data. The device may process the reference video frame, with a bird's eye view (BEV) model, to generate a rendered semantic segmentation and a BEV prediction, and may sample class probability values from the BEV prediction. The device may process the target video frame, with a geometry model, to generate densities, and may generate a target semantic segmentation based on the class probability values and the densities. The device may calculate a cross-entropy loss based on the rendered semantic segmentation and the target semantic segmentation, and may train the BEV model, with the cross-entropy loss, in order to generate a trained BEV model.

Abstract: A device may receive a query identifying one or more events to be captured by a camera associated with a vehicle, video data associated with the vehicle, and location data associated with the vehicle, and may generate a prompt requesting captions of video frames included in the video data. The device may process the video data, with a large language model and based on the prompt, to generate the captions of the video frames included in the video data, and may process the query and the captions, with a phrase model, to determine categories of words provided in the query and the captions. The device may calculate matching scores between the query and the captions based on the categories of words and a dictionary of words, and may identify an event based on the matching scores. The device may perform one or more actions based on the event.

Abstract: A device may receive video data associated with a vehicle experiencing an event, and may utilize an object detection model and an object tracking model to determine object data identifying bounding boxes, tracks, and classes for objects depicted in the video data. The device may process the object data, with an object backbone of a spatiotemporal multi-modal (ST-MM) neural network model, to determine object features associated with dynamics of the objects depicted in the video data, and may determine vehicle features associated with dynamics of the vehicle. The device may process the object features and the vehicle features, with a recurrent neural network of the ST-MM neural network model, to classify the event into a category, and may perform one or more actions based on the category of the event.

Abstract: A device may receive a query identifying one or more events to be captured by a camera associated with a vehicle, video data associated with the vehicle, and location data associated with the vehicle, and may generate a prompt requesting captions of video frames included in the video data. The device may process the video data, with a large language model and based on the prompt, to generate the captions of the video frames included in the video data, and may process the query and the captions, with a phrase model, to determine categories of words provided in the query and the captions. The device may calculate matching scores between the query and the captions based on the categories of words and a dictionary of words, and may identify an event based on the matching scores. The device may perform one or more actions based on the event.

Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may remove video frames from the video data to generate modified video data. The device may select objects and image regions of video frames of the modified video data, and may determine a current speed and a current turn angle of the vehicle based on the GPS data, the IMU data, and the modified video data. The device may mask the objects of the video frames of the modified video data to learn first features, and may mask the image regions of the video frames of the modified video data to learn second features. The device may generate a trained neural network model based on the current speed, the current turn angle, the first features, and the second features, and may implement the trained neural network model in the vehicle.

Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may remove video frames from the video data to generate modified video data. The device may select objects and image regions of video frames of the modified video data, and may determine a current speed and a current turn angle of the vehicle based on the GPS data, the IMU data, and the modified video data. The device may mask the objects of the video frames of the modified video data to learn first features, and may mask the image regions of the video frames of the modified video data to learn second features. The device may generate a trained neural network model based on the current speed, the current turn angle, the first features, and the second features, and may implement the trained neural network model in the vehicle.

Abstract: A device may receive, from a first device, first tracking data that includes first temporal data and first location data, and may receive, from a second device, second tracking data that includes second temporal data and second location data. The device may generate a first spatio-temporal object based on the first tracking data, and may generate a second spatio-temporal object based on the second tracking data. The device may calculate a matching score associated with the first spatio-temporal object and the second spatio-temporal object, and may determine whether the matching score satisfies a score threshold. The device may determine that the first device is associated with the second device based on determining that the matching score satisfies the score threshold, and may perform one or more actions based on determining that the first device is associated with the second device.

Abstract: A video summary device may determine, based on video data, a plurality of events that occurred during a period of time. The video summary device may determine first measures of relevance of a plurality of portions of the period of time. A first measure of relevance of a portion of time may be determined based on one or more events of the plurality of events. The video summary device may determine second measures of relevance of a plurality of ranges of time between different portions of time of the plurality of portions of the period of time. The second measures of relevance is determined based on the first measures of relevance determined for the different portions of time. The video summary device may determine particular frames of the video data, as a video summary of the video data, based on the second measures of relevance.

Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may remove video frames from the video data to generate modified video data. The device may select objects and image regions of video frames of the modified video data, and may determine a current speed and a current turn angle of the vehicle based on the GPS data, the IMU data, and the modified video data. The device may mask the objects of the video frames of the modified video data to learn first features, and may mask the image regions of the video frames of the modified video data to learn second features. The device may generate a trained neural network model based on the current speed, the current turn angle, the first features, and the second features, and may implement the trained neural network model in the vehicle.

Abstract: In some implementations, a device may receive image data and sensor data associated with the image data. The image data may depict a series of scenes that are captured over a time period and that are associated with an occurrence of an event. The device may determine a time of interest, within the time period, at which a value of the sensor data satisfies one or more criteria. The device may extract a portion of the image data based on the time of interest. The device may extract, from the portion of the image data, a slice of image data depicting a first set of scenes and a slice of image data depicting a second set of scenes. The device may train, based on the slices of image data, a model to predict a probability of an occurrence of an event.

Abstract: A device may process the video data, with a first machine learning model, to identify a driver of a vehicle and may process the video data associated with the driver, with a second machine learning model, to detect behavior data identifying a behavior of the driver. The device may process the behavior data, with a third machine learning model, to determine distraction data identifying whether the behavior is classified as a distracted behavior. The device may process the behavior data, with a fourth machine learning model, to determine policy compliance data identifying whether the behavior satisfies one or more policies. The device may calculate a distraction score based on the distraction data and the video data, and may calculate a policy compliance score based on the policy compliance data and vehicle data. The device may perform one or more actions based on the distraction score and the policy compliance score.

Abstract: A video summary device may determine, based on video data, a plurality of events that occurred during a period of time. The video summary device may determine first measures of relevance of a plurality of portions of the period of time. A first measure of relevance of a portion of time may be determined based on one or more events of the plurality of events. The video summary device may determine second measures of relevance of a plurality of ranges of time between different portions of time of the plurality of portions of the period of time. The second measures of relevance is determined based on the first measures of relevance determined for the different portions of time. The video summary device may determine particular frames of the video data, as a video summary of the video data, based on the second measures of relevance.

Abstract: In some implementations, a device may receive image data and sensor data associated with the image data. The image data may depict a series of scenes that are captured over a time period and that are associated with an occurrence of an event. The device may determine a time of interest, within the time period, at which a value of the sensor data satisfies one or more criteria. The device may extract a portion of the image data based on the time of interest. The device may extract, from the portion of the image data, a slice of image data depicting a first set of scenes and a slice of image data depicting a second set of scenes. The device may train, based on the slices of image data, a model to predict a probability of an occurrence of an event.

Abstract: A device may process the video data, with a first machine learning model, to identify a driver of a vehicle and may process the video data associated with the driver, with a second machine learning model, to detect behavior data identifying a behavior of the driver. The device may process the behavior data, with a third machine learning model, to determine distraction data identifying whether the behavior is classified as a distracted behavior. The device may process the behavior data, with a fourth machine learning model, to determine policy compliance data identifying whether the behavior satisfies one or more policies. The device may calculate a distraction score based on the distraction data and the video data, and may calculate a policy compliance score based on the policy compliance data and vehicle data. The device may perform one or more actions based on the distraction score and the policy compliance score.

Abstract: A device may receive vehicle operation data associated with operation of a plurality of vehicles, and may process the vehicle operation data to generate processed vehicle operation data. The device may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models that provide model outputs. The device may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, and may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. The device may select a set of features, from the sets of features, based on the indications and the reliabilities, where the set of features may be calculated by a device associated with a particular vehicle.

Abstract: A device may process the video data, with a first machine learning model, to identify a driver of a vehicle and may process the video data associated with the driver, with a second machine learning model, to detect behavior data identifying a behavior of the driver. The device may process the behavior data, with a third machine learning model, to determine distraction data identifying whether the behavior is classified as a distracted behavior. The device may process the behavior data, with a fourth machine learning model, to determine policy compliance data identifying whether the behavior satisfies one or more policies. The device may calculate a distraction score based on the distraction data and the video data, and may calculate a policy compliance score based on the policy compliance data and vehicle data. The device may perform one or more actions based on the distraction score and the policy compliance score.

Abstract: Embodiments described herein provide for the generation of models, which may predict or otherwise model road safety conditions. For example, embodiments may determine attributes of roads and/or segments of roads based on sensor data, statistical data, and/or other suitable data to determine road safety models associated with the roads or road segments. Such models may be used when determining navigation routes, controlling autonomous vehicles, city planning, and/or other suitable operations that may take road safety into account.

Abstract: A device may process the video data, with a first machine learning model, to identify a driver of a vehicle and may process the video data associated with the driver, with a second machine learning model, to detect behavior data identifying a behavior of the driver. The device may process the behavior data, with a third machine learning model, to determine distraction data identifying whether the behavior is classified as a distracted behavior. The device may process the behavior data, with a fourth machine learning model, to determine policy compliance data identifying whether the behavior satisfies one or more policies. The device may calculate a distraction score based on the distraction data and the video data, and may calculate a policy compliance score based on the policy compliance data and vehicle data. The device may perform one or more actions based on the distraction score and the policy compliance score.

Abstract: A device may receive vehicle operation data associated with operation of a plurality of vehicles, and may process the vehicle operation data to generate processed vehicle operation data. The device may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models that provide model outputs. The device may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, and may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. The device may select a set of features, from the sets of features, based on the indications and the reliabilities, where the set of features may be calculated by a device associated with a particular vehicle.

Abstract: A device may receive vehicle operation data associated with operation of a plurality of vehicles, and may process the vehicle operation data to generate processed vehicle operation data. The device may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models that provide model outputs. The device may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, and may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. The device may select a set of features, from the sets of features, based on the indications and the reliabilities, where the set of features may be calculated by a device associated with a particular vehicle.