Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may process the video data, with an object detector model, to identify objects and to generate a first feature vector. The device may process the GPS data and the IMU data, with a first CNN model, to generate a second feature vector, and may process the objects and the video data, with a tracking model, to identify positions and classes of the objects and to generate a third feature vector. The device may utilize a second CNN model to generate a matrix of object features based on the first, second, and third feature vectors, and may utilize a spatiotemporal attention selector model or a max pooled model with the matrix of object features to identify a classification of a maneuver of the vehicle. The device may perform actions based on the classification.

Abstract: In some implementations, a video system may receive, from a camera mounted to a vehicle, a video of a portion of a road on which the vehicle is traveling. The video system may extract, from each frame of a plurality of frames associated with the video of the road, a frame strip to form a plurality of frame strips, wherein each frame strip extends a predetermined width in a horizontal direction and a predetermined height in a vertical direction. The video system may form, from each frame strip, a single-pixel strip, to form a plurality of single-pixel strips. The video system may compile the plurality of single-pixel strips to form a motion profile. The video system may determine, using machine learning, one of: at least one driving maneuver associated with the vehicle based on the motion profile, or that no driving maneuvers are present in the motion profile.

Abstract: A device may receive video data and description data, and may extract image embeddings and video embeddings from the video data. The device may receive a query that includes a natural language description of a scene to be retrieved from the video data. The device may split the query into a first subquery and a second subquery, and may process the first and second subqueries, with a model, to generate a first query embedding and a second query embedding. The device may generate appearance similarity measures based on the image embeddings and the first query embedding, and may generate dynamic similarity measures based on the video embeddings and the second query embedding. The device may combine the appearance similarity measures and the dynamics similarity measures into combined similarity measures, and may retrieve video segments based on the combined similarity measures.

Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may process the video data, with an object detector model, to identify objects and to generate a first feature vector. The device may process the GPS data and the IMU data, with a first CNN model, to generate a second feature vector, and may process the objects and the video data, with a tracking model, to identify positions and classes of the objects and to generate a third feature vector. The device may utilize a second CNN model to generate a matrix of object features based on the first, second, and third feature vectors, and may utilize a spatiotemporal attention selector model or a max pooled model with the matrix of object features to identify a classification of a maneuver of the vehicle. The device may perform actions based on the classification.

Abstract: A device may receive video data identifying videos associated with one or more unsafe driving events by a driver of a vehicle, and may process the video data, with a machine learning model, to determine classifications for the videos. The device may assign tags to the videos based on the classifications, and may calculate event severity scores based on the classifications. The device may calculate tag scores based on the tags assigned to the videos, and may calculate time-to-contact scores, box cross scores, day/night scores, weather scores, and road condition scores based on the video data. The device may calculate video risk scores for the videos based on the event severity scores, the tag scores, the time-to-contact scores, the box cross scores, the day/night scores, the weather scores, and the road condition scores, and may provide one or more of the video risk scores for display.

Abstract: A device may receive a plurality of narratives associated with a plurality of scenes and an image identifying a scene not included in the plurality of scenes, and may process the image, with a classifier model, to detect a plurality of features in the image. The device may replace keywords in the plurality of narratives, with tags, to generate a plurality of sentences, and may group similar sentences of the plurality of sentences, based on a defined measure of dissimilarity, into clusters of templates. The device may select a candidate template from each of the clusters to generate a set of candidate templates, and may select a template from the set of candidate templates. The device may populate tags of the template with the plurality of features detected in the image to generate an image caption, and may provide the image and the image caption for display.

Abstract: A device may receive historical input data associated with trips traversed by a plurality of vehicles with vehicle tracking units (VTUs), and may process the historical input data to generate training data. The device may train a neural network model, with the training data, to generate a trained neural network model that provides a latent space representation of vectors, and may cluster the latent space representation of vectors to generate clusters. The device may receive input data associated with a trip traversed by a vehicle of the plurality of vehicles with the VTUs, and may process the input data to generate time series data. The device may compare the time series data and the clusters to determine whether the trip is anomalous or not anomalous, and may perform one or more actions based on the determination of whether the trip is anomalous or not anomalous.

Abstract: A device may receive driver facing video data associated with a driver of a vehicle and forward facing video data associated with the vehicle, and may process the driver facing video data, with a face model, to identify driver head orientation and driver gaze. The device may generate a first transformation matrix mapping the driver facing video data, the driver head orientation, and the driver gaze, and may generate a second transformation matrix mapping the driver facing video data and the forward facing video data. The device may utilize the first transformation matrix and the second transformation matrix to estimate image coordinates, and may aggregate the image coordinates to generate aggregated coordinates. The device may generate heat maps based on the aggregated coordinates, may train machine learning model, with the heat maps, to generate a trained machine learning model, and may perform actions based on the trained machine learning model.

Abstract: A system may determine, for each frame of a plurality of frames of image data associated with a vehicle, a probability that each frame of the plurality of frames that includes an image of a stop sign is relevant to the vehicle. The system may determine a longest sequence of consecutive frames of the plurality of frames for which the probability satisfies a probability threshold. The system may determine a maximum probability associated with a frame included in the longest sequence of consecutive frames. The system may determine a time window based on a time associated with the frame associated with the maximum probability. The system may determine location data and sensor data for the vehicle based on the time window. The system may determine an occurrence of a stop sign violation based on the location data and the sensor data.

Abstract: A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may process the video data, with an object detector model, to identify objects and to generate a first feature vector. The device may process the GPS data and the IMU data, with a first CNN model, to generate a second feature vector, and may process the objects and the video data, with a tracking model, to identify positions and classes of the objects and to generate a third feature vector. The device may utilize a second CNN model to generate a matrix of object features based on the first, second, and third feature vectors, and may utilize a spatiotemporal attention selector model or a max pooled model with the matrix of object features to identify a classification of a maneuver of the vehicle. The device may perform actions based on the classification.

Abstract: A device may receive a plurality of narratives associated with a plurality of scenes and an image identifying a scene not included in the plurality of scenes, and may process the image, with a classifier model, to detect a plurality of features in the image. The device may replace keywords in the plurality of narratives, with tags, to generate a plurality of sentences, and may group similar sentences of the plurality of sentences, based on a defined measure of dissimilarity, into clusters of templates. The device may select a candidate template from each of the clusters to generate a set of candidate templates, and may select a template from the set of candidate templates. The device may populate tags of the template with the plurality of features detected in the image to generate an image caption, and may provide the image and the image caption for display.

Abstract: A device may receive driver facing video data associated with a driver of a vehicle and forward facing video data associated with the vehicle, and may process the driver facing video data, with a face model, to identify driver head orientation and driver gaze. The device may generate a first transformation matrix mapping the driver facing video data, the driver head orientation, and the driver gaze, and may generate a second transformation matrix mapping the driver facing video data and the forward facing video data. The device may utilize the first transformation matrix and the second transformation matrix to estimate image coordinates, and may aggregate the image coordinates to generate aggregated coordinates. The device may generate heat maps based on the aggregated coordinates, may train machine learning model, with the heat maps, to generate a trained machine learning model, and may perform actions based on the trained machine learning model.

Abstract: In some implementations, a video system may receive, from a camera mounted to a vehicle, a video of a portion of a road on which the vehicle is traveling. The video system may extract, from each frame of a plurality of frames associated with the video of the road, a frame strip to form a plurality of frame strips, wherein each frame strip extends a predetermined width in a horizontal direction and a predetermined height in a vertical direction. The video system may form, from each frame strip, a single-pixel strip, to form a plurality of single-pixel strips. The video system may compile the plurality of single-pixel strips to form a motion profile. The video system may determine, using machine learning, one of: at least one driving maneuver associated with the vehicle based on the motion profile, or that no driving maneuvers are present in the motion profile.

Abstract: A system may determine, for each frame of a plurality of frames of image data associated with a vehicle, a probability that each frame of the plurality of frames that includes an image of a stop sign is relevant to the vehicle. The system may determine a longest sequence of consecutive frames of the plurality of frames for which the probability satisfies a probability threshold. The system may determine a maximum probability associated with a frame included in the longest sequence of consecutive frames. The system may determine a time window based on a time associated with the frame associated with the maximum probability. The system may determine location data and sensor data for the vehicle based on the time window. The system may determine an occurrence of a stop sign violation based on the location data and the sensor data.

Abstract: A device can receive a request for a schedule that assigns a fleet of vehicles to a set of deliveries. The device can determine that a parameter is not included in the request that is needed to generate a new schedule or that is needed to generate an existing schedule. The device can obtain the parameter using a historical user request, a historical schedule, or a scheduling template. The device can generate or obtain the schedule based on information included in the request and the obtained parameter. The device can provide the schedule to a user device and/or to one or more devices associated with the fleet of vehicles carrying out the set of deliveries. The device can modify the schedule based on a trigger. The device can provide the modified schedule to the user device and/or to the one or more devices associated with the fleet of vehicles.

Abstract: A device may receive video data identifying videos associated with one or more unsafe driving events by a driver of a vehicle, and may process the video data, with a machine learning model, to determine classifications for the videos. The device may assign tags to the videos based on the classifications, and may calculate event severity scores based on the classifications. The device may calculate tag scores based on the tags assigned to the videos, and may calculate time-to-contact scores, box cross scores, day/night scores, weather scores, and road condition scores based on the video data. The device may calculate video risk scores for the videos based on the event severity scores, the tag scores, the time-to-contact scores, the box cross scores, the day/night scores, the weather scores, and the road condition scores, and may provide one or more of the video risk scores for display.

Abstract: A device may receive a first image. The device may process the first image to identify an object in the first image and a location of the object within the first image. The device may extract a second image from the first image based on the location of the object within the first image. The device may process the second image to determine at least one of a coarse-grained viewpoint estimate or a fine-grained viewpoint estimate associated with the object. The device may determine an object viewpoint associated with the second vehicle based on the at least one of the coarse-grained viewpoint estimate or the fine-grained viewpoint estimate. The device may perform one or more actions based on the object viewpoint.

Abstract: A violation detection platform may obtain image data, location data, and sensor data associated with a vehicle. The violation detection platform may determine a probability that a frame of the image data includes an image of a stop sign. The violation detection platform may determine that the probability satisfies a probability threshold. The violation detection platform may identify location data and sensor data associated with the frame of the image data based on the probability satisfying the probability threshold. The violation detection platform may determine an occurrence of a type of a stop sign violation based on the probability, the location data, and the sensor data. The violation detection platform may perform one or more actions based on determining the occurrence of the type of the stop sign violation.

Abstract: A device may receive a video and corresponding sensor information associated with a vehicle, and may extract feature vectors associated with the corresponding sensor information and an appearance and a geometry of another vehicle captured in the video. The device may generate a tensor based on the feature vectors, and may process the tensor, with a convolutional neural network model, to generate a modified tensor. The device may select a decoder model from a plurality of decoder models, and may process the modified tensor, with the decoder model, to generate a caption for the video based on attributes associated with the video. The device may perform one or more actions based on the caption for the video.

Abstract: A device may receive vehicle operation data associated with operation of a plurality of vehicles, and may process the vehicle operation data to generate processed vehicle operation data. The device may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models that provide model outputs. The device may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, and may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. The device may select a set of features, from the sets of features, based on the indications and the reliabilities, where the set of features may be calculated by a device associated with a particular vehicle.