Abstract: Techniques are presented for the detection and management of collision warning (CW) events. A training dataset comprising videos of vehicle collisions and non-collisions, sensor readings, environmental conditions, and more is utilized to train a CW classification model for detecting potential collision events in vehicles. A backend CW classification model, with greater computational resources, employs a more complex neural network to review CW events received by the Behavioral Monitoring System (BMS) based on video data, achieving higher precision and reducing false positives. The CW model is installed in vehicles for real-time detection, while the backend model is deployed at the BMS. The BMS validates detected CW events, filters out false positives, and streamlines the review process for fleet administrators and customers. Additional BMS filtering operations include assessing non-proximity-related CW events and camera impairments, with the filtered CW events presented for review in the safety inbox.

Abstract: Methods, systems, and computer programs are presented for monitoring tailgating when a vehicle follows another vehicle at an unsafe distance. A method for enhancing a Following Distance (FD) machine learning (ML) model is disclosed. The method includes providing a management user interface (UI) for configuring FD parameters, followed by receiving FD events. A UI for manual FD annotation and another for customer review of filtered FD events are also provided. Annotations and customer review information are collected to improve the training set for the FD ML model. The FD model is then trained with the new data and downloaded to a vehicle. Once installed, the FD model is utilized to detect FD events within the vehicle, thereby enhancing the vehicle's safety and performance in driving scenarios by improving the accuracy and reliability of FD event predictions or detections.

Abstract: A computer-implemented method for machine learning model operation can include, by one or more processors executing program instructions: providing a first training dataset comprising a plurality of images and associated object detection labels, providing a second training dataset comprising a plurality of images and associated classification labels, and providing a machine learning model comprising a model backbone, an object detection task head, and a classification task head. The method can further include training the machine learning model by training the object detection task head using the first training dataset and training the classification task head using the second training dataset. The method can further include deploying the trained machine learning model that includes the trained model backbone and the trained object detection task head but does not include the trained classification task head.

Abstract: Methods, systems, and programs are presented for detecting impaired views in monitoring cameras. One method includes training a rotation classifier with unsupervised learning utilizing a first set of images. The rotation classifier is configured to receive an input image and generate a rotation feature embedding for the input image. In addition, the method includes training an impairment classifier with supervised learning utilizing a second set of images, impairment labels for each of the second set of images, and the rotation feature embedding, generated by the rotation classifier, for each of the second set of images. The method further includes accessing a vehicle image captured by a camera on a vehicle, and providing the vehicle image to the impairment classifier as input, and the impairment classifier outputs a camera impairment from a set of camera impairment categories. Further, the vehicle image and the camera impairment are presented on a user interface.

Abstract: Methods, systems, and computer programs are presented for the management of lane-departure (LD) events. One method includes training a classifier for LD events and loading the classifier into a vehicle. LD events are detected based on outward images using the classifier, while the turn signal is monitored to prevent false triggers. If an LD event is detected, rules are checked for alerting the driver and deciding whether to alert the driver or not. Subsequently, additional rules are checked for reporting the event and deciding whether to report the event to a Behavior Monitoring System (BMS) or to discard it. The method also includes a solid line departure model that identifies crossing dashed, solid-white, and solid-yellow lanes, delaying alerts and event generation until a significant portion of the vehicle crosses over the lane. The model also outputs a confidence score reflecting the amount of vehicle deviation from the driving lane.

Abstract: Techniques are presented for detecting when drivers drive while drowsy. In some implementations, a drowsiness model is trained with data associated with inward videos and outward videos captured during a trip. The inward videos capture the inside of the cabin with the driver, and the outward videos capture the view in front of the vehicle in the direction of travel. Further, a device at the vehicle periodically calculates a drowsiness scale index value that indicates the level of drowsiness of the driver. Calculating the drowsiness scale index value includes obtaining a set of inward frames from the inward videos; for each inward frame, creating a face image by cropping the inward frame; obtaining a set of outward frames from the outward videos; calculating inward embeddings of the face images and outward embeddings of the outward frames; and calculating, by the drowsiness model, the drowsiness scale index value.

Abstract: Methods, systems, and programs are presented for detecting impaired views in monitoring cameras. One method includes training a rotation classifier with unsupervised learning utilizing a first set of images. The rotation classifier is configured to receive an input image and generate a rotation feature embedding for the input image. In addition, the method includes training an impairment classifier with supervised learning utilizing a second set of images, impairment labels for each of the second set of images, and the rotation feature embedding, generated by the rotation classifier, for each of the second set of images. The method further includes accessing a vehicle image captured by a camera on a vehicle, and providing the vehicle image to the impairment classifier as input, and the impairment classifier outputs a camera impairment from a set of camera impairment categories. Further, the vehicle image and the camera impairment are presented on a user interface.

Abstract: A method for receiving a first user query from a user for searching an item, forming a first filter based on the first user query, and forming a first filtered item collection is provided. The method includes predicting a new query based on the first user query and a historical query log, forming a second filter for the new query, and applying the second filter to the first filtered item collection to form a second filtered item collection. Further, associating an item score to each of a plurality of items in the first and second filtered item collections, sorting the plurality of items in the first and second filtered item collections according to the item score associated to each of the plurality of items, and providing, to a user display, an item in the plurality of items in the first or second filtered item collections according to a sorting order.

Abstract: Methods for predicting network access probability of data files accessible over a computer network are provided. In one aspect, a method includes generating a primary data vector for a media file based on a stored data representation of the file, and providing the data vector for the file to an algorithm that uses past interaction information for at least one other media file from a collection of media files having a degree of similarity with the media file above a threshold similarity value. The method also includes receiving, as an output of the algorithm, a marketability score for the media file, the score indicative of a likelihood that a user will download the media file. Systems and machine-readable media are also provided.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for user drawing based image search. These aspects include an image retrieval system using a convolutional neural network trained to identify how users draw semantic concepts and using an image search engine to search against images having a similar concept. The aspects include mapping between concepts of the user drawing space and concepts of the image space such that images associated with the same concept are identified. For each input user drawing, the drawing is first processed through a concept classifier to identify a corresponding concept, and then through a feature extractor to form a corresponding feature vector. The results from the concept classifier and the feature extractor may be combined to search against a collection of images having a similar concept to determine a listing of images ranked by visual and semantic similarity to the input.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for language translation based on image search similarities. These aspects include an image retrieval system using a convolutional neural network that is trained to identify a correlation between an image and a language term, and using an image search engine to search against images corresponding to visual words that are responsive to a given search query in a given spoken language. These aspects include access to interaction probability data that identifies user interaction probabilities for the visual words to determine a correlation between the input language terms of the search query and the rate at which users interact with images of a corresponding visual word that is responsive to the search query. The system then provides a prioritized listing of images that is responsive to the given search query based on the identified user interaction probabilities.

Abstract: A computer-implemented method is provided for retrieving an image from a user in a desired format and for detecting a compression efficiency for the image. When the compression efficiency is above a pre-selected threshold the computer-implemented method includes obtaining a saliency representation of the image, capturing a feature description of a non-salient portion of the image, flattening the non-salient portion in a new image, storing the new image in a selected format in a memory and storing a background descriptor for the image in the memory.

Abstract: Methods for predicting network access probability of data files accessible over a computer network are provided. In one aspect, a method includes generating a primary data vector for a media file based on a stored data representation of the file, and providing the data vector for the file to an algorithm that uses past interaction information for at least one other media file from a collection of media files having a degree of similarity with the media file above a threshold similarity value. The method also includes receiving, as an output of the algorithm, a marketability score for the media file, the score indicative of a likelihood that a user will download the media file. Systems and machine-readable media are also provided.

Abstract: Methods for prioritizing a set of images identified as responsive to an image search query from a user based on features of the images identified as relevant to a geographic region of the user are provided. In one aspect, the method includes submitting a plurality of images to a computer-operated convolutional neural network that is configured to analyze image pixel data for each of the plurality of images to identify features, in each of the plurality of images, influencing a download probability of the corresponding image in a plurality of geographic regions. The method also includes receiving, from the neural network and for each of the plurality of images, a download probability of each image for each of the plurality of geographic regions. Systems and machine-readable media are also provided.

Abstract: A method for receiving a first user query from a user for searching an item, forming a first filter based on the first user query, and forming a first filtered item collection is provided. The method includes predicting a new query based on the first user query and a historical query log, forming a second filter for the new query, and applying the second filter to the first filtered item collection to form a second filtered item collection. Further, associating an item score to each of a plurality of items in the first and second filtered item collections, sorting the plurality of items in the first and second filtered item collections according to the item score associated to each of the plurality of items, and providing, to a user display, an item in the plurality of items in the first or second filtered item collections according to a sorting order.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for identification of a salient portion of an image. A system may receive user input identifying a search query for content from a client device. The system may determine a listing of images responsive to the search query from an image collection. The system may obtain one or more image crops for at least one image of the listing of images based on a saliency map of the at least one image. In one or more implementations, each of the one or more image crops indicates a salient region of a corresponding image. The system may provide a set of search results responsive to the search query to the client device. In one or more implementations, the set of search results includes the obtained one or more image crops in a prioritized listing of image crops.

Abstract: A computer-implemented method is provided for retrieving an image from a user in a desired format and for detecting a compression efficiency for the image. When the compression efficiency is above a pre-selected threshold the computer-implemented method includes obtaining a saliency representation of the image, capturing a feature description of a non-salient portion of the image, flattening the non-salient portion in a new image, storing the new image in a selected format in a memory and storing a background descriptor for the image in the memory.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for identification of a salient portion of an image. A system may receive user input identifying a search query for content from a client device. The system may determine a listing of images responsive to the search query from an image collection. The system may obtain one or more image crops for at least one image of the listing of images based on a saliency map of the at least one image. In one or more implementations, each of the one or more image crops indicates a salient region of a corresponding image. The system may provide a set of search results responsive to the search query to the client device. In one or more implementations, the set of search results includes the obtained one or more image crops in a prioritized listing of image crops.

Abstract: Methods for prioritizing a set of images identified as responsive to an image search query from a user based on features of the images identified as relevant to a geographic region of the user are provided. In one aspect, the method includes submitting a plurality of images to a computer-operated convolutional neural network that is configured to analyze image pixel data for each of the plurality of images to identify features, in each of the plurality of images, influencing a download probability of the corresponding image in a plurality of geographic regions. The method also includes receiving, from the neural network and for each of the plurality of images, a download probability of each image for each of the plurality of geographic regions. Systems and machine-readable media are also provided.

Abstract: Methods for predicting network access probability of data files accessible over a computer network are provided. In one aspect, a method includes generating a primary data vector for a media file based on a stored data representation of the file, and providing the data vector for the file to an algorithm that uses past interaction information for at least one other media file from a collection of media files having a degree of similarity with the media file above a threshold similarity value. The method also includes receiving, as an output of the algorithm, a marketability score for the media file, the score indicative of a likelihood that a user will download the media file. Systems and machine-readable media are also provided.