Abstract: Techniques for automated machine learning (ML) pipeline exploration and deployment are described. An automated ML pipeline generation system allows users to easily construct optimized ML pipelines by providing a dataset, identifying a target column in the dataset, and providing an exploration budget. Multiple candidate ML pipelines can be identified and evaluated through an exploration process, and a best ML pipeline can be provided to the requesting user or deployed for production inference. Users can configure, monitor, and adapt the exploration at multiple points in time throughout.

Abstract: Techniques for automated machine learning (ML) pipeline exploration and deployment are described. An automated ML pipeline generation system allows users to easily construct optimized ML pipelines by providing a dataset, identifying a target column in the dataset, and providing an exploration budget. Multiple candidate ML pipelines can be identified and evaluated through an exploration process, and a best ML pipeline can be provided to the requesting user or deployed for production inference. Users can configure, monitor, and adapt the exploration at multiple points in time throughout.

Abstract: Techniques for intelligent compute resource selection and utilization for machine learning training jobs are described. At least a portion of a machine learning (ML) training job is executed a plurality of times using a plurality of different resource configurations, where each of the plurality of resource configurations includes at least a different type or amount of compute instances. A performance metric is measured for each of the plurality of the executions, and can be used along with a desired performance characteristic to generate a recommended resource configuration for the ML training job. The ML training job is executed using the recommended resource configuration.

Abstract: Methods, systems, and computer-readable media for automated problem detection for machine learning models are disclosed. A machine learning analysis system receives data associated with use of a machine learning model. The data was collected by a machine learning inference system and comprises input to the model or a plurality of inferences representing output of the machine learning model. The machine learning analysis system performs analysis of the data associated with the use of the machine learning model. The machine learning analysis system detects one or more problems associated with the use of the machine learning model based at least in part on the analysis. The machine learning analysis system initiates one or more remedial actions associated with the one or more problems associated with the use of the machine learning model.

Abstract: Techniques for dynamic accuracy-based experimentation and deployment of machine learning (ML) models are described. Inference traffic flowing to ML models and the accuracy of the models is analyzed and used to ensure that better performing models are executed more often via model selection. A predictive component can evaluate which model is more likely to be accurate for certain input data elements. Ensemble techniques can combine inference results of multiple ML models to aim to achieve a better overall result than any individual model could on its own.

Abstract: Methods, systems, and computer-readable media for automated problem detection for machine learning models are disclosed. A machine learning analysis system receives data associated with use of a machine learning model. The data was collected by a machine learning inference system and comprises input to the model or a plurality of inferences representing output of the machine learning model. The machine learning analysis system performs analysis of the data associated with the use of the machine learning model. The machine learning analysis system detects one or more problems associated with the use of the machine learning model based at least in part on the analysis. The machine learning analysis system initiates one or more remedial actions associated with the one or more problems associated with the use of the machine learning model.

Abstract: Techniques for automated machine learning (ML) pipeline exploration and deployment are described. An automated ML pipeline generation system allows users to easily construct optimized ML pipelines by providing a dataset, identifying a target column in the dataset, and providing an exploration budget. Multiple candidate ML pipelines can be identified and evaluated through an exploration process, and a best ML pipeline can be provided to the requesting user or deployed for production inference. Users can configure, monitor, and adapt the exploration at multiple points in time throughout.

Abstract: Disclosed is a system and method for email management that leverages information derived from automatically generated templates in order to identify types of message and message content. The systems and methods discussed herein involve identifying messages matching specific template types and structures, and automatically extracting important data from email messages matching those templates. The extracted data enables improvements for a user's experience and increased monetization. That is, templates can be analyzed to determine a type of email message, which in turn can be presented to a receiving user within an automatic folder or tag designation. Additionally, email snippets or previews can be generated from the extracted data for display within a user's inbox. Also, the extracted data can be used for monetization purposes, by serving targeted advertisements based upon the data extracted from such messages.

Abstract: A computer system, computer program product, and computer-implemented method for communicating electronic messages over a communication network coupled thereto are provided. The computer system comprises a network interface for receiving messages sent over the network and addressed to a user of the computer system; and computer executable electronic message processing software. The software comprises instructions for directing the computer system to receive a message over the network, and to identify whether a sender of the received electronic message is a human or a machine. The identifying includes first and second phases of operation. The first phase includes an offline phase employing information and activities resident on the computer system. The second phase includes an online phase employing resources remotely accessible over the network.

Abstract: Method, system, and programs for estimating interests of a plurality of users with respect to a new piece of information are disclosed. In one example, historical interests of the plurality of users are obtained with respect to one or more existing pieces of information. One or more users are selected from the plurality of users. Historical interests of the one or more users can minimize an objective function over the plurality of users. Interests of the one or more users are obtained with respect to the new piece of information. Estimated interests of the plurality of users are generated with respect to the new piece of information based on the obtained interests of the one or more users.

Abstract: Disclosed is a system, method, and non-transitory computer readable storage medium for predicting future messages. A processor receives a message sent to a user operating a client device, analyzes the message in light of previously identified patterns and scores assigned to scanned messages, determines a future message that should be received by the client device based on the received message, and transmits an item of information based on the determined future message.

Abstract: As is disclosed herein, user behavior in connection with a number of electronic messages, such as electronic mail (email) messages, can be used to automatically learn from, and predict, whether a message is wanted or unwanted by the user, where an unwanted message is referred to herein as gray spam. A gray spam predictor is personalized for a given user in vertical learning that uses the user's electronic message behavior and horizontal learning that uses other users' message behavior. The gray spam predictor can be used to predict whether a new message for the user is, or is not, gray spam. A confidence in a prediction may be used in determining the disposition of the message, such as and without limitation placing the message in a spam folder, a gray spam folder and/or requesting input from the user regarding the disposition of the message, for example.

Abstract: Techniques for dynamic accuracy-based experimentation and deployment of machine learning (ML) models are described. Inference traffic flowing to ML models and the accuracy of the models is analyzed and used to ensure that better performing models are executed more often via model selection. A predictive component can evaluate which model is more likely to be accurate for certain input data elements. Ensemble techniques can combine inference results of multiple ML models to aim to achieve a better overall result than any individual model could on its own.

Abstract: A search query for searching electronic messages, such as email, may be used to search for different types of items, such as and without limitation electronic messages, contacts, photos, documents, such as and without limitation papers, presentations, etc., business entities, personal information extracted from messages, such as and without limitation purchase orders, shipments, reservations, travel itineraries, etc. Several sources of data, which may be indexed for searching, such as and without limitation a personal mail search index, contacts, or business entity, index, attachments index, extracted data index, etc. may be searched using the search query. A number of top search result items, which may include different types of items, may be presented apart from other search result items.

Abstract: Disclosed is a system and method for managing mailing list newsletter messages for a recipient user, and organizing such messages in accordance with a receiving user's interests. The present disclosure enables novel organizational tools for emails by intuitively organizing received newsletters and providing highly visible features within a user's inbox respective the newsletters. Organization of received newsletters is ensured through ranking users' mailing lists according to his/her interest in them, based on actions made by the respective user, in addition to actions of other users receiving the same newsletters. Additionally, upon reception of such newsletters, the present disclosure provides a specialized view, in addition to added functionality within a user's inbox, thereby enhancing a user's experience and engagement with received messages of a newsletter.

Abstract: As is disclosed herein, user behavior in connection with a number of electronic messages, such as electronic mail (email) messages, can be used to automatically learn from, and predict, whether a message is wanted or unwanted by the user, where an unwanted message is referred to herein as gray spam. A gray spam predictor is personalized for a given user in vertical learning that uses the user's electronic message behavior and horizontal learning that uses other users' message behavior. The gray spam predictor can be used to predict whether a new message for the user is, or is not, gray spam. A confidence in a prediction may be used in determining the disposition of the message, such as and without limitation placing the message in a spam folder, a gray spam folder and/or requesting input from the user regarding the disposition of the message, for example.

Abstract: A search query for searching electronic messages, such as email, may be used to search for different types of items, such as and without limitation electronic messages, contacts, photos, documents, such as and without limitation papers, presentations, etc., business entities, personal information extracted from messages, such as and without limitation purchase orders, shipments, reservations, travel itineraries, etc. Several sources of data, which may be indexed for searching, such as and without limitation a personal mail search index, contacts, or business entity, index, attachments index, extracted data index, etc. may be searched using the search query. A number of top search result items, which may include different types of items, may be presented apart from other search result items.

Abstract: Method, system, and programs for estimating interests of a plurality of users with respect to a new piece of information are disclosed. In one example, historical interests of the plurality of users are obtained with respect to one or more existing pieces of information. One or more users are selected from the plurality of users. Historical interests of the one or more users can minimize an objective function over the plurality of users. Interests of the one or more users are obtained with respect to the new piece of information. Estimated interests of the plurality of users are generated with respect to the new piece of information based on the obtained interests of the one or more users.

Abstract: Disclosed is a system, method, and non-transitory computer readable storage medium for predicting future messages. A processor receives a message sent to a user operating a client device, analyzes the message in light of previously identified patterns and scores assigned to scanned messages, determines a future message that should be received by the client device based on the received message, and transmits an item of information based on the determined future message.

Abstract: A computer system, computer program product, and computer-implemented method for communicating electronic messages over a communication network coupled thereto are provided. The computer system comprises a network interface for receiving messages sent over the network and addressed to a user of the computer system; and computer executable electronic message processing software. The software comprises instructions for directing the computer system to receive a message over the network, and to identify whether a sender of the received electronic message is a human or a machine. The identifying includes first and second phases of operation. The first phase includes an offline phase employing information and activities resident on the computer system. The second phase includes an online phase employing resources remotely accessible over the network.