Abstract: Systems and methods for forming collaboration recommendations. Techniques for forming event-based recommendations use time-decayed event values. A shared content management system supports a plurality of users that generate events by interacting with content objects of the shared content management system. Events over the content objects are captured as event objects. Method steps are invoked upon receiving event objects that describes user-to-object interaction events that arise from interactions by users over content objects. Different types of interactions carry different importance values. The importance values can be applied as weights when scoring user-to-object interaction activities. The importance can decay over time. As time progresses and as the importance of older interactions decay, score components of a user-to-object interaction can be updated based at least in part on a time decay function. The system emits collaboration recommendations based on the decayed user-to-user collaboration scores.

Abstract: Systems and methods for forming collaboration recommendations. Techniques for forming event-based recommendations use time-decayed event values. A shared content management system supports a plurality of users that generate events by interacting with content objects of the shared content management system. Events over the content objects are captured as event objects. Method steps are invoked upon receiving event objects that describes user-to-object interaction events that arise from interactions by users over content objects. Different types of interactions carry different importance values. The importance values can be applied as weights when scoring user-to-object interaction activities. The importance can decay over time. As time progresses and as the importance of older interactions decay, score components of a user-to-object interaction can be updated based at least in part on a time decay function. The system emits collaboration recommendations based on the decayed user-to-user collaboration scores.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.

Abstract: Systems, apparatus, and methods for determining unique contacts from a collection or pool of merchant data are discussed herein. Some embodiments may provide for an apparatus including circuitry configured to: access first merchant data associated with a first merchant; access second merchant data associated with a second merchant; determine a match score based the first merchant data and the second merchant data indicating a likelihood of the first merchant being the same as the second merchant; determine a match score threshold; determine whether the match score exceeds the match score threshold; and in response determining the match score fails to exceed the match score threshold, determine the first merchant as being different from the second merchant. Some embodiments may provide for techniques for machine learning with merchant data training sets to determine match scores.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.

Abstract: Embodiments provide for automated dynamic data quality assessment. A data quality job including a new data sample is received. If the new data sample is determined to be added to a reservoir of data samples, a quality verification request is sent to an oracle. A new data sample quality estimate is received from the oracle. The new data sample and estimate can be added to the reservoir. A judgment associated with a new data sample is received from a predictive model. Whether to send a new data sample quality verification request to an oracle is determined. If a new data sample quality estimate is received from the oracle, whether to add the new data sample and the judgment to the reservoir is determined.

Abstract: Systems, apparatus, and methods for determining unique contacts from a collection or pool of merchant data are discussed herein. Some embodiments may provide for an apparatus including circuitry configured to determine programmatic match results indicating whether different instances of merchant data match (e.g., describe the same contact). The circuitry may further determine probabilities of precision or recall errors with the programmatic match results. Programmatic match results having a high probability of error may be annotated by a user to generate user match results. The user match results may be used to generate a more reliable contacts database including unique contacts, as well as to train and/or update the match scoring algorithm. As such, the accuracy of machine-implemented binary classification is improved.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.

Abstract: Systems, apparatus, and methods for determining unique contacts from a collection or pool of merchant data are discussed herein. Some embodiments may provide for an apparatus including circuitry configured to: access first merchant data associated with a first merchant; access second merchant data associated with a second merchant; determine a match score based the first merchant data and the second merchant data indicating a likelihood of the first merchant being the same as the second merchant; determine a match score threshold; determine whether the match score exceeds the match score threshold; and in response determining the match score fails to exceed the match score threshold, determine the first merchant as being different from the second merchant. Some embodiments may provide for techniques for machine learning with merchant data training sets to determine match scores.

Abstract: Systems, apparatus, and methods for determining unique contacts from a collection or pool of merchant data are discussed herein. Some embodiments may provide for an apparatus including circuitry configured to determine programmatic match results indicating whether different instances of merchant data match (e.g., describe the same contact). The circuitry may further determine probabilities of precision or recall errors with the programmatic match results. Programmatic match results having a high probability of error may be annotated by a user to generate user match results. The user match results may be used to generate a more reliable contacts database including unique contacts, as well as to train and/or update the match scoring algorithm. As such, the accuracy of machine-implemented binary classification is improved.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for modeling multi-dimensional, dynamically evolving data using dynamic clustering. In one aspect, a method includes receiving a core group of clusters of objects, each object being represented by a corresponding instance of a multi-dimensional feature vector including a dimension k; receiving a stream of data points representing a group of objects, each data point respectively representing an instance of dimension k describing a feature of an object within the group of objects; and, for each data point, adding an object described by the data point to a first cluster of objects within the core group of clusters; updating properties of the first cluster of objects in response to adding the object; and determining whether to update the core group of clusters using the updated properties of the first cluster of objects.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for modeling multi-dimensional, dynamically evolving data using dynamic clustering. In one aspect, a method includes receiving a core group of clusters of objects, each object being represented by a corresponding instance of a multi-dimensional feature vector including a dimension k; receiving a stream of data points representing a group of objects, each data point respectively representing an instance of dimension k describing a feature of an object within the group of objects; and, for each data point, adding an object described by the data point to a first cluster of objects within the core group of clusters; updating properties of the first cluster of objects in response to adding the object; and determining whether to update the core group of clusters using the updated properties of the first cluster of objects.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for an adaptive oracle-trained learning framework for automatically building and maintaining models that are developed using machine learning algorithms. In embodiments, the framework leverages at least one oracle (e.g., a crowd) for automatic generation of high-quality training data to use in deriving a model. Once a model is trained, the framework monitors the performance of the model and, in embodiments, leverages active learning and the oracle to generate feedback about the changing data for modifying training data sets while maintaining data quality to enable incremental adaptation of the model.

Abstract: Systems for forming and maintaining spontaneous networks of collaborators in shared content management systems. A shared content management system supports user interactions with content objects. A service of the content management system monitors occurrences of interactions between users and objects. The users are associated with collaboration groups. To generate recommendations of groups other than the collaboration group or groups in which a particular user is already a member, a method embodiment receives entity relationship scores from the service. An entity relationship score quantifies a relationship between two subject entities that are common to a particular entity interaction event. The method then assigns the subject entities to one or more spontaneously-generated clusters. As clusters are formed and populated, cluster affinity scores are continuously calculated. Periodically, a recommended cluster is selected based on a corresponding cluster affinity score.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for an adaptive oracle-trained learning framework for automatically building and maintaining models that are developed using machine learning algorithms. In embodiments, the framework leverages at least one oracle (e.g., a crowd) for automatic generation of high-quality training data to use in deriving a model. Once a model is trained, the framework monitors the performance of the model and, in embodiments, leverages active learning and the oracle to generate feedback about the changing data for modifying training data sets while maintaining data quality to enable incremental adaptation of the model.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for modeling multi-dimensional, dynamically evolving data using dynamic clustering. In one aspect, a method includes receiving a core group of clusters of objects, each object being represented by a corresponding instance of a multi-dimensional feature vector including a dimension k; receiving a stream of data points representing a group of objects, each data point respectively representing an instance of dimension k describing a feature of an object within the group of objects; and, for each data point, adding an object described by the data point to a first cluster of objects within the core group of clusters; updating properties of the first cluster of objects in response to adding the object; and determining whether to update the core group of clusters using the updated properties of the first cluster of objects.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.

Abstract: Systems, apparatus, and methods for determining unique contacts from a collection or pool of merchant data are discussed herein. Some embodiments may provide for an apparatus including circuitry configured to determine programmatic match results indicating whether different instances of merchant data match (e.g., describe the same contact). The circuitry may further determine probabilities of precision or recall errors with the programmatic match results. Programmatic match results having a high probability of error may be annotated by a user to generate user match results. The user match results may be used to generate a more reliable contacts database including unique contacts, as well as to train and/or update the match scoring algorithm. As such, the accuracy of machine-implemented binary classification is improved.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automated dynamic data quality assessment. One aspect of the subject matter described in this specification includes the actions of receiving a data quality job including a new data sample; and, if the new data sample is determined to be added to a reservoir of data samples, sending a quality verification request to an oracle; receiving a new data sample quality estimate from the oracle; and adding the new data sample and estimate to the reservoir. A second aspect of the subject matter includes the actions of receiving, from a predictive model, a judgment associated with a new data sample; analyzing the new data sample based in part on the judgment to determine whether to send a new data sample quality verification request to an oracle; and, if a new data sample quality estimate is received from the oracle, determining whether to add the new data sample and the judgment to the reservoir.