Patents by Inventor Ralf Herbrich

Ralf Herbrich has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20120150771
    Abstract: Knowledge corroboration is described. In an embodiment many judges provide answers to many questions so that at least one answer is provided to each question and at least some of the questions have answers from more than one judge. In an example a probabilistic learning system takes features describing the judges or the questions or both and uses those features to learn an expertise of each judge. For example, the probabilistic learning system has a graphical assessment component which aggregates the answers in a manner which takes into account the learnt expertise in order to determine enhanced answers. In an example the enhanced answers are used for knowledge base clean-up or web-page classification and the learnt expertise is used to select judges for future questions. In an example the probabilistic learning system has a logical component that propagates answers according to logical relations between the questions.
    Type: Application
    Filed: December 8, 2010
    Publication date: June 14, 2012
    Applicant: Microsoft Corporation
    Inventors: Gjergji Kasneci, Jurgen Ann Francois Marie Van Gael, Thore Graepel, Ralf Herbrich, David Stern
  • Patent number: 8175726
    Abstract: Skill scores represent a ranking or other indication of the skill of the player based on the outcome of the game in a gaming environment. Skills scores can be used in matching compatible players on the same team and matching opposing players or teams to obtain an evenly-matched competition. An initial skill score of a player in a new gaming environment may be based in whole or in part on the skill score of that player in another game environment. The influence that the skill scores for these other game environments may have in the skill score seeding for the new game environment may be weighted based on a defined compatibility factor with the new game environment. The compatibility factor can be determined based on a game-to-game basis, compatible categories or features, game developer defined parameters, or any combination of considerations.
    Type: Grant
    Filed: September 29, 2006
    Date of Patent: May 8, 2012
    Assignee: Microsoft Corporation
    Inventors: Ralf Herbrich, Thore K. H. Graepel
  • Publication number: 20120101965
    Abstract: Machine learning techniques may be used to train computing devices to understand a variety of documents (e.g., text files, web pages, articles, spreadsheets, etc.). Machine learning techniques may be used to address the issue that computing devices may lack the human intellect used to understand such documents, such as their semantic meaning. Accordingly, a topic model may be trained by sequentially processing documents and/or their features (e.g., document author, geographical location of author, creation date, social network information of author, and/or document metadata). Additionally, as provided herein, the topic model may be used to predict probabilities that words, features, documents, and/or document corpora, for example, are indicative of particular topics.
    Type: Application
    Filed: October 26, 2010
    Publication date: April 26, 2012
    Applicant: Microsoft Corporation
    Inventors: Philipp Hennig, David Stern, Thore Graepel, Ralf Herbrich
  • Publication number: 20120089446
    Abstract: A publishing engine captures commercial information associated with a first user and automatically notifies other users in the first user's social network of this commercial information. The first user authorizes an e-commerce system to access his or her social network and to publish commercial information about the first user's commercial activity (e.g., a purchase or other commercial transaction) to users in the social network. By this automated notification, the notified users in the first user's social network can learn that the first user has completed a commercial transaction pertaining to a particular product or service. If a notified user is interested in a similar product or service, he or she can contact the first user to inquire about the first user's experience and information with the product or service.
    Type: Application
    Filed: October 7, 2010
    Publication date: April 12, 2012
    Applicant: Microsoft Corporation
    Inventors: Anoop Gupta, Thore Graepel, Ralf Herbrich
  • Publication number: 20120089581
    Abstract: A publishing engine captures capturing commercial events and other information (collectively, “commercial information”) associated with a first user and automatically notifies other users in the social network of the first user of this commercial information. The publishing engine also notifies one or more search engines of these events and information. Based on this commercial information, the search engine can augment search results of the members of the social network to include historical notifications relating to commercial transactions for similar products and/or services by others in their social network. In this manner, for example, the search engine can provide results directing the searcher to other users in their social network who have purchased such products and/or services.
    Type: Application
    Filed: October 7, 2010
    Publication date: April 12, 2012
    Applicant: Microsoft Corporation
    Inventors: Anoop Gupta, Thore Graepel, Ralf Herbrich
  • Publication number: 20120071239
    Abstract: A real-time stereo video signal of a captured scene with a physical foreground object and a physical background is received. In real-time, a foreground/background separation algorithm is used on the real-time stereo video signal to identify pixels from the stereo video signal that represent the physical foreground object. A video sequence may be produced by rendering a 3D virtual reality based on the identified pixels of the physical foreground object.
    Type: Application
    Filed: November 28, 2011
    Publication date: March 22, 2012
    Applicant: Microsoft Corporation
    Inventors: Thore KH Graepel, Andrew Blake, Ralf Herbrich
  • Patent number: 8094928
    Abstract: A real-time stereo video signal of a captured scene with a physical foreground object and a physical background is received. In real-time, a foreground/background separation algorithm is used on the real-time stereo video signal to identify pixels from the stereo video signal that represent the physical foreground object. A video sequence is produced by rendering a 3d virtual reality based on the identified pixels of the physical foreground object.
    Type: Grant
    Filed: November 14, 2005
    Date of Patent: January 10, 2012
    Assignee: Microsoft Corporation
    Inventors: Thore K H Graepel, Andrew Blake, Ralf Herbrich
  • Publication number: 20110320767
    Abstract: Methods, systems, and media are provided for a dynamic batch strategy utilized in parallelization of online learning algorithms. The dynamic batch strategy provides a merge function on the basis of a threshold level difference between the original model state and an updated model state, rather than according to a constant or pre-determined batch size. The merging includes reading a batch of incoming streaming data, retrieving any missing model beliefs from partner processors, and training on the batch of incoming streaming data. The steps of reading, retrieving, and training are repeated until the measured difference in states exceeds a set threshold level. The measured differences which exceed the threshold level are merged for each of the plurality of processors according to attributes. The merged differences which exceed the threshold level are combined with the original partial model states to obtain an updated global model state.
    Type: Application
    Filed: June 24, 2010
    Publication date: December 29, 2011
    Applicant: MICROSOFT CORPORATION
    Inventors: Taha Bekir Eren, Oleg Isakov, Weizhu Chen, Jeffrey Scott Dunn, Thomas Ivan Borchert, Joaquin Quinonero Candela, Thore Kurt Hartwig Graepel, Ralf Herbrich
  • Publication number: 20110313833
    Abstract: Online recommendations are tracked through a forwarding service. For example, a user may send an email to a friend recommending a product specified at a web site identified by a URI embedded in the email. Before sending the email, the user submits the URI to a forwarding service, which returns a new URI mapped to the original URI and to the recommending user. The recommending user can then recommend the web site by forwarding the new URI to the friend. If the friend selects the recommended URI to review the web site, the forwarding service records the decision to review the web site and directs the friend to the recommended web site. The forwarding service maintains a database of recommendations made by the recommending user, recommendation consumed by the friend, etc. Incentives can be provided to the recommending user and the friend to encourage recommendations.
    Type: Application
    Filed: June 18, 2010
    Publication date: December 22, 2011
    Applicant: Microsoft Corporation
    Inventors: Thore Graepel, Ralf Herbrich
  • Publication number: 20110313832
    Abstract: Online recommendations are tracked through a forwarding service. The forwarding service can provide such statistics to an ad service, which can provide incentives to the recommending user and a consuming user. Example incentives may include an accumulation of points by the recommending user, a discount to the consuming user if a purchase is made in response to the recommendation, etc. To determine how much of an incentive each participant in the recommendation flow receives, a graph is created to model the recommendation flow and incentives are allocated using a cooperative game description based on this graph that associates each participant with a power index that represents that participants share of the incentive.
    Type: Application
    Filed: June 18, 2010
    Publication date: December 22, 2011
    Applicant: Microsoft Corporation
    Inventors: Ralf Herbrich, Thore Graepel, Yoram Bachrach
  • Publication number: 20110231405
    Abstract: Data structures for collaborative filtering systems are described. In an embodiment sketches which extremely concisely represent a list of items that a user has rated are created and stored for use by a collaborative filtering system to recommend items. For example, the sketches are created by using several versions of a cryptographic hash function to permute the item list and store a minimal value from each permutation in the sketch together with a user rating. In examples the sketches are used to compute estimates of similarity measures between pairs of users such as rank correlations including Spearman's Rho and Kendall's Tau. For example, the similarity measures are used by a collaborative filtering system to accurately and efficiently recommend items to users. For example the sketches are so concise that massive amounts of data can be taken into account in order to give high quality recommendations in a practical manner.
    Type: Application
    Filed: March 17, 2010
    Publication date: September 22, 2011
    Applicant: Microsoft Corporation
    Inventors: Ralf Herbrich, Yoram Bachrach
  • Publication number: 20110218946
    Abstract: A user may request a presentation of a content item set, such as a social network comprising a set of status messages or an image database comprising a set of images. However, the volume and diversity of content items of the content item set may reduce the interest of the user in the presented content items. The potential interest of the user in the presented content items may be improved by selecting content items that are associated with one or more topics of potential interest to the user, and having a positive trending popularity among users of the content item set. Moreover, the interaction of the user with a presented content item may be monitored and used to determine the interest of the user in the topics associated with the presented content item and the popularity of the content item.
    Type: Application
    Filed: March 3, 2010
    Publication date: September 8, 2011
    Applicant: Microsoft Corporation
    Inventors: David Stern, Ralf Herbrich, Milad Shokouhi, Thore Kurt Hartwig Graepel
  • Publication number: 20110184778
    Abstract: Event prediction in dynamic environments is described. In an embodiment a prediction engine may use the learnt information to predict events in order to control a system such as for internet advertising, email filtering, fraud detection or other applications. In an example one or more variables exists for pre-specified features describing or associated with events and each variable is considered to have an associated weight and time stamp. For example, belief about each weight is represented using a probability distribution and a dynamics process is used to modify the probability distribution in a manner dependent on the time stamp for that weight. For example, the uncertainty about the associated variable's influence on prediction of future events is increased. Examples of different schedules for applying the dynamics process are given.
    Type: Application
    Filed: January 27, 2010
    Publication date: July 28, 2011
    Applicant: Microsoft Corporation
    Inventors: Thore Graepel, Joaquin Quinonero Candela, Thomas Ivan Borchert, Ralf Herbrich
  • Patent number: 7965295
    Abstract: Improved human-like realism of computer opponents in racing or motion-related games is provided by using a mixture model to determine a dynamically prescribed racing line that the AI driver is to follow for a given segment of the race track. This dynamically prescribed racing line may vary from segment to segment and lap to lap, roughly following an ideal line with some variation. As such, the AI driver does not appear to statically follow the ideal line perfectly throughout the race. Instead, within each segment of the course, the AI driver's path may smoothly follow a probabilistically-determined racing line defined relative to at least one prescribed racing line.
    Type: Grant
    Filed: April 2, 2009
    Date of Patent: June 21, 2011
    Assignee: Microsoft Corporation
    Inventors: Ralf Herbrich, Mark Hatton, Michael E Tipping
  • Publication number: 20110137629
    Abstract: An automatic algorithm for finding racing lines via computerized minimization of a measure of the curvature of a racing line is derived. Maximum sustainable speed of a car on a track is shown to be inversely proportional to the curvature of the line it is attempting to follow. Low curvature allows for higher speed given that a car has some maximum lateral traction when cornering. The racing line can also be constrained, or “pinned,” at arbitrary points on the track. Pinning may be performed randomly, deterministically, or manually and allows, for example, a line designer to pin the line at any chosen points on the track, such that when the automatic algorithm is run, it will produce the smoothest line that still passes through all the specified pins.
    Type: Application
    Filed: January 13, 2011
    Publication date: June 9, 2011
    Applicant: MICROSOFT CORPORATION
    Inventors: Michael E. Tipping, Mark Andrew Hatton, Ralf Herbrich
  • Publication number: 20110131163
    Abstract: Managing a portfolio of experts is described where the experts may be for example, automated experts or human experts. In an embodiment a selection engine selects an expert from a portfolio of experts and assigns the expert to a specified task. For example, the selection engine has a Bayesian machine learning system which is iteratively updated each time an experts performance on a task is observed. For example, sparsely active binary task and expert feature vectors are input to the selection engine which maps those feature vectors to a multi-dimensional trait space using a mapping learnt by the machine learning system. In examples, an inner product of the mapped vectors gives an estimate of a probability distribution over expert performance. In an embodiment the experts are automated problem solvers and the task is a hard combinatorial problem such as a constraint satisfaction problem or combinatorial auction.
    Type: Application
    Filed: December 1, 2009
    Publication date: June 2, 2011
    Applicant: Microsoft Corporation
    Inventors: David Stern, Horst Cornelius Samulowitz, Ralf Herbrich, Thore Graepel
  • Publication number: 20110066577
    Abstract: Machine learning using relational databases is described. In an embodiment a model of a probabilistic relational database is formed by augmenting relation schemas of a relational database with probabilistic attributes. In an example, the model comprises constraints introduced by linking the probabilistic attributes using factor statements. For example, a compiler translates the model into a factor graph data structure which may be passed to an inference engine to carry out machine learning. For example, this enables machine learning to be integrated with the data and it is not necessary to pre-process or reformat large scale data sets for a particular problem domain. In an embodiment a machine learning system for estimating skills of players in an online gaming environment is provided. In another example, a machine learning system for data mining of medical data is provided. In some examples, missing attribute values are filled using machine learning results.
    Type: Application
    Filed: September 15, 2009
    Publication date: March 17, 2011
    Applicant: Microsoft Corporation
    Inventors: Jurgen Anne Francois Marie Van Gael, Ralf Herbrich, Thore Graepel
  • Patent number: 7892078
    Abstract: An automatic algorithm for finding racing lines via computerized minimization of a measure of the curvature of a racing line is derived. Maximum sustainable speed of a car on a track is shown to be inversely proportional to the curvature of the line it is attempting to follow. Low curvature allows for higher speed given that a car has some maximum lateral traction when cornering. The racing line can also be constrained, or “pinned,” at arbitrary points on the track. Pinning may be randomly, deterministically, or manually and allows, for example, a line designer to pin the line at any chosen points on the track, such that when the automatic algorithm is run, it will produce the smoothest line that still passes through all the specified pins.
    Type: Grant
    Filed: December 30, 2005
    Date of Patent: February 22, 2011
    Assignee: Microsoft Corporation
    Inventors: Michael E. Tipping, Mark Andrew Hatton, Ralf Herbrich
  • Patent number: 7846024
    Abstract: Players in a gaming environment, particularly, electronic on-line gaming environments, may be scored relative to each other or to a predetermined scoring system. The scoring of each player may be based on the outcomes of games between players who compete against each other in one or more teams of one or more players. Each player's score may be represented as a distribution over potential scores which may indicate a confidence level in the distribution representing the player's score. The score distribution for each player may be modeled with a Gaussian distribution and may be determined through a Bayesian inference algorithm. The scoring may be used to track a player's progress and/or standing within the gaming environment, used in a leaderboard indication of rank, and/or may be used to match players with each other in a future game.
    Type: Grant
    Filed: November 17, 2006
    Date of Patent: December 7, 2010
    Assignee: Micorsoft Corporation
    Inventors: Thore KH Graepel, Ralf Herbrich
  • Patent number: 7837543
    Abstract: Adaptive agents are driven by rewards they receive based on the outcome of their behavior during actual game play. Accordingly, the adaptive agents are able to learn from experience within the gaming environment. Reward-driven adaptive agents can be trained at either or both of game-time or development time. Computer-controlled agents receive rewards (either positive or negative) at individual action intervals based on the effectiveness of the agents' actions (e.g., compliance with defined goals). The adaptive computer-controlled agent is motivated to perform actions that maximize its positive rewards and minimize is negative rewards.
    Type: Grant
    Filed: April 30, 2004
    Date of Patent: November 23, 2010
    Assignee: Microsoft Corporation
    Inventors: Kurt Hartwig Graepel, Ralf Herbrich, Julian Gold