Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting actions from large discrete action sets. One of the methods includes receiving a particular observation representing a particular state of an environment; and selecting an action from a discrete set of actions to be performed by an agent interacting with the environment, comprising: processing the particular observation using an actor policy network to generate an ideal point; determining, from the points that represent actions in the set, the k nearest points to the ideal point; for each nearest point of the k nearest points: processing the nearest point and the particular observation using a Q network to generate a respective Q value for the action represented by the nearest point; and selecting the action to be performed by the agent from the k actions represented by the k nearest points based on the Q values.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a machine learning model that has been trained through reinforcement learning to select a content item. One of the methods includes receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment; and providing the first data as input to a long-term engagement machine learning model, the model having been trained through reinforcement learning to: receive a plurality of inputs, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting actions from large discrete action sets. One of the methods includes receiving a particular observation representing a particular state of an environment; and selecting an action from a discrete set of actions to be performed by an agent interacting with the environment, comprising: processing the particular observation using an actor policy network to generate an ideal point; determining, from the points that represent actions in the set, the k nearest points to the ideal point; for each nearest point of the k nearest points: processing the nearest point and the particular observation using a Q network to generate a respective Q value for the action represented by the nearest point; and selecting the action to be performed by the agent from the k actions represented by the k nearest points based on the Q values.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a machine learning model that has been trained through reinforcement learning to select a content item. One of the methods includes receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment; and providing the first data as input to a long-term engagement machine learning model, the model having been trained through reinforcement learning to: receive a plurality of inputs, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context.

Abstract: A system to provide feedback for energy saving to a user of a property comprising a plurality of appliances, the system comprising: at least one sensor monitoring energy consumption of one or more appliances within the property; a user interface to provide feedback to the user; and a processor configured to receive input data from an energy bill for the property covering a predetermined period; receive input regional average statistics regarding energy consumption for a set of predetermined categories of energy usage; generate a model of energy usage within a plurality of categories over the predetermined period by combining the input data, energy consumption data from the at least one sensor and generic statistics regarding energy consumption and output, via the user interface, feedback to the user based on the generated model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a machine learning model that has been trained through reinforcement learning to select a content item. One of the methods includes receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment; and providing the first data as input to a long-term engagement machine learning model, the model having been trained through reinforcement learning to: receive a plurality of inputs, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context.

Abstract: A method for processing an image to generate a signature which is characteristic of a pattern within said image. The method comprising receiving an image; overlaying a window at multiple locations on said image to define a plurality of sub-images within said image, with each sub-image each having a plurality of pixels having a luminance level; determining a luminance value for each said sub-image, wherein said luminance value is derived from said luminance levels of said plurality of pixels; and combining said luminance values for each of said sub-images to form said signature. Said combining is such that said signature is independent of the location of each sub-image. A method of creating a database of images using said method of generating signatures is also described.

Abstract: The present invention relates to a method and system for characterizing an image. The characterization may then be used to conduct a search for similar images, for example using a learning system trained using previously characterized images. A face may be identified within the image and a subsection extracted from said image which does not contain said face. At least one fixed size patch is taken from said extracted subsection; and input into said learning network to characterize said image.

Abstract: The present invention relates to a method and system for characterizing an image. The characterization may then be used to conduct a search for similar images, for example using a learning system trained using previously characterized images. A face may be identified within the image and a subsection extracted from said image which does not contain said face. At least one fixed size patch is taken from said extracted subsection; and input into said learning network to characterize said image.

Abstract: A system to provide feedback for energy saving to a user of a property comprising a plurality of appliances, the system comprising: at least one sensor monitoring energy consumption of one or more appliances within the property; a user interface to provide feedback to the user; and a processor configured to receive input data from an energy bill for the property covering a predetermined period; receive input regional average statistics regarding energy consumption for a set of pre-determined categories of energy usage; generate a model of energy usage within a plurality of categories over the predetermined period by combining the input data, energy consumption data from the at least one sensor and generic statistics regarding energy consumption and output, via the user interface, feedback to the user based on the generated model.

Abstract: A method for processing an image to generate a signature which is characteristic of a pattern within said image. The method comprising receiving an image; overlaying a window at multiple locations on said image to define a plurality of sub-images within said image, with each sub-image each having a plurality of pixels having a luminance level; determining a luminance value for each said sub-image, wherein said luminance value is derived from said luminance levels of said plurality of pixels; and combining said luminance values for each of said sub-images to form said signature. Said combining is such that said signature is independent of the location of each sub-image. A method of creating a database of images using said method of generating signatures is also described.

Abstract: A method is described for processing an image to generate a signature which is characteristic of a pattern within the image. The method includes: receiving an image; overlaying a window at multiple locations on the image to define a plurality of sub-images within the image, with each sub-image each having a plurality of pixels having a luminance level; determining a luminance value for each sub-image where the luminance value is derived from the luminance levels of the plurality of pixels; and combining the luminance values for each of the sub-images to form the signature. The combining is such that the signature is independent of the location of each sub-image. A method of creating a database of images using the method of generating signatures is also described.

Abstract: A system and method for creating a search query and for searching based on said search query. A first user selected image and a user selection of a first feature within said first user selected image is received. A second user selected image and a user selection of a second feature within said second user selected image is received. Said first and second user selected features are combined to form a composite image to form the basis of a search query.

Abstract: A system (20) is provided which allows for definition of agents for discrimination and classification of data submitted thereto. Each agent is a collection of data defining a topic, or theme, of interest, in natural language. This definition is combined with classification rules which generate classification scores when applied to a document.