Abstract: A computer-implemented method comprising: receiving data associated with an operational control problem; formulating the operation control problem as an optimization problem; recursively generating a sequence of policies of operational control associated with the operational control problem, wherein each subsequent policy in the sequence is constructed by modifying one or more actions at a single state in a preceding policy in the sequence, and wherein the modifying monotonically changes a risk associated with the subsequent policy; constructing, from the sequence of policies, an optimal solution path, wherein each vertex on the optimal solution path represents an optimal solution to the operational control problem; calculating a ratio of reward to risk for each of the vertices on the path; and selecting one of the policies in the sequence to apply to the operational control problem, based, at least in part, on the calculated ratios.

Abstract: Computer hardware and/or software for generating training curricula for a plurality of reinforcement learning control agents, the hardware and/or software performing the following operations: (i) obtaining system data describing at least one operating parameter of a system based, at least in part, on at least one of a plurality of reinforcement learning control agents failing to satisfy a control criterion for the system; (ii) generating a set of training curricula based, at least in part, on at least one operating parameter of the system and at least one training policy for the plurality of reinforcement learning control agents; and (iii) communicating the set of training curricula to the plurality of reinforcement learning control agents.

Abstract: Method, apparatus and product for utilizing stochastic controller to provide user-controlled notification rate of wearable-based events. The method comprises obtaining events issued by a module based on analysis of multiple sensor readings of one or more sensors of a wearable device. The method further comprises determining by a stochastic controller whether to provide an alert to a user based on the events and based on a user preference, wherein the user preference is indicative of a desired notification rate of the user, wherein the stochastic controller comprises a stochastic model of an environment. Based on such determination, alerts are outputted to the user.

Abstract: A control system, computer program product, and method for generating a logically-represented policy for a control system operating based on a CMDP model are provided. The control system directs the operation of a controlled application system that is subject to a constraint. The method includes receiving, at the control system, data corresponding to control action variables and system state variables relating to the controlled application system, data corresponding to a cost/reward, and data corresponding to the constraint, and automatically training a CMDP model for the operation of the controlled application system based on the received data, where the CMDP model is formulated using dual linear programming, and where the CMDP model includes a policy corresponding to occupation measures that are decision variables of the dual linear programming formulation. The method also includes automatically generating a logically-represented policy for the control system based on the policy of the CMDP model.

Abstract: A computer-implemented method comprising: receiving data associated with an operational control problem; formulating the operation control problem as an optimization problem; recursively generating a sequence of policies of operational control associated with the operational control problem, wherein each subsequent policy in the sequence is constructed by modifying one or more actions at a single state in a preceding policy in the sequence, and wherein the modifying monotonically changes a risk associated with the subsequent policy; constructing, from the sequence of policies, an optimal solution path, wherein each vertex on the optimal solution path represents an optimal solution to the operational control problem; calculating a ratio of reward to risk for each of the vertices on the path; and selecting one of the policies in the sequence to apply to the operational control problem, based, at least in part, on the calculated ratios.

Abstract: A method, apparatus and a product for generating a dataset for a reinforcement model. The method comprises obtaining a plurality of different subsets of the set of features; for each subset of features, determining a policy using a Markov Decision Process; obtaining a state comprises a valuation of each feature of the set of features; applying the plurality of policies on the state, whereby obtaining a plurality of suggested actions for the state, based on different projections of the state onto different subsets of features; determining, for the state, one or more actions and corresponding scores thereof based on the plurality of suggested actions; and training a reinforcement learning model using the state and the one or more actions and corresponding scores thereof.

Abstract: Methods and systems for generating an environment include training transformer models from tabular data and relationship information about the training data. A directed acyclic graph is generated, that includes the transformer models as nodes. The directed acyclic graph is traversed to identify a subset of transformers that are combined in order. An environment is generated using the subset of transformers.

Abstract: A method for automatically reducing the dimensionality of a mathematical representation of a controlled application system is provided. The method includes receiving, at a control system, data corresponding to control action and system state variables relating to the controlled application system, fitting a constrained reinforcement learning (CRL) model to the controlled application system based on the data, and automatically identifying a subset of the system state variables by selecting control action variables of interest and identifying system state variables that drive the CRL model to recommend each control action variable of interest.

Abstract: Constructing a MARS prediction model using predictor variables at a first point in time within a time horizon, including directly-controllable variables of first physical characteristics of a system and that are associated with adjustable operational control settings for directly controlling the first physical characteristic, and including controllable variables of second physical characteristics that are affected by the first physical characteristics, recursively using the prediction model to define an optimization problem for later point in time within the time horizon, transforming the optimization problem into a MILP problem, and solving the MILP problem using an optimization engine to determine, for any given one of the directly-controllable variables and corresponding to at least one of the points in time, for adjusting, using the optimized value, the adjustable operational control setting corresponding to the given directly-controllable variable and thereby control the physical characteristic associate

Abstract: Embodiments may include novel techniques for intermediate model generation using historical data and domain knowledge for Reinforcement Learning (RL) training. Embodiments may start with gathering client data.

Abstract: A method for improving a machine operation are provided. The method includes receiving a plurality of domain specific heuristics and a set of states and a set of actions, where an immediate cost and/or reward is associated with a pair of state and action. The method also includes generating at least one of: a graph of state transitions for the actions, and a transition probability matrix. The method also includes executing a Markov Decision Process (MDP) model for solving an MDP problem, and outputting an MDP optimal policy of an optimal mapping of a given state to an action. The method also includes selecting one of the plurality of domain specific heuristics and heuristic input parameters thereof. The method also includes controlling the machine for solving a predefined optimization problem in a plurality of execution iterations.

Abstract: In some examples, a system for generating personalized recommendation includes a processor that can perform an initial training for a deep reinforcement learning (DRL) model using domain knowledge, available users data, and an items list. The processor also inputs users data and an items list to the trained DRL model to generate an initial list of recommended items. The processor also inputs the initial list of recommended items and a user profile to a content-based filter to generate a final list of recommendations for a target user.

Abstract: In an approach, a processor stores a dictionary set, including simplex dictionaries, for saving processing time when calculating an optimal control policy for at least one linearly time dependent value function of a plurality of variables complying with a plurality of linearly time dependent constraints. A processor calculates a storage limit for the dictionary set, based on a number of the plurality of variables, the plurality of constraints, and size of a memory. A processor removes at least one of the plurality of simplex dictionaries in accordance with the storage limit, while maintaining a neighbor density measure, where the neighbor density measure is based on a distance between the at least one of the simplex dictionaries and a non-removed simplex dictionary and the distance corresponds to a number of simplex pivots required to construct the at least one of the simplex dictionaries from the non-removed simplex dictionary.

Abstract: A method for automatically reducing the dimensionality of a mathematical representation of a controlled application system is provided. The method includes receiving, at a control system, data corresponding to control action and system state variables relating to the controlled application system, fitting a constrained reinforcement learning (CRL) model to the controlled application system based on the data, and automatically identifying a subset of the system state variables by selecting control action variables of interest and identifying system state variables that drive the CRL model to recommend each control action variable of interest.

Abstract: A computer implemented method of detecting complex user activities, comprising using processor(s) in each of a plurality of consecutive time intervals for: obtaining sensory data from wearable inertial sensor(s) worn by a user, computing an action score for continuous physical action(s) performed by the user, the continuous physical action(s) extending over multiple time intervals are indicated by repetitive motion pattern(s) identified by analyzing the sensory data, computing a gesture score for brief gesture(s) performed by the user, the brief gesture(s) bounded in a single basic time interval is identified by analyzing the sensory data, aggregating the action and gesture scores to produce an interval activity score of predefined activity(s) for a current time interval, adding the interval activity score to a cumulative activity score accumulated during a predefined number of preceding time intervals and identifying the predefined activity(s) when the cumulative activity score exceeds a predefined threshold

Abstract: A method, apparatus and a product for generating a dataset for a reinforcement model. The method comprises obtaining a plurality of different subsets of the set of features; for each subset of features, determining a policy using a Markov Decision Process; obtaining a state comprises a valuation of each feature of the set of features; applying the plurality of policies on the state, whereby obtaining a plurality of suggested actions for the state, based on different projections of the state onto different subsets of features; determining, for the state, one or more actions and corresponding scores thereof based on the plurality of suggested actions; and training a reinforcement learning model using the state and the one or more actions and corresponding scores thereof.

Abstract: A method for predicting at least one score for at least one item, comprising in at least one iteration of a plurality of iterations: receiving a user profile having a plurality of user attribute values; computing the at least one score according to a similarity between the user profile and a plurality of other user profiles by inputting the user profile and a plurality of items into a prediction model trained by: in each of a plurality of training iterations: receiving a training user profile of a plurality of training user profiles, the training user profile having a plurality of training user attribute values; computing by the prediction model a plurality of predicted scores, each for one of a plurality of training items, in response to the training user profile and the plurality of training items, where each of the plurality of training items has a plurality of training item.

Abstract: Automatic identification of features that drive a reinforcement learning model to recommend an action of interest. The identification is based on a calculation of occupation measures of state-action pairs associated with the reinforcement learning model. High occupation measures of certain action-state pairs indicate that the states of these pairs likely include the sought-after features.

Abstract: Embodiments of the present invention may provide the capability to detect complex events while providing improved detection and performance. In an embodiment of the present invention, a method for detecting an event may comprise receiving data representing measurement or detection of physical parameters, conditions, or actions, quantizing the received data and selecting a number of samples from the quantized data, generating a hidden Markov model representing events to be detected using initial model values based on ideal conditions, wherein a desired output is defined as a sequence of states, and wherein a number of states of the hidden Markov model is less than or equal to the number of samples of the quantized data, adjusting the quantized data and the initial model values to improve accuracy of the model, determining a state sequence of the hidden Markov model, and outputting an indication of a detected event.

Abstract: In some examples, a system for generating personalized recommendation includes a processor that can perform an initial training for a deep reinforcement learning (DRL) model using domain knowledge, available users data, and an items list. The processor also inputs users data and an items list to the trained DRL model to generate an initial list of recommended items. The processor also inputs the initial list of recommended items and a user profile to a content-based filter to generate a final list of recommendations for a target user.