Abstract: The present disclosure relates to generating and modifying recommended event sequences utilizing a dynamic user preference interface. For example, in one or more embodiments, the system generates a recommended event sequence using a recommendation model trained based on a plurality of historical event sequences. The system then provides, for display via a client device, the recommendation, a plurality of interactive elements for entry of user preferences, and a visual representation of historical event sequences. Upon detecting input of user preferences, the system can modify a reward function of the recommendation model and provide a modified recommended event sequence together with the plurality of interactive elements. In one or more embodiments, as a user enters user preferences, the system additionally modifies the visual representation to display subsets of the plurality of historical event sequences corresponding to the preferences.

Abstract: The present disclosure describes methods, systems, and non-transitory computer-readable media for generating a projected value metric that projects a performance of a target policy within a digital action space. For instance, in one or more embodiments, the disclosed systems identify a target policy for performing digital actions represented within a digital action space. The disclosed systems further determine a set of sampled digital actions performed according to a logging policy and represented within the digital action space. Utilizing an embedding model, the disclosed systems generate a set of action embedding vectors representing the set of sampled digital actions within an embedding space. Further, utilizing the set of action embedding vectors, the disclosed systems generate a projected value metric indicating a projected performance of the target policy.

Abstract: In implementations of item transfer control systems, a computing device implements a transfer system to receive input data describing types of requested items and corresponding quantities of the types of requested items to receive at each of a plurality of destination sites and types of available items and corresponding quantities of the types of available items that are available at each of a plurality of source sites. The transfer system constructs a flow network having a source node for each of the plurality of the source sites and a destination node for each of the plurality of the destination sites. An integral approximate solution is generated that transfers the corresponding quantities of the types of requested items to each of the plurality of the destination sites using a maximum flow solver and the flow network.

Abstract: Systems and methods for machine learning are described. Embodiments of the present disclosure receive state information that describes a state of a decision making agent in an environment; compute an action vector from an action embedding space based on the state information using a policy neural network of the decision making agent, wherein the policy neural network is trained using reinforcement learning based on a topology loss that constrains changes in a mapping between an action set and the action embedding space; and perform an action that modifies the state of the decision making agent in the environment based on the action vector, wherein the action is selected based on the mapping.

Abstract: A control system facilitates active management of a streaming data system. Given historical data traffic for each data stream processed by a streaming data system, the control system uses a machine learning model to predict future data traffic for each data stream. The control system selects a matching between data streams and servers for a future time that minimizes a cost comprising a switching cost and a server imbalance cost based on the predicted data traffic for the future time. In some configurations, the matching is selected using a planning window comprising a number of future time steps dynamically selected based on uncertainty associated with the predicted data traffic. Given the selected matching, the control system may manage the streaming data system by causing data streams to be moved between servers based on the matching.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing offline models to warm start online bandit learner models. For example, the disclosed system can determine relevant offline models for an environment based on reward estimate differences between the offline models and the online model. The disclosed system can then utilize the relevant offline models (if any) to select an arm for the environment. The disclosed system can update the online model based on observed rewards for the selected arm. Additionally, the disclosed system can also use entropy reduction of arms to determine the utility of the arms in differentiating relevant and irrelevant offline models. For example, the disclosed system can select an arm based on a combination of the entropy reduction of the arm and the reward estimate for the arm and use the observed reward to update an observation history.

Abstract: In implementations of item transfer control systems, a computing device implements a transfer system to receive input data describing types of requested items and corresponding quantities of the types of requested items to receive at each of a plurality of destination sites and types of available items and corresponding quantities of the types of available items that are available at each of a plurality of source sites. The transfer system constructs a flow network having a source node for each of the plurality of the source sites and a destination node for each of the plurality of the destination sites. An integral approximate solution is generated that transfers the corresponding quantities of the types of requested items to each of the plurality of the destination sites using a maximum flow solver and the flow network.

Abstract: The present disclosure relates to methods, systems, and non-transitory computer-readable media for determining a cognitive, action-selection bias of a user that influences how the user will select a sequence of digital actions for execution of a task. For example, the disclosed systems can identify, from a digital behavior log of a user, a set of digital action sequences that correspond to a set of sessions for a task previously executed by the user. The disclosed systems can utilize a machine learning model to analyze the set of sessions to generate session weights. The session weights can correspond to an action-selection bias that indicates an extent to which a future session for the task executed by the user is predicted to be influenced by the set of sessions. The disclosed systems can provide a visual indication of the action-selection bias of the user for display on a graphical user interface.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing offline models to warm start online bandit learner models. For example, the disclosed system can determine relevant offline models for an environment based on reward estimate differences between the offline models and the online model. The disclosed system can then utilize the relevant offline models (if any) to select an arm for the environment. The disclosed system can update the online model based on observed rewards for the selected arm. Additionally, the disclosed system can also use entropy reduction of arms to determine the utility of the arms in differentiating relevant and irrelevant offline models. For example, the disclosed system can select an arm based on a combination of the entropy reduction of the arm and the reward estimate for the arm and use the observed reward to update an observation history.

Abstract: An item recommendation system receives a set of recommendable items and a request to select, from the set of recommendable items, a contrast group. The item recommendation system selects a contrast group from the set of recommendable items by applying a recommendation model to the set of recommendable items. The recommendation model includes an item selection model configured to determine an unbiased conversion rate for each item of the set of recommendable items and select a recommended item from the set of recommendable items having a greatest unbiased conversion rate. The recommendation model includes a contrast group selection model configured to select, for the recommended item, a contrast group comprising the recommended item and one or more contrast items. The item recommendation system transmits the contrast group responsive to the request.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate digital item recommendations for client devices utilizing coagent recommendation models of a distributed asynchronous coagent network. Indeed, in one or more embodiments, the disclosed systems operate on an edge computing device of a distributed asynchronous coagent network. In some cases, the disclosed systems utilize recommendation scores generated at the edge computing device via local coagents and additional recommendation scores received from other coagents of other edge computing devices to generate a digital item recommendation. In some cases, the disclosed systems progressively refines the recommendation as delayed scores from the other coagents are received. Further, in some embodiments, the disclosed systems update parameters of the local coagents using local policy gradients determined from responses to the generated recommendations.

Abstract: Metric forecasting techniques and systems in a digital medium environment are described that leverage similarity of elements, one to another, in order to generate a forecast value for a metric for a particular element. In one example, training data is received that describes a time series of values of the metric for a plurality of elements. The model is trained to generate the forecast value of the metric, the training using machine learning of a neural network based on the training data. The training includes generating dimensional-transformation data configured to transform the training data into a simplified representation to determine similarity of the plurality of elements, one to another, with respect to the metric over the time series. The training also includes generating model parameters of the neural network based on the simplified representation to generate the forecast value of the metric.

Abstract: In implementations of item transfer control systems, a computing device implements a transfer system to receive input data describing types of requested items and corresponding quantities of the types of requested items to receive at each of a plurality of destination sites and types of available items and corresponding quantities of the types of available items that are available at each of a plurality of source sites. The transfer system constructs a flow network having a source node for each of the plurality of the source sites and a destination node for each of the plurality of the destination sites. An integral approximate solution is generated that transfers the corresponding quantities of the types of requested items to each of the plurality of the destination sites using a maximum flow solver and the flow network.

Abstract: Systems and methods are described for a decision-making process including actions characterized by stochastic availability, provide an Markov decision process (MDP) model that includes a stochastic action set based on the decision-making process, compute a policy function for the MDP model using a policy gradient based at least in part on a function representing the stochasticity of the stochastic action set, identify a probability distribution for one or more actions available at a time period using the policy function, and select an action for the time period based on the probability distribution.

Abstract: In implementations of item transfer control systems, a computing device implements a transfer system to receive input data describing types of requested items and corresponding quantities of the types of requested items to receive at each of a plurality of destination sites and types of available items and corresponding quantities of the types of available items that are available at each of a plurality of source sites. The transfer system constructs a flow network having a source node for each of the plurality of the source sites and a destination node for each of the plurality of the destination sites. An integral approximate solution is generated that transfers the corresponding quantities of the types of requested items to each of the plurality of the destination sites using a maximum flow solver and the flow network.

Abstract: This disclosure describes embodiments of methods, systems, and non-transitory-computer readable media that personalize visual content for display on digital signage near a projected location of a person by mapping visual content to physical items selected by the person. In some examples, the disclosed system identifies physical items selected by a person based on signals from the physical items, such as signals emitted by RFID tags affixed to (or other devices associated with) the physical items. The disclosed system analyzes the collection of physical items—as identified by the signals—to tailor digital signage content specific to the person. The disclosed system further tracks the location of the person as the person moves through a physical space and interacts with the physical items. Based on the tracked positions, the disclosed system determines a digital sign in proximity to a predicted location of the person to display the personalized visual content.

Abstract: Systems and methods are described for a decision-making process that includes an increasing set of actions, compute a policy function for a Markov decision process (MDP) for the decision-making process, wherein the policy function is computed based on a state conditional function mapping states into an embedding space, an inverse dynamics function mapping state transitions into the embedding space, and an action selection function mapping the elements of the embedding space to actions, identify an additional set of actions in the increasing set of actions, update the inverse dynamics function based at least in part on the additional set of actions, update the policy function based on the updated inverse dynamics function and parameters learned during the computing the policy function, and select an action based on the updated policy function.

Abstract: A system and method for automatically adjusting computing resources provisioned for a computer service or application by applying historical resource usage data to a predictive model to generate predictive resource usage. The predictive resource usage is then simulated for various service configurations, determining scaling requirements and resource wastage for each configuration. A cost value is generated based on the scaling requirement and resource wastage, with the cost value for each service configuration used to automatically select a configuration to apply to the service. Alternatively, the method for automatically adjusting computer resources provisioned for a service may include receiving resource usage data of the service, applying it to a linear quadratic regulator (LQR) to find an optimal stationary policy (treating the resource usage data as states and resource-provisioning variables as actions), and providing instructions for configuring the service based on the optimal stationary policy.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that intelligently generate and modify schedules of task sequences utilizing a graph neural network and/or reinforcement learning model. For example, the disclosed system utilizes a graph neural network to generate performance efficiency scores indicating predicted performances of the sets of tasks. Additionally, the disclosed systems utilizes the performance efficiency scores to rank sets of tasks and then determine a schedule including an ordered sequence of tasks. Furthermore, disclosed system generates modified schedules in response to detecting a modification to the schedule. For example, the disclosed system utilizes a reinforcement learning model to provide recommendations of new tasks or task sequences deviating from the schedule in the event of an interruption. The disclosed system also utilizes the reinforcement learning model to learn from user choices to inform future scheduling of tasks.

Abstract: The present disclosure relates to recommending points of interest to a plurality of users based on a type of each user as well as constraints associated with the points of interest. For example, one or more embodiments determine a user type for each user and determine user preferences based on the user type. Additionally, the system can determine resource constraints associated with each point of interest, indicating limitations on the capacity of each associated resource. The system can then provide recommendations to the plurality of users based on the user types and the resource constraints. In particular, the system can recommend points of interest that satisfy the preferences corresponding to each user type subject to the resource constraints of each point of interest. For example, one or more embodiments involve solving a linear program that takes into account user types to obtain recommendation policies subject to the resource constraints.