Abstract: An update to an encoder is implemented utilizing information regarding performance of a reinforcement learning (RL) agent. This allows the emphasis to be placed not only on improving the performance of the RL agent, but on providing that the data within the encoding is both required and in such a form that it is optimal for the RL agent to learn, thereby reducing complexity and increasing speed of learning.

Abstract: A procedure to train an online scheduling system using Reinforcement Learning agents to process any kind of product variant and any kind of machine configuration is disclosed. The novel approach of scheduling jobs or products in a flexible manufacturing system is to train Deep Reinforcement Learning agents with generated training data. One agent may represent a product and may autonomously guide the product through the manufacturing system, including decisions regarding resource allocations (which module should process which operation) and transport decisions. Dependent on the mode to be trained, the identical job-specification for same, job-specifications from the same cluster for similar, and job-specifications from different clusters for different are chosen. This solution may handle any product variant to be produced within the considered system.

Abstract: The invention relates to a method in which an information model having state graphs for the individual skills and general machine behaviour or error cases is created for the user in an automated manner. This drastically reduces the engineering effort for the subsequent implementation of skill interfaces and in many cases would make an economically viable implementation possible in the first place.

Abstract: Software systems of a plurality of components often require said components to be configured so that said components can perform their task in an optimal manner for a particular application. A software system which consists of a plurality of components is configured. To this end, two different alternatives are provided: a) mode 1, i.e., with offensive training, for quickly learning new situations: the range of values and the step size of the parameters are restricted to such an extent that only non-critical changes are possible with one action. Alternatively, b) mode 2 is used, I.e., defensive training, with continuous learning: the range of values and the step size of the parameters are restricted so that the changes do not significantly worsen the target variables; the Epsilon-Greedy values is set to a lower value.

Abstract: A method that is used for self-learning manufacturing scheduling for a flexible manufacturing system that is used to produce at least a product is provided. The manufacturing system consists of processing entities that are interconnected through handling entities. The manufacturing scheduling will be learned by a reinforcement learning system on a model of the flexible manufacturing system. The model represents at least a behavior and a decision making of the flexible manufacturing system. The model is realized as a petri net. An order of the processing entities and the handling entities is interchangeable, and therefore, the whole arrangement is very flexible.

Abstract: The method for self-learning manufacturing scheduling for a flexible manufacturing system (FMS) with processing entities that are interconnected through handling entities is disclosed. The manufacturing scheduling is learned by a reinforcement learning system on a model of the flexible manufacturing system. The model represents at least the behavior and the decision making of the flexible manufacturing system, and the model is transformed in a state matrix to simulate the state of the flexible manufacturing system. A self-learning system for online scheduling and resource allocation is also provided. The system is trained in a simulation and learns the best decision from a defined set of actions for many every situation within an FMS. A decision may be made in near real-time during a production process and the system finds the optimal way through the FMS for every product using different optimization goals.