Abstract: The techniques disclosed herein enable a machine learning model to learn a termination condition of a sub-task. A sub-task is one of a number of sub-tasks that, when performed in sequence, accomplish a long-running task. A machine learning model used to perform the sub-task is augmented to also provide a termination signal. The termination signal indicates whether the sub-task's termination condition has been met. Monitoring the termination signal while performing the sub-task enables subsequent sub-tasks to seamlessly begin at the appropriate time. A termination condition may be learned from the same data used to train other model outputs. In some configurations, the model learns whether a sub-task is complete by periodically attempting subsequent sub-tasks. If a subsequent sub-task can be performed, positive reinforcement is provided for the termination condition. The termination condition may also be trained using synthetic scenarios designed to test when the termination condition has been met.

Abstract: Systems and methods for controlling a performance variable of an engine system are provided. An apparatus includes a response model circuit structured to apply a constraint to a response model that represents a relationship regarding a manipulated variable or a relationship between the performance variable and the manipulated variable. The apparatus further includes an optimization circuit structured to determine a target for the manipulated variable via the response model such that the target of the manipulated variable satisfies the constraint of the response model. The performance variable is indicative of performance of operation of the engine system and the manipulated variable is capable of affecting the performance variable. Operation of the engine system is adjusted based upon the target of the manipulated variable by controlling at least one of a fuel system or an air handling system of the engine system.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors structured to inject fuel into the dedicated EGR cylinder and the non-dedicated EGR cylinder, and an electronic control system operatively coupled with the fueling system and the ignition system. The electronic control system is configured to evaluate engine operating parameters including an engine load and an engine speed. The electronic control system is responsive to variation of the engine operating parameters to control operation of the fueling system to vary combustion in the at least one dedicated cylinder between rich of stoichiometric and stoichiometric.

Abstract: The techniques disclosed herein enable systems to solve routing problems using machine learning augmented by optimization modules. To plot a route, a system receives a plurality of nodes from a problem space. The plurality of nodes is then analyzed by an optimization module and ranked based on various criteria such as distance from a reference node and deadline. Based on the ranking, the optimization module can select a smaller subset of nodes that is then processed by a machine learning model. The machine learning model can then select a node from the subset of nodes for addition to a route. This process can be repeated until a route is plotted for the full set of nodes within the problem space. In addition, the system can be configured to monitor current conditions of the problem space to modify the route in response to changes.

Abstract: Technologies for reinforcement learning-based optimization of manufacturing lines are disclosed. A reinforcement learning-based selector is trained utilizing reinforcement learning to select a reinforcement learning-based controller for controlling the operation of machines on a manufacturing line. The selection can be made based upon inputs from the machines on the manufacturing line indicating whether machines on the line are jammed, whether the manufacturing line is operating at a steady state, or other conditions. The selected reinforcement learning-based controller can generate outputs to adjust parameters of the machines on the manufacturing line, such as operating speed, in order to recover from the jamming of one or more machines, to transition to a steady state of operation, or to operate the manufacturing line in a steady state of operation. The selector and the reinforcement learning-based controllers can be trained using reinforcement learning on a simulation of the manufacturing line.

Abstract: Systems and methods for controlling a performance variable of an engine system are provided. An apparatus includes a response model circuit structured to apply a constraint to a response model that represents a relationship regarding a manipulated variable or a relationship between the performance variable and the manipulated variable. The apparatus further includes an optimization circuit structured to determine a target for the manipulated variable via the response model such that the target of the manipulated variable satisfies the constraint of the response model. The performance variable is indicative of performance of operation of the engine system and the manipulated variable is capable of affecting the performance variable. Operation of the engine system is adjusted based upon the target of the manipulated variable by controlling at least one of a fuel system or an air handling system of the engine system.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors, an ignition system including a plurality of spark plugs, an intake throttle, and an electronic control system. The electronic control system is configured to control combustion during transient operation of the engine by determining one or more combustion control parameters compensating for variation of one or more of inert matter, unburned air and unburned fuel in EGR output by the dedicated EGR cylinder during transient operation of the engine, and an effect of the EGR loop on inert matter, unburned air and unburned fuel provided to the plurality of cylinders, and controlling operation of at least one of the throttle, the ignition system and the plurality of injectors in response to at least one of the one or more combustion control parameters.

Abstract: Systems and methods for optimizing a performance variable for an engine system. The method includes applying constraints of manipulated variables as well as performance variables, mechanical constraints and other engine responses to response models. The response models each represent a piecewise linear relationship between the manipulated variables and other engine responses including performance variables and constraints. The method also comprises determining an optimal target for each of the manipulated variables by using a quasi-simplex optimization process on the response models. The optimal targets of the manipulated variables correspond to an optimal value of the performance variable.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors structured to inject fuel into the dedicated EGR cylinder and the non-dedicated EGR cylinder, and an electronic control system operatively coupled with the fueling system and the ignition system. The electronic control system is configured to evaluate engine operating parameters including an engine load and an engine speed. The electronic control system is responsive to variation of the engine operating parameters to control operation of the fueling system to vary combustion in the at least one dedicated cylinder between rich of stoichiometric and stoichiometric.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors structured to inject fuel into the dedicated EGR cylinder and the non-dedicated EGR cylinder, and an electronic control system operatively coupled with the fueling system and the ignition system. The electronic control system is configured to evaluate engine operating parameters including an engine load and an engine speed. The electronic control system is responsive to variation of the engine operating parameters to control operation of the fueling system to vary combustion in the at least one dedicated cylinder between rich of stoichiometric and stoichiometric.

Abstract: An engine system includes an air handling control unit which controls a plurality of air handling actuators responsible for maintaining flow of air and exhaust gas within the engine system. The engine system has a plurality of sensors whose sensor signals at least partially define a current state of the engine system. The air handling control unit includes a controller which controls the air handling actuators of the engine system as well as a processing unit coupled to the sensors and the controller. The processing unit includes an agent which learns a policy function that is trained to process the current state, determines a control signal to send to the controller by using the policy function after receiving the current state as an input, and outputs the control signal to the controller. Then, the agent receives a next state and a reward value from the processing unit and updates the policy function using a policy evaluation algorithm and a policy improvement algorithm based on the received reward value.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors structured to inject fuel into the dedicated EGR cylinder and the non-dedicated EGR cylinder, and an electronic control system operatively coupled with the fueling system and the ignition system. The electronic control system is configured to evaluate engine operating parameters including an engine load and an engine speed. The electronic control system is responsive to variation of the engine operating parameters to control operation of the fueling system to vary combustion in the at least one dedicated cylinder between rich of stoichiometric and stoichiometric.

Abstract: An engine system includes an air handling and fuel system whose states are managed by a reference managing unit. The engine system has a plurality of sensors whose sensor signals at least partially define a current state of the engine system. The reference managing unit includes a controller which controls the air handling and fuel system of the engine system as well as a processing unit coupled to the sensors and the controller. The processing unit includes an agent which learns a policy function that is trained to process the current state, determines air handling references and fuel system references by using the policy function after receiving the current state as an input, and outputs the air handling references and fuel system references to the controller. Then, the agent receives a next state and a reward value from the processing unit and updates the policy function using a policy evaluation algorithm and a policy improvement algorithm based on the received reward value.

Abstract: One embodiment is a system comprising an engine including a dedicated EGR cylinder configured to provide EGR to the engine via an EGR loop, a non-dedicated cylinder, a plurality of injectors, an ignition system including a plurality of spark plugs, an intake throttle, and an electronic control system. The electronic control system is configured to control combustion during transient operation of the engine by determining one or more combustion control parameters compensating for variation of one or more of inert matter, unburned air and unburned fuel in EGR output by the dedicated EGR cylinder during transient operation of the engine, and an effect of the EGR loop on inert matter, unburned air and unburned fuel provided to the plurality of cylinders, and controlling operation of at least one of the throttle, the ignition system and the plurality of injectors in response to at least one of the one or more combustion control parameters.

Abstract: An engine system includes an air handling and fuel system whose states are managed by a reference managing unit. The engine system has a plurality of sensors whose sensor signals at least partially define a current state of the engine system. The reference managing unit includes a controller which controls the air handling and fuel system of the engine system as well as a processing unit coupled to the sensors and the controller. The processing unit includes an agent which learns a policy function that is trained to process the current state, determines air handling references and fuel system references by using the policy function after receiving the current state as an input, and outputs the air handling references and fuel system references to the controller. Then, the agent receives a next state and a reward value from the processing unit and updates the policy function using a policy evaluation algorithm and a policy improvement algorithm based on the received reward value.

Abstract: An engine system includes an air handling control unit which controls a plurality of air handling actuators responsible for maintaining flow of air and exhaust gas within the engine system. The engine system has a plurality of sensors whose sensor signals at least partially define a current state of the engine system. The air handling control unit includes a controller which controls the air handling actuators of the engine system as well as a processing unit coupled to the sensors and the controller. The processing unit includes an agent which learns a policy function that is trained to process the current state, determines a control signal to send to the controller by using the policy function after receiving the current state as an input, and outputs the control signal to the controller. Then, the agent receives a next state and a reward value from the processing unit and updates the policy function using a policy evaluation algorithm and a policy improvement algorithm based on the received reward value.

Abstract: Systems and methods for optimizing a performance variable for an engine system. The method includes applying constraints of manipulated variables as well as performance variables, mechanical constraints and other engine responses to response models. The response models each represent a piecewise linear relationship between the manipulated variables and other engine responses including performance variables and constraints. The method also comprises determining an optimal target for each of the manipulated variables by using a quasi-simplex optimization process on the response models. The optimal targets of the manipulated variables correspond to an optimal value of the performance variable.