Abstract: The technology described herein provides a cloud reinforcement-learning architecture that allows a single reinforcement-learning model to interact with multiple live software environments. The live software environments and the single reinforcement-learning model run in a distributed computing environment (e.g., cloud environment). The single reinforcement-learning model may run on a first computing device(s) with a graphical processing unit (GPU) to aid in training the single reinforcement-learning model. At a high level, the single reinforcement-learning model may receive state telemetry data from the multiple live environments. The single reinforcement-learning model selects an available action for each set of state telemetry data received and communicates the selection to appropriate the test agent. The test agent then facilitates completion of the action within the software instance being tested in the live environment. A reward is then determined for the action.

Abstract: The technology described herein provides an automated software-testing platform that uses reinforcement learning to discover how to perform tasks used in testing. The technology described herein is able to perform quality testing even when prescribed paths to completing tasks are not provided. The reinforcement-learning agent is not directly supervised to take actions in any given situation, but rather learns which sequences of actions generate the most rewards through the observed states and rewards from the environment. In the software-testing environment, the state can be user interface features and actions are interactions with user interface elements. The testing system may recognize when a sought after state is achieved by comparing a new state to a reward criteria.

Abstract: The technology described herein provides an automated software-testing platform that functions in an undefined action space. The technology described herein starts with an undefined action space but begins to learn about the action space through random exploration. Both the action taken during testing and the resulting state may be communicated to a centralized testing service. The technology described herein also mines the action telemetry data and state telemetry data to identify action patterns that produce a sought after result. Once a plurality of action patterns is identified and, at least, a partial model of the action space is built, the testing on the test machines may be split into random test mode, replay test mode, and a pioneering test mode.

Abstract: The technology described herein trains a reinforcement-learning model in a simulated environment. A simulated environment contrasts with a live environment. A live environment is a computing environment with which the reinforcement-learning model will interact once it is deployed. In order to be effective, the simulated environment may provide inputs to the reinforcement-learning model in the same format as the reinforcement-learning model receives from the live environment. In aspects, the training in the simulated environment may act as pre-training for training in the live environment. Once pre-trained, the reinforcement-learning model may be deployed in a live environment and continue to learn how to perform the same task in different ways, learn how to perform additional tasks, and/or improve performance of a task learned in pre-training. In aspects, the reinforcement-learning model may be used to discover unhealthy conditions in software by performing the tasks it has learned.

Abstract: Methods, associated products and apparatus are described for the production of biodegradable foam products using a controlled pressure increase due to compressed air and a controlled pressure decrease in pressure as key variables during a microwave heating cycle to produce a foamed product. The biodegradable product formed has improved characteristics including a density from 10 to 100 kg/m3; a soft and resilient structure; cushioning G-value characteristics to cushion an object with a fragility of 15 to 115; and a surface abrasion comparable to polystyrene.

Abstract: Methods, associated products and apparatus are described for the production of biodegradable foam products using a controlled pressure increase due to compressed air and a controlled pressure decrease in pressure as key variables during a microwave heating cycle to produce a foamed product. The biodegradable product formed has improved characteristics including a density from 10 to 100 kg/m3; a soft and resilient structure; cushioning G-value characteristics to cushion an object with a fragility of 15 to 115; and a surface abrasion comparable to polystyrene.

Abstract: A method for the production of a biodegradable foamed product 7 from a base material and a blowing agent 1. The base materials are mixed with a blowing agent and any other required additives 1. The mixture 3, after extrusion 2, is placed in a microwave transparent mold 6 and processed in a microwave via distinct steps. The first step preheats the extrudate to a temperature just below the flash point 6. The second step 7 rapidly heats the extrudate beyond the flash point causing the extrudate to foam in the mold 7. By utilizing this method it is possible to produce shaped articles with uniform properties and with packaging properties such as compressibility, resilience and shock absorption.

Abstract: The method provided produces a bio-degradable foamed material with qualities of uniformity of mechanical and physical properties throughout the product including a foam thickness of up to one meter and a finished foam surface suitable for packaging applications. The parameters for producing such a product are selected from a range of variables which includes wall thickness, mold material, use of a susceptor and the type and composition of a susceptor, the number and arrangement of magnetrons and mold shape. Complex shapes produced by the process are also disclosed.

Abstract: A method for the production of a biodegradable foamed product 7 from a base material and a blowing agent 1. The base materials are mixed with a blowing agent and any other required additives 1. The mixture 3, after extrusion 2, is placed in a microwave transparent mold 6 and processed in a microwave via distinct steps. The first step preheats the extrudate to a temperature just below the flash point 6. The second step 7 rapidly heats the extrudate beyond the flash point causing the extrudate to foam in the mold 7. By utilizing this method it is possible to produce shaped articles with uniform properties and with packaging properties such as compressibility, resilience and shock absorption.