Abstract: An extension device for one or more automation devices in an industrial system is provided. Industrial data processing units capable of performing data processing based on one or more artificial neural networks are provided. To enable and/or accelerate one or more computations in an industrial system, thereby simplifying integration of artificial intelligence into the industrial system, and to simplify data exchange between an extension device capable of processing data using artificial intelligence and an automation device, one or more results of the one or more computations are obtained. The results indicate one or more states of the industrial system. The one or more results are provided via a process state model shared with the automation device to monitor and/or control the industrial system.

Abstract: Processor system, apparatus and method generate improved model of an industrial or chemical process. A multiple input variable multiple output variable (MIMO) model of a subject industrial process is translated into a custom modified neural network. The custom model is modular (componentized) and is formed of plural multiple input single output (MISO) models. Each MISO model represents a respective input variable-output variable relationship of a subset of the input variables and associated one output variable of the initial MIMO model. The plural MISO models enable modeling relatively simple input variable-output variable relationships with a minimal number of parameters while modeling other input variable-output variable relationships with relatively complex representation on an as need basis. Architecture of each MISO model is automatically assigned. The architecture is optimally selected from a library of machine learning or neural network basis model architectures.

Abstract: System and method are disclosed for training a generative adversarial network pipeline that can produce realistic artificial depth images useful as training data for deep learning networks used for robotic tasks. A generator network receives a random noise vector and a computer aided design (CAD) generated depth image and generates an artificial depth image. A discriminator network receives either the artificial depth image or a real depth image in alternation, and outputs a predicted label indicating a discriminator decision as to whether the input is the real depth image or the artificial depth image. Training of the generator network is performed in tandem with the discriminator network as a generative adversarial network. A generator network cost function minimizes correctly predicted labels, and a discriminator cost function maximizes correctly predicted labels.

Abstract: It is recognized herein that current approaches to autonomous operations are often limited to grasping and manipulation operations that can be performed in a single step. It is further recognized herein that there are various operations in robotics (e.g., assembly tasks) that require multiple steps or a sequence of motions to be performed. To determine or plan a sequence of motions for fulfilling a task, an autonomous system that includes a robot can perform object recognition, pose estimation, affordance analysis, decision-making, probabilistic task or motion planning, and object manipulation.

Abstract: A computer-implemented method for building an object detection module uses mesh representations of objects belonging to specified object classes of interest to render images by a physics-based simulator. Each rendered image captures a simulated environment containing objects belonging to multiple object classes of interest placed in a bin or on a table. The rendered images are generated by randomizing a set of parameters by the simulator to render a range of simulated environments. The randomized parameters include environmental and sensor-based parameters. A label is generated for each rendered image, which includes a two-dimensional representation indicative of location and object classes of objects in that rendered image frame. Each rendered image and the respective label constitute a data sample of a synthetic training dataset. A deep learning model is trained using the synthetic training dataset to output object classes from an input image of a real-world physical environment.

Abstract: Fully flexible kitting processes can be automated by generating pick and place motions for multi-robot, multi-gripper, robotic systems.

Abstract: In some cases, grasp point algorithms can be implemented so as to compute grasp points on an object that enable a stable grasp. It is recognized herein, however, that in practice a robot in motion can drop the object or otherwise have grasp issues when the object is grasped at the computed stable grasp points. Path constraints that can differ based on a given object are generated while generating the trajectory for a robot, so as to ensure that a grasp remains stable throughout the motion of the robot.

Abstract: A covariate shift generally refers to the change of the distribution of the input data (e.g., noise distribution) between the training and inference regimes. Such covariate shifts can degrade the performance grasping neural networks, and thus robotic grasping operations. As described herein, an output of a grasp neural network can be transformed, so as to determine appropriate locations on a given object for a robot or autonomous machine to grasp.

Abstract: In described embodiments of method for executing autonomous bin picking, a physical environment comprising a bin containing a plurality of objects is perceived by one or more sensors. Multiple artificial intelligence (AI) modules feed from the sensors to compute grasping alternatives, and in some embodiments, detected objects of interest. Grasping alternatives and their attributes are computed based on the outputs of the AI modules in a high-level sensor fusion (HLSF) module. A multi-criteria decision making (MCDM) module is used to rank the grasping alternatives and select the one that maximizes the application utility while satisfying specified constraints.

Abstract: A controller system includes a CPU module, one or more technology modules, and a backplane bus. The CPU module comprises a processor executing a control program. The technology modules include an artificial intelligence (AI) accelerator processor configured to (a) receive input data values related to one or more machine learning models, and (b) apply the machine learning models to the input data values to generate one or more output data values. The backplane bus connects the CPU module and the technology modules. The technology modules transfer the output data values to the processor over the backplane bus and the processor uses output data values during execution of the control program.

Abstract: A system controller for visual servoing includes a technology module with dedicated hardware acceleration for deep neural network that retrieves a desired configuration of a workpiece object being manipulated by a robotic device and receives visual feedback information from one or more sensors on or near the robotic device that includes a current configuration of the workpiece object. The hardware accelerator executes a machine learning model trained to process the visual feedback information and determine a configuration error based on a difference between the current configuration of the workpiece object and the desired configuration of the workpiece object. A servo control module adapts a servo control signal to the robotic device for manipulation of the workpiece object in response to the configuration error.

Abstract: An edge device can be configured to perform industrial control operations within a production environment that defines a physical location. The edge device can include a plurality of neural network layers that define a deep neural network. The edge device be configured to obtain data from one or more sensors at the physical location defined by the production environment. The edge device can be further configured to perform one or more matrix operations on the data using the plurality of neural network layers so as to generate a large scale matrix computation at the physical location defined by the production environment. In some examples, the edge device can send the large scale matrix computation to a digital twin simulation model associated with the production environment, so as to update the digital twin simulation model in real time.

Abstract: A computer-implemented method includes operating a controllable physical device to perform a task. The method also includes miming forward simulations of the task by a physics engine based on one or more physics parameters. The physics engine communicates with a parameter data layer where each of the one or more physics parameters is modeled with a probability distribution. For each forward simulation run, a tuple of parameter values is sampled from the probability distribution of the one or more physics parameters and fed to the physics engine. The method includes obtaining an observation pertaining to the task from the physical environment and a corresponding forward simulation outcome associated with each sampled tuple of parameter values. The method then includes updating the probability distribution of the one or more physics parameters in the parameter data layer based on the observation from the physical environment and the corresponding forward simulation outcomes.

Abstract: A sensor housing for being assembled to a vehicle and a connector integrated with the housing, wherein the sensor housing is further adapted to house at least one sensor to be connected to the connector means and for monitoring environmental conditions of the vehicle, wherein a sensor adapter resides in the housing to hold the at least one sensor, wherein the adapter has one or more assembly-fittings to hold each of the at least one sensors in a monitoring position, and wherein a placeholder is integrated with the housing to hold a washer-body of a cleaning-device in position as to afford cleaning of the at least one sensor.

Abstract: Distributed neural network boosting is performed by a neural network system through operating at least one processor. A method comprises providing a boosting algorithm that distributes a model among a plurality of processing units each being a weak learner of multiple weak learners that can perform computations independent from one another yet process data concurrently. The method further comprises enabling a distributed ensemble learning which enables a programmable logic controller (PLC) to use more than one processing units of the plurality of processing units to scale an application and training the multiple weak learners using the boosting algorithm. The multiple weak learners are machine learning models that do not capture an entire data distribution and are purposefully designed to predict with a lower accuracy. The method further comprises using the multiple weak learners to vote for a final hypothesis based on a feed forward computation of neural networks.

Abstract: Computerized system and method are provided. A robotic manipulator (12) is arranged to grasp objects (20). A gripper (16) is attached to robotic manipulator (12), which includes an imaging sensor (14). During motion of robotic manipulator (12), imaging sensor (14) is arranged to capture images providing different views of objects in the environment of the robotic manipulator. A processor (18) is configured to find, based on the different views, candidate grasp locations and trajectories to perform a grasp of a respective object in the environment of the robotic manipulator. Processor (18) is configured to calculate respective values indicative of grasp quality for the candidate grasp locations, and, based on the calculated respective values indicative of grasp quality for the candidate grasp locations, processor (18) is configured to select a grasp location likely to result in a successful grasp of the respective object.

Abstract: Autonomous operations, such as robotic grasping and manipulation, in unknown or dynamic environments present various technical challenges. For example, three-dimensional (3D) reconstruction of a given object often focuses on the geometry of the object without considering how the 3D model of the object is used in solving or performing a robot operation task. As described herein, in accordance with various embodiments, models are generated of objects and/or physical environments based on tasks that autonomous machines perform with the objects or within the physical environments. Thus, in some cases, a given object or environment may be modeled differently depending on the task that is performed using the model. Further, portions of an object or environment may be modeled with varying resolutions depending on the task associated with the model.

Abstract: A system for supporting artificial intelligence inference in an edge computing device associated with a physical process or plant includes a neural network training module, a neural network testing module and a digital twin of the physical process or plant. The neural network training module is configured to train a neural network model for deployment to the edge computing device based on data including baseline training data and field data received from the edge computing device. The neural network testing module configured to validate the trained neural network model prior to deployment to the edge computing device by leveraging the digital twin of the physical process or plant.

Abstract: According to an aspect of the present disclosure, a computer-implemented includes creating a plurality of basic skill functions for a controllable physical device of an autonomous system. Each basic skill function includes a functional description for using the controllable physical device to interact with a physical environment to perform a defined objective. The method further includes selecting one or more basic skill functions to configure the controllable physical device to perform a defined task. The method also includes determining a decorator skill function specifying at least one constraint. The decorator skill function is configured to impose, at run-time, the at least one constraint, on the one or more basic skill functions. The method further includes generating executable code by applying the decorator skill function to the one or more basic skill functions, and actuating the controllable physical device using the executable code.

Abstract: Over the past several decades, rapid advances in semiconductors, automation, and control systems have resulted in the adoption of programmable logic controllers (PLCs) in an immense variety of environments. Machine learning techniques help train replacement PLCs when a legacy PLC must be replaced, e.g., due to aging or failure. The techniques facilitate the efficient adoption and correct operation of replacement PLCs in the industrial environment.