Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a self-learning robotic process automation are disclosed. In one aspect, a method includes receiving an automated script that includes one or more commands and that is configured to interact with graphical elements that appear on a user interface. The method further includes executing a command of the one or more commands of the automated script. The method further includes determining that an error occurred during execution of the command of the one or more commands of the automated script. The method further includes determining a modification for the command by applying a script repair model. The method further includes executing the modified command. The method further includes determining whether the error or another error occurred during execution of the modified command. The method further includes determining whether to update the automated script with the modified command.

Abstract: A system for object recognition and segmentation from digital images provides an intelligent object recognition and segmentation using one or more multilayer convolutional neural network (CNN) models trained in multiple-stages and in a parallel and distributed manner to improve training speed and efficiency. The training dataset used in each of the multiple training stages for the CNN models are generated, expanded, self-validated from a preceding stage. The trained final CNN models are augmented with post-model filters to enhance prediction accuracy by removing false positive object recognition and segmentation. The system provides improved accuracy to predict object labels to append to unlabeled image blocks in digital images. In one embodiment, the system may be useful for enhancing a digital landmark registry by appending identifying labels on new infrastructure improvements recognized in aerial or satellite land images.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a cognitive robotics analyzer are disclosed. In one aspect, a method includes the actions of receiving, for each user characteristic of a plurality of user characteristics, first data that identifies one or more first actions that perform a first process and second data that identifies one or more second actions that perform a second process that is labeled as similar to the first process. The actions further include training a predictive model. The actions further include receiving data that identifies actions performed by a user. The actions further include applying the predictive model to one or more of the actions. The actions further include classifying a process performed by the one or more actions as similar to a particular process. The actions further include associating the user with the particular user characteristic.

Abstract: A system for object recognition and segmentation from digital images provides an intelligent object recognition and segmentation using one or more multilayer convolutional neural network (CNN) models trained in multiple-stages and in a parallel and distributed manner to improve training speed and efficiency. The training dataset used in each of the multiple training stages for the CNN models are generated, expanded, self-validated from a preceding stage. The trained final CNN models are augmented with post-model filters to enhance prediction accuracy by removing false positive object recognition and segmentation. The system provides improved accuracy to predict object labels to append to unlabeled image blocks in digital images. In one embodiment, the system may be useful for enhancing a digital landmark registry by appending identifying labels on new infrastructure improvements recognized in aerial or satellite land images.

Abstract: A system for object recognition and segmentation from digital images provides an intelligent object recognition and segmentation using one or more multilayer convolutional neural network (CNN) models trained in multiple-stages and in a parallel and distributed manner to improve training speed and efficiency. The training dataset used in each of the multiple training stages for the CNN models are generated, expanded, self-validated from a preceding stage. The trained final CNN models are augmented with post-model filters to enhance prediction accuracy by removing false positive object recognition and segmentation. The system provides improved accuracy to predict object labels to append to unlabeled image blocks in digital images. In one embodiment, the system may be useful for enhancing a digital landmark registry by appending identifying labels on new infrastructure improvements recognized in aerial or satellite land images.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a self-learning robotic process automation are disclosed. In one aspect, a method includes receiving an automated script that includes one or more commands and that is configured to interact with graphical elements that appear on a user interface. The method further includes executing a command of the one or more commands of the automated script. The method further includes determining that an error occurred during execution of the command of the one or more commands of the automated script. The method further includes determining a modification for the command by applying a script repair model. The method further includes executing the modified command. The method further includes determining whether the error or another error occurred during execution of the modified command. The method further includes determining whether to update the automated script with the modified command.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a self-learning robotic process automation are disclosed. In one aspect, a method includes receiving an automated script that includes one or more commands and that is configured to interact with graphical elements that appear on a user interface. The method further includes executing a command of the one or more commands of the automated script. The method further includes determining that an error occurred during execution of the command of the one or more commands of the automated script. The method further includes determining a modification for the command by applying a script repair model. The method further includes executing the modified command. The method further includes determining whether the error or another error occurred during execution of the modified command. The method further includes determining whether to update the automated script with the modified command.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a self-learning robotic process automation are disclosed. In one aspect, a method includes receiving an automated script that includes one or more commands and that is configured to interact with graphical elements that appear on a user interface. The method further includes executing a command of the one or more commands of the automated script. The method further includes determining that an error occurred during execution of the command of the one or more commands of the automated script. The method further includes determining a modification for the command by applying a script repair model. The method further includes executing the modified command. The method further includes determining whether the error or another error occurred during execution of the modified command. The method further includes determining whether to update the automated script with the modified command.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a cognitive robotics analyzer are disclosed. In one aspect, a method includes the actions of receiving, for each user characteristic of a plurality of user characteristics, first data that identifies one or more first actions that perform a first process and second data that identifies one or more second actions that perform a second process that is labeled as similar to the first process. The actions further include training a predictive model. The actions further include receiving data that identifies actions performed by a user. The actions further include applying the predictive model to one or more of the actions. The actions further include classifying a process performed by the one or more actions as similar to a particular process. The actions further include associating the user with the particular user characteristic.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a cognitive robotics analyzer are disclosed. In one aspect, a method includes the actions of receiving, for each user characteristic of a plurality of user characteristics, first data that identifies one or more first actions that perform a first process and second data that identifies one or more second actions that perform a second process that is labeled as similar to the first process. The actions further include training a predictive model. The actions further include receiving data that identifies actions performed by a user. The actions further include applying the predictive model to one or more of the actions. The actions further include classifying a process performed by the one or more actions as similar to a particular process. The actions further include associating the user with the particular user characteristic.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for automating a manual process. The methods, systems, and apparatus include actions of identifying a process that (i) is manually performed by a user interacting with a computer, and (ii) is to be automated for performance by a robot that is configured to interact with another computer. Additional actions include obtaining one or more images taken of a display of the computer while the user is interacting with the computer in manually performing the process and applying a computer vision technique to identify one or more activities associated with the process. Further actions include, for each of the one or more identified activities, generating activity information associated with the activity and generating a process definition for use in causing the robot to automatically perform the process.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for automating a manual process. The methods, systems, and apparatus include actions of identifying a process that (i) is manually performed by a user interacting with a computer, and (ii) is to be automated for performance by a robot that is configured to interact with another computer. Additional actions include obtaining one or more images taken of a display of the computer while the user is interacting with the computer in manually performing the process and applying a computer vision technique to identify one or more activities associated with the process. Further actions include, for each of the one or more identified activities, generating activity information associated with the activity and generating a process definition for use in causing the robot to automatically perform the process.