Abstract: A method for generating a three-dimensional model of an asset includes receiving input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around an asset, generating the mission plan based on the input parameters including information of a representative asset type, wherein the mission plan includes waypoints identifying locations and orientations of one or more image sensors of the unmanned vehicle, generating a flight path for the unmanned vehicle connecting the waypoints that satisfy one or more predefined criteria, monitoring a vehicle state of the unmanned vehicle during execution of the flight path from one waypoint to the next waypoint, determining, at each waypoint, a local geometry of the asset sensed by the one or more image sensors, changing the mission plan on-the-fly based on the local geometry, and capturing images of the asset along waypoints of the changed mission plan.

Abstract: Provided are systems and methods for generating an autonomous 3D inspection plan for an unmanned robot. In an example, the method may include receiving a selection of a plurality of regions of interest with respect to a virtual asset displayed in virtual space, detecting a 3D position of the regions of interest within a coordinate frame of the virtual space, auto-generating a travel path about a physical asset corresponding to the virtual asset by generating a virtual 3D travel path with respect to the virtual asset based on the detected 3D positions of the selected regions of interest within the coordinate frame, aligning the virtual 3D travel path in the virtual space with a physical travel path in a physical space, and outputting a robotic inspection plan comprising the auto-generated physical travel path for the unmanned robot.

Abstract: Provided are systems and methods for generating an autonomous 3D inspection plan for an unmanned robot. In an example, the method may include receiving a selection of a plurality of regions of interest with respect to a virtual asset displayed in virtual space, detecting a 3D position of the regions of interest within a coordinate frame of the virtual space, auto-generating a travel path about a physical asset corresponding to the virtual asset by generating a virtual 3D travel path with respect to the virtual asset based on the detected 3D positions of the selected regions of interest within the coordinate frame, aligning the virtual 3D travel path in the virtual space with a physical travel path in a physical space, and outputting a robotic inspection plan comprising the auto-generated physical travel path for the unmanned robot.

Abstract: A robotic system includes one or more optical sensors configured to separately obtain two dimensional (2D) image data and three dimensional (3D) image data of a brake lever of a vehicle, a manipulator arm configured to grasp the brake lever of the vehicle, and a controller configured to compare the 2D image data with the 3D image data to identify one or more of a location or a pose of the brake lever of the vehicle. The controller is configured to control the manipulator arm to move toward, grasp, and actuate the brake lever of the vehicle based on the one or more of the location or the pose of the brake lever.

Abstract: A method for generating a three-dimensional model of an asset includes receiving input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around an asset, generating the mission plan based on the input parameters including information of a representative asset type, wherein the mission plan includes waypoints identifying locations and orientations of one or more image sensors of the unmanned vehicle, generating a flight path for the unmanned vehicle connecting the waypoints that satisfy one or more predefined criteria, monitoring a vehicle state of the unmanned vehicle during execution of the flight path from one waypoint to the next waypoint, determining, at each waypoint, a local geometry of the asset sensed by the one or more image sensors, changing the mission plan on-the-fly based on the local geometry, and capturing images of the asset along waypoints of the changed mission plan.

Abstract: The present approach employs a context-aware simulation platform to facilitate control of a robot remote from an operator. Such a platform may use the prior domain/task knowledge along with the sensory feedback from the remote robot to infer context and may use inferred context to dynamically change one or both of simulation parameters and a robot-environment-task state being simulated. In some implementations, the simulator instances make forward predictions of their state based on task and robot constraints. In accordance with this approach, an operator may therefore issue a general command or instruction to a robot and based on this generalized guidance, the actions taken by the robot may be simulated, and the corresponding results visually presented to the operator prior to evaluate prior to the action being taken.

Abstract: In accordance with certain implementations of the present approach, a reduced, element-by-element, data set is transmitted between a robot having a sensor suite and a control system remote from the robot that is configured to display a representation of the environment local to the robot. Such a scheme may be useful in allowing a human operator remote from the robot to perform an inspection using the robot while the robot is on-site with an asset and the operator is off-site. In accordance with the present approach, an accurate representation of the environment in which the robot is situated is provided for the operator to interact with.

Abstract: A system for performing industrial tasks includes a robot and a computing device. The robot includes one or more sensors that collect data corresponding to the robot and an environment surrounding the robot. The computing device includes a user interface, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to receive the collected data from the robot, generate a virtual recreation of the robot and the environment surrounding the robot, receive inputs from a human operator controlling the robot to demonstrate an industrial task. The system is configured to learn how to perform the industrial task based on the human operator's demonstration of the task, and perform, via the robot, the industrial task autonomously or semi-autonomously.

Abstract: System and methods may evaluate and/or improve target aiming accuracy for a sensor of an Unmanned Aerial Vehicle (“UAV”). According to some embodiments, a position and orientation measuring unit may measure a position and orientation associated with the sensor. A pose estimation platform may execute a first order calculation using the measured position and orientation as the actual position and orientation to create a first order model. A geometry evaluation platform may receive planned sensor position and orientation from a targeting goal data store and calculate a standard deviation for a target aiming error utilizing: (i) location and geometry information associated with the industrial asset, (ii) a known relationship between the sensor and a center-of-gravity of the UAV, (iii) the first order model as a transfer function, and (iv) an assumption that the position and orientation of the sensor have Gaussian-distributed noises with zero mean and a pre-determined standard deviation.

Abstract: The present approach relates to streaming data derived from inspection data acquired using one or more robots performing inspections of an asset or assets. Such inspections may be fully or partially automated, such as being controlled by one or more computer-based routines, and may be planned or dynamically altered in response to inputs or requirements associated with an end-user of the inspection data, such as a subscriber to the data in a publication/subscription distribution scheme. Thus, an inspection may be planned or altered based on the data needs or subscription levels of the user or customers.

Abstract: A robotic system includes a controller configured to obtain image data from one or more optical sensors and to determine one or more of a location and/or pose of a vehicle component based on the image data. The controller also is configured to determine a model of an external environment of the robotic system based on the image data and to determine tasks to be performed by components of the robotic system to perform maintenance on the vehicle component. The controller also is configured to assign the tasks to the components of the robotic system and to communicate control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component.

Abstract: A three-dimensional model data store may contain a three-dimensional model of an industrial asset, including points of interest associated with the industrial asset. An inspection plan data store may contain an inspection plan for the industrial asset, including a path of movement for an autonomous inspection robot. An industrial asset inspection platform may receive sensor data from an autonomous inspection robot indicating characteristics of the industrial asset and determine a current location of the autonomous inspection robot along the path of movement in the inspection plan along with current context information. A forward simulation of movement for the autonomous inspection robot may be executed from the current location, through a pre-determined time window, to determine a difference between the path of movement in the inspection plan and the forward simulation of movement along with future context information.

Abstract: Provided are systems and methods for monitoring an asset via an autonomous model-driven inspection. In an example, the method may include storing an inspection plan including a virtually created three-dimensional (3D) model of a travel path with respect to a virtual asset that is created in virtual space, converting the virtually created 3D model of the travel path about the virtual asset into a physical travel path about a physical asset corresponding to the virtual asset, autonomously controlling vertical and lateral movement of the unmanned robot in three dimensions with respect to the physical asset based on the physical travel path and capturing data at one or more regions of interest, and capturing data at one or more regions of interest, and storing information concerning the captured data about the asset.

Abstract: Provided are systems and methods for generating an autonomous 3D inspection plan for an unmanned robot. In an example, the method may include receiving a selection of a plurality of regions of interest with respect to a virtual asset displayed in virtual space, detecting a 3D position of the regions of interest within a coordinate frame of the virtual space, auto-generating a travel path about a physical asset corresponding to the virtual asset by generating a virtual 3D travel path with respect to the virtual asset based on the detected 3D positions of the selected regions of interest within the coordinate frame, aligning the virtual 3D travel path in the virtual space with a physical travel path in a physical space, and outputting a robotic inspection plan comprising the auto-generated physical travel path for the unmanned robot.

Abstract: A robotic system includes one or more optical sensors configured to separately obtain two dimensional (2D) image data and three dimensional (3D) image data of a brake lever of a vehicle, a manipulator arm configured to grasp the brake lever of the vehicle, and a controller configured to compare the 2D image data with the 3D image data to identify one or more of a location or a pose of the brake lever of the vehicle. The controller is configured to control the manipulator arm to move toward, grasp, and actuate the brake lever of the vehicle based on the one or more of the location or the pose of the brake lever.

Abstract: A robotic system includes a controller configured to obtain image data from one or more optical sensors and to determine one or more of a location and/or pose of a vehicle component based on the image data. The controller also is configured to determine a model of an external environment of the robotic system based on the image data and to determine tasks to be performed by components of the robotic system to perform maintenance on the vehicle component. The controller also is configured to assign the tasks to the components of the robotic system and to communicate control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component.

Abstract: A processing system having at least one processor operatively coupled to at least one memory. The processor receives sensor data from the at least one sensor indicating one or more characteristics of the asset. The processor generates, updates, or maintains a digital representation that models the one or more characteristics of the asset. The processor detects a defect of the asset based at least in part on the one or more characteristics. The processor generate an output signal encoding or conveying instructions to provide a recommendation to an operator, to control the at least one robot to address the defect on the asset, or both, based on the defect and the digital representation of the asset.