Abstract: A system includes sensors, memory, and a processor. The sensors are configured to capture images of a workspace, and the memory is configured to store the images and a model of machinery located within the workspace. The processor is configured to generate a spatial representation of the workspace based on the captured images. The processor is also configured to identify non-machinery entities within the workspace and, for each of the identified entities, generate an object that specifies a spatial extent of the respective entity and a protective separation distance for the entity. Further, the processor is configured to recognize an interaction between the entities and merge the corresponding objects into a merged object specifying a protective separation distance therefor. Moreover, the processor is configured to control operation of the machinery in accordance with a safety protocol and the protective separation distance for the merged object.

Abstract: Image sensors distributed about a workcell including industrial machinery are registered using a registration object and an information tag associated therewith. The tag contains information specifying the location of the object and/or the pose of the object. This information is acquired along with images of the registration object, and the sensors are registered based at least in part on the images and the acquired information.

Abstract: Systems and methods utilize one or more 3D cameras (e.g., ToF cameras) in industrial safety applications. The 3D camera generates a depth map that may be used by external hardware and software to classify objects in a workcell and generate control signals for machinery. To facilitate sensor-specific calibration and coordination among sensors in a workcell, the sensors may store calibration data in a boot file that is loaded upon start-up. During initialization, the calibration data is loaded and, as the sensor operates, corrections are made to sensed data (e.g., pixel depth values) using the calibration data.

Abstract: A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Abstract: In various embodiments, systems and methods for safely operating machinery such as industrial robots signal intrusions into and/or above computationally defined safeguarded volumes in a workspace during sensor monitoring of the workspace. Various related actions may be taken in response thereto, including restricting operation of the machinery and/or issuing safety or occlusion signals.

Abstract: A monitoring system for monitoring a portal between a three-dimensional workcell and an adjacent three-dimensional workcells each of which includes controlled machinery, the portal being traversable by humans from either of the workcells into the adjacent workcell, the monitoring system including a plurality of cameras distributed throughout at least one of the workcell or the adjacent workcell and a controller for determining a proximity of a human between the adjacent workcells, where the controller is configured to electronically signal determined expected entry of the human into the adjacent workcell, and where the controller ignores detected features at the portal unassociated with humans except following receipt of the signal indicating expected entry of the human into the adjacent workcell through the portal.

Abstract: Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These “potential occupancy envelopes” (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

Abstract: Systems and methods utilize one or more 3D cameras (e.g., ToF cameras) in industrial safety applications. The 3D camera generates a depth map that may be used by external hardware and software to classify objects in a workcell and generate control signals for machinery. To facilitate sensor-specific calibration and coordination among sensors in a workcell, the sensors may store calibration data in a boot file that is loaded upon start-up. During initialization, the calibration data is loaded and, as the sensor operates, corrections are made to sensed data (e.g., pixel depth values) using the calibration data.

Abstract: A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Abstract: One or more computational strategies is used to reduce the complexity of handling detected point clusters appearing at a portal of a monitored workcell. If the other side of the portal is a second, adjacent workcell, the monitoring system for a first workcell predicts when a human in that workcell may pass through the portal to a second workcell and alerts the monitoring system for the second workcell. The prediction may be based on absolute proximity of a human to the portal or movement toward it. Other strategies are employed if the workcells overlap, or if the portal leads to an unmonitored space.

Abstract: In various embodiments, systems and methods for acquiring depth images utilize an architecture suited to safety-rated applications, and may include more than sensor (such as time-of-flight sensors) operating along different optical paths and a comparison module for ensuring proper sensor operation. Error metrics may be associated with pixel-level depth values for purposes of allowing safe control based on imperfectly known depths.

Abstract: An image processing system includes at least two three-dimensional sensors and a controller. Each of the sensors being for illuminating a corresponding field of view of the sensor, and generating an output array of pixelwise values indicative of distances to illuminated objects in the field of view, each sensor being configured to generate illumination at least at two different frequencies so that objects in the corresponding field of view are illuminated at the two different frequencies. The controller is communicably connected to the at least two sensors to receive pixelwise data from each sensor embodying intensity and distance information from the illuminated objects. The controller is configured to disambiguate a distance to each of the illuminated objects, resolve error in the received pixelwise data due to periodic distance ambiguity, and determine corrected pixelwise values indicative of true distance via each sensor illumination at the at least two different frequencies.

Abstract: A system includes sensors, memory, and a processor. The sensors are configured to capture images of a workspace, and the memory is configured to store the images and a model of machinery located within the workspace. The processor is configured to generate a spatial representation of the workspace based on the captured images. The processor is also configured to identify non-machinery entities within the workspace and, for each of the identified entities, generate an object that specifies a spatial extent of the respective entity and a protective separation distance for the entity. Further, the processor is configured to recognize an interaction between the entities and merge the corresponding objects into a merged object specifying a protective separation distance therefor. Moreover, the processor is configured to control operation of the machinery in accordance with a safety protocol and the protective separation distance for the merged object.

Abstract: This Application regards a system for continuously monitoring a workcell during operation of industrial machinery. The system includes a safety system, a monitoring system, and a controller. The safety system includes a sensor and supporting software or hardware for acquiring image data associated with the workcell. The monitoring system is for detecting a parameter value associated with the safety system. And the controller is configured to determine whether the image data is valid based at least in part on the detected parameter value, and cause an alert to be issued responsive to determining that the image data is invalid.

Abstract: A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Abstract: Systems and methods for determining safe and unsafe zones in a workspace—where safe actions are calculated in real time based on all relevant objects (e.g., some observed by sensors and others computationally generated based on analysis of the sensed workspace) and on the current state of the machinery (e.g., a robot) in the workspace—may utilize a variety of workspace-monitoring approaches as well as dynamic modeling of the robot geometry. The future trajectory of the robot(s) and/or the human(s) may be forecast using, e.g., a model of human movement and other forms of control. Modeling and forecasting of the robot may, in some embodiments, make use of data provided by the robot controller that may or may not include safety guarantees.

Abstract: Systems and methods for identifying a robot end effector in a processing environment may utilize one or more sensors for digitally recording visual information and providing that information to an industrial workflow. The sensor(s) may be positioned to record at least one image of the robot including the end effector. A processor may determine the identity of the end effector from the recorded image(s) and a library or database stored digital models.

Abstract: Systems and methods for continuously monitoring a workcell during operation of industrial machinery are disclosed. The system may comprise a safety system that includes at least one sensor and supporting software and/or hardware for acquiring image data associated with the workcell; a monitoring system for detecting a parameter value associated with the safety system; and a controller configured to determine a status of the safety system based at least in part on the detected parameter value and cause an alert to be issued if the status of the safety system does not satisfy a target objective.

Abstract: An integrated system and method for monitoring a workspace, classifying regions therein, dynamically identifying safe states, and identifying and tracking workpieces utilizes semantic analysis of a robot in the workspace and identification of the workpieces with which it interacts, as well as a semantic understanding of entity properties both governing and arising from interaction with other entities for control purposes. These properties may be stored in a profile corresponding to the entity; depending on the implementation, the profile and methods associated with the entity may be stored in a data structure corresponding to the entity “class.” Volumetric regions (e.g., voxels) corresponding to each entity may be tracked and used to implement object methods and/or control methods. For example, machinery may be controlled in accordance with the attributes specified by the objects corresponding to identified entities in a monitored workspace.

Abstract: Crosstalk mitigation among cameras in neighboring monitored workcells is achieved by computationally defining a noninterference scheme that respects the independent monitoring and operation of each workcell. The scheme may involve communication between adjacent cells to adjudicate non-interfering camera operation or system-wide mapping of interference risks and mitigation thereof. Mitigation strategies can involve spread-spectrum techniques.