Abstract: A vision analytics and validation (VAV) system for providing an improved inspection of robotic assembly, the VAV system comprising a trained neural network three-way classifier, to classify each component as good, bad, or do not know, and an operator station configured to enable an operator to review an output of the trained neural network, and to determine whether a board including one or more “bad” or a “do not know” classified components passes review and is classified as good, or fails review and is classified as bad. In one embodiment, a retraining trigger to utilize the output of the operator station to train the trained neural network, based on the determination received from the operator station.

Abstract: A vision analytics and validation (VAV) system for providing an improved inspection of robotic assembly, the VAV system comprising a trained neural network three-way classifier, to classify each component as good, bad, or do not know, and an operator station configured to enable an operator to review an output of the trained neural network, and to determine whether a board including one or more “bad” or a “do not know” classified components passes review and is classified as good, or fails review and is classified as bad. In one embodiment, a retraining trigger to utilize the output of the operator station to train the trained neural network, based on the determination received from the operator station.

Abstract: A method for vision-based tool localization (VTL) in a robotic assembly system including one or more calibrated cameras, the method comprising capturing a plurality of images of the tool contact area from a plurality of different vantage points, determining an estimated position of the tool contact area based on an image, and refining the estimated position based on another image from another vantage point. The method further comprises providing the refined position to the robotic assembly system to enable accurate control of the tool by the robotic assembly system.

Abstract: A robotic cell calibration method comprising a robotic cell system having elements comprising: one or more cameras, one or more sensors, components, and a robotic arm. The method comprises localizing positions of the one or more cameras and components relative to a position of the robotic arm using a common coordinate frame, moving the robotic arm in a movement pattern, and using the cameras and sensors to determine robotic arm position at multiple times during the movement. The method includes identifying a discrepancy in robotic arm position between a predicted position and the determined position in real time, and computing, by an auto-calibrator, a compensation for the identified discrepancy, the auto-calibrator solving for the elements in the robotic cell system as a system. The method includes modifying actions of the robotic arm in real time during the movement based on the compensation.

Abstract: A method, system, and apparatus create a 3D CAD model. Scan data from two or more structured scans of a real-world scene are acquired and each scan processed independently by segmenting the scan data into multiple segments, filtering the scan data, and fitting an initial model that is used as a model candidate. Model candidates are clustered into groups and a refined model is fit onto the model candidates in the same group. A grid of cells representing points is mapped over the refined model. Each of the grid cells is labeled by processing each scan independently, labeling each cell located within the refined model as occupied, utilizing back projection to label remaining cells as occluded or empty. The labels from multiple scans are then combined. Based on the labeling, model details are extracted to further define and complete the refined model.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: A method, system, apparatus, article of manufacture, and computer-readable storage medium provide the ability to merge multiple point cloud scans. A first raw scan file and a second raw scan file (each including multiple points) are imported. The scan files are segmented by extracting segments based on geometry in the scene. The segments are filtered. A set of candidate matching feature pairs are acquired by registering features from one scan to features from another scan. The two raw scan files are merged based on the candidate matching feature pairs.

Abstract: A vision analytics and validation (VAV) system for providing an improved inspection of robotic assembly, the VAV system comprising a trained neural network three-way classifier, to classify each component as good, bad, or do not know, and an operator station configured to enable an operator to review an output of the trained neural network, and to determine whether a board including one or more “bad” or a “do not know” classified components passes review and is classified as good, or fails review and is classified as bad. In one embodiment, a retraining trigger to utilize the output of the operator station to train the trained neural network, based on the determination received from the operator station.

Abstract: A method, system, and apparatus create a 3D CAD model. Scan data from two or more structured scans of a real-world scene are acquired and each scan processed independently by segmenting the scan data into multiple segments, filtering the scan data, and fitting an initial model that is used as a model candidate. Model candidates are clustered into groups and a refined model is fit onto the model candidates in the same group. A grid of cells representing points is mapped over the refined model. Each of the grid cells is labeled by processing each scan independently, labeling each cell located within the refined model as occupied, utilizing back projection to label remaining cells as occluded or empty. The labels from multiple scans are then combined. Based on the labeling, model details are extracted to further define and complete the refined model.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: A method, system, and apparatus create a 3D CAD model. Scan data from two or more structured scans of a real-world scene are acquired and each scan processed independently by segmenting the scan data into multiple segments, filtering the scan data, and fitting an initial model that is used as a model candidate. Model candidates are clustered into groups and a refined model is fit onto the model candidates in the same group. A grid of cells representing points is mapped over the refined model. Each of the grid cells is labeled by processing each scan independently, labeling each cell located within the refined model as occupied, utilizing back projection to label remaining cells as occluded or empty. The labels from multiple scans are then combined. Based on the labeling, model details are extracted to further define and complete the refined model.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Abstract: A method, apparatus, and system provides the ability to process and render a point cloud. The points in the point cloud are grouped into three-dimensional (3D) voxels. A position of each of the points is stored in the point data file. The position is with respect to a location of the point's corresponding 3D voxel. Surface normal data for a surface normal associated with each of the points is also stored in the point data file. The points are organized into levels of details (LODs). The point data file is provided to a graphics processing unit (GPU) that processes the point data file to render the point cloud. During rendering, a LOD is selected to determine the points in the point cloud to render.

Abstract: A method, apparatus, system, and computer readable storage medium provide the ability to pre-segment point cloud data. Point cloud data is obtained and segmented. Based on the segment information, a determination is made regarding points needed for shape extraction. Needed points are fetched and used to extract shapes. The extracted shapes are used to cull points from the point cloud data.

Abstract: A method, system, apparatus, article of manufacture, and computer-readable storage medium provide the ability to merge multiple point cloud scans. A first raw scan file and a second raw scan file (each including multiple points) are imported. The scan files are segmented by extracting segments based on geometry in the scene. The segments are filtered. A set of candidate matching feature pairs are acquired by registering features from one scan to features from another scan.

Abstract: A method, system, apparatus, article of manufacture, and computer-readable storage medium provide the ability to merge multiple point cloud scans. A first raw scan file and a second raw scan file (each including multiple points) are imported. The scan files are segmented by extracting segments based on geometry in the scene. The segments are filtered to reduce a number of segments and identify features. A set of candidate matching feature pairs are acquired by coarsely registering features from one scan to features from another scan. The candidate pairs are refined by improving alignment based on corresponding points in the features. The candidate pairs are scored and then merged based on the scores.

Abstract: A method, apparatus, and system provides the ability to process and render a point cloud. The points in the point cloud are grouped into three-dimensional (3D) voxels. A position of each of the points is stored in the point data file. The position is with respect to a location of the point's corresponding 3D voxel. Surface normal data for a surface normal associated with each of the points is also stored in the point data file. The points are organized into levels of details (LODs). The point data file is provided to a graphics processing unit (GPU) that processes the point data file to render the point cloud. During rendering, a LOD is selected to determine the points in the point cloud to render.

Abstract: A method, system, apparatus, article of manufacture, and computer-readable storage medium provide the ability to merge multiple point cloud scans. A first raw scan file and a second raw scan file (each including multiple points) are imported. The scan files are segmented by extracting segments based on geometry in the scene. The segments are filtered to reduce a number of segments and identify features. A set of candidate matching feature pairs are acquired by coarsely registering features from one scan to features from another scan. The candidate pairs are refined by improving alignment based on corresponding points in the features. The candidate pairs are scored and then merged based on the scores.

Abstract: A method, system, apparatus, article of manufacture, and computer-readable storage medium provide the ability to merge multiple point cloud scans. A first raw scan file and a second raw scan file (each including multiple points) are imported. The scan files are segmented by extracting segments. Features are extracted from the segments. A set of candidate matching feature pairs are acquired by registering/matching/pairing features from one scan to features from another scan. The candidate pairs are refined based on an evaluation of all of the matching pairs. The candidate pairs are further refined by extracting sample points from the segments (within the matched pairs) and refining the pairs based on the points. The feature pairs are scored and then merged based on the scores.