Abstract: A method of performing open world object detection includes receiving object data, that includes embeddings data corresponding to a plurality of embeddings for known objects in a first input image, projecting the embeddings into a hyperbolic embedding space that includes embeddings in a plurality of categories of objects each including one or more classes of objects, regularizing the projected embeddings within the hyperbolic embedding space by moving each of the projected embeddings closer to embeddings in a same category of the plurality of categories and further away from embeddings in different categories of the plurality of categories, receiving an unmatched query corresponding to an object in a second input image, and generating, based on the hyperbolic embedding space including the regularized embeddings, an output signal that indicates whether the object in the second input image corresponds to an unknown object in one of the classes of objects.

Abstract: A method of obtaining an uncertainty attribution of a prediction of objects in an input image includes receiving an input image, generating a prediction of objects in the input image, estimating an uncertainty associated with the prediction of the objects in the input image, calculating an uncertainty attribution that represents regions of the input image that cause the estimated uncertainty, including generating a plurality of adversarial gradients each corresponding to a modification of the input image configured to change the estimated uncertainty, and generating an output indicative of the calculated uncertainty attribution.

Abstract: A method of performing data pre-selection for an object detection system includes receiving a first dataset that includes unlabeled data corresponding to one or more images, providing the first dataset and a plurality of learnable prompt vectors to a pre-training model. The learnable prompt vectors include text inputs. The method further includes generating, using the pre-training model, an unsupervised learning prompt based on the first dataset and the plurality of learnable prompt vectors. The unsupervised learning prompt corresponds to a multi-modal feature of the one or more images of the first dataset. The method further includes extracting features from either of the first dataset and a second dataset based on the unsupervised learning prompt, selecting and labeling a subset of instances of the extracted features, and generating and outputting a labeled dataset based on the labeled subset of instances.

Abstract: In some implementations, the device may receive, for a plurality of stations, processing times indicating a time required for a part to be processed by each station, and waiting times indicating how long the part waited before moving to a subsequent one of the plurality of stations. In addition, the device may determine, cycle times for a predetermined window of time, where the cycle times indicates an average number of parts processed by the plurality of stations during the predetermined window of time. The device may determine one of the stations as a potential bottleneck station. Moreover, the device may display, to a user, the potential bottleneck station as a visualization which includes the processing time, the waiting time, and the cycle time of the potential bottleneck station. Also, the device may receive, from the user, a user feedback related to the potential bottleneck station.

Abstract: Methods and systems for providing an interactive image scene graph pattern search are provided. A user is provide with an image having a plurality of selectable segmented regions therein. The user selects one or more of the segmented regions to build a query graph. Via a graph neural network, matching target graphs are retrieved that contain the query graph from a target graph database. Each matching target graph has matching target nodes that match with the query nodes of the query graph. Matching target images from an image database are associated with the matching target graphs. Embeddings of each of the query nodes and the matching target nodes are extracted. A comparison of the embeddings of each query node with the embeddings of each matching target node is performed. The user interface displays the matching target images that are associated with the matching target graphs.

Abstract: A method for performing at least one perception task associated with autonomous vehicle control includes receiving a first dataset and identifying a first object category of objects associated with the plurality of images, the first object category including a plurality of object types. The method also includes identifying a current statistical distribution of a first object type of the plurality of object types and determining a first distribution difference between the current statistical distribution of the first object type and a standard statistical distribution associated with the first object category. The method also includes, in response to a determination that the first distribution difference is greater than a threshold, generating first object type data corresponding to the first object type, configuring at least one attribute of the first object type data, and generating a second dataset by augmenting the first dataset using the first object type data.

Abstract: A computer-implemented system and method relates to language-guided self-supervised semantic segmentation. A modified image is generated by performing data augmentation on a source image. A machine learning model generates first pixel embeddings based on the modified image. First segment embeddings are generated using the first pixel embeddings. A pretrained vision-language model generates second pixel embeddings based on the source image. Second segment embeddings are generated by applying segment contour data from the first pixel embeddings to the second pixel embeddings after the data augmentation is performed on the second pixel embeddings. Embedding consistent loss data is generated by comparing the first segment embeddings in relation to the second segment embeddings. Combined loss data is generated that includes the embedding consistent loss data. Parameters of the machine learning model are updated based on the combined loss data.

Abstract: A method for training a model for determining graph similarity is disclosed. The method comprises receiving a first graph and a second graph as training inputs, the first graph and the second graph each including nodes connected by edges. The method further comprises applying a model to the first graph and the second graph to determine (i) pairs of aligned nodes between the first graph and the second graph and (ii) a first training loss. The method further comprises generating a first augmented graph by modifying the first graph depending on the pairs of aligned nodes. The method further comprises applying the model to the first graph and the first augmented graph to determine a second training loss. The method further comprises refining the model based on the first training loss and the second training loss.

Abstract: A computer-implemented system and method includes generating first pseudo segment data from a first augmented image and generating second pseudo segment data from a second augmented image. The first augmented image and the second augmented image are in a dataset along with other augmented images. A machine learning system is configured to generate pixel embeddings based on the dataset. The first pseudo segment data and the second pseudo segment data are used to identify a first set of segments that a given pixel belongs with respect to the first augmented image and the second augmented image. A second set of segments is identified across the dataset. The second set of segments do not include the given pixel. A local segmentation loss is computed for the given pixel based on the corresponding pixel embedding that involves attracting the first set of segments while repelling the second set of segments.

Abstract: A computer-implemented system and method relate to gesture recognition. A machine learning model includes a first subnetwork, a second subnetwork, and a third subnetwork. The first subnetwork generates feature data based on sensor data, which includes a gesture. The feature data is divided into a set of patches. The second subnetwork selects a target patch of feature data from among the set of patches. The third subnetwork generates gesture data based on the target patch of feature data. The gesture data identifies the gesture of the sensor data. Command data is generated based on the gesture data. A device is controlled based on the command data.

Abstract: A computer-implemented system and method relate to gesture recognition. A machine learning system is trained using a training dataset of sensor data that include a set of gestures. The training dataset includes at least a first subset that displays a first gesture. Loss data is generated based on a first loss function that includes a first cross entropy loss and a second cross entropy loss. Parameters of the machine learning system are updated based on the loss data. The machine learning system is outputted and configured for gesture recognition of the set of gestures. The machine learning system includes (i) a first subnetwork to generate feature data based on the sensor data, (ii) a second subnetwork to extract a selected patch of the feature data, and (iii) a third subnetwork to generate gesture data based on a classification of the corresponding feature data of the selected patch. The first cross entropy loss is based on a first performance of the second subnetwork in relation to the training dataset.

Abstract: A method is disclosed for improving a mobile robot that is configured to perform a task in an environment using an operating procedure. Data is received that was recorded by the mobile robot using one or more sensors as the mobile robot navigates the environment to perform the task. A database and/or a model associated with the environment is updated to incorporate the recorded data. The operating procedure of the mobile robot can be modified, based on the database and/or the model, to generate a modified operating procedure for performing the task in the environment that improves a performance of the mobile robot. Additionally, a recommendation for improving the performance of the mobile robot when performing the task in the environment can be determined, based on the database and/or the model, and displayed to a user for consideration.

Abstract: A computer-implemented method for a machine-learning network includes receiving an input dataset, wherein the input dataset is indicative of image information, tabular information, radar information, sonar information, or sound information, sending the input dataset to the machine-learning model to output predictions associated with the input data, identifying one or more slices associated with the input dataset and the machine learning model in a first iteration, wherein each of the one or more slices include input data from the input dataset and common attributes associated with each slice, outputting an interface that includes information associated with the one or more slices and performance measurements of the one or more slices of the first iteration and subsequent iterations identifying subsequent slices, wherein the performance measurements relate to the predictions associated with the first iteration and subsequent iterations.

Abstract: A computer-implemented method for a machine-learning network that includes receiving an input dataset, sending the input dataset to a first machine-learning model to output predictions associated with the input data, identifying one or more slices associated with the input dataset and a first machine learning model in a first iteration, wherein each of the one or more slices include input data from the input dataset and common attributes associated with each slice; upon selecting one or more slices of the input dataset, training a shallow regressor model configured to predict residuals associated with the model, create a representation associated with a ground-truth label and a second representation associated with a model prediction associated with each sample associated with each of the one or more slices, determine residuals associated with every prediction of the first machine learning model, training the shallow regressor to compute one or more predicted residuals of the selected slices, generate an opti

Abstract: A method for training a machine learning model. The method comprises receiving a training dataset that includes a plurality of images. The method also includes identifying, by a machine learning model, at least one portion of at least one image of the plurality of images in the training dataset associated with a first object type. The method further includes identifying other images having at least one portion that includes the first object type. The method also includes grouping the identified other images into a first image group. The method also includes generating for display a first user interface, that at least includes a rank matrix, wherein a first row of the rank matrix represents the images of the first image object. The user may provide feedback for the visualization using the first interface. The method may also include training the machine learning model based on the user feedback.

Abstract: A system and method is disclosed having an end-to-end two-stage depth estimation deep learning framework that takes one spherical color image and estimate dense spherical depth maps. The contemplated framework may include a view synthesis (stage 1) and a multi-view stereo matching (stage 2). The combination of the two-stage process may provide the advantage of the geometric constraints from stereo matching to improve depth map quality, without the need of additional input data. It is also contemplated that a spherical warping layer may be used to integrate multiple spherical features volumes to one cost volume with uniformly sampled inverse depth for the multi-view spherical stereo matching stage. The two-stage spherical depth estimation system and method may be used in various applications including virtual reality, autonomous driving and robotics.

Abstract: Methods and systems for performing function testing for moveable objects. One system includes an electronic processor configured to access a driving scene including a moveable object. The electronic processor is also configured to perform spatial representation learning on the driving scene. The electronic processor is also configured to generate an adversarial example based on the learned spatial representation. The electronic processor is also configured to retrain the deep learning model using the adversarial example and the driving scene.

Abstract: A method for performing at least one perception task associated with autonomous vehicle control includes receiving a first dataset and identifying a first object category of objects associated with the plurality of images, the first object category including a plurality of object types. The method also includes identifying a current statistical distribution of a first object type of the plurality of object types and determining a first distribution difference between the current statistical distribution of the first object type and a standard statistical distribution associated with the first object category. The method also includes, in response to a determination that the first distribution difference is greater than a threshold, generating first object type data corresponding to the first object type, configuring at least one attribute of the first object type data, and generating a second dataset by augmenting the first dataset using the first object type data.

Abstract: Methods and systems for performing concept-based adversarial generation with steerable and diverse semantics. One system includes an electronic processor configured to access an input image. The electronic processor is also configured to perform concept-based semantic image generation based on the input image. The electronic processor is also configured to perform concept-based semantic adversarial learning using a set of semantic latent spaces generated as part of performing the concept-based semantic image generation. The electronic processor is also configured to generate an adversarial image based on the concept-based semantic adversarial learning. The electronic processor is also configured to test a target model using the adversarial image.

Abstract: A computer-implemented system and method for semantic localization of various objects includes obtaining an image from a camera. The image displays a scene with a first object and a second object. A first set of 2D keypoints are generated with respect to the first object. First object pose data is generated based on the first set of 2D keypoints. Camera pose data is generated based on the first object pose data. A keypoint heatmap is generated using the camera pose data. A second set of 2D keypoints is generated with respect to the second object based on the keypoint heatmap. Second object pose data is generated based on the second set of 2D keypoints. First coordinate data of the first object is generated in world coordinates using the first object pose data and the camera pose data. Second coordinate data of the second object is generated in the world coordinates using the second object pose data and the camera pose data. The first object is tracked based on the first coordinate data.