Abstract: Systems and methods for training a visual question answer model include training a teacher model by performing image conditional visual question generation on a visual language model (VLM) and a targeted visual question answer dataset using images to generate question and answer pairs. Unlabeled images are pseudolabeled using the teacher model to decode synthetic question and answer pairs for the unlabeled images. The synthetic question and answer pairs for the unlabeled images are merged with real data from the targeted visual question answer dataset to generate a self-augmented training set. A student model is trained using the VLM and the self-augmented training set to return visual answers to text queries.

Abstract: A 3D printer includes a nozzle configured to jet a drop of liquid metal therethrough. The 3D printer also includes a light source configured to illuminate the drop with a pulse of light. A duration of the pulse of light is from about 0.0001 seconds to about 0.1 seconds. The 3D printer also includes a camera configured to capture an image, video, or both of the drop. The 3D printer also includes a computing system configured to detect the drop in the image, the video, or both. The computing system is also configured to characterize the drop after the drop is detected. Characterizing the drop includes determining a size of the drop, a location of the drop, or both in the image, the video, or both.

Abstract: A computer-implemented method for training a neural network to predict object categories without manual annotation is provided. The method includes feeding training datasets including at least images and data annotations to an object detection neural network, converting, by a text prompter, the data annotations into natural text inputs, converting, by a text embedder, the natural text inputs into embeddings, minimizing objective functions during training to adjust parameters of the object detection neural network, and predicting, by the object detection neural network, objects within images and videos.

Abstract: A computer-implemented method for detecting objects within an advanced driver assistance system (ADAS) is provided. The method includes obtaining road scene datasets from a plurality of cameras, including at least road scene images and road scene data annotations, to be provided to an object detection neural network communicating with an open-vocabulary detector of a vehicle, converting, by a text prompter, the road scene data annotations into natural text inputs, converting, by a text embedder, the natural text inputs into embeddings, minimizing objective functions during training to adjust parameters of the object detection neural network, and detecting, by the object detection neural network, objects within the road scene datasets to provide alerts or notifications to a driver of the vehicle pertaining to the detected objects.

Abstract: Methods and systems for training a model include pre-training a backbone model with a pre-training decoder, using an unlabeled dataset with multiple distinct sensor data modalities that derive from different sensor types. The backbone model is fine-tuned with an output decoder after pre-training, using a labeled dataset with the multiple modalities.

Abstract: A method includes illuminating a drop with a pulse of light from a light source. A duration of the pulse of light is from about 0.0001 seconds to about 0.1 seconds. The method also includes capturing an image, video, or both of the drop. The method also includes detecting the drop in the image, the video, or both. The method also includes characterizing the drop after the drop is detected. Characterizing the drop includes determining a size of the drop, a location of the drop, or both in the image, the video, or both.

Abstract: A method includes capturing a video of a plurality of drops being jetted through a nozzle of a printer. The method also includes measuring a signal proximate to the nozzle based at least partially upon the video. The method also includes determining one or more metrics that characterize a behavior of the drops based at least partially upon the signal.

Abstract: A method for object detection obtains, from a set of RGB images lacking annotations, a set of regions that include potential objects, a bounding box, and an objectness score indicating a region prediction confidence. The method obtains, by a region scorer for each region in the set, a category from a fixed set of categories and a confidence for the category responsive to the objectness score. The method duplicates each region in the set to obtain a first and a second patch. The method encodes the patches to obtain an image vector. The method encodes a template sentence using the category to obtain a text vector for each category. The method compares the image vector to the text vector via a similarity function to obtain a similarity probability based on the confidence. The method defines a final set of pseudo labels based on the similarity probability being above a threshold.

Abstract: A method is provided for pretraining vision and language models that includes receiving image-text pairs, each including an image and a text describing the image. The method encodes an image into a set of feature vectors corresponding to input image patches and a CLS token which represents a global image feature. The method parses, by a text tokenizer, the text into a set of feature vectors as tokens for each word in the text. The method encodes the CLS token from the NN based visual encoder and the tokens from the text tokenizer into a set of features by a NN based text and multimodal encoder that shares weights for encoding both the CLS token and the tokens. The method accumulates the weights from multiple iterations as an exponential moving average of the weights during the pretraining until a predetermined error threshold is reduced to be under a threshold amount.

Abstract: A 3D printer includes a nozzle and a camera configured to capture an image, a video, or both of a plurality of drops of liquid metal being jetted through the nozzle. The 3D printer also includes a computing system configured to measure a signal proximate to the nozzle based at least partially upon the image, the video, or both. The computing system is also configured to determine one or more metrics that characterize a behavior of the drops based at least partially upon the signal.

Abstract: One embodiment can provide a system for detecting occlusion at an orifice of a three-dimensional (3D) printer nozzle while the printer nozzle is jetting liquid droplets. During operation, the system uses one or more cameras to capture an image of the orifice of the printer nozzle while the 3D printer nozzle is jetting liquid droplets. The system performs an image-analysis operation on the captured image to identify occluded regions within the orifice of the 3D printer nozzle, compute an occlusion fraction based on the determined occluded regions, and generate an output based on the computed occlusion fraction, thereby facilitating effective maintenance of the 3D printer.

Abstract: Systems and methods is provided for road hazard analysis. The method includes obtaining sensor data of a road environment including a road and observable surroundings, and applying labels to the sensor data. The method further includes training a first neural network model to identify road hazards, training a second neural network model to identify faded lane markings, and training a third neural network model to identify overhanging trees and blocking foliage. The method further includes implementing the trained neural network models to detect road hazards in a real road setting.

Abstract: A computer-implemented method for rut detection is provided. The method includes detecting, by a rut detection system, areas in a road-scene image that include ruts with pixel-wise probability values, wherein a higher value indicates a better chance of being a rut. The method further includes performing at least one of rut repair and vehicle rut avoidance responsive to the pixel-wise probability values. The detecting step includes performing neural network-based, pixel-wise semantic segmentation with context information on the road-scene image to distinguish rut pixels from non-rut pixels on a road depicted in the road-scene image.

Abstract: A multi-task text infilling system receives a digital image and identifies a region of interest of the image that contains original text. The system uses a machine learning model to determine, in parallel: a foreground image that includes the original text; a background image that omits the original text; and a binary mask that distinguishes foreground pixels from background pixels, The system receives a target mask that contains replacement text. The system then applies the target mask to blend the background image with the foreground layer image and yield a modified digital image that includes the replacement text and omits the original text.

Abstract: To replace text in a digital video image sequence, a system will process frames of the sequence to: define a region of interest (ROI) with original text in each of the frames; use the ROIs to select a reference frame from the sequence; select a target frame from the sequence; determine a transform function between the ROI of the reference frame and the ROI of the target frame; replace the original text in the ROI of the reference frame with replacement text to yield a modified reference frame ROI; and use the transform function to transform the modified reference frame ROI to a modified target frame ROI in which the original text is replaced with the replacement text. The system will then insert the modified target frame ROI into the target frame to produce a modified target frame. This process may repeat for other target frames of the sequence.

Abstract: A system is provided for generating a custom article to fit a target surface. During operation, the system compares an input dataset with a number of cut template cut meshes. A respective cut template cut mesh includes one or more cutting paths that correspond to a boundary of the mesh. Next, the system identifies a template cut mesh that produces a closest match with the input dataset, and applies global geometric transformations to the identified template cut mesh to warp the template cut mesh to conform to the input dataset. The system further refines and projects a set of boundary and landmark points from the template cut mesh to the input dataset to define cutting paths for the input dataset. Next, the system applies cutting paths to the input dataset to produce a cut-and-trimmed mesh.

Abstract: A 3D printer includes a nozzle configured to jet a drop of liquid metal therethrough. The 3D printer also includes a light source configured to illuminate the drop with a pulse of light. A duration of the pulse of light is from about 0.0001 seconds to about 0.1 seconds. The 3D printer also includes a camera configured to capture an image, video, or both of the drop. The 3D printer also includes a computing system configured to detect the drop in the image, the video, or both. The computing system is also configured to characterize the drop after the drop is detected. Characterizing the drop includes determining a size of the drop, a location of the drop, or both in the image, the video, or both.

Abstract: A method includes illuminating a drop with a pulse of light from a light source. A duration of the pulse of light is from about 0.0001 seconds to about 0.1 seconds. The method also includes capturing an image, video, or both of the drop. The method also includes detecting the drop in the image, the video, or both. The method also includes characterizing the drop after the drop is detected. Characterizing the drop includes determining a size of the drop, a location of the drop, or both in the image, the video, or both.

Abstract: A 3D printer includes a nozzle and a camera configured to capture an image, a video, or both of a plurality of drops of liquid metal being jetted through the nozzle. The 3D printer also includes a computing system configured to measure a signal proximate to the nozzle based at least partially upon the image, the video, or both. The computing system is also configured to determine one or more metrics that characterize a behavior of the drops based at least partially upon the signal.

Abstract: A method includes capturing a video of a plurality of drops being jetted through a nozzle of a printer. The method also includes measuring a signal proximate to the nozzle based at least partially upon the video. The method also includes determining one or more metrics that characterize a behavior of the drops based at least partially upon the signal.