Abstract: The present disclosure describes a model for large scale color prediction of objects identified in images. Embodiments of the present disclosure include an object detection network, an attention network, and a color classification network. The object detection network generates object features for an object in an image and may include a convolutional neural network (CNN), region proposal network, or a ResNet. The attention network generates an attention vector for the object based on the object features, wherein the attention network takes a query vector based on the object features, and a plurality of key vector and a plurality of value vectors corresponding to a plurality of colors as input. The color classification network generates a color attribute vector based on the attention vector, wherein the color attribute vector indicates a probability of the object including each of the plurality of colors.

Abstract: Methods, systems, and media for image searching are described. Images comprising one query image and a plurality of candidate images are received. For each candidate image, a first model similarity measure from an output of a first model configured for scene classification to perceive scenes in the images is determined. Further, for each candidate image of the plurality of candidate images, a second model similarity measure from the output of a second model configured for attribute classification to perceive attributes in the images is determined. For each candidate image of the plurality of candidate images, a similarity agglomerate index of a weighted aggregate of the first model similarity measure and the second model similarity measure is computed. The plurality of candidate images based on the respective similarity agglomerate index of each candidate image are ranked and a first ranked candidate images corresponding to the searched images are generated.

Abstract: A human detection device, which analyzes a fisheye image obtained by a fisheye camera installed above a detection target area to detect a person present in the detection target area, includes: a detector configured to detect a human body candidate and an object from a fisheye image; and an erroneous detection determination unit configured to determine whether or not the human body candidate has been erroneously detected based on a positional relationship between a detection position of the human body candidate and a detection position of the object present around the human body candidate.

Abstract: Systems and methods described herein relate to jointly training a machine-learning-based monocular optical flow, depth, and scene flow estimator. One embodiment processes a pair of temporally adjacent monocular image frames using a first neural network structure to produce an optical flow estimate and to extract, from at least one image frame in the pair of temporally adjacent monocular image frames, a set of encoded image context features; triangulates the optical flow estimate to generate a depth map; extracts a set of encoded depth context features from the depth map using a depth context encoder; and combines the set of encoded image context features and the set of encoded depth context features to improve performance of a second neural network structure in estimating depth and scene flow.

Abstract: Keyword localization digital image search techniques are described. These techniques support an ability to indicate “where” a corresponding keyword is to be expressed with respect to a layout in a respective digital image resulting from a search query. The search query may also include an indication of a size of the keyword as expressed in the digital image, a number of instances of the keyword, and so forth. Additionally, the techniques and systems as described herein support real time search through use of keyword signatures.

Abstract: Devices, methods, and non-transitory program storage devices are disclosed to: obtain an input image; identify a first face in the input image; divide the first face into a plurality of regions; identify obstructions in one or more of the plurality of regions; select a subset of regions, based on the identified obstructions; and determine a white point for the first face based, at least in part, on the selected subset of regions. In some embodiments, obstructions may be identified using a face obstruction detection model (e.g., a logistic regression model trained based on a large dataset of annotated facial images with and without obstructions). The identified obstructions may comprise, e.g., facial hair, head hair, glasses, a facial covering, a head covering, a face mask, or clothing. In some embodiments, a white balancing operation may be performed on the input image based, at least in part, on the determined white point.

Abstract: A management system including at least one processor, wherein the processor is configured to acquire an image obtained by imaging an outer surface of each of plural sample containers which contains a sample and in which name information of a subject from whom the sample is collected and subject discrimination information for discriminating plural subjects having the same surname and the same name are given to the outer surface, recognize the name information and the subject discrimination information given to each of the sample containers based on the image, and associate a test result related to the sample contained in each of the sample containers with a test order including the name information and the subject discrimination information, based on a result of the recognition and the test order.

Abstract: A contextual query system is configured to perform operations that include: causing display of a graphical user interface at a client device, the graphical user interface including a display of image data that comprises a set of image features; generating a query based on the set of image features of the image data; accessing media content based on the query at a repository, the repository comprising a collection of media content; and causing display of a presentation of the media content within the graphical user interface at the client device.

Abstract: A system, apparatus, method, and non-transitory computer readable medium for performing image search verification using an online platform may include a memory storing computer readable instructions and a database corresponding to a neural network associated with the online platform, and processing circuitry configured to execute the computer readable instructions. The processing circuitry may cause the server to receive an image-based search query from a first user device, the image-based search query including at least one image including a search object, and search query parameters related to the search object, analyze the search object using the neural network to determine an object type of the search object and at least one object type specific characteristic of the search object, and receive at least one search result response from at least one second user device.

Abstract: A system for performing automated inspection of a sewer line using machine learning. The system comprises a inspection data database storing sewer inspection data including images and metadata associated therewith; an inspection upload module receiving and uploading the sewer inspection data to the inspection data database; and an inspection module receiving the sewer inspection data from the inspection data database and generating therefrom identification data including characteristics of the sewer line identified and categorized.

Abstract: A method, apparatus and computer program product train a signature encoding module and a query processing module. In a method, at least one of the signature encoding module and the query processing module is trained by providing the signature encoding module with a reference image containing a particular object of interest that is marked therewithin. The method generates a digital signature of the object of interest and at least some context associated therewith and provides the query processing module with a query image and the digital signature representing the object of interest and at least some of the context associated therewith. The method additionally identifies the object of interest within the query image based upon the digital signature and modifies at least one of the signature encoding module or the query processing module based upon a qualitative or quantitative difference between the objects of interest identified within the query image and marked in the reference image.

Abstract: The present disclosure provides a method and system for processing multi-modality images. The method may include obtaining multi-modality images; registering the multi-modality images; fusing the multi-modality images; generating a reconstructed image based on a fusion result of the multi-modality images; and determining a removal range with respect to a focus based on the reconstructed image. The multi-modality images may include at least three modalities. The multi-modality images may include a focus.

Abstract: Systems and methods here may be used for pre-processing images, including using a computer for receiving a pixelated image of a paper document of an original size, downscaling the received pixelated image, employing a neural network algorithm to the downscaled image to identify four corners of the paper document in the received pixelated image, re-enlarging the downscaled image to the original size, identifying each of four corners of the paper document in the pixelated image, determining a quadrilateral composed of lines that intersect at four angles at the four corners of the paper document in the pixelated image, defining a projective plane of the pixelated image, and determining an inverse transformation of the pixelated image to transform the projective plane quadrilateral into a right angled rectangle.

Abstract: An image selector selects images for visual similarity in some aspects and visual distinction in other aspects. A user submits a query to a search engine. Images identified by the search engine as relevant to the query are clustered into clusters having at least one similar attribute. Selection mechanisms organize images selected from each cluster based on a similarity metric into image rows. A selection metric is calculated for each image row and a row to be displayed is selected based on the selection metric. In some embodiments, images representative of a subset of image rows are displayed to a user and responsive to selection of a representative image, the corresponding image row is displayed.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for determining a glyph and a font from a vector outline by applying various combinations of hash-based querying, path-descriptor matching, or anchor-point matching. For example, the disclosed systems can select a subset of candidate glyphs for a vector outline based on (i) comparing hash keys of candidate glyphs with a point-order-agnostic hash key corresponding to the vector outline and (ii) comparing a path descriptor for a primary path of the vector outline to path descriptors corresponding to candidate glyphs. By further comparing anchor points between the vector outline and the subset of candidate glyphs, the disclosed systems can select both a glyph and a font matching the vector outline.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing instance segmentation by detecting and segmenting individual objects in an image. In one aspect, a method comprises: processing an image to generate data identifying a region of the image that depicts a particular object; obtaining data defining a plurality of example object segmentations; generating a respective weight value for each of the example object segmentations; for each of a plurality of pixels in the region of the image, determining a score characterizing a likelihood that the pixel is included in the particular object depicted in the region of the image using: (i) the example object segmentations, and (ii) the weight values for the example object segmentations; and generating a segmentation of the particular object depicted in the region of the image using the scores for the pixels in the region of the image.

Abstract: A recognition system and an image augmentation and training method thereof are provided. The image augmentation and training method of a recognition system includes the following steps. A plurality of image frames are obtained, wherein each of the image frames includes an object pattern. A plurality of environmental patterns are obtained. The object pattern is separated from each of the image frames. A plurality of image parameters are set. The image frames, based on the object patterns and the environmental patterns, are augmented according to the image parameters to increase the number of the image frames. A recognition model is trained using the image frames.

Abstract: In methods and systems for graph-based panoptic segmentation of point clouds, points of a point cloud are received with a semantic label from a first category. Further, a plurality of unified cluster feature vectors from a second category are received, each being extracted from a cluster of points in the point cloud. Nodes of a constructed graph represent the unified feature vectors, and edges indicate the relationship between pairs of nodes. The edges are represented as an adjacency matrix indicating the existence or absence of an edge between pairs of nodes. A graph convolutional neural network uses the graph to predict an instance label for each node or an attribute for each edge, wherein the attribute of each edge is used for assigning the instance label to each node.

Abstract: A method of image annotation includes selecting a plurality of annotation models related to an annotation task for an image, obtaining a candidate annotation map for the image from each of the plurality of annotation models, and selecting at least one of the candidate annotation maps to be displayed via a user interface, the candidate annotation maps comprising suggested annotations for the image. The method further includes receiving user selections or modifications of at least one of the suggested annotations from the candidate annotation map and generating a final annotation map based on the user selections or modifications.

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.