Abstract: Disclosed herein are systems and method for classifying objects in an image using a color-based neural network. A method may include: training a neural network to classify an object in a given image into a color class from a set of color classes; determining, from the set of color classes, a subset of color classes that are anticipated to be in a received input image based on image metadata; generating a matched mask input indicating the subset; inputting both the input image and the matched mask input into the neural network, wherein the neural network is configured to: determine a first semantic embedding of the input image and the matched mask input; outputting a color class associated with a second semantic embedding with a least amount of distance to the first semantic embedding from a plurality of semantic embeddings.

Abstract: Disclosed herein are systems and method for determining environment dimensions based on environment pose. In one aspect, the method may include training, with a dataset including a plurality of images featuring an environment and labelled landmarks in the environment, a neural network to identify a pose of an environment. The method may comprise receiving an input image depicting the environment, generating an input tensor based on the input image, and inputting the input tensor into the neural network, which may be configured to generate an output tensor including a position of each identified landmark, a confidence level associated with each position, and a pose confidence score. The method may include calculating a homography matrix between each position in the output tensor along a camera plane and a corresponding position in an environment plane in order to output an image that visually connects each landmark along the environment plane.

Abstract: Disclosed herein are systems and method for detecting small objects in an image using a neural network (NN). An exemplary method may include: receiving a first NN that is trained on a dataset including a plurality of images depicting various objects; identifying a first structure of the first NN, the first structure indicative of each layer and layer size in the first NN; determining, based on the first structure, whether the first NN can classify an object less than a threshold size in an input image; in response to determining that the first NN cannot classify the object, identifying a subset of detection layers in the first NN; generating and training a second NN that has a second structure in which the subset of detection layers are replaced by at least one layer not in the subset; and receiving, from the second NN, a classification of the object.

Abstract: Disclosed herein are systems and method for classifying objects in an image using a color-based neural network. A method may include: training a neural network to classify an object in a given image into a color class from a set of color classes; determining, from the set of color classes, a subset of color classes that are anticipated to be in a received input image based on image metadata; generating a matched mask input indicating the subset; inputting both the input image and the matched mask input into the neural network, wherein the neural network is configured to: determine a first semantic embedding of the input image and the matched mask input; outputting a color class associated with a second semantic embedding with a least amount of distance to the first semantic embedding from a plurality of semantic embeddings.

Abstract: Disclosed herein are systems and method for classifying objects in an image using a color-based machine learning classifier. A method may include: training, with a dataset including a plurality of images, a machine learning classifier to classify an object in a given image into a color class from a set of color classes of a first size; receiving an input image depicting at least one object belonging to the set of color classes; determining a subset of color classes that are anticipated to be in the input image based on metadata of the input image; generating a matched mask input indicating the subset set of color classes in the input image, wherein the subset of color classes is of a second size that is smaller than the first size; and inputting both the input image and the matched mask input into the machine learning classifier.

Abstract: Disclosed herein are systems and method for determining environment dimensions based on environment pose. In one aspect, the method may include training, with a dataset including a plurality of images featuring an environment and labelled landmarks in the environment, a neural network to identify a pose of an environment. The method may comprise receiving an input image depicting the environment, generating an input tensor based on the input image, and inputting the input tensor into the neural network, which may be configured to generate an output tensor including a position of each identified landmark, a confidence level associated with each position, and a pose confidence score. The method may include calculating a homography matrix between each position in the output tensor along a camera plane and a corresponding position in an environment plane in order to output an image that visually connects each landmark along the environment plane.

Abstract: Disclosed herein are systems and method for detecting small objects in an image using a neural network (NN). An exemplary method may include: receiving a first NN that is trained on a dataset including a plurality of images depicting various objects; identifying a first structure of the first NN, the first structure indicative of each layer and layer size in the first NN; determining, based on the first structure, whether the first NN can classify an object less than a threshold size in an input image; in response to determining that the first NN cannot classify the object, identifying a subset of detection layers in the first NN; generating and training a second NN that has a second structure in which the subset of detection layers are replaced by at least one layer not in the subset; and receiving, from the second NN, a classification of the object.

Abstract: Disclosed herein are systems and method for classifying objects in an image using a neural network. In one exemplary aspect, the techniques described herein relate to a method including: training, with a dataset including a plurality of images, a neural network to identify objects of a set of classes, wherein the neural network includes: a shared convolutional backbone with feature extraction layers, and a plurality of heads with fully connected layers, wherein there is a respective distinct head for each of the set of classes; receiving an input image depicting at least one object from the set of classes; inputting the input image into the neural network, wherein the neural network is configured to classify the at least one object into at least one class of the set of classes; and outputting the at least one class.

Abstract: Disclosed herein are systems and method for determining environment dimensions based on landmark detection, the method including: training, with a dataset including a plurality of images featuring an environment and labelled landmarks in the environment, a neural network to identify the labelled landmarks in an arbitrary image of the environment; receiving an input image depicting the environment; generating an input tensor based on the received input image; inputting the input tensor into the neural network, wherein the neural network is configured to generate an output tensor including a position of each identified landmark and a visibility score associated with each position; calculating a homography matrix between each position in the output tensor along a camera plane and a corresponding position in an environment plane, based on a pre-built model of the environment; and outputting an image that visually connects each landmark along the environment plane based on the homography matrix.