Abstract: A dense captioning system and method is provided for analyzing an image to generate proposed bounding regions for a plurality of visual concepts within the image, generating a region feature for each proposed bounding region to generate a plurality of region features of the image, and determining a context feature for the image using a proposed bounding region that is a largest in size of the proposed bounding regions. For each region feature of the plurality of region features of the image, the dense captioning system and method further provides for analyzing the region feature to determine for the region feature a detection score that indicates a likelihood that the region feature comprises an actual object, and generating a caption for a visual concept in the image using the region feature and the context feature when a detection score is above a specified threshold value.

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

Abstract: A dense captioning system and method is provided for analyzing an image to generate proposed bounding regions for a plurality of visual concepts within the image, generating a region feature for each proposed bounding region to generate a plurality of region features of the image, and determining a context feature for the image using a proposed bounding region that is a largest in size of the proposed bounding regions. For each region feature of the plurality of region features of the image, the dense captioning system and method further provides for analyzing the region feature to determine for the region feature a detection score that indicates a likelihood that the region feature comprises an actual object, and generating a caption for a visual concept in the image using the region feature and the context feature when a detection score is above a specified threshold value.

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

Abstract: A dense captioning system and method is provided for processing an image to produce a feature map of the image, analyzing the feature map to generate proposed bounding boxes for a plurality of visual concepts within the image, analyzing the feature map to determine a plurality of region features of the image, and analyzing the feature map to determine a context feature for the image. For each region feature of the plurality of region features of the image, the dense captioning system further provides for analyzing the region feature to determine a detection score for the region feature, calculating a caption for a bounding box for a visual concept in the image using the region feature and the context feature, and localizing the visual concept by adjusting the bounding box around the visual concept based on the caption to generate an adjusted bounding box for the visual concept.