Abstract: The present disclosure describes methods and devices for generating a vector line drawing. A vector line drawing network may include a machine learning-based model that is trained to convert a raster image to a vector line drawing directly. The vector line drawing network may be trained end-to-end, using supervised learning, where only raster images are used as training data. A vector line drawing is generated stroke by stroke, over a series of time steps. In each time step, a dynamic drawing window is moved and scaled across the input raster image to sample a patch of the raster image, and a drawing stroke is predicted to draw a stroke in a corresponding patch in the canvas for the vector line drawing. The image patches are pasted in the canvas to assemble a final vector line drawing that corresponds to the input raster image.

Abstract: An apparatus, method, and computer-readable medium for rendering a 3D image using a neural network for implicit representation of the image. UV coordinates of a texture map corresponding to a point on the image are calculated using a texture neural network. The image is rendered by applying a color value located at the UV coordinates of the texture map to the point.

Abstract: The present disclosure describes methods and devices for generating a vector line drawing. A vector line drawing network may include a machine learning-based model that is trained to convert a raster image to a vector line drawing directly. The vector line drawing network may be trained end-to-end, using supervised learning, where only raster images are used as training data. A vector line drawing is generated stroke by stroke, over a series of time steps. In each time step, a dynamic drawing window is moved and scaled across the input raster image to sample a patch of the raster image, and a drawing stroke is predicted to draw a stroke in a corresponding patch in the canvas for the vector line drawing. The image patches are pasted in the canvas to assemble a final vector line drawing that corresponds to the input raster image.

Abstract: Methods and systems for rendering a virtual overlay in an extended reality (XR) environment are described. Image data is received, representing a current field of view (FOV) in the XR environment. A spatial boundary is defined in the current FOV, based on user input. An image label representing a region of interest (ROI) within the defined spatial boundary, and one or more object labels representing one or more objects within the defined spatial boundary are generated. At least one relevant virtual object is identified. The virtual object is relevant to a semantic meaning of the ROI based on the image label and/or the one or more object labels. Identification of the at least one relevant virtual object is outputted, to be renderable in the XR environment.

Abstract: Devices, methods, systems, and media are described for providing an extended screen distributed user interface in an augmented reality environment. GUI layout information for laying out a conventional 2D GUI is processed in order to generate an extended screen DUI for display partially on a 2D display device and partially on one or more virtual screens of an AR environment viewed using an AR display device, such as a head mounted display. GUI elements are laid out in the DUI based on a primary modality of the GUI element (input or output), and/or based on spatial dependencies between GUI elements encoded in the GUI layout information. Methods for switching focus between two software application instances displayed in the DUI are also disclosed.

Abstract: Methods and systems for rendering a virtual overlay in an extended reality (XR) environment are described. Image data is received, representing a current field of view (FOV) in the XR environment. A spatial boundary is defined in the current FOV, based on user input. An image label representing a region of interest (ROI) within the defined spatial boundary, and one or more object labels representing one or more objects within the defined spatial boundary are generated. At least one relevant virtual object is identified. The virtual object is relevant to a semantic meaning of the ROI based on the image label and/or the one or more object labels. Identification of the at least one relevant virtual object is outputted, to be renderable in the XR environment.

Abstract: A sketch-based image retrieval method, device and system, to improve accuracy of image searching from a scene sketch image. For example, the image retrieval method, device and system can be used to retrieve a target scene image from a collection of stored images in a storage (i.e., an image collection). The image retrieval method includes: segmenting the scene sketch image using an image segmentation module into semantic object-level instances, and fine-grained features are obtained for each object instance, generating an attribute graph which integrates the fine-grained features for each semantic object instance detected from the query scene sketch image, generating a feature graph by using a graph encoder module from the attribute graph, and computing a similarity or distance between the feature graphs of the query scene sketch image and the scene images in the image collection by a graph matching module and the most similar scene images are returned.

Abstract: Devices, methods, systems, and media are described for selecting virtual objects for user interaction in an extended reality environment. Distant virtual objects are brought closer to the user within a virtual 3D space to situate the selected virtual object in virtual proximity to the user's hand for direct manipulation. A virtual object is selected by the user based on movements of the user's hand and/or head that are correlated or associated with an intent to select a specific virtual object within the virtual 3D space. As the user's hand moves in a way that is consistent with this intent, the virtual object is brought closer to the user's hand within the virtual 3D space. To predict the user's intent, hand and head trajectory data may be compared to a library of kinematic trajectory templates to identify a best-matched trajectory template.

Abstract: A sketch-based image retrieval method, device and system, to improve accuracy of image searching from a scene sketch image. For example, the image retrieval method, device and system can be used to retrieve a target scene image from a collection of stored images in a storage (i.e., an image collection). The image retrieval method includes: segmenting the scene sketch image using an image segmentation module into semantic object-level instances, and fine-grained features are obtained for each object instance, generating an attribute graph which integrates the fine-grained features for each semantic object instance detected from the query scene sketch image, generating a feature graph by using a graph encoder module from the attribute graph, and computing a similarity or distance between the feature graphs of the query scene sketch image and the scene images in the image collection by a graph matching module and the most similar scene images are returned.