Abstract: Artificial neural networks (ANN) may be trained to output estimated floor plans from 3D spaces that would be challenging or impossible for existing techniques to estimate. In embodiments, an ANN may be trained using a supervised approach where top-down views of 3D meshes or point clouds are provided to the ANN as input, with ground truth floor plans provided as output for comparison. A suitably large training set may be used to fully train the ANN on challenging scenarios such as open loop scans and/or unusual geometries. The trained ANN may then be used to accurately estimate floor plans for such 3D spaces. Other embodiments are described.

Abstract: Embodiments include methods for synchronizing an augmented reality (AR) object placed in a 3D mesh onto a video feed. A computing device may first receive a video feed including a sequence of frames of a scene and depth or motion data captured by a camera. The computing device may generate a three-dimensional (3D) mesh based on the depth or motion data. The computing device may texture the 3D mesh to create a 3D model. Upon performing object recognition, the computing device may identify anchor points in the 3D model and anchor points in the video feed. The anchor points are used to calculate the location of the AR object.

Abstract: Embodiments include systems and methods for generating a 3D mesh on a server from one or more images and other data of a physical space that is captured by a device. The device may capture images with RGB data that includes depth information for pixels of the one or more images. The server may then create or update a 3D mesh based upon the images and depth information. The server may also use received images to provide textures to the various surfaces of the 3D mesh. The server may then cause a 3D model of the physical space to be viewed by another device, either by transmitting the 3D mesh for the other device used to construct a 3D model, or by rendering the 3D model directly on the other device. Other embodiments may be described and/or claimed.

Abstract: Embodiments include systems and methods for creation of a 3D mesh from a video stream or a sequence of frames. A sparse point cloud is first created from the video stream, which is then densified per frame by comparison with spatially proximate frames. A 3D mesh is then created from the densified depth maps, and the mesh is textured by projecting the images from the video stream or sequence of frames onto the mesh. Metric scale of the depth maps may be estimated where direct measurements are not able to be measured or calculated using a machine learning depth estimation network.

Abstract: Embodiments of devices and techniques of obtaining a three dimensional (3D) representation of an area are disclosed. In one embodiment, a two dimensional (2D) frame is obtained of an array of pixels of the area. Also, a depth frame of the area is obtained. The depth frame includes an array of depth estimation values. Each of the depth estimation values in the array of depth estimation values corresponds to one or more corresponding pixels in the array of pixels. Furthermore, an array of confidence scores is generated. Each confidence score in the array of confidence scores corresponds to one or more corresponding depth estimation values in the array of depth estimation values. Each of the confidence scores in the array of confidence scores indicates a confidence level that the one or more corresponding depth estimation values in the array of depth estimation values is accurate.

Abstract: Frames for texturing a 3D mesh may be selected to minimize the number of frames required to completely texture the mesh, thus reducing the overhead of texturing. Keyframes are selected from a video stream on the basis of amount of overlap from previously selected keyframes, with the amount of overlap held below a predetermined threshold. The 3D mesh may also be refined and corrected to ensure a higher quality mesh application, including color correction of the selected keyframes. Other embodiments are described.

Abstract: A mesh model of a 3D space is provided with improved accuracy by refining the locations of edges of objects in the space. The mesh model includes vertices which define surfaces of triangles. Triangles are identified which have two vertices in one plane and another, outlier vertex in another, adjacent plane. A line is fitted to the outlier vertices to define an edge of an object, and the outlier vertices are moved to the line, referred to as a mesh-based line. Texture data from images of the space can be used to further refine the edge. In one approach, gradients in grayscale pixels which correspond the vertices of the mesh-based line are used to define a grayscale-based line. The two line definitions can be combined or otherwise used to provide a final definition of the edge. The object can be measured based on the length and position of the edge.

Abstract: A platform for synchronizing augmented reality (AR) views and information between two or more network connected devices is disclosed. A first device captures a video stream and associated essential meta-data, embeds the essential meta-data into the video stream and transmits it to a second device. The second device receives the video stream, extracts the essential meta-data, inserts one or more AR objects into the video stream with reference to the enhanced meta-data, and transmits to the first device the AR objects and reference to the essential meta-data. The first device renders the one or more AR objects into the video stream, using the essential meta-data references to locate the AR objects in each video stream frame. The second device may also determine and transmit a modified video stream to the first device.

Abstract: Embodiments include systems and methods for generating 2D and 3D layouts from a physical 3D space captured by a capturing device, the layouts having an identical scale to the physical 3D space, and estimating measurements of the physical 3D space from the layouts. The capturing device captures a point cloud or 3D mesh of the physical 3D space, from which one or more planes are identified. These one or more planes can then be used to create a virtual 3D reconstruction of the captured 3D space. In other embodiments, one plane may be identified as a floor plane, and features from the point cloud or 3D mesh that are above the floor plane may be projected onto the floor plane to create a top down view and 2D layout. Other embodiments are described.

Abstract: Embodiments include systems and methods for remotely measuring distances in an environment captured by a device. A device captures a video stream of a device along with AR data that may include camera pose information and/or depth information, and transmits the video stream and AR data to a remote device. The environment may be analyzed to identify objects such as lines, edges, curves, shapes, anchors/corners, products, e.g., appliances, and other things of interest. Some or all of objects may be identified to the remote device to facilitate selecting an object or region of interest. Selected points for an object may be more precisely located by snapping them to corresponding anchor points for an object. Using anchor points facilitates more precise identification and/or measurement of an aspect of an object, such as one of its dimensions, the volume of a space, or performing other actions such as replacement of an object.

Abstract: Embodiments include systems and methods for offloading media service operation to one or graphical processing unit (GPUs). In some embodiments, a computer system, includes a first computer device to provide a media service that involves implementing media service operations and transmit a first media service request for a first media service operation of the media service operations. In addition, the computer system includes a second computer device that includes one or more GPUs. The second computer device is to implement the first media service operation with the one or more GPUs in response to the first computer device transmitting the first media service request for the first media service operation.

Abstract: A mesh model of a 3D space is modified based on semantic segmentation data to more accurately represent boundaries of an object in the 3D space. In one aspect, semantic segmentation images define one or more boundaries of the object. The semantic segmentation images are projected to a 3D mesh representation of the 3D space, and the 3D mesh representation is updated based on the one or more boundaries in the projected semantic segmentation image. In another aspect, the 3D mesh representation is updated based on one or more boundaries defined by the semantic segmentation images as applied to a point cloud of the 3D space.

Abstract: Methods and systems for automatic detection and recognition of visual tags on equipment are disclosed. The make and model of an object such as an appliance or consumer device is recognized from an image or video using object detection. This make and model information may then be used to direct a user to locate an equipment information tag that includes model and serial number information. Object recognition and optical character recognition can then be employed to extract the model and serial number from the tag, along with any other relevant information. The extracted information may then be used to locate service and/or operation information.

Abstract: The present invention relates to methods for searching for two-dimensional or three-dimensional objects. More particularly, the present invention relates to searching for two-dimensional or three-dimensional objects in a collection by using a multi-modal query of image and/or tag data. Aspects and/or embodiments seek to provide a method of searching for digital objects using any combination of images, three-dimensional shapes and text by embedding the vector representations for these multiple modes in the same space. Aspects and/or embodiments can be easily extensible to any other type of modality, making it more general.

Abstract: A sequence of frames including color and depth data is processed to identify key frames while minimizing redundancy. A sparse 3D point cloud is obtained for each frame and represented by a set of voxels. Each voxel has associated data indicating, e.g., a depth and a camera viewing angle. When a new frame is processed, a new sparse 3D point cloud is obtained. For points which are not encompassed by the existing voxels, new voxels are created. For points which are encompassed by the existing voxels, a comparison determines whether the depth data of the new frame is more accurate than the existing depth data. A frame is selected as a key frame based on factors such as a number of new voxels which are created, a number of existing voxels for which the depth data is updated, and accuracy scores.

Abstract: Methods for placement of location-persistent 3D objects or annotations in an augmented reality scene are disclosed. By capturing location data along with device spatial orientation and the placement of 3D objects or annotations, the augmented reality scene can be recreated and manipulated. Placed 3D objects or annotations can reappear in a subsequent capture by the same or a different device when brought back to the location of the initial capture and placement of objects. Still further, the placed 3D objects or annotations may be supplemented with additional objects or annotations, or the placed objects or annotations may be removed or modified.

Abstract: A mesh model of a 3D space is provided with improved accuracy by refining the locations of edges of objects in the space. The mesh model includes vertices which define surfaces of triangles. Triangles are identified which have two vertices in one plane and another, outlier vertex in another, adjacent plane. A line is fitted to the outlier vertices to define an edge of an object, and the outlier vertices are moved to the line, referred to as a mesh-based line. Texture data from images of the space can be used to further refine the edge. In one approach, gradients in grayscale pixels which correspond the vertices of the mesh-based line are used to define a grayscale-based line. The two line definitions can be combined or otherwise used to provide a final definition of the edge. The object can be measured based on the length and position of the edge.

Abstract: A neural network architecture is provided for reconstructing, in real-time, a 3D scene with additional attributes such as color and segmentation, from a stream of camera-tracked RGB images. The neural network can include a number of modules which process image data in sequence. In an example implementation, the processing can include capturing frames of color data, selecting key frames, processing a set of key frames to obtain partial 3D scene data, including a mesh model and associated voxels, fusing the partial 3D scene data into existing scene data, and extracting a 3D colored and segmented mesh from the 3D scene data.

Abstract: A sequence of frames including color and depth data is processed to identify key frames while minimizing redundancy. A sparse 3D point cloud is obtained for each frame and represented by a set of voxels. Each voxel has associated data indicating, e.g., a depth and a camera viewing angle. When a new frame is processed, a new sparse 3D point cloud is obtained. For points which are not encompassed by the existing voxels, new voxels are created. For points which are encompassed by the existing voxels, a comparison determines whether the depth data of the new frame is more accurate than the existing depth data. A frame is selected as a key frame based on factors such as a number of new voxels which are created, a number of existing voxels for which the depth data is updated, and accuracy scores.

Abstract: Embodiments herein may relate to a technique to be performed by surface partition logic. The technique may include identifying a first mesh portion that is related to a first plane of a three-dimensional (3D) space, and a second mesh portion that is related to a second plane of the 3D space. The technique may include identifying, based on a linear representation of a border between the first mesh portion and the second mesh portion, an element of the first mesh portion that at least partially overlaps the second mesh portion. The technique may further include altering the element of the first mesh portion to reduce the amount that the element overlaps the second mesh portion. Other embodiments may be described and/or claimed.