Abstract: Systems and techniques are described for Light Detection and Ranging (LiDAR) noise modeling using machine learning (ML). An example method can include collecting, using one or more sensors, a first set of data for a simulation environment and generating, using the first set of data, a point cloud that represents and/or describes the simulation environment. The method can further include collecting, using the one or more sensors, a second set of data for a real-world environment and generating a noise model using the second set of data and a neural network. The method can also include generating, using the noise model and the point cloud, a noisy point cloud that represents and/or describes the real-world environment.

Abstract: The system generates real-time augmented reality video for TV broadcast, cinema or video games. The system includes a monoscopic video camera including a body, a stereoscopic video camera, and a processor. The system includes sensors, including multiple non-optical sensors, which provide real-time positioning data defining the 3D position and 3D orientation of the monoscopic video camera, or enable the 3D position and 3D orientation of the monoscopic video camera to be calculated. The processor is configured to use the real-time positioning data automatically to create, recall, render or modify computer generated 3D objects. The processor is configured to determine the 3D position and orientation of the monoscopic video camera with reference to a 3D map of the real-world generated whilst the camera is being used to capture video. The processor is configured to track the scene without a requirement for an initial or prior survey of the scene.

Abstract: A device which generates a speech moving image includes a first encoder, a second encoder, a combination unit, and an image reconstruction unit. The first encoder receives a person background image in which a portion related to speech of a person that is a video part of the speech moving image of the person is covered with a mask, extracts an image feature vector from the person background image, and compresses the extracted image feature vector. The second encoder receives a speech audio signal that is an audio part of the speech moving image, extracts a voice feature vector from the speech audio signal, and compresses the extracted voice feature vector. The combination unit generates a combination vector of the compressed image feature vector and the compressed voice feature vector. The image reconstruction unit reconstructs the speech moving image of the person with the combination as an input.

Abstract: Implementations of the subject technology relate to generative scene networks (GSNs) that are able to generate realistic scenes that can be rendered from a free moving camera at any location and orientation. A GSN may be implemented using a global generator and a locally conditioned radiance field. GSNs may employ a spatial latent representation as conditioning for a grid of locally conditioned radiance fields, and may be trained using an adversarial learning framework. Inverting a GSN may allow free navigation of a generated scene conditioned on one or more observations.

Abstract: Systems and methods are disclosed for adjusting plane positions in multi-dimensional models. Disclosed is moving a plane associated with an architectural element based on a scale and a translation positional error, wherein the scaled is determined based on the architectural element, and the translation position error is based on a position of the architectural element, and reconstructing the multi-dimensional building model based on the moved plane.

Abstract: A system and method is provided for measurements of building façade elements by combining ground-level and orthogonal imagery. The measurements of the dimension of building façade elements are based on ground-level imagery that is scaled and geo-referenced using orthogonal imagery. The method continues by creating a tabular dataset of measurements for one or more architectural elements such as siding (e.g., aluminum, vinyl, wood, brick and/or paint), windows or doors. The tabular dataset can be part of an estimate report.

Abstract: Virtual spaces can be added to both 2D and 3D representations of physical spaces allowing for enhancements of those spaces. The virtual spaces can belong to business locations, such as a bakery or coffee shop, and content for those spaces can be controlled by the owner or user of that location. Virtual spaces belonging to public spaces can be provided as digital space for content providers. Valuation of the virtual space can occur based on various requests and use metrics of that virtual space and provide a marketplace for the addition of virtual content. Layers or groups of virtual spaces can be created and enabled or disabled by a user viewing the representation with the virtual space.

Abstract: The disclosed technology provides solutions for generating accurate virtual representations of real-world environments. A process of the disclosed technology can include steps for: processing image sensor data to identify one or more images of road paint; identifying a geographic location that is associated with the one or more images of road paint; and generating, within a simulated environment corresponding to the geographic location, at least one road paint mesh object that is based on the one or more images of road paint.

Abstract: The invention concerns a method implemented by computer means for visualizing at least a zone of an object in at least one interface, said method comprising the following steps: obtaining at least one image of said zone, said image comprising at least one channel, said image being a 2-dimensional or 3-dimensional image comprising pixels or voxels, a value being associated to each channel of each pixel or voxel of said image, a representation of said image being displayed in the interface, obtaining at least one annotation from a user, said annotation defining a group of selected pixels or voxels of said image, calculating a transfer function based on said selected pixels or voxels and applying said transfer function to the values of each channel of the image, updating said representation of the image in the interface, in which the colour and the transparency of the pixels or voxels of said representation are dependent on the transfer function.

Abstract: A positioning of a digitally modeled 3D object is assisted in a particular method which also includes obtaining a first digitally modeled 3D object having a 3D position in a 3D scene, rendering a projection of said first digitally modeled 3D object on a screen according to a first axis and a first viewpoint, and while modifying, upon user action, the 3D position of the first digitally modeled 3D object along the first axis, automatically scaling the first 3D object in order to keep constant the projection of the moved object on the screen.

Abstract: A shader auto-simplifying method and system include: obtaining a rendering instruction flow, extracting a target shader from the rendering instruction flow, and creating a simplifying shader differing from the target shader in code only; intercepting a current frame of a rendering instruction with a rendering initiating instruction of the target shader as a particular frame; obtaining time consumed by the simplifying shader by measuring time needed for rendering the particular frame with the simplifying shader; obtaining error(s) of the simplifying shader by measuring a pixel difference value between a rendering frame drawn by the simplifying shader and the particular frame when a rendering instruction corresponding to the particular frame is executed; and screening an optimal simplifying shader according to the time consumed by the simplifying shader and the error of the simplifying shader.

Abstract: A computing system and method to generate an avatar wearing a piece of clothing. Given a generic clothing model acquired for the avatar, the system generates a customized clothing model based on uniformly scaling the generic clothing model according to a size of the avatar, identifies one or more edges or boundaries of the clothing, deforms a clothing mesh of the scaled clothing model at the one or more edges by stretching and/or shrinking the edges, and performs a physical simulation of fitting the deformed clothing model on the avatar to generate a fitted clothing model for the avatar. User interfaces can be optionally provided to interactively adjust the scaling, deforming, and/or physical simulation of the clothing models.

Abstract: Provided are a method and a device for generating a virtual face, the method including: performing, by a device for generating the virtual face, comparison and learning on an inferred face and an existent face through deep learning after receiving a plurality of pieces of face source data and at least one piece of face background data; and generating, after receiving one piece of face background data, virtual face data by combining the face inferred from the plurality of pieces of face source data with a feature of the one piece of face background data through a model generated by comparison and learning.

Abstract: A method and system provide the ability to design a terrain surface. A triangular surface mesh representative of an existing surface is obtained and consists of triangles that are connected by vertices and edges. A drain intention is specified for the terrain surface through a geometry that is a point or line. The drain intention defines a drainage flow that influences a shape of the terrain surface. The mesh is modified using a Voronoi diagram that prevents a drain conflict between mesh triangles. A drain direction is autonomously determined a for each of the mesh triangles based on the drain intention. The determination generates a drain pattern that is used to shape the terrain surface.

Abstract: An information processing apparatus that generates an image to be displayed, based on shape data indicating a shape of an object and surface characteristics data indicating surface characteristics of the object includes an acquisition unit configured to obtain a plurality of pieces of the shape data with different resolutions and the surface characteristics data, a setting unit configured to set a display area including at least part of the object, and a generation unit configured to generate the image to be displayed, based on one of the plurality of pieces of shape data according to the display area and the surface characteristics data.

Abstract: Partial simulation of forming wrinkles on a garment is performed by receiving selection of a partial region of a three-dimensional (3D) model of a garment that includes an a first region and a second region. The outer coordinates of polygons in the periphery of the selected first region are fixed while polygons in the second region within the first region are moved to simulate wrinkles formed in a middle region between the first region and the second region based on an external force applied to the second region.

Abstract: A method of generating a map for visual localization includes specifying a virtual camera pose by using 3-dimensional (3D) model data which is based on an image of an outdoor space captured from the air; rendering the image of the outdoor space from a perspective of the virtual camera, by using the virtual camera pose and the 3D model data; and generating a feature point map by using the rendered image and the virtual camera pose.

Abstract: A system and method is provided for constructing a labeled and dimensioned multidimensional (e.g., 3D) building model from building object imagery (e.g., ground-level imagery). The method begins by retrieve building object imagery, the building object imagery collected based on directed capture with a mobile device. The method continues by constructing a scaled multi-dimensional building model, the scale based on sizing of at least one selected architectural feature. The method continues by identifying architectural elements within facades of the multi-dimensional building model. The method continues by determining dimensions of at least one of the architectural elements, the dimensions based on the scale. The method continues by determining dimensions (e.g., area) of at least one of the architectural elements. The method continues by labeling each identified architectural element with at least an identifier and by labeling at least one of the architectural elements with the determined dimensions.

Abstract: A system and method is provided for measurements of building façade elements by combining ground-level and orthogonal imagery. The measurements of the dimension of building façade elements are based on ground-level imagery that is scaled and geo-referenced using orthogonal imagery. The method continues by creating a tabular dataset of measurements for one or more architectural elements such as siding (e.g., aluminum, vinyl, wood, brick and/or paint), windows or doors. The tabular dataset can be part of an estimate report.

Abstract: Systems, apparatuses, and methods for implementing a downsampler in a single compute shader pass are disclosed. A central processing unit (CPU) issues a single-pass compute shader kernel to perform downsampling of a texture on a graphics processing unit (GPU). The GPU includes a plurality of compute units for executing thread groups of the kernel. Each thread group fetches a patch of the texture, and each individual thread downsamples four quads of texels to compute mip levels 1 and 2 independently of the other threads. For mip level 3, texel data is written back over one of the local data share (LDS) entries from which the texel data was loaded. This eliminates the need for a barrier between loads and stores for computing mip level 3. The remaining mip levels are computed in a similar fashion by the thread groups of the single-pass kernel.