Abstract: Rendering an avatar in a selected environment may include determining as inputs into an inferred shading network, an expression geometry to be represented by an avatar, head pose, and camera angle, along with a lighting representation for the selected environment. The inferred shading network may then generate a texture of a face to be utilized in rendering the avatar. The lighting representation may be obtained as lighting latent variables which are obtained from an environment autoencoder trained on environment images with various lighting conditions.

Abstract: A methods and system are provided for dynamically adjusting a camera angle in a video game is provided. One example method includes receiving a dataset of a plurality of video sequences that are associated with tagging data in metadata that identify characteristics of actions occurring in each of the plurality of video sequences. The method includes generating an artificial intelligence (AI) model from the dataset. The method includes accessing the AI model during gameplay of the video game. The accessing includes providing snapshots of video sequences generated during said gameplay of the video game. The method includes adjusting a game camera angle dynamically during the gameplay. The adjusting is responsive to adjustment instructions derived from said AI model.

Abstract: A camera is used to capture image data of representations of a physical environment. Planes and surfaces are determined from a representation. The planes and the surfaces are analyzed using relationships there between to obtain shapes and depth information for available spaces within the physical environment. Locations of the camera with respect to the physical environment are determined. The shapes and the depth information are analyzed using a trained neural network to determine items fitting the available spaces. A live camera view is overlaid with a selection from the items to provide an augmented reality (AR) view of the physical environment from an individual location of the locations. The AR view is enabled so that a user can port to a different location than the individual location by an input received to the AR view while the selection from the items remains anchored to the individual location.

Abstract: A computer system maintains structure data indicating geometrical constraints for each structure category of a plurality of structure categories. The computer system generates a virtual 3D representation of a structure based on a set of images depicting the structure. For each image in the set of images, one or more landmarks are identified. Based on the landmarks, a candidate structure category is selected. The virtual 3D representation is generated based on the geometrical constraints of the candidate structure category and the landmarks identified in the set of images.

Abstract: A method and a device for gigapixel-level light field intelligent reconstruction of a large-scale scene are provided. The method includes: obtaining a coarse three-dimensional geometric model based on a multi-view three-dimensional reconstruction system; constructing an implicit representation of the meta-deformed manifold on the coarse three-dimensional geometric model; and optimizing the implicit representation of the meta-deformed manifold to obtain the light field reconstruction in the form of free viewpoint rendering of the large-scale scene.

Abstract: A method for 3D imaging of a sample, in a vessel having a longitudinal axis orthogonal to a horizontal plane, includes capturing, by at least three cameras located at different positions around the vessel, respective 2D images of the sample. Each image comprises pixels having associated pixel values. The optical axis of a first camera is inclined or declined at a first angle relative to the horizontal plane, with the first angle being greater than or equal to zero degrees. The optical axis of a second camera is inclined or declined at a second, larger angle relative to the horizontal plane. The method also includes generating a 3D image of the sample based on the pixel values associated with the 2D image pixels, and one or more look-up tables that collectively indicate, for pixels in each image, expected paths for light traversing the vessel and the sample.

Abstract: Disclosed is surface guided cropping in volume rendering of 3D volumetric data from intervening anatomical structures in the patient's body. A digital 3D representation expressing the topography of a first anatomical structure is used to define a clipping surface or a bounding volume which then is used in the volume rendering to exclude data from an intervening structure when generating a 2D projection of the first anatomical structure.

Abstract: A control apparatus for work machines includes a position designation reception unit configured to identify designation of a position with respect to a state image displayed on a display panel and a screen control unit configured to perform screen control according to an image displayed at the position that has been identified, among partial images constituting parts of the state image.

Abstract: Various embodiments are generally directed to techniques of overlaying a virtual object on a physical object in augmented reality (AR). A computing device may receive one or more images of the physical object, perform analysis on the images (such as image segmentation) to generate a digital outline, and determine a position and a scale of the physical object based at least in part on the digital outline. The computing device may configure (e.g., rotate, scale) a 3D model of the physical object to match the determined position and scale of the physical object. The computing device may place or overlay a 3D virtual object on the physical object in AR based on a predefined location relation between the 3D virtual object and the 3D model of the physical object, and further, generate a composite view of the placement or overlay.

Abstract: A display device including a controller and a display panel. The controller receives original image data and output a display image signal. The display panel receives the display image signal and displays a display image corresponding to the display image signal. The controller includes an image shift controller and a memory. The image shift controller generates shifted image data by modulating the original image data to shift the display image sequentially along a preset shift path on the display panel. The memory stores a shift path value indicating a distance by which the display image has been shifted on the preset shift path. The image shift controller generates the display image signal by processing the shifted image data. When the display device is powered on, the image shift controller generates shifted image data corresponding to a shift path value stored in the memory.

Abstract: Novel tools and techniques are provided for implementing augmented reality (AR)-based assistance within a work environment. In various embodiments, a computing system might receive, from a camera having a field of view of a work environment, first images of at least part of the work environment, the first images overlapping with a field of view of a user wearing an AR headset; might analyze the received first images to identify objects; might query a database(s) to determine a task associated with a first object(s) among the identified objects; might generate an image overlay providing at least one of graphical icon-based, text-based, image-based, and/or highlighting-based instruction(s) each indicative of instructions presented to the user to implement the task associated with the first object(s); and might display, to the user's eyes through the AR headset, the generated first image overlay that overlaps with the field of view of the user's eyes.

Abstract: Systems and methods are disclosed for enhancing and developing a damage scene virtual reality (VR) visualization. Annotated immersive multimedia image(s) may be received from a first user, where the annotated immersive multimedia image(s) can be associated with a damage scene. A VR visualization of the annotated immersive multimedia image(s) may be rendered using a VR device associated with a second user. The VR visualization may be used to determine a damage amount, where the damage amount is determined from one or more damaged items identifiable in the annotated immersive multimedia image(s).

Abstract: A method for displaying guidance information during an image-guided medical procedure includes receiving, by one or more hardware processors, data from a tracking system associated with an elongate device comprising a flexible body, calculating a bend radius along a length of the flexible body, determining supplemental guidance information that includes an indication of whether the bend radius is less than a minimum allowable bend radius, augmenting images with the supplemental guidance information to produce augmented images by applying a scheme in which a portion of a graphical representation of the flexible body having a radius less than the minimum is augmented with a visual property, displaying the augmented images on a display device at a console, determining a direction to steer the flexible body to increase the bend radius, and displaying actuation information on the display device. The actuation information includes a directional indicator to display the direction.

Abstract: Disclosed is a system and associated methods for generating a mutable tree to efficiently access data within a three-dimensional (“3D”) environment. The system generates the mutable tree with a root node defined at a root node position, a first branch with nodes for each of a first set of subdivided regions that are a first distance from the root node position, and a second branch with nodes for each of a second set of subdivided regions that are a second distance from the root node position. The system sorts the mutable tree in response to a request to access data from a first position within the 3D environment so that the first node in the first branch is the first subtree node that is closest to the first position, and the first node in the second branch is the second subtree node that is closest to the first position.

Abstract: Systems and methods are provided for allowing stakeholders in the video production industry to perform location scouting and other film-related processes via virtual reality devices. Imagery can be collected from a plurality of different physical locations. Subsequently, directors, location managers, directors of photography, production designers, and others, can virtually view each of the different physical locations using a virtual reality device. Attributes regarding the physical locations that are relative to filming can also be collected and provided to the users.

Abstract: Various implementations of the present application set forth a method comprising generating, three-dimensional data and two-dimensional data representing a physical space that includes a real-world asset, generating an adaptable three-dimensional (3D) representation of the physical space based on the two-dimensional and three-dimensional data, where the adaptable 3D representation includes a plurality of coordinates representing different positions in 3D coordinate space corresponding to the physical space, transforming the adaptable 3D representation into geometry data comprising a set of vertices, faces comprising edges between pairs of vertices, and texture data, transmitting the geometry data to a remote device, wherein the remote device, constructs, based on the geometry data, the adaptable 3D representation of the physical space for display at a location of the remote device in a remote environment, and modifies, based on an input, at least one of a dimension or a position of the adaptable 3D representa

Abstract: A computer-implemented method for generating a mask for a light source in a virtual scene includes determining a bounding box for the scene based on a frustum of a virtual camera and generating a path-traced image of the scene within the bounding box. Light paths emitted by the camera and exiting at the light source are stored, and objects poorly sampled by the light source are removed from the scene. An initial mask for the light source is generated from the density of light paths exiting at that position on the light source. The initial mask is refined by averaging in the light path density at each point on the light source for subsequent images.

Abstract: The disclosure relates to a 3D reconstruction method based on an on-site edge-cloud collaboration for a cultivated land. An edge-cloud collaborative computing architecture is used, such that the edge computing device performs advance calculations after image data is acquired. The edge computing device measures performances of itself and a cloud data center, and arranges and deploys multiple 3D reconstruction containers in the cloud data center for the 3D reconstruction. Multiple reconstruction containers in the cloud data center perform reconstruction tasks in parallel to quickly obtain 3D reconstruction results, and provide them to the edge computing device for retrieval and download. This method is mainly oriented to agricultural project monitoring scenes, to reduce reconstruction time and a data transmission amount of 3D models, in order to improve a response speed and a quality of 3D reconstruction results, for large-scale on-site monitoring, acceptance, and review purposes of agricultural projects.

Abstract: Computer animation involving pose disambiguation is disclosed. Two or more source segmentation masks are generated from corresponding contemporaneous video images of a character from different points of view. A three-dimensional model of an animation character corresponding to the character in the two or more contemporaneous video images is generated. Two or more different target segmentation masks corresponding to different views of the animation character corresponding to the character in the two or more video images. Each target segmentation mask is compared to a corresponding source segmentation mask from the comparison it is determined whether a pose of the three-dimensional model corresponds to a pose of the character in the video images. The model is used to generate a frame of animation of the animated character when the pose of model corresponds to the pose of the character in the video images.

Abstract: Light contribution information can be determined and cached for use in rendering image frames for a scene. In at least one embodiment, a spatial hash data structure can be used to split the scene into regions, such as octahedral voxels. Using cast light rays, an average light contribution can be computed for each individual voxel. Those light values can then be used to build a cumulative distribution function for each voxel that can be used to select which lights to sample for a given frame during rendering. The sampling for a region or voxel can be based at least in part upon the number of contributing lights for that region, as well as the relative contributions of those lights. Such an approach can be very bandwidth and cache efficient, while providing high image quality.