Abstract: Procedural generation of simulated environments is described herein. In an example, a computing device can access road network data associated with a real environment. Further, the computing device can generate, based on the sensor data, a road mesh associated with the real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment and can procedurally render at least one object into the simulated environment based at least in part on at least one rule. The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous vehicle for at least one of navigating, planning, or decision making.

Abstract: Techniques for delivering virtualized content are disclosed. In some embodiments, source content is virtualized by mapping elements in the source content to existing database objects, and a specification of the virtualized version of the source content is provided to an output device in response to a request from the output device for the source content. In some such cases, a format of the source content, a format of the virtualized version of the source content, and a native format in which the virtualized version of the source content is rendered at the output device all comprise different formats.

Abstract: In one embodiment, a method includes accessing a first image and a second image, where at least part of the first image overlaps with at least part of the second image. The first and second images are divided into portions associated with a first set of grid points, where each grid point in the first set corresponds to a portion of the first image or the second image. Differences in the region of overlap between the first and second images are determined. One or more grid points in the first set and the corresponding portions of the first image or the second image are moved relative to one or more other grid points in the first set based on the determined differences.

Abstract: Disclosed herein are methods and systems for combining, in real time, images of one or more real life objects that have been captured in real time based on a comprehensive characteristic-based mechanism with a virtual environment to provide an augmented reality environment to a user or user device. In particular, the captured images are provided to a remote server that is capable of performing the processing steps to create a plurality of integrated images. The plurality of integrated images can then be provided to the user device. Advantageously, the remote server can create the plurality of integrated images much more quickly due to the server's enhanced computing power and storage space as compared to the user device. The remote server may be an intermediate server between multiple user devices. The comprehensive content matching mechanism is based on a plurality of factors comprising advertisement content, user preference information, and context information.

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media which retarget 2D screencast video tutorials into an active VR host application. VR-embedded widgets can render on top of a VR host application environment while the VR host application is active. Thus, VR-embedded widgets can provide various interactive tutorial interfaces directly inside the environment of the VR host application. For example, VR-embedded widgets can present external video content, related information, and corresponding interfaces directly in a VR painting environment, so a user can simultaneously access external video (e.g., screencast video tutorials) and a VR painting. Possible VR-embedded widgets include a VR-embedded video player overlay widget, a perspective thumbnail overlay widget (e.g., a user-view thumbnail overlay, an instructor-view thumbnail overlay, etc.), an awareness overlay widget, a tutorial steps overlay widget, and/or a controller overlay widget, among others.

Abstract: Techniques are disclosed relating to low-level instruction storage in a processing unit. In some embodiments, a graphics unit includes execution circuitry, decode circuitry, hazard circuitry, and caching circuitry. In some embodiments the execution circuitry is configured to execute clauses of graphics instructions. In some embodiments, the decode circuitry is configured to receive graphics instructions and a clause identifier for each received graphics instruction and to decode the received graphics instructions. In some embodiments, the caching circuitry includes a plurality of entries each configured to store a set of decoded instructions in the same clause. A given clause may be fetched and executed multiple times, e.g., for different SIMD groups, while stored in the caching circuitry.

Abstract: Methods and devices providing augmented reality (AR) content. The method includes monitoring viewing activity of a user with respect to one or more of objects, determining, based on the viewing activity of the user, a viewing time of AR content associated with a next appearing object from among the one or more of objects, determining a size of the AR content associated with the next appearing object based on the viewing activity of the user, and displaying the AR content of the next appearing object based on the size of the AR content.

Abstract: There is provided an image processing apparatus including an image processing unit which combines a virtual object with a captured image. The image processing unit determines the virtual object based on a state or a type of an object shown in the captured image.

Abstract: Techniques are disclosed for analyzing a scene depicted in an input stream of video frames captured by a video camera. The techniques include evaluating sequence pairs representing segments of object trajectories. Assuming the objects interact, each of the sequences of the sequence pair may be mapped to a sequence cluster of an adaptive resonance theory (ART) network. A rareness value for the pair of sequence clusters may be determined based on learned joint probabilities of sequence cluster pairs. A statistical anomaly model, which may be specific to an interaction type or general to a plurality of interaction types, is used to determine an anomaly temperature, and alerts are issued based at least on the anomaly temperature. In addition, the ART network and the statistical anomaly model are updated based on the current interaction.

Abstract: The present invention relates to a device, a method and a program for generating a multidimensional reaction-type image, and a method and a program for reproducing a multidimensional reaction-type image. The method for generating a multidimensional reaction-type image, according to one embodiment of the present invention, comprises the steps of: a computer acquiring a plurality of multidimensional image frames forming a base image (S200); generating a three-dimensional cell combination on the basis of a configuration region in the base image and a particular frame density in the configuration region (S600); and matching image frames corresponding to respective detailed cells (S1000).

Abstract: An apparatus is configured to render graphics content to reduce latency of the graphics content. The apparatus includes a display configured to present graphics content including a first portion corresponding to an area of interest and further including a second portion. The apparatus further includes a fovea estimation engine configured to generate an indication of the area of interest based on scene information related to the graphics content. The apparatus further includes a rendering engine responsive to the fovea estimation engine. The rendering engine is configured to perform a comparison of a first result of an evaluation metric on part of the area of interest with a second result of the evaluation metric with another part of the area of interest. The rendering engine is further configured to render the graphics content using predictive adjustment to reduce latency based on the comparison.

Abstract: The present disclosure is directed to supervising displayed content. In particular, the methods and systems of the present disclosure may: generate data representing a plurality of images of interfaces displayed by a computing device configured to supervise content displayed to a user; determine, based at least in part on one or more machine learning (ML) models and the data representing the plurality of images, whether the interfaces displayed by the computing device include content of a type designated by a content supervisor of the user for identification; and generate data representing a graphical user interface (GUI) for presentation to the content supervisor, the GUI indicating whether the interfaces displayed by the computing device include content of the type designated for identification.

Abstract: In accordance with an embodiment, described herein is a system and method for providing a live lyrics overlay in a social messaging environment. The system can utilize advances in three-dimensional mapping technology that allow social messaging services, to offer real time video lenses or overlays to their users, and extends this three-dimensional mapping technology to support for lyrics. During creation of a video with lyrics lens overlay, the lyrics corresponding to a selected song are retrieved from a lyrics source, and are displayed within the video. For example, with the lyrics lens, a user can record an image of themselves on live video, singing along to a song clip, with the lyrics of the song displayed as if they appear to be coming from their mouths. The created live lyrics content can also be shared with other users of a social messaging environment.

Abstract: Techniques are disclosed relating to synchronizing access to pixel resources. Examples of pixel resources include color attachments, a stencil buffer, and a depth buffer. In some embodiments, hardware registers are used to track status of assigned pixel resources and pixel wait and pixel release instruction are used to synchronize access to the pixel resources. In some embodiments, other accesses to the pixel resources may occur out of program order. Relative to tracking and ordering pass groups, this weak ordering and explicit synchronization may improve performance and reduce power consumption. Disclosed techniques may also facilitate coordination between fragment rendering threads and auxiliary mid-render compute tasks.

Abstract: A disaggregated processor package can be configured to accept interchangeable chiplets. Interchangeability is enabled by specifying a standard physical interconnect for chiplets that can enable the chiplet to interface with a fabric or bridge interconnect. Chiplets from different IP designers can conform to the common interconnect, enabling such chiplets to be interchangeable during assembly. The fabric and bridge interconnects logic on the chiplet can then be configured to confirm with the actual interconnect layout of the on-board logic of the chiplet. Additionally, data from chiplets can be transmitted across an inter-chiplet fabric using encapsulation, such that the actual data being transferred is opaque to the fabric, further enable interchangeability of the individual chiplets. With such an interchangeable design, higher or lower density memory can be inserted into memory chiplet slots, while compute or graphics chiplets with a higher or lower core count can be inserted into logic chiplet slots.

Abstract: In a ray tracer, to prevent any long-running query from hanging the graphics processing unit, a traversal coprocessor provides a preemption mechanism that will allow rays to stop processing or time out early. The example non-limiting implementations described herein provide such a preemption mechanism, including a forward progress guarantee, and additional programmable timeout options that can be time or cycle based. Those programmable options provide a means for quality of service timing guarantees for applications such as virtual reality (VR) that have strict timing requirements.

Abstract: Systems and methods for receiving audio data; identifying one or more graphical interface elements that correspond to the audio data; generating a display of the identified one or more graphical interface elements, wherein a first portion of the one or more graphical interface elements is persistently displayed, and wherein a second portion of the one or more graphical interface elements is temporarily displayed for a predetermined period of time together with the first portion of the one or more graphical interface elements; and at expiry of the predetermined period of time, ceasing display of the second portion while maintaining display of the first portion.

Abstract: In one embodiment, a method includes receiving image data corresponding to an external environment of a user. The image data is captured at a first time and comprises a body part of the user. The method also includes receiving a first tracking data generated based on measurements made at the first time by at least one motion sensor associated with the body part; generating, based at least on the image data, a model representation associated with the body part; receiving a second tracking data generated based on measurements made at a second time by the at least one motion sensor associated with the body part; determining a deformation of the model representation associated with the body part based on the first tracking data and the second tracking data; and displaying the deformation of the model representation associated with the body part of the user.

Abstract: An intelligent VR (IVR) system receives a VR content item and automatically augments the VR content item with additional or ancillary content. The intelligent VR system initially identifies within the VR content item, valid textual and non-textual objects that have ancillary content associated therewith. The IVR system provides hotspots that signify presence of the ancillary content for the valid objects during a display of the VR content item. The hotspots and the ancillary content are positioned for display corresponding to pixel blocks including the valid objects. The ancillary content is selected for the hotspots based on a spatial and temporal extent of display of the valid objects. The hotspots with respective ancillary content items and the VR content item are streamed to a head mounted display (HMD) for display to a viewer.

Abstract: A processor employs a hierarchical register file for a graphics processing unit (GPU). A top level of the hierarchical register file is stored at a local memory of the GPU (e.g., a memory on the same integrated circuit die as the GPU). Lower levels of the hierarchical register file are stored at a different, larger memory, such as a remote memory located on a different die than the GPU. A register file control module monitors the status of in-flight wavefronts at the GPU, and in particular whether each in-flight wavefront is active, predicted to be become active, or inactive. The register file control module places execution data for active and predicted-active wavefronts in the top level of the hierarchical register file and places execution data for inactive wavefronts at lower levels of the hierarchical register file.