Abstract: Real-time animation of a virtual character can be provided by pre-calculating a control policy that identifies a suitable animation clip to animate the next movement of the virtual character. The control policy can be calculated using a Markov decision process (MDP) and can specify an action to take (e.g., a movement) when the virtual character is in a particular state (e.g., in a particular position or pose). The control policy can be determined based on a discounted set of rewards or punishments associated with various actions that can occur in each state. The control policies can be pre-calculated, offline from a runtime animation engine (e.g., in an augmented reality display). The runtime animation engine can use the control policies to select an animation clip to move the virtual character through an environment in real time.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-readable storage media, for machine learning to generate and evaluate visualizations. In some implementations, a system determines properties of a dataset. The system generates visualization specifications that each define a different visualization for the dataset, wherein the visualization specifications specify different subsets of the dataset being illustrated with different visualization formats. The system evaluates the visualization specifications using a machine learning model trained based on user feedback for visualizations for multiple datasets. The system selects a subset of the visualization specifications based on output of the machine learning model. The system provides, for display, visualization data for the subset of visualization specifications that were selected based on the output of the machine learning model.

Abstract: A graphics processing system for a head mounted display (or other non-standard projection display) comprises a low latency distortion unit which is separate from a graphics processing unit in the graphics processing system. The low latency distortion unit receives pixel data generated by the graphics processing system using a standard projection and performs a mapping operation to introduce distortion which is dependent upon the optical properties of the optical arrangement within the head mounted display. The distorted pixel data which is generated by the low latency distortion unit is then output to the display in the head mounted display.

Abstract: Augmented reality (AR) and virtual reality (VR) environment enhancement using an eyewear device. The eyewear device includes an image capture system, a display system, and a position detection system. The image capture system and position detection system identify feature points within a point cloud that represents captured images of an environment. The display system presents image overlays to a user including enhancement graphics positioned at the feature points within the environment.

Abstract: In some embodiments, a display system comprising a head-mountable, augmented reality display is configured to perform a neurological analysis and to provide a perception aid based on an environmental trigger associated with the neurological condition. Performing the neurological analysis may include determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and identifying a neurological condition associated with the reaction. In some embodiments, the perception aid may include a reminder, an alert, or virtual content that changes a property, e.g. a color, of a real object. The augmented reality display may be configured to display virtual content by outputting light with variable wavefront divergence, and to provide an accommodation-vergence mismatch of less than 0.5 diopters, including less than 0.25 diopters.

Abstract: An augmented reality system for procedural guidance identifies a fiducial marker object in a frame of a first field of view generated by a camera, determines a pose of the fiducial marker object, applies the fiducial marker pose to generate a first transformation between a first coordinate system of the fiducial marker object and a second coordinate system of the camera, and applies a pose of a headset to determine a second transformation between the first coordinate system and a third coordinate system of the headset.

Abstract: A method and system for transforming simple user input into customizable animated images for use in text-messaging applications.

Abstract: An analog to digital converter comprises an input for receiving an analog input signal; a plurality of outputs for outputting parallel bits of a digital signal that represents said analog input signal; and a neural network layer providing connections between each of said outputs respectively, each connection having an adjustable weighting. The synapses of the neural networks may be memristors and training may use online gradient descent.

Abstract: Systems and methods for generating and displaying heat maps are provided. A heat map generation computing device includes a memory and a processor. The processor is programmed to receive trap data for a plurality of pest traps in a geographic location, the trap data including current and historical pest pressure values at each of the plurality of pest traps, receive weather data for the geographic location, receive image data for the geographic location, apply a machine learning algorithm to generate predicted future pest pressure values at each of the plurality of pest traps, generate a first heat map for a first point in time and a second heat map for a second point in time, and transmit the first and second heat maps to a mobile computing device to cause a user interface on the mobile computing device to display a time lapse heat map.

Abstract: A technique for compressing an original image is disclosed. According to the technique, an original image is obtained and a delta-encoded image is generated based on the original image. Next, a segregated image is generated based on the delta-encoded image and then the segregated image is compressed to produce a compressed image. The segregated image is generated because the segregated image may be compressed more efficiently than the original image and the delta image.

Abstract: An extended reality (XR) support system establishes a communicative link between an XR presentation system and a multi-access edge compute (MEC) system operating on a cellular network. The XR support system identifies a parameter associated with a computing task to be performed in connection with an XR experience presented by the XR presentation system. The XR support system also identifies a condition associated with a capability of the XR presentation system or the MEC system to perform the computing task. Based on the identified parameter and the identified condition, the XR support system selects at least one of the XR presentation system or the MEC system to perform the computing task and then directs the selected system to perform the computing task. Corresponding methods and systems are also disclosed.

Abstract: In a graphics processing unit (GPU), receiving an input image comprising an array of pixels. Each pixel having a grayscale value from a range of N grayscale values. For each particular input patch of pixels of a set of input patches partitioning the input image and in parallel for each particular grayscale value the range, counting the number of pixels in the particular input patch having the particular grayscale value. In parallel for each particular input patch of pixels of a set of input patches partitioning the input image, creating an output image patch as an ordered sequence of N pixels, with the color value of the nth pixel in each corresponding output patch representing the count of pixels in the particular input patch having the nth grayscale value. Combining the output image patches into a single composite output image of N pixels, the pixel value of the nth pixel in the single composite output image corresponding to the count of pixels in the input image having the nth grayscale value.

Abstract: Disclosed are systems and methods for rapid generation of simulations of a patient's spinal morphology that enable pre-operative viewing of a patient's condition and to assist surgeons in determining the best corrective procedure and with any of the selection, augmentation or manufacture of spinal devices based on the patient specific simulated condition. The simulation is generated by morphing a generic spine model with a three-dimensional curve representation of the patient's particular spinal morphology derived from existing images of the patient's condition.

Abstract: A mobile computing device comprises an AR display, an image capture device that generates image data of a face of a viewer of the AR display, and a processing device. The processing device receives the image data; processes the image data to identify a position of a dental arch in the image data; determines a treatment outcome for the dental arch; generates a post-treatment image of the dental arch that shows the treatment outcome; generates updated image data comprising a superimposition of the post-treatment image of the dental arch over the received image data depicting the face of the viewer; and outputs the updated image data to the AR display, wherein the post-treatment image of the dental arch is superimposed over the dental arch in the received image data such that the post-treatment image is visible in the AR display rather than a true depiction of the dental arch.

Abstract: A system is provided for use by athletes and their coaches during practice. The system includes an electronic system configured for inclusion in a helmet. The electronic system includes a transparent display screen coupled to the helmet, a processor coupled to the transparent display screen, a camera coupled to the processor and a first wireless transceiver coupled to the processor. The system also includes a handheld processing device. The handheld processing device includes a second wireless transceiver configured to communicate video content with the first wireless transceiver, an interactive display screen configured to display the first video and a video editing application configured to edit the first video to generate a second video including a simulated player technique superimposed in the first video. The second video is communicated from the handheld processing device to the electronic system for display to the athlete via the transparent display screen.

Abstract: A method and system for generating a three-dimensional representation of a vehicle to assess damage to the vehicle. A mobile device may capture multispectral scans of a vehicle from each a plurality of cameras configured to scan the vehicle at a different wavelength of the electromagnetic spectrum. A virtual model of the vehicle may be generated from the multispectral scan of the vehicle, such that anomalous conditions or errors in individual wavelength data are omitted from model generation. A representation of the virtual model may be presented to the user via the display of the mobile device. The virtual model of the vehicle may further be analyzed to assess damage to the vehicle.

Abstract: An artificial reality system is configured to more accurately and efficiently construct a 3D virtual representation of a real-world environment from a set of 2D images. The system identifies points and/or lines within the images that define a plane along an orientation and then performs a planar sweep along a perpendicular path to identify surfaces in which the plane intersects with multiple points. Points that appear to be in the same plane are “collapsed” into a cohesive plane to conserve processing power by estimating and/or storing parameters for the cohesive plane, rather than all of the individual 3D points. In this way, the system also “averages out” random variation in the planar surface that would otherwise result from random noise in the estimation of the individual 3D points. The system may then generate a 3D map from a constrained alignment of all of the identified planes.

Abstract: Introduced here computer programs and associated computer-implemented techniques for establishing the dimensions of interior spaces. These computer programs are able to accomplish this by combining knowledge of these interior spaces with spatial information that is output by an augmented reality (AR) framework. Such an approach allows two-dimensional (2D) layouts to be seamlessly created through guided corner-to-corner measurement of interior spaces.

Abstract: A ray is cast into a volume described by a volumetric data structure, which describes the volume at a plurality of levels of detail. A first entry in the volumetric data structure includes a first set of bits representing voxels at a lowest one of the plurality of levels of detail, and values of the first set of bits indicate whether a corresponding one of the voxels is at least partially occupied by respective geometry. A set of second entries in the volumetric data structure describe voxels at a second level of detail, which represent subvolumes of the voxels at the first lowest level of detail. The ray is determined to pass through a particular subset of the voxels at the first level of detail and at least a particular one of the particular subset of voxels is determined to be occupied by geometry.

Abstract: A system or method includes a platform to allow users to coordinate images captured by a separated camera with images captured by a first camera, displaying such images at real-time to a user through a user interface on the first camera or a user interface on a separate device. The separate camera is contemplated to provide a separate image feed to provide one or more augmentations to an image captured by the first camera.