Patents Assigned to Magic Leap, Inc.
  • Patent number: 11508141
    Abstract: A method to reconstruct an environment is provided. The method makes available to a wide variety of XR applications fresh and accurate 3D reconstruction data of environments with low processing time and low usage of computational resources and storage spaces. The 3D reconstruction data are structured in a way to be efficiently shared between users for multi-user experiences. The method includes obtaining plane segments of an environment, identifying surface planes of the environment by, for example, filtering and grouping the plane segments or ad hoc selection of the plane segments by a user, and inferring corner points of the environment based on the surface planes. The corner points are used to build a 3D representation of the environment when an XR application requires.
    Type: Grant
    Filed: June 25, 2020
    Date of Patent: November 22, 2022
    Assignee: Magic Leap, Inc.
    Inventor: Benjamin Joseph Uscinski
  • Patent number: 11510027
    Abstract: This disclosure describes techniques for receiving information that is wirelessly transmitted to a mobile computing system by wireless devices that are proximal to a route being travelled by the mobile computing system, and presenting at least a portion of the received information through a display of the mobile computing system. The information can be displayed according to a computing experience that is determined for a user of the mobile computing system (e.g., user-selected, inferred based on a stored schedule of the user, and so forth). Different sets of location-based information can be transmitted to the mobile computing system from different wireless devices that the mobile computing system comes into proximity with while traveling along a route. In some instances, the information can be locally stored on the wireless device(s) to reduce latency.
    Type: Grant
    Filed: July 3, 2019
    Date of Patent: November 22, 2022
    Assignee: Magic Leap, Inc.
    Inventor: David Charles Lundmark
  • Patent number: 11507193
    Abstract: Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. The system may comprise an image capturing device to capture one or more images, the one or more images corresponding to a field of the view of a user of a head-mounted augmented reality device, and a processor communicatively coupled to the image capturing device to extract a set of map points from the set of images, to identify a set of sparse points and a set of dense points from the extracted set of map points, and to perform a normalization on the set of map points.
    Type: Grant
    Filed: October 19, 2020
    Date of Patent: November 22, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Gary R. Bradski, Samuel A. Miller, Rony Abovitz
  • Patent number: 11507204
    Abstract: Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.
    Type: Grant
    Filed: November 12, 2021
    Date of Patent: November 22, 2022
    Assignee: Magic Leap, Inc.
    Inventors: James M. Powderly, Savannah Niles, Frank Hamilton, Marshal A. Fontaine, Rony Abovitz, Alysha Naples
  • Publication number: 20220369044
    Abstract: A hearing aid comprises a sensor configured for detecting a focus of an end user on a real sound source, a microphone assembly configured for converting sounds into electrical signals, a speaker configured for converting the electrical signals into sounds, and a control subsystem configured for modifying the direction and/or distance of a greatest sensitivity of the microphone assembly based on detected focus. A virtual image generation system comprises memory storing a three-dimensional scene, a sensor configured for detecting a focus of the end user on a sound source, a speaker configured for conveying sounds to the end user, and a control subsystem configured for causing the speaker to preferentially convey a sound originating from the sound source in response to detection of the focus, and for rendering image frames of the scene, and a display subsystem configured for sequentially displaying the image frames to the end user.
    Type: Application
    Filed: July 29, 2022
    Publication date: November 17, 2022
    Applicant: MAGIC LEAP, INC.
    Inventors: George Alistair Sanger, Samuel A. Miller, Brian Schmidt, Anastasia Andreyevna Tajik
  • Publication number: 20220368763
    Abstract: Disclosed are methods and systems for a scripting framework and implementations therefor for mixed reality software applications of heterogeneous systems. These methods or systems create a mixed-reality software application that executes across heterogeneous platforms on a server-side instance of a scripting framework and manage a change in the mixed-reality software application using the server-side instance of the scripting framework. Moreover, the change in the mixed-reality software application using a client-side instance of the scripting framework; and the mixed-reality software application may be interactively executed on a mixed-reality device.
    Type: Application
    Filed: July 15, 2022
    Publication date: November 17, 2022
    Applicant: MAGIC LEAP, INC.
    Inventors: Nikolay Ivanov GROZDANOV, Konrad PIASCIK, Leonid ZOLOTAREV, Timothy Dean CASWELL
  • Publication number: 20220366688
    Abstract: An apparatus configured for head-worn by a user, includes: a screen configured to present graphics for the user; a camera system configured to view an environment in which the user is located; and a processing unit coupled to the camera system, the processing unit configured to: obtain a first image with a first resolution, the first image having a first corner, determine a second image with a second resolution, the second image having a second corner that corresponds with the first corner in the first image, wherein the second image is based on the first image, the second resolution being less than the first resolution, detect the second corner in the second image, determine a position of the second corner in the second image, and determine a position of the first corner in the first image based at least in part on the determined position of the second corner in the second image.
    Type: Application
    Filed: July 22, 2022
    Publication date: November 17, 2022
    Applicant: MAGIC LEAP, INC.
    Inventor: Hélder Toshiro SUZUKI
  • Publication number: 20220366635
    Abstract: Systems and methods of generating a three-dimensional (3D) reconstruction of a scene or environment surrounding a user of a spatial computing system, such as a virtual reality, augmented reality or mixed reality system, using only multiview images comprising RGB images, and without the need for depth sensors or depth data from sensors. Features are extracted from a sequence of frames of RGB images and back-projected using known camera intrinsics and extrinsic s into a 3D voxel volume wherein each pixel of the voxel volume is mapped to a ray in the voxel volume. The back-projected features are fused into the 3D voxel volume. The 3D voxel volume is passed through a 3D convolutional neural network to refine the and regress truncated signed distance function values at each voxel of the 3D voxel volume.
    Type: Application
    Filed: June 30, 2022
    Publication date: November 17, 2022
    Applicant: MAGIC LEAP, INC.
    Inventor: Zachary Paul MUREZ
  • Patent number: 11498261
    Abstract: Systems, apparatus, and methods for double-sided imprinting are provided. An example system includes first rollers for moving a first web including a first template having a first imprinting feature, second rollers for moving a second web including a second template having a second imprinting feature, dispensers for dispensing resist, a locating system for locating reference marks on the first and second webs for aligning the first and second templates, a light source for curing the resist, such that a cured first resist has a first imprinted feature corresponding to the first imprinting feature on one side of the substrate and a cured second resist has a second imprinted feature corresponding to the second imprinting feature on the other side of the substrate, and a moving system for feeding in the substrate between the first and second templates and unloading the double-imprinted substrate from the first and second webs.
    Type: Grant
    Filed: January 20, 2021
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Roy Patterson, Charles Scott Carden, Satish Sadam, Ryan Christiansen, Matthew S. Shafran, Christopher John Fleckenstein, Vikramjit Singh, Michael Nevin Miller, Kang Luo
  • Patent number: 11501489
    Abstract: An extended or cross reality system includes a computing device communicably connected to a plurality of portable electronic devices via a network component, a repository accessible by the computing device and the plurality of portable electronic devices, and a dense map merge component. The extended or cross reality system determines a representation for multiple portions of a 3D environment based at least in part upon on a set of dense maps received from the plurality of portable devices, wherein the set of dense maps is grouped into multiple subgroups based at least in part upon pose data pertaining to the set of dense maps or surface information in the set of dense maps. The extended or cross reality system storing the representation as at least a portion of a shared persistent dense map.
    Type: Grant
    Filed: February 25, 2021
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Yilun Cao, Mohan Babu Kandra, David Geoffrey Molyneaux, Daniel Olshansky, David Paul Pena, Frank Thomas Steinbrücker, Rafael Domingos Torres
  • Patent number: 11500208
    Abstract: Images perceived to be substantially full color or multi-colored may be formed using component color images that are distributed in unequal numbers across a plurality of depth planes. The distribution of component color images across the depth planes may vary based on color. In some embodiments, a display system includes a stack of waveguides that each output light of a particular color, with some colors having fewer numbers of associated waveguides than other colors. The stack of waveguides may include by multiple pluralities (e.g., first and second pluralities) of waveguides, each configured to produce an image by outputting light corresponding to a particular color. The total number of waveguides in the second plurality of waveguides is less than the total number of waveguides in the first plurality of waveguides, and may be more than the total number of waveguides in a third plurality of waveguides, in embodiments where three component colors are utilized.
    Type: Grant
    Filed: January 8, 2021
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Brian T. Schowengerdt, Hong Hua, Hui-Chuan Cheng, Christophe Peroz
  • Patent number: 11501529
    Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.
    Type: Grant
    Filed: March 5, 2021
    Date of Patent: November 15, 2022
    Assignee: MAGIC LEAP, INC.
    Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
  • Patent number: 11500206
    Abstract: An example waveguide can include a polymer layer having substantially optically transparent material with first and second major surfaces configured such that light containing image information can propagate through the polymer layer being guided therein by reflecting from the first and second major surfaces via total internal reflection. The first surface can include first smaller and second larger surface portions monolithically integrated with the polymer layer and with each other. The first smaller surface portion can include at least a part of an in-coupling optical element configured to couple light incident on the in-coupling optical element into the polymer layer for propagation therethrough by reflection from the second major surface and the second larger surface portion of the first major surface.
    Type: Grant
    Filed: April 1, 2019
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Christophe Peroz, Victor Kai Liu, Samarth Bhargava
  • Patent number: 11503204
    Abstract: A method includes capturing an image using a content capture device with an initial image setting. The image includes a plurality of pixel groups, and a pixel group can have one or more pixels. A plurality of edge pixel groups is identified and then classified into two subsets. A first subset of saturated edge pixel groups includes edge pixel groups that have at least one neighboring pixel group with an image intensity exceeding a saturated intensity value. A second subset of non-saturated edge pixel groups includes edge pixel groups that have no neighboring pixel groups with an image intensity exceeding a saturated intensity value. An adjustment value to the image setting is determined based on a total number of saturated edge pixel groups and a total number of non-saturated edge pixel groups. An updated image is captured with an updated image setting based on the adjustment value.
    Type: Grant
    Filed: December 18, 2020
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Shih-Che Huang, Christian Ivan Robert Moore, Brian Keith Smith
  • Patent number: 11503275
    Abstract: An improved method, system, and apparatus is provided to perform camera calibration, where cameras are mounted onto a moving conveyance apparatus to capture images of a multi-planar calibration target. The calibration process is optimized by reducing the number of images captured while simultaneously preserving overall information density.
    Type: Grant
    Filed: December 19, 2019
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Jeffrey Steven Kranski, Frederick Dennis Zyda, Grace Vesom, Grace Shin-Yee Tsai, Jeremy A. Grata, Zhiheng Jia, Li Jiang
  • Patent number: 11501680
    Abstract: Embodiments provide a computer implemented method for warping multi-field color virtual content for sequential projection. First and second color fields having different first and second colors are obtained. A first time for projection of a warped first color field is determined. A first pose corresponding to the first time is predicted. For each one color among the first colors in the first color field, (a) an input representing the one color among the first colors in the first color field is identified; (b) the input is reconfigured as a series of pulses creating a plurality of per-field inputs; and (c) each one of the series of pulses is warped based on the first pose. The warped first color field is generated based on the warped series of pulses. Pixels on a sequential display are activated based on the warped series of pulses to display the warped first color field.
    Type: Grant
    Filed: February 2, 2021
    Date of Patent: November 15, 2022
    Assignee: Magic Leap, Inc.
    Inventor: Marshall Charles Capps
  • Publication number: 20220357580
    Abstract: An augmented reality system includes a light source to generate a virtual light beam, the virtual light beam carrying information for a virtual object. The system also includes a light guiding optical element, the light guiding optical element allowing a first portion of a first real-world light beam to pass therethrough, where the virtual light beam enters the light guiding optical element, propagates through the light guiding optical element by substantially total internal reflection (TIR), and exits the light guiding optical element. The system further includes a lens disposed adjacent and exterior to a surface of the light guiding optical element, the lens comprising a light modulating mechanism to absorb a second portion of the real-world light beam and to allow the first portion of the real-world light to pass through the lens.
    Type: Application
    Filed: July 20, 2022
    Publication date: November 10, 2022
    Applicant: MAGIC LEAP, INC.
    Inventor: Andrew Ian RUSSELL
  • Publication number: 20220358733
    Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps of very large scale environments and render virtual content specified in relation to those maps. The cross reality system may build a persisted map, which may be in canonical form, by merging tracking maps from the multiple devices. A map merge process determines mergibility of a tracking map with a canonical map and merges a tracking map with a canonical map in accordance with mergibility criteria, such as, when a gravity direction of the tracking map aligns with a gravity direction of the canonical map. Refraining from merging maps if the orientation of the tracking map with respect to gravity is not preserved avoids distortions in persisted maps and results in multiple devices, which may use the maps to determine their locations, to present more realistic and immersive experiences for their users.
    Type: Application
    Filed: July 1, 2022
    Publication date: November 10, 2022
    Applicant: Magic Leap, Inc.
    Inventors: Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez, Mukta Prasad, Eran Guendelman, Ali Shahrokni, Ashwin Swaminathan
  • Patent number: 11494995
    Abstract: Disclosed are methods, systems, and computer program products for mixed-reality systems. These methods or systems determine a three-dimensional model for at least a portion of a physical environment in which a user is located; and present, by a spatial computing system, a mixed-reality representation to the user. In addition, these methods or systems determine a first object model for a first object in the mixed-reality representation and update, by the spatial computing system, the mixed-reality representation into an updated mixed-reality representation that reflects an interaction pertaining to the first object.
    Type: Grant
    Filed: October 29, 2020
    Date of Patent: November 8, 2022
    Assignee: Magic Leap, Inc.
    Inventor: Robert David Berkebile
  • Patent number: 11493989
    Abstract: A mixed reality (MR) device can allow a user to switch between input modes to allow interactions with a virtual environment via devices such as a six degrees of freedom (6DoF) handheld controller and a touchpad input device. A default input mode for interacting with virtual content may rely on the user's head pose, which may be difficult to use in selecting virtual objects that are far away in the virtual environment. Thus, the system may be configured to allow the user to use a 6DoF cursor, and a visual ray that extends from the handheld controller to the cursor, to enable precise targeting. Input via a touchpad input device (e.g., that allows three degrees of freedom movements) may also be used in conjunction with the 6DoF cursor.
    Type: Grant
    Filed: November 4, 2020
    Date of Patent: November 8, 2022
    Assignee: Magic Leap, Inc.
    Inventors: Daniel Stephen Speelman, Rodrigo Cano, Kara Lauren Gundersen, Griffith Buckley Hazen, Lorena Pazmino