Patents by Inventor Martin Georg Zahnert
Martin Georg Zahnert has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240143068Abstract: A high-resolution image sensor suitable for use in an augmented reality (AR) system to provide low latency image analysis with low power consumption. The AR system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object, selectively output imaging information for that region, and synchronously output high-resolution image frames. The region may be updated dynamically as the image sensor and/or the object moves. The image sensor may output the high-resolution image frames less frequently than the region being updated when the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an AR scene can be developed.Type: ApplicationFiled: September 28, 2023Publication date: May 2, 2024Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander IIic
-
Publication number: 20240137664Abstract: An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.Type: ApplicationFiled: January 3, 2024Publication date: April 25, 2024Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic, Erik Fonseka
-
Publication number: 20240135707Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: ApplicationFiled: October 8, 2023Publication date: April 25, 2024Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Publication number: 20240137665Abstract: A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.Type: ApplicationFiled: December 15, 2023Publication date: April 25, 2024Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic, Miguel Andres Granados Velasquez, Javier Victorio
-
Patent number: 11935180Abstract: Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.Type: GrantFiled: April 7, 2022Date of Patent: March 19, 2024Assignee: Magic Leap, Inc.Inventors: Yu-Hsiang Huang, Evan Gregory Levine, Igor Napolskikh, Dominik Michael Kasper, Manel Quim Sanchez Nicuesa, Sergiu Sima, Benjamin Langmann, Ashwin Swaminathan, Martin Georg Zahnert, Blazej Marek Czuprynski, Joao Antonio Pereira Faro, Christoph Tobler, Omid Ghasemalizadeh
-
Patent number: 11902677Abstract: An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.Type: GrantFiled: October 30, 2019Date of Patent: February 13, 2024Assignee: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic, Erik Fonseka
-
Patent number: 11885871Abstract: An augmented reality device has a radar system that generates radar maps of locations of real world objects. An inertial measurement unit detects measurement values such as acceleration, gravitational force and inclination ranges. The values from the measurement unit drift over time. The radar maps are processed to determine fingerprints and the fingerprints are combined with the values from the measurement unit to store a pose estimate. Pose estimates at different times are compared to determine drift of the measurement unit. A measurement unit filter is adjusted to correct for the drift.Type: GrantFiled: May 24, 2019Date of Patent: January 30, 2024Assignee: Magic Leap, Inc.Inventors: Alexander Ilic, Martin Georg Zahnert, Koon Keong Shee
-
Patent number: 11889209Abstract: A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.Type: GrantFiled: February 7, 2020Date of Patent: January 30, 2024Assignee: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic, Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez
-
Patent number: 11823450Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: GrantFiled: October 14, 2022Date of Patent: November 21, 2023Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Patent number: 11809613Abstract: A high-resolution image sensor suitable for use in an augmented reality (AR) system to provide low latency image analysis with low power consumption. The AR system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object, selectively output imaging information for that region, and synchronously output high-resolution image frames. The region may be updated dynamically as the image sensor and/or the object moves. The image sensor may output the high-resolution image frames less frequently than the region being updated when the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an AR scene can be developed.Type: GrantFiled: October 30, 2019Date of Patent: November 7, 2023Assignee: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic
-
Patent number: 11598957Abstract: Systems and methods for synthesizing an image of the face by a head-mounted display (HMD), such as an augmented reality device, are disclosed. The HMD may be able to observe only a portion of the face with an inward-facing imaging system, e.g., the periocular region. The systems and methods described herein can generate a mapping of a conformation of the portion of the face that is not imaged based at least partly on a conformation of the portion of the face that is imaged. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.Type: GrantFiled: April 20, 2022Date of Patent: March 7, 2023Assignee: MAGIC LEAP, INC.Inventors: Alexander Ilic, Martin Georg Zahnert
-
Publication number: 20230034363Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: ApplicationFiled: October 14, 2022Publication date: February 2, 2023Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Patent number: 11501529Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: GrantFiled: March 5, 2021Date of Patent: November 15, 2022Assignee: MAGIC LEAP, INC.Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Publication number: 20220244532Abstract: Systems and methods for synthesizing an image of the face by a head-mounted display (HMD), such as an augmented reality device, are disclosed. The HMD may be able to observe only a portion of the face with an inward-facing imaging system, e.g., the periocular region. The systems and methods described herein can generate a mapping of a conformation of the portion of the face that is not imaged based at least partly on a conformation of the portion of the face that is imaged. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.Type: ApplicationFiled: April 20, 2022Publication date: August 4, 2022Inventors: Alexander Ilic, Martin Georg Zahnert
-
Publication number: 20220230382Abstract: Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.Type: ApplicationFiled: April 7, 2022Publication date: July 21, 2022Inventors: Yu-Hsiang HUANG, Evan Gregory LEVINE, Igor NAPOLSKIKH, Dominik Michael KASPER, Manel Quim SANCHEZ NICUESA, Sergiu SIMA, Benjamin LANGMANN, Ashwin SWAMINATHAN, Martin Georg ZAHNERT, Blazej Marek CZUPRYNSKI, Joao Antonio Pereira FARO, Christoph TOBLER, Omid GHASEMALIZADEH
-
Patent number: 11347051Abstract: Systems and methods for synthesizing an image of the face by a head-mounted display (HMD), such as an augmented reality device, are disclosed. The HMD may be able to observe only a portion of the face with an inward-facing imaging system, e.g., the periocular region. The systems and methods described herein can generate a mapping of a conformation of the portion of the face that is not imaged based at least partly on a conformation of the portion of the face that is imaged. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.Type: GrantFiled: July 30, 2020Date of Patent: May 31, 2022Assignee: Magic Leap, Inc.Inventors: Alexander Ilic, Martin Georg Zahnert
-
Patent number: 11328475Abstract: Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a first preintegration term and second preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the first and second preintegration terms, and the wearable head device presents the virtual content based on the position of the device.Type: GrantFiled: October 16, 2020Date of Patent: May 10, 2022Assignee: Magic Leap, Inc.Inventors: Yu-Hsiang Huang, Evan Gregory Levine, Igor Napolskikh, Dominik Michael Kasper, Manel Quim Sanchez Nicuesa, Sergiu Sima, Benjamin Langmann, Ashwin Swaminathan, Martin Georg Zahnert, Blazej Marek Czuprynski, Joao Antonio Pereira Faro, Christoph Tobler, Omid Ghasemalizadeh
-
Publication number: 20220129066Abstract: A wearable display system for a cross reality (XR) system may have a dynamic vision sensor (DVS) camera and a color camera. At least one of the cameras may be a plenoptic camera. The wearable display system may dynamically restrict processing of image data from either or both cameras based on detected conditions and XR function being performed. For tracking an object, image information may be processed for patches of a field of view of either or both cameras corresponding to the object. The object may be tracked based on asynchronously acquired events indicating changes within the patches. Stereoscopic or other types of image information may be used when event-based object tacking yields an inadequate quality metric. The tracked object may be a user's hand or a stationary object in the physical world, enabling calculation of the pose of the wearable display system and of the wearer's head.Type: ApplicationFiled: February 7, 2020Publication date: April 28, 2022Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander llic
-
Publication number: 20220132056Abstract: A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.Type: ApplicationFiled: February 7, 2020Publication date: April 28, 2022Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander Ilic, Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez
-
Publication number: 20220051441Abstract: A wearable display system with a limited number of cameras. Two cameras can be arranged to provide an overlapping central view field and a peripheral view field associated with one of the two cameras. A third camera can be arranged to provide a color view field overlapping the central view field. The wearable display system may be coupled to a processor configured to generate a world model and track hand motion in the central view field using the two cameras. The processor may be configured to perform a calibration routine to compensate for distortions during use of the wearable display system. The processor may be configured to identify and address portions of the world model including incomplete depth information by obtaining additional depth information, such as by enabling emitters, detecting planar surfaces in the physical world, or identifying relevant object templates in the world model.Type: ApplicationFiled: December 19, 2019Publication date: February 17, 2022Applicant: Magic Leap, Inc.Inventors: Martin Georg Zahnert, Alexander llic, Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez