Patents by Inventor Yifu Zhang
Yifu Zhang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11688138Abstract: A method for updating camera poses includes receiving a set of captured depth maps associated with a scene and detecting a first shape and a second shape present in the scene. The method also includes, for each of the first shape and the second shape, creating a 3D mesh, creating a virtual camera associated with the 3D mesh, and rendering a depth map associated with the virtual camera. The method further includes identifying a subset of the captured depth maps such that each captured depth map in the subset contains at least a portion of either the first shape or the second shape. The method additionally includes updating the physical camera poses by jointly solving for the physical camera poses by optimizing an alignment between the first depth map, the second depth map, and the subset of the set of captured depth maps.Type: GrantFiled: October 28, 2021Date of Patent: June 27, 2023Assignee: Magic Leap, Inc.Inventors: Xiaolin Wei, Yifu Zhang
-
Publication number: 20230153185Abstract: Exemplary embodiments may programmatically determine the number and locations of breakpoints and table values in a lookup table so as to reduce their counts. In some exemplary embodiments, the minimum number of breakpoints and breakpoint locations that conform with the error tolerance may be determined along with table values for those breakpoints. The exemplary embodiments may determine if a lookup table that conforms with the error tolerance is feasible and may inform a user of the feasibility or infeasibility of generating the lookup table. Where it is feasible to generate the lookup table, the exemplary embodiments may provide automatic programmatic generation of a lookup table with minimal breakpoints and optimal or near-optimal table values.Type: ApplicationFiled: January 12, 2023Publication date: May 18, 2023Inventors: Yifu Zhang, Yuan Yuan
-
Publication number: 20230147759Abstract: A method to culling parts of a 3D reconstruction volume is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data with low usage of computational resources and storage spaces. The method includes culling parts of the 3D reconstruction volume against a depth image. The depth image has a plurality of pixels, each of which represents a distance to a surface in a scene. In some embodiments, the method includes culling parts of the 3D reconstruction volume against a frustum. The frustum is derived from a field of view of an image sensor, from which image data to create the 3D reconstruction is obtained.Type: ApplicationFiled: January 4, 2023Publication date: May 11, 2023Applicant: Magic Leap, Inc.Inventors: Frank Thomas Steinbrücker, David Geoffrey Molyneaux, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Patent number: 11580705Abstract: A method to culling parts of a 3D reconstruction volume is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data with low usage of computational resources and storage spaces. The method includes culling parts of the 3D reconstruction volume against a depth image. The depth image has a plurality of pixels, each of which represents a distance to a surface in a scene. In some embodiments, the method includes culling parts of the 3D reconstruction volume against a frustum. The frustum is derived from a field of view of an image sensor, from which image data to create the 3D reconstruction is obtained.Type: GrantFiled: April 26, 2021Date of Patent: February 14, 2023Assignee: Magic Leap, Inc.Inventors: Frank Thomas Steinbrücker, David Geoffrey Molyneaux, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Patent number: 11579956Abstract: Exemplary embodiments may perform feasibility analysis to determine whether it is possible to generate a lookup table that conforms to an error tolerance given a specification of a function or a set of data points that the lookup table attempts to approximate, an indication of breakpoint positions, and a specification of a data type for table values. Where it is determined that it is feasible to generate the lookup table, the lookup table may be automatically programmatically generated. Suggestions of how to modify the breakpoint positions and/or error tolerance may be provided. In addition, a visualization of approximation error and error tolerance, such as a visualization showing a feasibility margin, may be output. New data points may be processed to update table values for an already generated lookup table.Type: GrantFiled: March 24, 2021Date of Patent: February 14, 2023Assignee: The MathWorks, Inc.Inventors: Yifu Zhang, Aditya Dodda
-
Patent number: 11492766Abstract: The present invention discloses a method of hot recycling repairing by optimizing a proportion of an asphalt mixture on a pavement alignment variation section. According to the method of the present invention, by regulating and optimizing the proportion of a recycled asphalt mixture, internal frictional resistance and cohesion of the recycled asphalt mixture are improved, elastic modulus and shear strength of the recycled asphalt mixture are increased, overall bearing capacity of an asphalt pavement is improved, and diseases such as tracks caused by vehicle loads are effectively resisted. The pavement asphalt mixture is made to adapt to and be enough to bear additional loads increased manifold without generating early defects and road diseases. Therefore, the overall service life of a road is further prolonged.Type: GrantFiled: July 10, 2019Date of Patent: November 8, 2022Assignee: FREETECH THERMAL POWER CO., LTDInventors: Wai Pan Sze, Yifu Zhang, Kai King Chan, Min Chen, Wei Zhu, Quan Niu
-
Patent number: 11398081Abstract: An augmented reality/mixed reality system that provides a more immersive user experience. That experience is provided with increased speed of update for occlusion data by using depth sensor data augmented with lower-level reconstruction data. When operating in real-time dynamic environments, changes in the physical world can be reflected quickly in the occlusion data. Occlusion rendering using live depth data augmented with lower-level 3D reconstruction data, such as a raycast point cloud, can greatly reduce the latency for visual occlusion processing. Generating occlusion data in this way may provide faster operation of an XR system using less computing resources and enabling the system to be packaged in a battery operated wearable device.Type: GrantFiled: December 17, 2020Date of Patent: July 26, 2022Assignee: Magic Leap, Inc.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Publication number: 20220179728Abstract: Exemplary embodiments may perform feasibility analysis to determine whether it is possible to generate a lookup table that conforms to an error tolerance given a specification of a function or a set of data points that the lookup table attempts to approximate, an indication of breakpoint positions, and a specification of a data type for table values. Where it is determined that it is feasible to generate the lookup table, the lookup table may be automatically programmatically generated. Suggestions of how to modify the breakpoint positions and/or error tolerance may be provided. In addition, a visualization of approximation error and error tolerance, such as a visualization showing a feasibility margin, may be output. New data points may be processed to update table values for an already generated lookup table.Type: ApplicationFiled: March 24, 2021Publication date: June 9, 2022Inventors: Yifu Zhang, Aditya Dodda
-
Patent number: 11321924Abstract: A method to efficiently update and manage outputs of real time or offline 3D reconstruction and scanning in a mobile device having limited resource and connection to the Internet is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data, in either single user applications or multi-user applications sharing and updating the same 3D reconstruction data. The method includes a block-based 3D data representation that allows local update and maintains neighbor consistency at the same time, and a multi-layer caching mechanism that retrieves, prefetches, and stores 3D data efficiently for XR applications.Type: GrantFiled: July 9, 2020Date of Patent: May 3, 2022Assignee: Magic Leap, Inc.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Publication number: 20220122322Abstract: A method for updating camera poses includes receiving a set of captured depth maps associated with a scene and detecting a first shape and a second shape present in the scene. The method also includes, for each of the first shape and the second shape, creating a 3D mesh, creating a virtual camera associated with the 3D mesh, and rendering a depth map associated with the virtual camera. The method further includes identifying a subset of the captured depth maps such that each captured depth map in the subset contains at least a portion of either the first shape or the second shape. The method additionally includes updating the physical camera poses by jointly solving for the physical camera poses by optimizing an alignment between the first depth map, the second depth map, and the subset of the set of captured depth maps.Type: ApplicationFiled: October 28, 2021Publication date: April 21, 2022Applicant: Magic Leap, Inc.Inventors: Xiaolin Wei, Yifu Zhang
-
Publication number: 20220106746Abstract: The present invention discloses a method of hot recycling repairing by optimizing a proportion of an asphalt mixture on a pavement alignment variation section. According to the method of the present invention, by regulating and optimizing the proportion of a recycled asphalt mixture, internal frictional resistance and cohesion of the recycled asphalt mixture are improved, elastic modulus and shear strength of the recycled asphalt mixture are increased, overall bearing capacity of an asphalt pavement is improved, and diseases such as tracks caused by vehicle loads are effectively resisted. The pavement asphalt mixture is made to adapt to and be enough to bear additional loads increased manifold without generating early defects and road diseases. Therefore, the overall service life of a road is further prolonged.Type: ApplicationFiled: July 10, 2019Publication date: April 7, 2022Applicant: FREETECH THERMAL POWER CO.,LTDInventors: Weibin SHI, Yifu ZHANG, Qijing CHEN, Min CHEN, Wei ZHU, Quan NIU
-
Patent number: 11263820Abstract: A method of operating a computing system to generate a model of an environment represented by a mesh is provided. The method allows to update 3D meshes to client applications in real time with low latency to support on the fly environment changes. The method provides 3D meshes adaptive to different levels of simplification requested by various client applications. The method provides local update, for example, updating the mesh parts that are changed since last update. The method also provides 3D meshes with planarized surfaces to support robust physics simulations. The method includes segmenting a 3D mesh into mesh blocks. The method also includes performing a multi-stage simplification on selected mesh blocks. The multi-stage simplification includes a pre-simplification operation, a planarization operation, and a post-simplification operation.Type: GrantFiled: October 26, 2020Date of Patent: March 1, 2022Assignee: Magic Leap, Inc.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Patent number: 11189093Abstract: A method for forming a reconstructed 3D mesh includes receiving a set of captured depth maps associated with a scene, performing an initial camera pose alignment associated with the set of captured depth maps, and overlaying the set of captured depth maps in a reference frame. The method also includes detecting one or more shapes in the overlaid set of captured depth maps and updating the initial camera pose alignment to provide a shape-aware camera pose alignment. The method further includes performing shape-aware volumetric fusion and forming the reconstructed 3D mesh associated with the scene.Type: GrantFiled: December 18, 2019Date of Patent: November 30, 2021Assignee: Magic Leap, Inc.Inventors: Xiaolin Wei, Yifu Zhang
-
Publication number: 20210248829Abstract: A method to culling parts of a 3D reconstruction volume is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data with low usage of computational resources and storage spaces. The method includes culling parts of the 3D reconstruction volume against a depth image. The depth image has a plurality of pixels, each of which represents a distance to a surface in a scene. In some embodiments, the method includes culling parts of the 3D reconstruction volume against a frustum. The frustum is derived from a field of view of an image sensor, from which image data to create the 3D reconstruction is obtained.Type: ApplicationFiled: April 26, 2021Publication date: August 12, 2021Applicant: Magic Leap, Inc.Inventors: Frank Thomas Steinbrücker, David Geoffrey Molyneaux, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Patent number: 11024095Abstract: A method to culling parts of a 3D reconstruction volume is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data with low usage of computational resources and storage spaces. The method includes culling parts of the 3D reconstruction volume against a depth image. The depth image has a plurality of pixels, each of which represents a distance to a surface in a scene. In some embodiments, the method includes culling parts of the 3D reconstruction volume against a frustum. The frustum is derived from a field of view of an image sensor, from which image data to create the 3D reconstruction is obtained.Type: GrantFiled: March 4, 2020Date of Patent: June 1, 2021Assignee: Magic Leap, Inc.Inventors: Frank Thomas Steinbrücker, David Geoffrey Molyneaux, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Publication number: 20210142581Abstract: An augmented reality/mixed reality system that provides a more immersive user experience. That experience is provided with increased speed of update for occlusion data by using depth sensor data augmented with lower-level reconstruction data. When operating in real-time dynamic environments, changes in the physical world can be reflected quickly in the occlusion data. Occlusion rendering using live depth data augmented with lower-level 3D reconstruction data, such as a raycast point cloud, can greatly reduce the latency for visual occlusion processing. Generating occlusion data in this way may provide faster operation of an XR system using less computing resources and enabling the system to be packaged in a battery operated wearable device.Type: ApplicationFiled: December 17, 2020Publication date: May 13, 2021Applicant: MAGIC LEAP, INC.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Publication number: 20210068646Abstract: An otoscope configured to be used together with a host device, such as a smart phone or other handheld mobile devices. The otoscope can utilize features on the hand-held mobile device, such as camera, software, and display, to allow a user view a person's ear canal and tympanic membrane or eardrum. In some embodiments, the otoscope may comprise a housing; a light source; an speculum; an optical redirection component; one or more image sensors; image-processing circuitry; and communication circuitry configured to transmit the sensed image to the host device.Type: ApplicationFiled: April 2, 2019Publication date: March 11, 2021Inventors: Jane Yuqian ZHANG, Yuk Kwan LEE, Yifu ZHANG
-
Patent number: 10937246Abstract: A method of operating a computing system to generate a model of an environment represented by a mesh is provided. The method allows to update 3D meshes to client applications in real time with low latency to support on the fly environment changes. The method provides 3D meshes adaptive to different levels of simplification requested by various client applications. The method provides local update, for example, updating the mesh parts that are changed since last update. The method also provides 3D meshes with planarized surfaces to support robust physics simulations. The method includes segmenting a 3D mesh into mesh blocks. The method also includes performing a multi-stage simplification on selected mesh blocks. The multi-stage simplification includes a pre-simplification operation, a planarization operation, and a post-simplification operation.Type: GrantFiled: December 21, 2018Date of Patent: March 2, 2021Assignee: Magic Leap, Inc.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang
-
Publication number: 20210056763Abstract: A method of operating a computing system to generate a model of an environment represented by a mesh is provided. The method allows to update 3D meshes to client applications in real time with low latency to support on the fly environment changes. The method provides 3D meshes adaptive to different levels of simplification requested by various client applications. The method provides local update, for example, updating the mesh parts that are changed since last update. The method also provides 3D meshes with planarized surfaces to support robust physics simulations. The method includes segmenting a 3D mesh into mesh blocks. The method also includes performing a multi-stage simplification on selected mesh blocks. The multi-stage simplification includes a pre-simplification operation, a planarization operation, and a post-simplification operation.Type: ApplicationFiled: October 26, 2020Publication date: February 25, 2021Applicant: Magic Leap, Inc.Inventors: David Geoffrey MOLYNEAUX, Frank Thomas STEINBRÜCKER, Zhongle WU, Xiaolin WEI, Jianyuan MIN, Yifu ZHANG
-
Patent number: 10902679Abstract: An augmented reality/mixed reality system that provides a more immersive user experience. That experience is provided with increased speed of update for occlusion data by using depth sensor data augmented with lower-level reconstruction data. When operating in real-time dynamic environments, changes in the physical world can be reflected quickly in the occlusion data. Occlusion rendering using live depth data augmented with lower-level 3D reconstruction data, such as a raycast point cloud, can greatly reduce the latency for visual occlusion processing. Generating occlusion data in this way may provide faster operation of an XR system using less computing resources and enabling the system to be packaged in a battery operated wearable device.Type: GrantFiled: December 21, 2018Date of Patent: January 26, 2021Assignee: MAGIC LEAP, INC.Inventors: David Geoffrey Molyneaux, Frank Thomas Steinbrücker, Zhongle Wu, Xiaolin Wei, Jianyuan Min, Yifu Zhang