Patents by Inventor Ashwin Swaminathan
Ashwin Swaminathan has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11978159Abstract: A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.Type: GrantFiled: December 3, 2021Date of Patent: May 7, 2024Assignee: Magic Leap, Inc.Inventors: Anush Mohan, Rafael Domingos Torres, Daniel Olshansky, Samuel A. Miller, Jehangir Tajik, Joel David Holder, Jeremy Dwayne Miranda, Robert Blake Taylor, Ashwin Swaminathan, Lomesh Agarwal, Hiral Honar Barot, Helder Toshiro Suzuki, Ali Shahrokni, Eran Guendelman, Prateek Singhal, Xuan Zhao, Siddharth Choudhary, Nicholas Atkinson Kramer, Kenneth William Tossell, Christian Ivan Robert Moore
-
Patent number: 11978174Abstract: A method includes extracting, using a backbone of a machine learning model, a plurality of features from an image of a foot and predicting, using a first portion of the machine learning model and based on one or more features of the plurality of features, a first aspect of the foot. The method also includes predicting, using a second portion of the machine learning model and based on one or more features of the plurality of features, a second aspect of the foot different from the first aspect, generating, using at least the first aspect and the second aspect, a two-dimensional model of a shoe, and superimposing the two-dimensional model of the shoe onto the image of the foot.Type: GrantFiled: March 28, 2022Date of Patent: May 7, 2024Assignee: Amazon Technologies, Inc.Inventors: Yuelong Li, Gitika Karumuri, Miriam Bellver Bueno, Sunil Sharadchandra Hadap, Ashwin Swaminathan, Amogh Gupta, Xin Shen
-
Publication number: 20240135707Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: ApplicationFiled: October 8, 2023Publication date: April 25, 2024Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Patent number: 11956222Abstract: One or more computing devices, systems, and/or methods for end-to-end encryption for multiple recipient devices are provided. A first registration, comprising a first device public key, is created for a first device. A second registration, comprising a second device public key, is created for a second device. A first notify message of the second registration and second device public key is provided to the first device. A second notify message is provided to the second device of the first registration and first device public key. A secure communication invite is routed to the first device. An encrypted message, comprising a first device private key, is routed from the first device to the second device. End-to-end encrypted communication between a sender device and the first device and the second device using the first device private key is facilitated.Type: GrantFiled: September 3, 2021Date of Patent: April 9, 2024Assignee: Verizon Patent and Licensing Inc.Inventors: Andrew Wahlon Lam, Ashwin Swaminathan, Wen Zea Kuo, Rezwanul Azim
-
Patent number: 11935180Abstract: Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.Type: GrantFiled: April 7, 2022Date of Patent: March 19, 2024Assignee: Magic Leap, Inc.Inventors: Yu-Hsiang Huang, Evan Gregory Levine, Igor Napolskikh, Dominik Michael Kasper, Manel Quim Sanchez Nicuesa, Sergiu Sima, Benjamin Langmann, Ashwin Swaminathan, Martin Georg Zahnert, Blazej Marek Czuprynski, Joao Antonio Pereira Faro, Christoph Tobler, Omid Ghasemalizadeh
-
Publication number: 20240087258Abstract: A cross reality system enables any of multiple devices to efficiently access previously stored maps. Both stored maps and tracking maps used by portable devices may have any of multiple types of location metadata associated with them. The location metadata may be used to select a set of candidate maps for operations, such as localization or map merge, that involve finding a match between a location defined by location information from a portable device and any of a number of previously stored maps. The types of location metadata may prioritized for use in selecting the subset. To aid in selection of candidate maps, a universe of stored maps may be indexed based on geo-location information. A cross reality platform may update that index as it interacts with devices that supply geo-location information in connection with location information and may propagate that geo-location information to devices that do not supply it.Type: ApplicationFiled: November 15, 2023Publication date: March 14, 2024Applicant: Magic Leap, Inc.Inventors: Xuan Zhao, Christian Ivan Robert Moore, Sen Lin, Ali Shahrokni, Ashwin Swaminathan
-
Publication number: 20240062403Abstract: Disclosed here in are systems and methods for mapping environment information. In some embodiments, the systems and methods are configured for mapping information in a mixed reality environment. In some embodiments, the system is configured to perform a method including scanning an environment including capturing, with a sensor, a plurality of points of the environment; tracking a plane of the environment; updating observations associated with the environment by inserting a keyframe into the observations; determining whether the plane is coplanar with a second plane of the environment; in accordance with a determination that the plane is coplanar with the second plane, performing planar bundle adjustment on the observations associated with the environment; and in accordance with a determination that the plane is not coplanar with the second plane, performing planar bundle adjustment on a portion of the observations associated with the environment.Type: ApplicationFiled: February 11, 2022Publication date: February 22, 2024Inventors: Lipu ZHOU, Ashwin SWAMINATHAN, Lomesh AGARWAL
-
Publication number: 20240062491Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps of very large scale environments and render virtual content specified in relation to those maps. The cross reality system may build a persisted map, which may be in canonical form, by merging tracking maps from the multiple devices. A map merge process determines mergibility of a tracking map with a canonical map and merges a tracking map with a canonical map in accordance with mergibility criteria, such as, when a gravity direction of the tracking map aligns with a gravity direction of the canonical map. Refraining from merging maps if the orientation of the tracking map with respect to gravity is not preserved avoids distortions in persisted maps and results in multiple devices, which may use the maps to determine their locations, to present more realistic and immersive experiences for their users.Type: ApplicationFiled: August 28, 2023Publication date: February 22, 2024Applicant: Magic Leap, Inc.Inventors: Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez, Mukta Prasad, Eran Guendelman, Ali Shahrokni, Ashwin Swaminathan
-
Publication number: 20240029301Abstract: A method of efficiently and accurately computing a pose of an image with respect to other image information. The image may be acquired with a camera on a portable device and the other information may be a map, such that the computation of pose localizes the device relative to the map. Such a technique may be applied in a cross reality system to enable devices to efficiently and accurately access previously persisted maps. Localizing with respect to a map may enable multiple cross reality devices to render virtual content at locations specified in relation to those maps, providing an enhanced experience for uses of the system. The method may be used in other devices and for other purposes, such as for navigation of autonomous vehicles.Type: ApplicationFiled: July 17, 2023Publication date: January 25, 2024Applicant: Magic Leap, Inc.Inventors: Lipu Zhou, Ashwin Swaminathan, Frank Thomas Steinbruecker, Daniel Esteban Koppel
-
Patent number: 11830149Abstract: A cross reality system enables any of multiple devices to efficiently access previously stored maps. Both stored maps and tracking maps used by portable devices may have any of multiple types of location metadata associated with them. The location metadata may be used to select a set of candidate maps for operations, such as localization or map merge, that involve finding a match between a location defined by location information from a portable device and any of a number of previously stored maps. The types of location metadata may prioritized for use in selecting the subset. To aid in selection of candidate maps, a universe of stored maps may be indexed based on geo-location information. A cross reality platform may update that index as it interacts with devices that supply geo-location information in connection with location information and may propagate that geo-location information to devices that do not supply it.Type: GrantFiled: February 11, 2021Date of Patent: November 28, 2023Assignee: Magic Leap, Inc.Inventors: Xuan Zhao, Christian Ivan Robert Moore, Sen Lin, Ali Shahrokni, Ashwin Swaminathan
-
Patent number: 11823450Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: GrantFiled: October 14, 2022Date of Patent: November 21, 2023Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Patent number: 11790619Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps of very large scale environments and render virtual content specified in relation to those maps. The cross reality system may build a persisted map, which may be in canonical form, by merging tracking maps from the multiple devices. A map merge process determines mergibility of a tracking map with a canonical map and merges a tracking map with a canonical map in accordance with mergibility criteria, such as, when a gravity direction of the tracking map aligns with a gravity direction of the canonical map. Refraining from merging maps if the orientation of the tracking map with respect to gravity is not preserved avoids distortions in persisted maps and results in multiple devices, which may use the maps to determine their locations, to present more realistic and immersive experiences for their users.Type: GrantFiled: July 1, 2022Date of Patent: October 17, 2023Assignee: Magic Leap, Inc.Inventors: Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez, Mukta Prasad, Eran Guendelman, Ali Shahrokni, Ashwin Swaminathan
-
Patent number: 11748905Abstract: A method of efficiently and accurately computing a pose of an image with respect to other image information. The image may be acquired with a camera on a portable device and the other information may be a map, such that the computation of pose localizes the device relative to the map. Such a technique may be applied in a cross reality system to enable devices to efficiently and accurately access previously persisted maps. Localizing with respect to a map may enable multiple cross reality devices to render virtual content at locations specified in relation to those maps, providing an enhanced experience for uses of the system. The method may be used in other devices and for other purposes, such as for navigation of autonomous vehicles.Type: GrantFiled: March 2, 2021Date of Patent: September 5, 2023Assignee: Magic Leap, Inc.Inventors: Lipu Zhou, Ashwin Swaminathan, Frank Thomas Steinbruecker, Daniel Esteban Koppel
-
Publication number: 20230127303Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.Type: ApplicationFiled: December 20, 2022Publication date: April 27, 2023Applicant: Magic Leap, Inc.Inventors: Ali Shahrokni, Daniel Olshansky, Xuan Zhao, Rafael Domingos Torres, Joel David Holder, Keng-Sheng Lin, Ashwin Swaminathan, Anush Mohan
-
Publication number: 20230119217Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.Type: ApplicationFiled: December 7, 2022Publication date: April 20, 2023Applicant: Magic Leap, Inc.Inventors: Miguel Andres Granados Velasquez, Javier Victorio Gomez Gonzalez, Danying Hu, Eran Guendelman, Ali Shahrokni, Ashwin Swaminathan, Mukta Prasad
-
Publication number: 20230108794Abstract: A portable electronic system receives a set of one or more canonical maps and determines the sparse map based at least in part upon one or more anchors pertaining to the physical environment. The sparse map is localized to at least one canonical map in the set of one or more canonical maps, and a new canonical map is created at least by merging sparse map data of the sparse map into the at least one canonical map. The set of one or more canonical maps may be determined from a universe of canonical maps comprising a plurality of canonical maps by applying a hierarchical filtering scheme to the universe. The sparse map may be localized to the at least one canonical map at least by splitting the sparse map into a plurality of connected components and by one or more merger operations.Type: ApplicationFiled: November 17, 2022Publication date: April 6, 2023Applicant: MAGIC LEAP, INC.Inventors: Moshe BOUHNIK, Ben WEISBIH, Miguel Andres GRANADOS VELASQUEZ, Ali SHAHROKNI, Ashwin SWAMINATHAN
-
Publication number: 20230075085Abstract: One or more computing devices, systems, and/or methods for end-to-end encryption for multiple recipient devices are provided. A first registration, comprising a first device public key, is created for a first device. A second registration, comprising a second device public key, is created for a second device. A first notify message of the second registration and second device public key is provided to the first device. A second notify message is provided to the second device of the first registration and first device public key. A secure communication invite is routed to the first device. An encrypted message, comprising a first device private key, is routed from the first device to the second device. End-to-end encrypted communication between a sender device and the first device and the second device using the first device private key is facilitated.Type: ApplicationFiled: September 3, 2021Publication date: March 9, 2023Inventors: Andrew Wahlon LAM, Ashwin SWAMINATHAN, Wen Zea Kuo, Rezwanul AZIM
-
Publication number: 20230034363Abstract: To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.Type: ApplicationFiled: October 14, 2022Publication date: February 2, 2023Inventors: Martin Georg Zahnert, Joao Antonio Pereira Faro, Miguel Andres Granados Velasquez, Dominik Michael Kasper, Ashwin Swaminathan, Anush Mohan, Prateek Singhal
-
Patent number: 11567950Abstract: A confidentiality preserving system and method for performing a rank-ordered search and retrieval of contents of a data collection. The system includes at least one computer system including a search and retrieval algorithm using term frequency and/or similar features for rank-ordering selective contents of the data collection, and enabling secure retrieval of the selective contents based on the rank-order. The search and retrieval algorithm includes a baseline algorithm, a partially server oriented algorithm, and/or a fully server oriented algorithm. The partially and/or fully server oriented algorithms use homomorphic and/or order preserving encryption for enabling search capability from a user other than an owner of the contents of the data collection. The confidentiality preserving method includes using term frequency for rank-ordering selective contents of the data collection, and retrieving the selective contents based on the rank-order.Type: GrantFiled: December 4, 2020Date of Patent: January 31, 2023Assignee: UNIVERSITY OF MARYLAND, COLLEGE PARKInventors: Ashwin Swaminathan, Yinian Mao, Guan-Ming Su, Hongmei Gou, Avinash Varna, Shan He, Min Wu, Douglas W. Oard
-
Patent number: 11568605Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.Type: GrantFiled: October 15, 2020Date of Patent: January 31, 2023Assignee: Magic Leap, Inc.Inventors: Ali Shahrokni, Daniel Olshansky, Xuan Zhao, Rafael Domingos Torres, Joel David Holder, Keng-Sheng Lin, Ashwin Swaminathan, Anush Mohan