Patents by Inventor Samuel Alan Johnson
Samuel Alan Johnson has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11972040Abstract: A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.Type: GrantFiled: January 11, 2023Date of Patent: April 30, 2024Assignee: Meta Platforms Technologies, LLCInventors: Samuel Alan Johnson, Shaik Shabnam Nizamudeen Basha, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux
-
Publication number: 20230400914Abstract: A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may determine contextual information associated with the user. The computing system may generate a lower body pose corresponding to a second portion of the body of the user comprising a leg of the user based on the upper body pose and the contextual information. The computing system may generate a full body pose of the user based on the first upper body pose and the lower body pose.Type: ApplicationFiled: August 23, 2023Publication date: December 14, 2023Inventors: Benjamin Antoine Georges Lefaudeux, Samuel Alan Johnson, Carsten Sebastian Stoll, Kishore Venkateshan
-
Patent number: 11809617Abstract: A system includes processing circuitry configured to receive sensor data regarding a user operating a head mounted display (HMD). The processing circuitry is configured to identify a plurality of reference points of a pose of the user based at least on the sensor data. The processing circuitry is configured to apply one or more models to the plurality of reference points to determine a type of the pose of the user. The processing circuitry is configured to select a mode of operation of the HMD responsive to the type of the pose.Type: GrantFiled: August 23, 2021Date of Patent: November 7, 2023Assignee: Meta Platforms Technologies, LLCInventors: Samuel Alan Johnson, Mahdi Salmani Rahimi
-
Patent number: 11747892Abstract: A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate a first upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may process the upper body pose of the user to generate a temporal sequence of lower body poses comprising a first lower body pose associated with the first time and a second lower body pose associated with a second time, which may correspond to a second portion of the body of the user comprising a leg of the user. The computing system may generate one or more full body poses of the user based on at least the first upper body pose and the temporal sequence of lower body poses.Type: GrantFiled: November 21, 2022Date of Patent: September 5, 2023Assignee: Meta Platforms Technologies, LLCInventors: Benjamin Antoine Georges Lefaudeux, Samuel Alan Johnson, Carsten Sebastian Stoll, Kishore Venkateshan
-
Publication number: 20230245499Abstract: A computing system may receive an image captured by a camera, and determine based on (1) one or more regions in the image that include a portion of a body of a user, and (2) a camera pose of the camera, a three-dimensional volume constraint that stems from the camera pose in which an elbow of the user is likely to be located. The system may infer, based on at least the three-dimensional volume constraint, an elbow pose of the user, wherein the elbow pose includes a location associated with an elbow of the user in three-dimensional space. The system may determine the location associated with the elbow is outside the three-dimensional volume constraint, and in response to the determination, adjust the inferred elbow pose. The system may generate, based on the adjusted elbow pose, a body pose of the user that includes at least the adjusted elbow pose.Type: ApplicationFiled: April 6, 2023Publication date: August 3, 2023Inventors: Samuel Alan Johnson, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux, Kishore Venkateshan
-
Publication number: 20230168734Abstract: A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.Type: ApplicationFiled: January 11, 2023Publication date: June 1, 2023Inventors: Samuel Alan JOHNSON, Shaik Shabnam NIZAMUDEEN BASHA, Mahdi SALMANI RAHIMI, Benjamin Antoine Georges LEFAUDEUX
-
Patent number: 11651625Abstract: A computing system may determine a head pose and a wrist pose of a user based on sensor data captured by a head-mounted device. The computing system may further receive an image captured by a camera of the head-mounted device that includes at least portion of the body of the user. Using this image, the computing system may determine one or more regions in the image that correspond to the body of the user, and determine, based on these regions and a camera pose of the camera, a three-dimensional volume constraint in which an elbow of the user is likely to be located when the image was captured. From this information, the computing system may utilize the head pose, the wrist pose, and the three-dimensional volume constraint to infer a body pose of the user that includes at least an inferred elbow pose associated with the user.Type: GrantFiled: September 17, 2020Date of Patent: May 16, 2023Assignee: Meta Platforms Technologies, LLCInventors: Samuel Alan Johnson, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux, Kishore Venkateshan
-
Patent number: 11609625Abstract: A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.Type: GrantFiled: October 13, 2021Date of Patent: March 21, 2023Assignee: Meta Platforms Technologies, LLCInventors: Samuel Alan Johnson, Shaik Shabnam Nizamudeen Basha, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux
-
Publication number: 20230077772Abstract: A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate a first upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may process the upper body pose of the user to generate a temporal sequence of lower body poses comprising a first lower body pose associated with the first time and a second lower body pose associated with a second time, which may correspond to a second portion of the body of the user comprising a leg of the user. The computing system may generate one or more full body poses of the user based on at least the first upper body pose and the temporal sequence of lower body poses.Type: ApplicationFiled: November 21, 2022Publication date: March 16, 2023Inventors: Benjamin Antoine Georges Lefaudeux, Samuel Alan Johnson, Carsten Sebastian Stoll, Kishore Venkateshan
-
Publication number: 20220413434Abstract: A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.Type: ApplicationFiled: June 28, 2021Publication date: December 29, 2022Inventors: Albert PARRA POZO, Joseph VIRSKUS, Ganesh VENKATESH, Kai LI, Shen-Chi CHEN, Amit KUMAR, Rakesh RANJAN, Brian Keith CABRAL, Samuel Alan JOHNSON, Wei YE, Michael Alexander SNOWER, Yash PATEL
-
Publication number: 20220413433Abstract: A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.Type: ApplicationFiled: June 28, 2021Publication date: December 29, 2022Inventors: Albert PARRA POZO, Joseph VIRSKUS, Ganesh VENKATESH, Kai LI, Shen-Chi CHEN, Amit KUMAR, Rakesh RANJAN, Brian Keith CABRAL, Samuel Alan JOHNSON, Wei YE, Michael Alexander SNOWER, Yash PATEL
-
Patent number: 11507179Abstract: A computing system may receive sensor data from one or more sensors coupled to a user. Based on this sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which may comprise a head and an arm of the user. The computing system may process the upper body pose of the user using a machine learning model to generate a lower body pose that corresponds to a second portion of the body of the user, which may comprise a leg of the user. The computing system may generate a full body pose of the user based on the upper body pose and the lower body pose.Type: GrantFiled: September 17, 2020Date of Patent: November 22, 2022Assignee: Meta Platforms Technologies, LLCInventors: Benjamin Antoine Georges Lefaudeux, Samuel Alan Johnson, Carsten Sebastian Stoll, Kishore Venkatehsan
-
Patent number: 11461962Abstract: A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.Type: GrantFiled: June 28, 2021Date of Patent: October 4, 2022Assignee: Meta Platforms Technologies, LLCInventors: Albert Parra Pozo, Joseph Virskus, Ganesh Venkatesh, Kai Li, Shen-Chi Chen, Amit Kumar, Rakesh Ranjan, Brian Keith Cabral, Samuel Alan Johnson, Wei Ye, Michael Alexander Snower, Yash Patel
-
Publication number: 20220083768Abstract: A computing system may determine a head pose and a wrist pose of a user based on sensor data captured by a head-mounted device. The computing system may further receive an image captured by a camera of the head-mounted device that includes at least portion of the body of the user. Using this image, the computing system may determine one or more regions in the image that correspond to the body of the user, and determine, based on these regions and a camera pose of the camera, a three-dimensional volume constraint in which an elbow of the user is likely to be located when the image was captured. From this information, the computing system may utilize the head pose, the wrist pose, and the three-dimensional volume constraint to infer a body pose of the user that includes at least an inferred elbow pose associated with the user.Type: ApplicationFiled: September 17, 2020Publication date: March 17, 2022Inventors: Samuel Alan Johnson, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux, Kishore Venkateshan
-
Publication number: 20220083125Abstract: A computing system may receive sensor data from one or more sensors coupled to a user. Based on this sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which may comprise a head and an arm of the user. The computing system may process the upper body pose of the user using a machine learning model to generate a lower body pose that corresponds to a second portion of the body of the user, which may comprise a leg of the user. The computing system may generate a full body pose of the user based on the upper body pose and the lower body pose.Type: ApplicationFiled: September 17, 2020Publication date: March 17, 2022Inventors: Benjamin Antoine Georges Lefaudeux, Samuel Alan Johnson, Carsten Sebastian Stoll, Kishore Venkatehsan
-
Publication number: 20220035444Abstract: A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.Type: ApplicationFiled: October 13, 2021Publication date: February 3, 2022Inventors: Samuel Alan JOHNSON, Shaik Shabnam NIZAMUDEEN BASHA, Mahdi SALMANI RAHIMI, Benjamin Antoine Georges LEFAUDEUX
-
Patent number: 11200745Abstract: In particular embodiments, a computing system may display a virtual-reality scene on a device worn by a user. The system may capture an image of a physical environment surrounding the user and detect an object of interest in the image. The system may determine, based on the detected object, that one or more alert criteria are satisfied. The system may generate depth measurements of at least the object. The system may generate, based on the depth measurements, a model representing the object. The system may render, based on a viewpoint of the user and the model, an output image depicting a visualization of the object. The system may display the output image on the device worn by the user in response to the determination that the one or more alert criteria are satisfied.Type: GrantFiled: January 27, 2020Date of Patent: December 14, 2021Assignee: Facebook Technologies, LLC.Inventors: Samuel Alan Johnson, Mahdi Salmani Rahimi, Shaik Shabnam Nizamudeen Basha, Belinda Heywood
-
Publication number: 20210382547Abstract: A system includes processing circuitry configured to receive sensor data regarding a user operating a head mounted display (HMD). The processing circuitry is configured to identify a plurality of reference points of a pose of the user based at least on the sensor data. The processing circuitry is configured to apply one or more models to the plurality of reference points to determine a type of the pose of the user. The processing circuitry is configured to select a mode of operation of the HMD responsive to the type of the pose.Type: ApplicationFiled: August 23, 2021Publication date: December 9, 2021Inventors: Samuel Alan Johnson, Mahdi Salmani Rahimi
-
Patent number: 11175730Abstract: A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.Type: GrantFiled: December 6, 2019Date of Patent: November 16, 2021Assignee: Facebook Technologies, LLCInventors: Samuel Alan Johnson, Shaik Shabnam Nizamudeen Basha, Mahdi Salmani Rahimi, Benjamin Antoine Georges Lefaudeux
-
Patent number: 11156830Abstract: Artificial reality (AR) systems track pose and skeletal positioning for multiple co-located participants, each having a head mounted display (HMD). Participants can join a shared artificial reality event or experience with others in the same location. Each participant's HMD can independently render AR content for the participant based on the participant's pose and pose information obtained from other participants' HMDs. A participating HMD may broadcast tracking estimates for skeletal points of interest (e.g., joints, finger tips, knees, ankle points, etc.) that are within the field-of-view of the HMD's cameras and/or sensors. A participating HMD may receive skeletal position information determined by other HMDs, and aggregate the received tracking information along with internal tracking information to construct an accurate, full estimate of its own pose and skeletal positioning information for its corresponding participant.Type: GrantFiled: November 14, 2019Date of Patent: October 26, 2021Assignee: Facebook Technologies, LLCInventors: Samuel Alan Johnson, Carsten Sebastian Stoll, Benjamin Antoine Georges Lefaudeux