Patents by Inventor Thorsten Gernoth
Thorsten Gernoth has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11915460Abstract: A device implementing a system for providing predicted RGB images includes at least one processor configured to obtain an infrared image of a subject, and to obtain a reference RGB image of the subject. The at least one processor is further configured to provide the infrared image and the reference RGB image to a machine learning model, the machine learning model having been trained to output predicted RGB images of subjects based on infrared images and reference RGB images of the subjects. The at least one processor is further configured to provide a predicted RGB image of the subject based on output by the machine learning model.Type: GrantFiled: July 7, 2022Date of Patent: February 27, 2024Assignee: Apple Inc.Inventors: Carlos E. Guestrin, Leon A. Gatys, Shreyas V. Joshi, Gustav M. Larsson, Kory R. Watson, Srikrishna Sridhar, Karla P. Vega, Shawn R. Scully, Thorsten Gernoth, Onur C Hamsici
-
Publication number: 20240062488Abstract: Various implementations disclosed herein include devices, systems, and methods that generates a three-dimensional (3D) model of an object based on images and tracked positions of a device during acquisition of the images. For example, an example process may include acquiring sensor data during movement of the device in a physical environment including an object, the sensor data including images of a physical environment acquired via a camera on the device, identifying the object in at least some of the images, tracking positions of the device during acquisition of the images based on identifying the object in the at least some of the images, the positions identifying positioning of the device with respect to a coordinate system defined based on a position and orientation of the object, and generating a 3D model of the object based on the images and positions of the device during acquisition of the images.Type: ApplicationFiled: November 1, 2023Publication date: February 22, 2024Inventors: Thorsten Gernoth, Chen Huang, Onur C. Hamsici, Shuo Feng, Hao Tang, Tobias Rick
-
Patent number: 11868455Abstract: Techniques are disclosed relating to biometric authentication, e.g., facial recognition. In some embodiments, a device is configured to verify that image data from a camera unit exhibits a pseudo-random sequence of image capture modes and/or a probing pattern of illumination points (e.g., from lasers in a depth capture mode) before authenticating a user based on recognizing a face in the image data. In some embodiments, a secure circuit may control verification of the sequence and/or the probing pattern. In some embodiments, the secure circuit may verify frame numbers, signatures, and/or nonce values for captured image information. In some embodiments, a device may implement one or more lockout procedures in response to biometric authentication failures. The disclosed techniques may reduce or eliminate the effectiveness of spoofing and/or replay attacks, in some embodiments.Type: GrantFiled: February 22, 2021Date of Patent: January 9, 2024Assignee: Apple Inc.Inventors: Deepti S. Prakash, Lucia E. Ballard, Jerrold V. Hauck, Feng Tang, Etai Littwin, Pavan Kumar Anasosalu Vasu, Gideon Littwin, Thorsten Gernoth, Lucie Kucerova, Petr Kostka, Steven P. Hotelling, Eitan Hirsh, Tal Kaitz, Jonathan Pokrass, Andrei Kolin, Moshe Laifenfeld, Matthew C. Waldon, Thomas P. Mensch, Lynn R. Youngs, Christopher G. Zeleznik, Michael R. Malone, Ziv Hendel, Ivan Krstic, Anup K. Sharma
-
Publication number: 20230377299Abstract: Generating a three-dimensional virtual representation of a three-dimensional physical object can be based on capturing or receiving a capture bundle or a set of images. In some examples, generating the virtual representation of the physical object can be facilitated by user interfaces for identifying a physical object and capturing a set of images of the physical object. Generating the virtual representation can include previewing or modifying a set of images. In some examples, generating the virtual representation of the physical object can include generating a first representation of the physical object (e.g., a point cloud) and/or generating a second three-dimensional virtual representation of the physical object (e.g., a mesh reconstruction). In some examples, a visual indication of the progress of the image capture process and/or the generation of the virtual representation of the three-dimensional object can be displayed, such as in a capture user interface.Type: ApplicationFiled: May 15, 2023Publication date: November 23, 2023Inventors: Zachary Z. BECKER, Michelle CHUA, Thorsten GERNOTH, Michael P. JOHNSON
-
Publication number: 20230377259Abstract: Generating a three-dimensional virtual representation of a three-dimensional physical object can be based on capturing or receiving a capture bundle or a set of images. In some examples, generating the virtual representation of the physical object can be facilitated by user interfaces for identifying a physical object and capturing a set of images of the physical object. Generating the virtual representation can include previewing or modifying a set of images. In some examples, generating the virtual representation of the physical object can include generating a first representation of the physical object (e.g., a point cloud) and/or generating a second three-dimensional virtual representation of the physical object (e.g., a mesh reconstruction). In some examples, a visual indication of the progress of the image capture process and/or the generation of the virtual representation of the three-dimensional object can be displayed, such as in a capture user interface.Type: ApplicationFiled: May 15, 2023Publication date: November 23, 2023Inventors: Zachary Z. BECKER, Michelle CHUA, Thorsten GERNOTH, Michael P. JOHNSON, Allison W. DRYER
-
Publication number: 20230377300Abstract: Generating a three-dimensional virtual representation of a three-dimensional physical object can be based on capturing or receiving a capture bundle or a set of images. In some examples, generating the virtual representation of the physical object can be facilitated by user interfaces for identifying a physical object and capturing a set of images of the physical object. Generating the virtual representation can include previewing or modifying a set of images. In some examples, generating the virtual representation of the physical object can include generating a first representation of the physical object (e.g., a point cloud) and/or generating a second three-dimensional virtual representation of the physical object (e.g., a mesh reconstruction). In some examples, a visual indication of the progress of the image capture process and/or the generation of the virtual representation of the three-dimensional object can be displayed, such as in a capture user interface.Type: ApplicationFiled: May 15, 2023Publication date: November 23, 2023Inventors: Zachary Z. BECKER, Michelle CHUA, Thorsten GERNOTH, Michael P. JOHNSON, Allison W. DRYER
-
Patent number: 11693937Abstract: An operation of a facial recognition authentication process may fail to authenticate a user even if the user is an authorized user of the device. In such cases, the facial recognition authentication process may automatically re-initiate to provide another attempt to authenticate the user using additional captured images. For the new attempt (e.g., the retry) to authenticate the user, one or more criteria for the images used in the facial recognition authentication process may be adjusted. For example, criteria for distance between the camera and the user's face and/or occlusion of the user's face in the images may be adjusted before the new attempt to authenticate the user. Adjustment of these criteria may increase the likelihood that the authorized user will be successfully authenticated in the new attempt.Type: GrantFiled: April 26, 2019Date of Patent: July 4, 2023Assignee: Apple Inc.Inventors: Marcel Van Os, Thorsten Gernoth, Kelsey Y. Ho
-
Publication number: 20230096119Abstract: Various implementations disclosed herein provide feedback to a user during object scanning based on how well sensor data of the scanned object has been captured. During object scanning a user may move an electronic device with sensors (e.g., cameras, depth sensors, etc.) around an object to capture sensor data for use in generating a final 3D model of the object. Live feedback during the scanning process is enabled by assessing how well the captured sensor data represents different portions of the object. In some implementations, a 3D model is generated, updated, and assessed based on the sensor data live during the scanning process. This live 3D model may be coarser (i.e., having fewer details) than the final 3D model.Type: ApplicationFiled: September 15, 2022Publication date: March 30, 2023Inventors: Thorsten Gernoth, Cheng Lu, Hao Tang, Michael P. Johnson
-
Patent number: 11561621Abstract: Intelligent systems are disclosed that respond to user intent and desires based upon activity that may or may not be expressly directed at the intelligent system. In some embodiments, the intelligent system acquires a depth image of a scene surrounding the system. A scene geometry may be extracted from the depth image and elements of the scene may be monitored. In certain embodiments, user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system. The interpretation of the user's intent as well as the system's response may be affected by the scene geometry surrounding the user and/or the system. In some embodiments, techniques and systems are disclosed for interpreting express user communication, e.g., expressed through hand gesture movements. In some embodiments, such gesture movements may be interpreted based on real-time depth information obtained from, e.g., optical or non-optical type depth sensors.Type: GrantFiled: October 14, 2019Date of Patent: January 24, 2023Assignee: Apple Inc.Inventors: Feng Tang, Chong Chen, Haitao Guo, Xiaojin Shi, Thorsten Gernoth
-
Publication number: 20220414543Abstract: A device implementing a system for providing predicted RGB images includes at least one processor configured to obtain an infrared image of a subject, and to obtain a reference RGB image of the subject. The at least one processor is further configured to provide the infrared image and the reference RGB image to a machine learning model, the machine learning model having been trained to output predicted RGB images of subjects based on infrared images and reference RGB images of the subjects. The at least one processor is further configured to provide a predicted RGB image of the subject based on output by the machine learning model.Type: ApplicationFiled: July 7, 2022Publication date: December 29, 2022Inventors: Carlos E. GUESTRIN, Leon A. GATYS, Shreyas V. JOSHI, Gustav M. LARSSON, Kory R. WATSON, Srikrishna SRIDHAR, Karla P. VEGA, Shawn R. SCULLY, Thorsten GERNOTH, Onur C HAMSICI
-
Patent number: 11521423Abstract: Occlusion of facial features may be detected and assessed in an image captured by a camera on a device. Landmark heat maps may be used to estimate the location of landmarks such as the eyes, mouth, and nose of a user's face in the captured image. An occlusion heat map may also be generated for the captured image. The occlusion heat map may include values representing the amount of occlusion in regions of the face. The estimated locations of the eyes, mouth, and nose may be used in combination with the occlusion heat map to assess occlusion scores for the landmarks. The occlusion scores for the landmarks may be used control one or more operations of the device.Type: GrantFiled: January 15, 2021Date of Patent: December 6, 2022Assignee: Apple Inc.Inventors: Thorsten Gernoth, Ian R. Fasel, Touraj Tajbakhsh, Jia-Chi Wu
-
Patent number: 11386355Abstract: A device implementing a system for providing predicted RGB images includes at least one processor configured to obtain an infrared image of a subject, and to obtain a reference RGB image of the subject. The at least one processor is further configured to provide the infrared image and the reference RGB image to a machine learning model, the machine learning model having been trained to output predicted RGB images of subjects based on infrared images and reference RGB images of the subjects. The at least one processor is further configured to provide a predicted RGB image of the subject based on output by the machine learning model.Type: GrantFiled: December 6, 2019Date of Patent: July 12, 2022Assignee: Apple Inc.Inventors: Carlos E. Guestrin, Leon A. Gatys, Shreyas V. Joshi, Gustav M. Larsson, Kory R. Watson, Srikrishna Sridhar, Karla P. Vega, Shawn R. Scully, Thorsten Gernoth, Onur C Hamsici
-
Patent number: 11151235Abstract: Techniques are disclosed relating to biometric authentication, e.g., facial recognition. In some embodiments, a device is configured to verify that image data from a camera unit exhibits a pseudo-random sequence of image capture modes and/or a probing pattern of illumination points (e.g., from lasers in a depth capture mode) before authenticating a user based on recognizing a face in the image data. In some embodiments, a secure circuit may control verification of the sequence and/or the probing pattern. In some embodiments, the secure circuit may verify frame numbers, signatures, and/or nonce values for captured image information. In some embodiments, a device may implement one or more lockout procedures in response to biometric authentication failures. The disclosed techniques may reduce or eliminate the effectiveness of spoofing and/or replay attacks, in some embodiments.Type: GrantFiled: July 31, 2018Date of Patent: October 19, 2021Assignee: Apple Inc.Inventors: Deepti S. Prakash, Lucia E. Ballard, Jerrold V. Hauck, Feng Tang, Etai Littwin, Pavan Kumar Anasosalu Vasu, Gideon Littwin, Thorsten Gernoth, Lucie Kucerova, Petr Kostka, Steven P. Hotelling, Eitan Hirsh, Tal Kaitz, Jonathan Pokrass, Andrei Kolin, Moshe Laifenfeld, Matthew C. Waldon, Thomas P. Mensch, Lynn R. Youngs, Christopher G. Zeleznik, Michael R. Malone, Ziv Hendel, Ivan Krstic, Anup K. Sharma, Kelsey Y. Ho
-
Publication number: 20210286865Abstract: Techniques are disclosed relating to biometric authentication, e.g., facial recognition. In some embodiments, a device is configured to verify that image data from a camera unit exhibits a pseudo-random sequence of image capture modes and/or a probing pattern of illumination points (e.g., from lasers in a depth capture mode) before authenticating a user based on recognizing a face in the image data. In some embodiments, a secure circuit may control verification of the sequence and/or the probing pattern. In some embodiments, the secure circuit may verify frame numbers, signatures, and/or nonce values for captured image information. In some embodiments, a device may implement one or more lockout procedures in response to biometric authentication failures. The disclosed techniques may reduce or eliminate the effectiveness of spoofing and/or replay attacks, in some embodiments.Type: ApplicationFiled: February 22, 2021Publication date: September 16, 2021Inventors: Deepti S. Prakash, Lucia E. Ballard, Jerrold V. Hauck, Feng Tang, Etai Littwin, Pavan Kumar Ansosalu Vasu, Gideon Littwin, Thorsten Gernoth, Lucie Kucerova, Petr Kostka, Steven P. Hotelling, Eitan Hirsh, Tal Kaitz, Jonathan Pokrass, Andrei Kolin, Moshe Laifenfeld, Matthew C. Waldon, Thomas P. Mensch, Lynn R. Youngs, Christopher G. Zeleznik, Michael R. Malone, Ziv Hendel, Ivan Krstic, Anup K. Sharma
-
Publication number: 20210279967Abstract: Various implementations disclosed herein include devices, systems, and methods that generates a three-dimensional (3D) model of an object based on images and tracked positions of a device during acquisition of the images. For example, an example process may include acquiring sensor data during movement of the device in a physical environment including an object, the sensor data including images of a physical environment acquired via a camera on the device, identifying the object in at least some of the images, tracking positions of the device during acquisition of the images based on identifying the object in the at least some of the images, the positions identifying positioning of the device with respect to a coordinate system defined based on a position and orientation of the object, and generating a 3D model of the object based on the images and positions of the device during acquisition of the images.Type: ApplicationFiled: February 19, 2021Publication date: September 9, 2021Inventors: Thorsten Gernoth, Chen Huang, Onur C. Hamsici, Shuo Feng, Hao Tang, Tobias Rick
-
Publication number: 20210133428Abstract: Occlusion of facial features may be detected and assessed in an image captured by a camera on a device. Landmark heat maps may be used to estimate the location of landmarks such as the eyes, mouth, and nose of a user's face in the captured image. An occlusion heat map may also be generated for the captured image. The occlusion heat map may include values representing the amount of occlusion in regions of the face. The estimated locations of the eyes, mouth, and nose may be used in combination with the occlusion heat map to assess occlusion scores for the landmarks. The occlusion scores for the landmarks may be used control one or more operations of the device.Type: ApplicationFiled: January 15, 2021Publication date: May 6, 2021Inventors: Thorsten Gernoth, Ian R. Fasel, Touraj Tajbakhsh, Jia-Chi Wu
-
Patent number: 10929515Abstract: Techniques are disclosed relating to biometric authentication, e.g., facial recognition. In some embodiments, a device is configured to verify that image data from a camera unit exhibits a pseudo-random sequence of image capture modes and/or a probing pattern of illumination points (e.g., from lasers in a depth capture mode) before authenticating a user based on recognizing a face in the image data. In some embodiments, a secure circuit may control verification of the sequence and/or the probing pattern. In some embodiments, the secure circuit may verify frame numbers, signatures, and/or nonce values for captured image information. In some embodiments, a device may implement one or more lockout procedures in response to biometric authentication failures. The disclosed techniques may reduce or eliminate the effectiveness of spoofing and/or replay attacks, in some embodiments.Type: GrantFiled: July 31, 2018Date of Patent: February 23, 2021Assignee: Apple Inc.Inventors: Deepti S. Prakash, Lucia E. Ballard, Jerrold V. Hauck, Feng Tang, Etai Littwin, Pavan Kumar Ansosalu Vasu, Gideon Littwin, Thorsten Gernoth, Lucie Kucerova, Petr Kostka, Steven P. Hotelling, Eitan Hirsh, Tal Kaitz, Jonathan Pokrass, Andrei Kolin, Moshe Laifenfeld, Matthew C. Waldon, Thomas P. Mensch, Lynn R. Youngs, Christopher G. Zeleznik, Michael R. Malone, Ziv Hendel, Ivan Krstic, Anup K. Sharma
-
Patent number: 10896318Abstract: Occlusion of facial features may be detected and assessed in an image captured by a camera on a device. Landmark heat maps may be used to estimate the location of landmarks such as the eyes, mouth, and nose of a user's face in the captured image. An occlusion heat map may also be generated for the captured image. The occlusion heat map may include values representing the amount of occlusion in regions of the face. The estimated locations of the eyes, mouth, and nose may be used in combination with the occlusion heat map to assess occlusion scores for the landmarks. The occlusion scores for the landmarks may be used control one or more operations of the device.Type: GrantFiled: March 23, 2018Date of Patent: January 19, 2021Assignee: Apple Inc.Inventors: Thorsten Gernoth, Ian R. Fasel, Touraj Tajbakhsh, Jia-Chi Wu
-
Patent number: 10769414Abstract: A neural network may implement a face detection process on a mobile device or computer system. An image captured using a camera on the device may be processed using the face detection process. The face detection process may provide a bounding box for the face detected in the image. The face detected in the image may have any orientation. The face detected in the image may include either a whole face of the user or only a partial face of the user. The face detection process may provide face detection that is substantially invariant to exposure in the captured image. The face may also be detected over a wide range of distances between the camera and the face of the user.Type: GrantFiled: August 31, 2018Date of Patent: September 8, 2020Assignee: Apple Inc.Inventors: Atulit Kumar, Thorsten Gernoth, Feng Tang
-
Patent number: 10762335Abstract: Detection of a user paying attention to a device may be used to enable or support biometric security (e.g., facial recognition) enabled features on the device. Images captured by a camera on the device may be used to determine if the user is paying attention to the device. Facial features of the user's face in the images may be assessed to determine if the user is paying attention to the device. Facial features may be assessed through comparison of feature vectors generated from the captured image feature to a set of known feature vectors. The known feature vectors for attention may be generated using a machine learning process.Type: GrantFiled: March 23, 2018Date of Patent: September 1, 2020Assignee: Apple Inc.Inventors: Thorsten Gernoth, Joerg A. Liebelt, Onur C. Hamsici, Kelsey Y. Ho