Patents by Inventor Charles MAALOUF
Charles MAALOUF has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240103633Abstract: Embodiments are disclosed for hold gesture recognition using machine learning (ML). In an embodiment, a method comprises: receiving sensor signals indicative of a hand gesture made by a user, the sensor data obtained from at least one sensor of a wearable device worn by the user; generating a first embedding of first features extracted from the sensor signals; predicting a first part of a hold gesture based on a first ML gesture classifier and the first embedding; generating a second embedding of second features extracted from the sensor signals; predicting a second part of the hold gesture based on a second ML gesture classifier and the second embedding; predicting a hold gesture based at least in part on outputs of the first and second ML gesture classifiers and a prediction policy; and performing an action on the wearable device or other device based on the predicted hold gesture.Type: ApplicationFiled: September 20, 2023Publication date: March 28, 2024Inventors: Bongsoo Suh, Behrooz Shahsavari, Charles Maalouf, Hojjat Seyed Mousavi, Laurence Lindsey, Shivam Kumar Gupta
-
Publication number: 20240094819Abstract: In some embodiments, the present disclosure includes techniques and user interfaces for performing operations using air gestures. In some embodiments, the present disclosure includes techniques and user interfaces for audio playback adjustment using gestures. In some embodiments, the present disclosure includes techniques and user interfaces for conditionally responding to inputs.Type: ApplicationFiled: September 6, 2023Publication date: March 21, 2024Inventors: Yiqiang NIE, Giovanni M. AGNOLI, Allison W. DRYER, Jules K. FENNIS, Charles MAALOUF, Camille MOUSSETTE, Giancarlo YERKES
-
Patent number: 11907436Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.Type: GrantFiled: September 16, 2022Date of Patent: February 20, 2024Assignee: Apple Inc.Inventors: Stephen O. Lemay, Michael R. Bastian, Roman Holenstein, Minwoo Jeong, Charles Maalouf, Brandon J. Newendorp, Heriberto Nieto, Timothy Paek, Joanna Peterson, Shawn Scully, Srikrishna Sridhar, Brandt M. Westing, Shiwen Zhao
-
Publication number: 20230376193Abstract: The present disclosure generally relates to displaying user interfaces with device controls.Type: ApplicationFiled: May 15, 2023Publication date: November 23, 2023Inventors: Elizabeth HAN, Joanna ARREAZA-TAYLOR, Hannah G. COLEMAN, Caroline J. CRANDALL, Christopher B. FLEIZACH, Charles MAALOUF, Tu K. NGUYEN, Jennifer D. PATTON
-
Publication number: 20230325719Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.Type: ApplicationFiled: May 26, 2023Publication date: October 12, 2023Inventors: Charles MAALOUF, Shawn R. SCULLY, Christopher B. FLEIZACH, Tu K. NGUYEN, Lilian H. LIANG, Warren J. SETO, Julian QUINTANA, Michael J. BEYHS, Hojjat SEYED MOUSAVI, Behrooz SHAHSAVARI
-
Patent number: 11699104Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.Type: GrantFiled: July 20, 2022Date of Patent: July 11, 2023Assignee: Apple Inc.Inventors: Charles Maalouf, Shawn R. Scully, Christopher B. Fleizach, Tu K. Nguyen, Lilian H. Liang, Warren J. Seto, Julian Quintana, Michael J. Beyhs, Hojjat Seyed Mousavi, Behrooz Shahsavari
-
Publication number: 20230195237Abstract: The present disclosure generally relates to navigating user interfaces using hand gestures.Type: ApplicationFiled: February 14, 2023Publication date: June 22, 2023Inventors: Tu K. NGUYEN, James N. CARTWRIGHT, Elizabeth C. CRANFILL, Christopher B. FLEIZACH, Joshua R. FORD, Jeremiah R. JOHNSON, Charles MAALOUF, Heriberto NIETO, Jennifer D. PATTON, Hojjat SEYED MOUSAVI, Shawn R. SCULLY, Ibrahim G. YUSUF
-
Publication number: 20230160922Abstract: Individual health related events (e.g., handwashing events) can be detected based on multiple sensors including motion and audio sensors. Detecting a qualifying handwashing event can include detecting a qualifying scrubbing event based on motion data (e.g., accelerometer data) and a qualifying rinsing event based on audio data. In some examples, power consumption can be reduced by implementing one or more power saving mitigations.Type: ApplicationFiled: January 23, 2023Publication date: May 25, 2023Inventors: Gierad LAPUT, Jared LeVan ZERBE, William C. ATHAS, Andreas Edgar SCHOBEL, Shawn R. SCULLY, Brian H. TSANG, Kevin LYNCH, Charles MAALOUF, Shiwen ZHAO
-
Patent number: 11639944Abstract: Individual health related events (e.g., handwashing events) can be detected based on multiple sensors including motion and audio sensors. Detecting a qualifying handwashing event can include detecting a qualifying scrubbing event based on motion data (e.g., accelerometer data) and a qualifying rinsing event based on audio data. In some examples, power consumption can be reduced by implementing one or more power saving mitigations.Type: GrantFiled: August 14, 2020Date of Patent: May 2, 2023Assignee: Apple Inc.Inventors: Gierad Laput, Jared LeVan Zerbe, William C. Athas, Andreas Edgar Schobel, Shawn R. Scully, Brian H. Tsang, Kevin Lynch, Charles Maalouf, Shiwen Zhao
-
Publication number: 20230040703Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.Type: ApplicationFiled: September 16, 2022Publication date: February 9, 2023Inventors: Stephen O. LEMAY, Michael R. BASTIAN, Roman HOLENSTEIN, Minwoo JEONG, Charles MAALOUF, Brandon J. NEWENDORP, Heriberto NIETO, Timothy PAEK, Joanna PETERSON, Shawn SCULLY, Srikrishna SRIDHAR, Brandt M. WESTING, Shiwen ZHAO
-
Patent number: 11561239Abstract: Individual health related events (e.g., handwashing events) can be detected based on multiple sensors including motion and audio sensors. Detecting a qualifying handwashing event can include detecting a qualifying scrubbing event based on motion data (e.g., accelerometer data) and a qualifying rinsing event based on audio data. In some examples, power consumption can be reduced by implementing one or more power saving mitigations.Type: GrantFiled: August 14, 2020Date of Patent: January 24, 2023Assignee: Apple Inc.Inventors: Gierad Laput, Jared LeVan Zerbe, William C. Athas, Andreas Edgar Schobel, Shawn R. Scully, Brian H. Tsang, Kevin Lynch, Charles Maalouf, Shiwen Zhao
-
Publication number: 20220391697Abstract: Embodiments are disclosed for a machine learning (ML) gesture recognition with a framework for adding user-customized gestures. In an embodiment, a method comprises: receiving sensor data indicative of a gesture made by a user, the sensor data obtained from at least one sensor of a wearable device worn on a limb of the user; generating a current encoding of features extracted from the sensor data using a machine learning model with the features as input; generating similarity metrics between the current encoding and each encoding in a set of previously generated encodings for gestures; generating similarity scores based on the similarity metrics; predicting the gesture made by the user based on the similarity scores; and performing an action on the wearable device or other device based on the predicted gesture.Type: ApplicationFiled: May 9, 2022Publication date: December 8, 2022Inventors: Hojjat Seyed Mousavi, Behrooz Shahsavari, Nima Ferdosi, Charles Maalouf, Xuhai Xu
-
Publication number: 20220374085Abstract: The present disclosure generally relates to navigating user interfaces using hand gestures.Type: ApplicationFiled: May 18, 2022Publication date: November 24, 2022Inventors: Tu K. NGUYEN, James N. CARTWRIGHT, Elizabeth C. CRANFILL, Christopher B. FLEIZACH, Joshua R. FORD, Jeremiah R. JOHNSON, Charles MAALOUF, Heriberto NIETO, Jennifer D. PATTON, Hojjat SEYED MOUSAVI, Shawn R. SCULLY, Ibrahim G. YUSUF
-
Publication number: 20220351086Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.Type: ApplicationFiled: July 20, 2022Publication date: November 3, 2022Inventors: Charles MAALOUF, Shawn R. SCULLY, Christopher B. FLEIZACH, Tu K. NGUYEN, Lilian H. LIANG, Warren J. SETO, Julian QUINTANA, Michael J. BEYHS, Hojjat SEYED MOUSAVI, Behrooz SHAHSAVARI
-
Patent number: 11487364Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.Type: GrantFiled: September 29, 2021Date of Patent: November 1, 2022Assignee: Apple Inc.Inventors: Stephen O. Lemay, Michael R. Bastian, Roman Holenstein, Minwoo Jeong, Charles Maalouf, Brandon J. Newendorp, Heriberto Nieto, Timothy Paek, Joanna Peterson, Shawn Scully, Srikrishna Sridhar, Brandt M. Westing, Shiwen Zhao
-
Patent number: 11449802Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.Type: GrantFiled: July 23, 2020Date of Patent: September 20, 2022Assignee: Apple Inc.Inventors: Charles Maalouf, Shawn R. Scully, Christopher B. Fleizach, Tu K. Nguyen, Lilian H. Liang, Warren J. Seto, Julian Quintana, Michael J. Beyhs, Hojjat Seyed Mousavi, Behrooz Shahsavari
-
Publication number: 20220019292Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.Type: ApplicationFiled: September 29, 2021Publication date: January 20, 2022Inventors: Stephen O. LEMAY, Michael R. BASTIAN, Roman HOLENSTEIN, Minwoo JEONG, Charles MAALOUF, Brandon J. NEWENDORP, Heriberto NIETO, Timothy PAEK, Joanna PETERSON, Shawn SCULLY, Srikrishna SRIDHAR, Brandt M. WESTING, Shiwen ZHAO
-
Patent number: 11169616Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.Type: GrantFiled: September 25, 2020Date of Patent: November 9, 2021Assignee: Apple Inc.Inventors: Stephen O. Lemay, Michael R. Bastian, Roman Holenstein, Minwoo Jeong, Charles Maalouf, Brandon J. Newendorp, Heriberto Nieto, Timothy Paek, Joanna Peterson, Shawn Scully, Srikrishna Sridhar, Brandt M. Westing, Shiwen Zhao
-
Publication number: 20210142214Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.Type: ApplicationFiled: July 23, 2020Publication date: May 13, 2021Inventors: Charles MAALOUF, Shawn R. SCULLY, Christopher B. FLEIZACH, Tu K. NGUYEN, Lilian H. LIANG, Warren J. SETO, Julian QUINTANA, Michael J. BEYHS, Hojjat SEYED MOUSAVI, Behrooz SHAHSAVARI
-
Publication number: 20210063434Abstract: Individual health related events (e.g., handwashing events) can be detected based on multiple sensors including motion and audio sensors. Detecting a qualifying handwashing event can include detecting a qualifying scrubbing event based on motion data (e.g., accelerometer data) and a qualifying rinsing event based on audio data. In some examples, power consumption can be reduced by implementing one or more power saving mitigations.Type: ApplicationFiled: August 14, 2020Publication date: March 4, 2021Inventors: Gierad LAPUT, Jared LeVan ZERBE, William C. ATHAS, Andreas Edgar SCHOBEL, Shawn R. SCULLY, Brian H. TSANG, Kevin LYNCH, Charles MAALOUF, Shiwen ZHAO