Patents by Inventor Hrvoje Benko

Hrvoje Benko has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11334157
    Abstract: A wearable device and system for detecting data generated by a user's contact with a surface and using that data to determine that a user contacted the surface and, in some examples, the location of the contact. The location of the contact may then be used by another device, such as an artificial reality headset, to select a user interface element, activate a function or perform a task in an artificial reality environment, or execute another type of function typically performed in response to an input to a user interface. In one example, the wearable device includes an ultrasound transmitter (Tx) and an ultrasound receiver (Rx). In some embodiments, the wearable device may also include a processing element capable of processing the signals detected or received by the receiver to determine that a contact occurred.
    Type: Grant
    Filed: March 25, 2020
    Date of Patent: May 17, 2022
    Assignee: Facebook Technologies, LLC
    Inventors: Jun Gong, Aakar Gupta, Hrvoje Benko
  • Publication number: 20220129088
    Abstract: An electronic device tracks, for a user performing a target acquisition movement within a 3D space, movement parameters of a plurality of input devices of the user. The electronic device predicts, for the user, a region of interest within the 3D space, based on the movement parameters. The region of interest includes a plurality of targets in close proximity. The electronic device predicts an endpoint of the target acquisition movement, within the region of interest. In some embodiments, the plurality of input devices includes an eye tracking input device, each input device corresponds to a predefined input device type, and the movement parameters include gaze data from the eye tracking input device. In some embodiments, input devices includes an eye tracking input device, a head-mounted display, and a hand-held controller, and the user's eye, hand, and head movements are coordinated.
    Type: Application
    Filed: January 5, 2022
    Publication date: April 28, 2022
    Inventors: Rorik Henrikson, Tovi Samuel Grossman, Sean Edwin Trowbridge, Hrvoje Benko, Daniel John Wigdor, Marcello Giordano, Michael Glueck, Tanya Renee Jonker, Aakar Gupta, Stephanie Santosa, Carolina Brum Medeiros, Daniel Clarke
  • Patent number: 11287885
    Abstract: An apparatus for determining the position of artificial reality wearables may include a wearable dimensioned to be donned by a user of an artificial reality system. The apparatus may also include a sensor incorporated into the wearable. The sensor may detect physical interactions between the wearable and at least one specific body part of the user. The apparatus may further include a processing device communicatively coupled to the sensor. The processing device may determine, based at least in part on one or more physical interactions between the wearable and the specific body part of the user, a change in a position of the wearable relative to the specific body part of the user. Various other apparatuses, systems, and methods are also disclosed.
    Type: Grant
    Filed: January 22, 2019
    Date of Patent: March 29, 2022
    Assignee: Facebook Technologies, LLC
    Inventors: Wolf Kienzle, Hrvoje Benko, Joseph Davis Greer, Tristan Thomas Trutna, Raymond King, Yanjun Ma
  • Patent number: 11256342
    Abstract: An electronic device tracks, for a user performing a target acquisition movement within a 3D space, movement parameters of a plurality of input devices of the user. The electronic device predicts, for the user, a region of interest within the 3D space, using a regression model, based on the movement parameters. The region of interest includes a plurality of targets in close proximity. The electronic device predicts an endpoint of the target acquisition movement, within the region of interest, using a pointer facilitation technique. In some embodiments, the plurality of input devices includes an eye tracking input device, each input device corresponds to a predefined input device type, and the movement parameters include gaze data from the eye tracking input device. In some embodiments, input devices includes an eye tracking input device, a head-mounted display, and a hand-held controller, and the user's eye, hand, and head movements are coordinated.
    Type: Grant
    Filed: September 15, 2020
    Date of Patent: February 22, 2022
    Assignee: FACEBOOK TECHNOLOGIES, LLC
    Inventors: Rorik Henrikson, Tovi Samuel Grossman, Sean Edwin Trowbridge, Hrvoje Benko, Daniel John Wigdor, Marcello Giordano, Michael Glueck, Tanya Renee Jonker, Aakar Gupta, Stephanie Santosa, Carolina Brum Medeiros, Daniel Clarke
  • Patent number: 11175731
    Abstract: An apparatus for directional acoustic sensing may include a wearable dimensioned to be donned by a user of an artificial reality system. The wearable may include a signal generator that propagates an acoustic wave via a body of the user toward a specific body part. The wearable may also include an acoustic sensor that detects an acoustic reflection of the acoustic wave propagated by the signal generator. The wearable may further include at least one processing device that is communicatively coupled to the acoustic sensor. The processing device may determine that the specific body part has made physical contact with an object based at least in part on the acoustic reflection. In response, the processing device may generate an input command for the artificial reality system that accounts for the specific body part making physical contact with the object. Various other apparatuses, systems, and methods are also disclosed.
    Type: Grant
    Filed: January 14, 2021
    Date of Patent: November 16, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Eric Michael Whitmire, Wolf Kienzle, David R. Perek, Hrvoje Benko
  • Publication number: 20210311553
    Abstract: An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a wearable device, a set of signals that establish a signal pathway to the wearable device. The method also includes, while receiving the set of signals, determining baseline characteristics for the signal pathway, and sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage. The method further includes, in accordance with a determination that the sensed change in the baseline characteristics satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device that creates the user interface or is in communication with another electronic device that creates the user interface.
    Type: Application
    Filed: June 22, 2021
    Publication date: October 7, 2021
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko
  • Patent number: 11132058
    Abstract: The disclosed system may include (1) a processor that generates an artificial environment that includes a virtual object, (2) a display that presents the artificial environment, (3) an input subsystem that detects tracks positioning of a body, and (4) a plurality of haptic actuators that are arranged to apply haptic feedback to a first plurality of locations on the body, where the processor (a) determines, based on the positioning of the body, a virtual contact of one of a second plurality of locations on the body with the virtual object, where the second plurality of locations is different from the first plurality of locations, and (b) activates, in response to the virtual contact, at least one of the plurality of haptic actuators based on a mapping of the second plurality of locations to the plurality of haptic actuators. Various other systems and methods are also disclosed.
    Type: Grant
    Filed: April 13, 2020
    Date of Patent: September 28, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Aakar Gupta, Majed Jamal Samad, Hrvoje Benko
  • Patent number: 11093035
    Abstract: The disclosed system may include a support structure dimensioned for a user's hand. The system may also include transmitting electrodes coupled to a first finger portion of the support structure and may further include receiving electrodes coupled to a second, different finger portion of the support structure. The system may also include a controller that is coupled to the support structure and that is communicatively connected to the transmitting and receiving electrodes. The controller may also be configured to cause the transmitting electrodes to transmit a signal, detect at least some of the transmitted signal at the receiving electrodes and, based on the detected signal, determine that at least two fingers of the user's hand are touching each other. Various other methods, systems, and computer-readable media are also disclosed.
    Type: Grant
    Filed: May 21, 2019
    Date of Patent: August 17, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Shiu Sang Ng, Yanjun Ma, Wolf Kienzle, Hrvoje Benko
  • Patent number: 11042221
    Abstract: An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a wearable device attached to an appendage of the user, a set of signals that propagate through the user's appendage and establish a signal pathway to the wearable device. The method also includes, while receiving the set of signals, determining baseline characteristics for the signal pathway, and sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage. The method further includes, in accordance with a determination that the sensed change in the baseline characteristics satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device that creates the user interface or is in communication with another electronic device that creates the user interface.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: June 22, 2021
    Assignee: FACEBOOK TECHNOLOGIES, LLC
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko
  • Patent number: 10948976
    Abstract: A system may include a wearable dimensioned to be donned by a user of an artificial reality system. The system may also include an energy measuring device incorporated into the wearable. The energy measuring device may measure an electrical energy differential between a body of the user and a surface within an environment occupied by the user. The system may further include at least one processing device communicatively coupled to the energy measuring device. The processing device may detect, based at least in part on the electrical energy differential measured by the energy measuring device, an interaction between the user and the surface within the environment. In response to this detection, the processing device may generate an input command for the artificial reality system based at least in part on the interaction detected between the user and the surface. Various other systems and methods are also disclosed.
    Type: Grant
    Filed: July 8, 2019
    Date of Patent: March 16, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Eric Michael Whitmire, Wolf Kienzle, David R. Perek, Hrvoje Benko
  • Patent number: 10928889
    Abstract: An apparatus for directional acoustic sensing may include a wearable dimensioned to be donned by a user of an artificial reality system. The wearable may include a signal generator that propagates an acoustic wave via a body of the user toward a specific body part. The wearable may also include an acoustic sensor that detects an acoustic reflection of the acoustic wave propagated by the signal generator. The wearable may further include at least one processing device that is communicatively coupled to the acoustic sensor. The processing device may determine that the specific body part has made physical contact with an object based at least in part on the acoustic reflection. In response, the processing device may generate an input command for the artificial reality system that accounts for the specific body part making physical contact with the object. Various other apparatuses, systems, and methods are also disclosed.
    Type: Grant
    Filed: January 22, 2019
    Date of Patent: February 23, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Eric Michael Whitmire, Wolf Kienzle, David R. Perek, Hrvoje Benko
  • Publication number: 20210011555
    Abstract: An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a wearable device attached to an appendage of the user, a set of signals that propagate through the user's appendage and establish a signal pathway to the wearable device. The method also includes, while receiving the set of signals, determining baseline characteristics for the signal pathway, and sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage. The method further includes, in accordance with a determination that the sensed change in the baseline characteristics satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device that creates the user interface or is in communication with another electronic device that creates the user interface.
    Type: Application
    Filed: September 29, 2020
    Publication date: January 14, 2021
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko
  • Publication number: 20200409481
    Abstract: An electronic device tracks, for a user performing a target acquisition movement within a 3D space, movement parameters of a plurality of input devices of the user. The electronic device predicts, for the user, a region of interest within the 3D space, using a regression model, based on the movement parameters. The region of interest includes a plurality of targets in close proximity. The electronic device predicts an endpoint of the target acquisition movement, within the region of interest, using a pointer facilitation technique. In some embodiments, the plurality of input devices includes an eye tracking input device, each input device corresponds to a predefined input device type, and the movement parameters include gaze data from the eye tracking input device. In some embodiments, input devices includes an eye tracking input device, a head-mounted display, and a hand-held controller, and the user's eye, hand, and head movements are coordinated.
    Type: Application
    Filed: September 15, 2020
    Publication date: December 31, 2020
    Inventors: Rorik Henrikson, Tovi Samuel Grossman, Sean Trowbridge, Hrvoje Benko, Daniel Wigdor
  • Patent number: 10824247
    Abstract: An electronic device with a display selects a subset of templates from a library of templates. Each of the templates comprises logged movements of one or more predefined input device types in a 3D space and each template includes one or more movement parameters. The electronic device tracks, for a user, movement parameters of at least one input device of the user. The at least one input device corresponds to one of the predefined input device types. The electronic device compares the tracked movement parameters of the at least one input device of the user to the movement parameters for the subset of the templates selected from the library of templates. The electronic device predicts, based on the comparison, a goal target ray, within the 3D space. In some embodiments, the goal target ray is highlighted or visually emphasized according to the prediction.
    Type: Grant
    Filed: September 4, 2019
    Date of Patent: November 3, 2020
    Assignee: FACEBOOK TECHNOLOGIES, LLC
    Inventors: Rorik Henrikson, Tovi Grossman, Sean Trowbridge, Hrvoje Benko, Daniel Wigdor
  • Patent number: 10824235
    Abstract: An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a first wearable device attached to a first appendage of the user, a set of signals transmitted by a second wearable device attached to the user, wherein (i) receiving the set of signals creates a signal pathway between the first and second wearable devices, and (ii) signals in the set of signals propagate through at least the user's first appendage. The method also includes determining baseline characteristics for the signal pathway created between the first wearable device and the second wearable device, sensing a change in the baseline characteristics while receiving the set of signals, and in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, report a candidate touch event on the user's first appendage.
    Type: Grant
    Filed: January 7, 2019
    Date of Patent: November 3, 2020
    Assignee: FACEBOOK TECHNOLOGIES, LLC
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko
  • Patent number: 10795445
    Abstract: A method of determining contact on a user of a virtual reality and/or augmented reality device includes a first device including a plurality of transducers and a control circuit coupled to the plurality of transducers. The method includes activating one or more of the first plurality of transducers and generating waves that couple into at least a portion of a first appendage of a user wearing the first wearable device. The method further includes at a second wearable device having a second plurality of transducers, receiving at least a portion of the waves generated by the first plurality of transducers when the first appendage of the user is within a threshold distance from the second wearable device.
    Type: Grant
    Filed: January 7, 2019
    Date of Patent: October 6, 2020
    Assignee: FACEBOOK TECHNOLOGIES, LLC
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko
  • Patent number: 10788897
    Abstract: A system for vibration-driven sensing may include a wearable dimensioned to be donned by a user of an artificial reality system. The system may also include a vibration sensor that is incorporated into the wearable and generates an electrical response that corresponds to a vibration detected at the wearable. The system may further include at least one processing device communicatively coupled to the vibration sensor. The processing device may determine, based at least in part on the electrical response generated by the vibration sensor, that the user has made a specific gesture with at least one body part. In response to this determination, the processing device may generate an input command for the artificial reality system based at least in part on the specific gesture made by the user. Various other systems and methods are also disclosed.
    Type: Grant
    Filed: January 15, 2019
    Date of Patent: September 29, 2020
    Assignee: Facebook Technologies, LLC
    Inventors: Hrvoje Benko, Wolf Kienzle, Neil Weiss, Yu-Te Wang, Yanjun Ma, David R. Perek
  • Patent number: 10789760
    Abstract: Methods, systems, and computer-readable media providing focal feedback and control in a three-dimensional display. Focal anchors are provided at different depths and used to determine at what depth the user is currently focusing. The focal anchors are also used to receive input from the user. By looking at a focal anchor, the use can cause the portion of content associated with the focal anchor to be displayed more prominently relative to content displayed at different depths. In one embodiment, predictive feedback is provided at a depth associated with one of the focal anchors.
    Type: Grant
    Filed: July 2, 2018
    Date of Patent: September 29, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Hrvoje Benko, Daniel Chaim Robbins
  • Publication number: 20200249762
    Abstract: A method creates haptic stimulations on a user of an artificial reality system. The system includes a head-mounted display (HMD) and a wearable device. The HMD includes a display and speakers. The wearable device includes a plurality of transducers that can each generate waves to provide haptic feedback to a user. The system displays media content on the display and, in accordance with the displayed media content, determines a virtual object location in the displayed media content corresponding to a physical object location. The system provides, to the user, audio directed to the virtual object location. The system activates one or more transducers to provide haptic feedback at a target location on the user, distinct from the physical object location, to produce haptic feedback whose perceptual interpretation is at the physical object location based on a combination of the displayed media, the provided audio, and the haptic feedback.
    Type: Application
    Filed: April 23, 2020
    Publication date: August 6, 2020
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko, Raymond King, Andrew Arthur Stanley, Massimiliano Di Luca, Yon Visell, Yitian Shao, Bharat Dandu
  • Patent number: 10684690
    Abstract: A method of creating localized haptic stimulations on a user includes a wearable device including a plurality of transducers that can each generate one or more waves that propagate away from the wearable device through a medium. The method includes activating two or more transducers of the plurality of transducers, selecting values for characteristics of waves to be generated by the two or more transducers based at least in part on a known impedance of the medium. The method further includes generating, by the two or more transducers, waves that constructively interfere at a target location to create a haptic stimulation on a user of the wearable device, the waves having the selected values.
    Type: Grant
    Filed: January 7, 2019
    Date of Patent: June 16, 2020
    Assignees: Facebook Technologies, LLC, The Regents of the University of California
    Inventors: Sean Jason Keller, Tristan Thomas Trutna, Hrvoje Benko, Raymond King, Andrew Arthur Stanley, Massimiliano Di Luca, Yon Visell, Yitian Shao, Bharat Dandu