SYSTEMS, ARTICLES, AND METHODS FOR HUMAN-ELECTRONICS INTERFACES
Human-electronics interfaces in which a wearable electromyography (“EMG”) device is operated to control virtually any electronic device are described. In response to detected muscle activity and/or motions of a user, the wearable EMG device transmits generic gesture identification flags that are not specific to the particular electronic device(s) being controlled. An electronic device being controlled is programmed with user-definable instructions for how to interpret and respond to the gesture identification flags.
1. Technical Field
The present systems, articles, and methods generally relate to human-electronics interfaces and particularly relate to electromyographic control of electronic devices.
2. Description of the Related Art
Wearable Electronic DevicesElectronic devices are commonplace throughout most of the world today. Advancements in integrated circuit technology have enabled the development of electronic devices that are sufficiently small and lightweight to be carried by the user. Such “portable” electronic devices may include on-board power supplies (such as batteries or other power storage systems) and may be designed to operate without any wire-connections to other electronic systems; however, a small and lightweight electronic device may still be considered portable even if it includes a wire-connection to another electronic system. For example, a microphone may be considered a portable electronic device whether it is operated wirelessly or through a wire-connection.
The convenience afforded by the portability of electronic devices has fostered a huge industry. Smartphones, audio players, laptop computers, tablet computers, and ebook readers are all examples of portable electronic devices. However, the convenience of being able to carry a portable electronic device has also introduced the inconvenience of having one's hand(s) encumbered by the device itself. This problem is addressed by making an electronic device not only portable, but wearable.
A wearable electronic device is any portable electronic device that a user can carry without physically grasping, clutching, or otherwise holding onto the device with their hand(s). For example, a wearable electronic device may be attached or coupled to the user by a strap or straps, a band or bands, a clip or clips, an adhesive, a pin and clasp, an article of clothing, tension or elastic support, an interference fit, an ergonomic form, etc. Examples of wearable electronic devices include digital wristwatches, electronic armbands, electronic rings, electronic ankle-bracelets or “anklets,” head-mounted electronic display units, hearing aids, and so on.
Human-Electronics InterfacesA wearable electronic device may provide direct functionality for a user (such as audio playback, data display, computing functions, etc.) or it may provide electronics to interact with, receive information from, or control another electronic device. For example, a wearable electronic device may include sensors that detect inputs effected by a user and transmit signals to another electronic device based on those inputs. Sensor-types and input-types may each take on a variety of forms, including but not limited to: tactile sensors (e.g., buttons, switches, touchpads, or keys) providing manual control, acoustic sensors providing voice-control, electromyography sensors providing gesture control, and/or accelerometers providing gesture control.
A human-computer interface (“HCI”) is an example of a human-electronics interface. The present systems, articles, and methods may be applied to HCIs, but may also be applied to any other form of human-electronics interface.
Electromyography DevicesElectromyography (“EMG”) is a process for detecting and processing the electrical signals generated by muscle activity. EMG devices employ EMG sensors that are responsive to the range of electrical potentials (typically μV-mV) involved in muscle activity. EMG signals may be used in a wide variety of applications, including: medical monitoring and diagnosis, muscle rehabilitation, exercise and training, prosthetic control, and even in controlling functions of electronic devices.
Human-electronics interfaces that employ EMG have been proposed in the art. For example, U.S. Pat. No. 6,244,873 and U.S. Pat. No. 8,170,656 describe such systems. Characteristics that are common to these known proposals will now be described.
Typically, such systems (including the two examples listed above) employ a wearable EMG device that exclusively controls specific, pre-defined functions of a specific, pre-defined “receiving” electronic device. The term “pre-defined” here refers to information that is programmed into the wearable EMG device (or with which the wearable EMG device is programmed) in advance of a following interaction with a receiving device. The wearable EMG device typically includes built-in EMG sensors that detect muscle activity of a user and an on-board processor that determines when the detected muscle activity corresponds to a pre-defined gesture. The on-board processor maps each pre-defined gesture to a particular pre-defined function of the pre-defined receiving device. In other words, the wearable EMG device stores and executes pre-defined mappings between detected gestures and receiving device functions. The receiving device function(s) is/are then controlled by one or more “command(s)” that is/are output by the wearable EMG device. Each command that is output by the wearable EMG device has already been formulated to control (and is therefore limited to exclusively controlling) a specific function of a specific receiving device prior to being transmitted by the wearable EMG device.
The wearable EMG devices proposed in the art are hard-coded to map pre-defined gestures to specific, pre-defined commands controlling specific, pre-defined functions of a specific, pre-defined receiving device. The wearable EMG devices proposed in the art are programmed with information about the specific receiving device (and/or about a specific application within the specific receiving device) under their control such that the wearable EMG devices proposed in the art output commands that include instructions that are specifically formulated for the specific receiving device (and/or the specific application within the specific receiving device). Thus, existing proposals for human-electronics interfaces that employ EMG are limited in their versatility because they employ a wearable EMG device that is hard-coded to control a specific electronic device (and/or a specific application within a specific electronic device). For such systems, the wearable EMG device needs to be modified/adapted for each distinct use (e.g., the wearable EMG device needs to be programmed with command signals that are specific to the receiving device and/or specific to the application within the receiving device). Because the outputs (i.e., commands) provided by such wearable EMG devices are hard-coded with information about the function(s) of the receiving device(s), a user cannot use such a wearable EMG device to control any generic electronic device (or any generic application within an electronic device) without reprogramming/reconfiguring the wearable EMG device itself. A user who wishes to control multiple electronic devices (or multiple applications within a single electronic device, either simultaneously or in sequence) must use multiple such wearable EMG devices with each wearable EMG device separately controlling a different electronic device, or the user must re-program a single such wearable EMG device in between uses. There is a need in the art for a human-electronics interface employing EMG that overcomes these limitations.
BRIEF SUMMARYA wearable electromyography (“EMG”) device may be summarized as including: at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity; a processor communicatively coupled to the at least one EMG sensor, the processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor; and an output terminal communicatively coupled to the processor to in use transmit the gesture identification flag. The gesture identification flag may be independent of any downstream processor-based device and generic to a variety of end user applications executable by a variety of downstream processor-based devices useable with the wearable EMG device.
The wearable EMG device may further include a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags. The non-transitory processor-readable storage medium may store processor-executable instructions that embody and/or produce/effect a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the processor, the processor-executable instructions may cause the processor to determine a gesture identification flag in accordance with the mapping. The non-transitory processor-readable storage medium may store processor-executable instructions that, when executed by the processor, cause the processor to determine a gesture identification flag based at least in part on at least one signal provided by the at least one EMG sensor.
The wearable EMG device may further include at least one accelerometer communicatively coupled to the processor, the at least one accelerometer to in use detect motion effected by the user of the wearable EMG device and provide at least one signal in response to the detected motion, and the processor may in use determine the gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least one signal provided by the at least one accelerometer.
The processor may be selected from the group consisting of: a digital microprocessor, a digital microcontroller, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a programmable gate array, and a programmable logic unit. The at least one EMG sensor may include a plurality of EMG sensors, and the wearable EMG device may further include a set of communicative pathways to route signals provided by the plurality of EMG sensors to the processor, wherein each EMG sensor in the plurality of EMG sensors is communicatively coupled to the processor by at least one communicative pathway from the set of communicative pathways. The wearable EMG device may further include a set of pod structures that form physically coupled links of the wearable EMG device, wherein each pod structure in the set of pod structures is positioned adjacent and physically coupled to at least one other pod structure in the set of pod structures, and wherein the set of pod structures comprises at least two sensor pods and a processor pod, each of the at least two sensor pods comprising a respective EMG sensor from the plurality of EMG sensors and the processor pod comprising the processor. Each pod structure in the set of pod structures may be positioned adjacent and in between two other pod structures in the set of pod structures and physically coupled to the two other pod structures in the set of pod structures, and the set of pod structures may form a perimeter of an annular configuration. The wearable EMG device may further include at least one adaptive coupler, wherein each respective pod structure in the set of pod structures is adaptively physically coupled to at least one adjacent pod structure in the set of pod structures by at least one adaptive coupler.
The output terminal of the wearable EMG device may include at least one of a wireless transmitter and/or a tethered connector port. The at least one EMG sensor may include at least one capacitive EMG sensor.
A method of operating a wearable electromyography (“EMG”) device to provide electromyographic control of an electronic device, wherein the wearable EMG device includes at least one EMG sensor, a processor, and an output terminal, may be summarized as including: detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor; providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity; determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor, wherein the gesture identification flag is independent of the electronic device; and transmitting the gesture identification flag to the electronic device by the output terminal. Detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor may include detecting muscle activity of the user of the wearable EMG device by a first EMG sensor and by at least a second EMG sensor. Providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity may include providing at least a first signal from the first EMG sensor to the processor in response to the detected muscle activity and providing at least a second signal from the second EMG sensor to the processor in response to the detected muscle activity. Determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include determining, by the processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from the at least a second EMG sensor to the processor.
The wearable EMG device may further include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include executing the processor-executable instructions by the processor to cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor.
The wearable EMG device may further include at least one accelerometer, and the method may further include: detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and providing at least one signal from the at least one accelerometer to the processor in response to the detected motion. Determining a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor. The wearable EMG device may include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor may includes executing the processor-executable instructions by the processor to cause the processor to determine the gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
The output terminal of the wearable EMG device may include a wireless transmitter, and transmitting the gesture identification flag to the electronic device by the output terminal may include wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
A system that enables electromyographic control of an electronic device may be summarized as including: a wearable electromyography (“EMG”) device comprising: at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity, a first processor communicatively coupled to the at least one EMG sensor, the first processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor, and an output terminal communicatively coupled to the first processor, the output terminal to in use transmit the gesture identification flag; and an electronic device comprising: an input terminal to in use receive the gesture identification flag, and a second processor communicatively coupled to the input terminal, the second processor to in use determine a function of the electronic device based at least in part on the gesture identification flag. The gesture identification flag may be independent of the electronic device and generic to a variety of end user applications executable by the electronic device.
The wearable EMG device of the system may further include a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags. The non-transitory processor-readable storage medium of the wearable EMG device may store processor-executable instructions that embody and/or produce/effect a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the first processor, the processor-executable instructions may cause the first processor to determine a gesture identification flag in accordance with the mapping.
The wearable EMG device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the first processor, cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor.
The wearable EMG device of the system may include at least one accelerometer communicatively coupled to the first processor, the at least one accelerometer to in use detect motion effected by the user of the wearable EMG device and provide at least one signal in response to the detected motion, and the first processor may in use determine a gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least on signal provided by the at least one accelerometer.
The electronic device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores at least a set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
The electronic device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores: a first application executable by the electronic device; at least a second application executable by the electronic device; a first set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the first application based at least in part on a gesture identification flag; and a second set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the second application based at least in part on a gesture identification flag.
The output terminal of the wearable EMG device may include a first tethered connector port, the input terminal of the electronic device may include a second tethered connector port, and the system may further include a communicative pathway to in use communicatively couple the first tethered connector port to the second tethered connector port and to route the gesture identification flag from the output terminal of the wearable EMG device to the input terminal of the electronic device.
The output terminal of the wearable EMG device may include a wireless transmitter to in use wirelessly transmit the gesture identification flag, the input terminal of the electronic device may include a tethered connector port, and the system may include a wireless receiver to in use communicatively couple to the tethered connector port of the electronic device and to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
The output terminal of the wearable EMG device may include a wireless transmitter to in use wirelessly transmit the gesture identification flag and the input terminal of the electronic device may include a wireless receiver to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
The electronic device may be selected from the group consisting of: a computer, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and a vehicle.
A method of electromyographically controlling at least one function of an electronic device by a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes at least one EMG sensor, a first processor, and an output terminal and the electronic device includes an input terminal and a second processor, may be summarized as including: detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor; providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity; determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor, wherein the gesture identification flag is independent of the electronic device; transmitting the gesture identification flag by the output terminal of the wearable EMG device; receiving the gesture identification flag by the input terminal of the electronic device; determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag; and performing the function by the electronic device. Detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor may include detecting muscle activity of the user of the wearable EMG device by a first EMG sensor of the wearable EMG device and by at least a second EMG sensor of the wearable EMG device. Providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity may include providing at least a first signal from the first EMG sensor to the first processor in response to the detected muscle activity and providing at least a second signal from the send EMG sensor to the first processor in response to the detected muscle activity. Determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include determining, by the first processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the first processor and the at least a second signal provided from the at least a second EMG sensor to the first processor.
The wearable EMG device may include a non-transitory processor-readable medium that stores processor-executable instructions, and determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include executing the processor-executable instructions by the first processor to cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor.
The wearable EMG device may include at least one accelerometer, and the method may include: detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and providing at least one signal from the at least one accelerometer to the first processor in response to the detected motion. Determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor and the at least one signal provided by the at least one accelerometer to the first processor.
The output terminal of the wearable EMG device may include a wireless transmitter and the input terminal of the electronic device may include a wireless receiver. Transmitting the gesture identification flag by the output terminal of the wearable EMG device may include wirelessly transmitting the gesture identification flag by the wireless transmitter of the wearable EMG device, and receiving the gesture identification flag by the input terminal of the electronic device may include wirelessly receiving the gesture identification flag by the wireless receiver of the electronic device.
The electronic device may include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag may include executing the processor-executable instructions by the second processor to cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve drawing legibility. Further, the particular shapes of the elements as drawn are not intended to convey any information regarding the actual shape of the particular elements, and have been solely selected for ease of recognition in the drawings.
In the following description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with electronic devices, and in particular portable electronic devices such as wearable electronic devices, have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the embodiments.
Unless the context requires otherwise, throughout the specification and claims which follow, the word “comprise” and variations thereof, such as, “comprises” and “comprising” are to be construed in an open, inclusive sense, that is as “including, but not limited to.”
Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the content clearly dictates otherwise. It should also be noted that the term “or” is generally employed in its broadest sense, that is as meaning “and/or” unless the content clearly dictates otherwise.
The headings and Abstract of the Disclosure provided herein are for convenience only and do not interpret the scope or meaning of the embodiments.
The various embodiments described herein provide systems, articles, and methods for human-electronics interfaces employing a generalized wearable EMG device that may be readily implemented in a wide range of applications. The human-electronics interfaces described herein employ a wearable EMG device that controls functions of another electronic device not by outputting “commands” as in the known proposals previously described, but by outputting generic gesture identification signals, or “flags,” that are not specific to the particular electronic device being controlled. In this way, the wearable EMG device may be used to control virtually any other electronic device if, for example, the other electronic device (or multiple other electronic devices) is (are) programmed with instructions for how to respond to the gesture identification flags.
Throughout this specification and the appended claims, the term “gesture identification flag” is used to refer to at least a portion of a data signal (e.g., a bit string) that is defined by and transmitted from a wearable EMG device in response to the wearable EMG device identifying that a user thereof has performed a particular gesture. The gesture identification flag may be received by a “receiving” electronic device, but the “gesture identification flag” portion of the data signal does not contain any information that is specific to the receiving electronic device. A gesture identification flag is a general, universal, and/or ambiguous signal that is substantially independent of the receiving electronic device (e.g., independent of any downstream processor-based device) and/or generic to a variety of applications run on any number of receiving electronic devices (e.g., generic to a variety of end user applications executable by one or more downstream processor-based device(s) useable with the wearable EMG device). A gesture identification flag may carry no more information than the definition/identity of the flag itself. For example, a set of three gesture identification flags may include a first flag simply defined as “A,” a second flag simply defined as “B,” and a third flag simply defined as “C.” Similarly, a set of four binary gesture identification flags may include a 00 flag, a 01 flag, a 10 flag, and a 11 flag. In accordance with the present systems, articles, and methods, a gesture identification flag may be defined and output by a wearable EMG device with little to no regard for the nature or functions of the receiving electronic device. The receiving electronic device may be programmed with specific instructions for how to interpret and/or respond to one or more gesture identification flag(s). As will be understood by a person of skill in the art, in some applications a gesture identification flag may be combined with authentication data, encryption data, device ID data (i.e., transmitting electronic device ID data and/or receiving electronic device ID data), pairing data, and/or any other data to enable and/or facilitate telecommunications between the wearable EMG device and the receiving electronic device in accordance with known telecommunications protocols (e.g., Bluetooth®). For greater certainty, throughout this specification and the appended claims, the term “gesture identification flag” refers to at least a portion of a data signal that is defined by a wearable EMG device based (at least in part) on EMG and/or accelerometer data and is substantially independent of the receiving electronic device. For the purposes of transmission, a gesture identification flag may be combined with other data that is at least partially dependent on the receiving electronic device. For example, a gesture identification flag may be a 2-bit component of an 8-bit data byte, where the remaining 6 bits are used for telecommunication purposes, as in: 00101101, where the exemplary first six bits “001011” may correspond to telecommunications information such as transmitting/receiving device IDs, encryption data, pairing data, and/or the like, and the exemplary last two bits “01” may correspond to a gesture identification flag. While a bit-length of two bits is used to represent a gesture identification flag in this example, in practice a gesture identification flag may comprise any number of bits (or other measure of signal length of a scheme not based on bits is employed).
Device 100 includes a set of eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 that form physically coupled links of the wearable EMG device 100. Each pod structure in the set of eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 is positioned adjacent and in between two other pod structures in the set of eight pod structures and the set of pod structures forms a perimeter of an annular or closed loop configuration. For example, pod structure 101 is positioned adjacent and in between pod structures 102 and 108 at least approximately on a perimeter of the annular or closed loop configuration of pod structures, pod structure 102 is positioned adjacent and in between pod structures 101 and 103 at least approximately on the perimeter of the annular or closed loop configuration, pod structure 103 is positioned adjacent and in between pod structures 102 and 104 at least approximately on the perimeter of the annular or closed loop configuration, and so on. Each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 is physically coupled to the two adjacent pod structures by at least one adaptive coupler (not visible in
Throughout this specification and the appended claims, the term “pod structure” is used to refer to an individual link, segment, pod, section, structure, component, etc. of a wearable EMG device. For the purposes of the present systems, articles, and methods, an “individual link, segment, pod, section, structure, component, etc.” (i.e., a “pod structure”) of a wearable EMG device is characterized by its ability to be moved or displaced relative to another link, segment, pod, section, structure component, etc. of the wearable EMG device. For example, pod structures 101 and 102 of device 100 can each be moved or displaced relative to one another within the constraints imposed by the adaptive coupler providing adaptive physical coupling therebetween. The desire for pod structures 101 and 102 to be movable/displaceable relative to one another specifically arises because device 100 is a wearable EMG device that advantageously accommodates the movements of a user and/or different user forms.
Device 100 includes eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 that form physically coupled links thereof. Wearable EMG devices employing pod structures (e.g., device 100) are used herein as exemplary wearable EMG device designs, while the present systems, articles, and methods may be applied to wearable EMG devices that do not employ pod structures (or that employ any number of pod structures). Thus, throughout this specification, descriptions relating to pod structures (e.g., functions and/or components of pod structures) should be interpreted as being applicable to any wearable EMG device design, even wearable EMG device designs that do not employ pod structures (except in cases where a pod structure is specifically recited in a claim).
In exemplary device 100 of
Details of the components contained within the housings (i.e., within the inner volumes of the housings) of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 are not visible in
Throughout this specification and the appended claims the term “communicative” as in “communicative pathway,” “communicative coupling,” and in variants such as “communicatively coupled,” is generally used to refer to an engineered arrangement for transferring and/or exchanging information. Exemplary communicative pathways include, but are not limited to, electrically conductive pathways (e.g., electrically conductive wires, electrically conductive traces), magnetic pathways (e.g., magnetic media), and/or optical pathways (e.g., optical fiber), and exemplary communicative couplings include, but are not limited to, electrical couplings and/or optical couplings.
Each individual pod structure within a wearable EMG device may perform a particular function, or particular functions. For example, in device 100, each of pod structures 101, 102, 103, 104, 105, 106, and 107 includes a respective EMG sensor 110 (only one called out in
Pod structure 108 of device 100 includes a processor 140 that in use processes the signals provided by the EMG sensors 110 of sensor pods 101, 102, 103, 104, 105, 106, and 107 in response to detected muscle activity. Pod structure 108 may therefore be referred to as a “processor pod.” Throughout this specification and the appended claims, the term “processor pod” is used to denote an individual pod structure that includes at least one processor to process signals. The processor may be any type of processor, including but not limited to: a digital microprocessor or microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a graphics processing unit (GPU), a programmable gate array (PGA), a programmable logic unit (PLU), or the like, that in use analyzes the signals to determine at least one output, action, or function based on the signals.
As used throughout this specification and the appended claims, the terms “sensor pod” and “processor pod” are not necessarily exclusive. A single pod structure may satisfy the definitions of both a “sensor pod” and a “processor pod” and may be referred to as either type of pod structure. For greater clarity, the term “sensor pod” is used to refer to any pod structure that includes a sensor and performs at least the function(s) of a sensor pod, and the term processor pod is used to refer to any pod structure that includes a processor and performs at least the function(s) of a processor pod. In device 100, processor pod 108 includes an EMG sensor 110 (not visible in
Processor 140 includes and/or is communicatively coupled to a non-transitory processor-readable storage medium or memory 141. As will be described in more detail later, memory 141 may store, for example, a set of gesture identification flags to be transmitted by device 100 and/or, for example, processor-executable instructions to be executed by processor 140. For transmitting gesture identification flags, a wearable EMG device may include at least one output terminal communicatively coupled to processor 140. Throughout this specification and the appended claims, the term “terminal” is generally used to refer to any physical structure that provides a telecommunications link through which a data signal may enter and/or leave a device. The term “output terminal” is used to describe a terminal that provides at least a signal output link and the term “input terminal” is used to describe a terminal that provides at least a signal input link; however unless the specific context requires otherwise, an output terminal may also provide the functionality of an input terminal and an input terminal may also provide the functionality of an output terminal. In general, a “communication terminal” represents the end (or “terminus”) of communicative signal transfer within a device and the beginning of communicative signal transfer to/from an external device (or external devices). As examples, communication terminal 151 of device 100 may include a wireless transmitter that implements a known wireless communication protocol, such as Bluetooth®, WiFi®, or Zigbee®, while communication terminal 152 may include a tethered communication port such as Universal Serial Bus (USB) port, a micro-USB port, a Thunderbolt® port, and/or the like.
For some applications, device 100 may also include at least one accelerometer 160 (e.g., an inertial measurement unit, or “IMU,” that includes at least one accelerometer and/or at least one gyroscope) communicatively coupled to processor 140. In use, the at least one accelerometer may detect, sense, and/or measure motion effected by a user and provide signals in response to the detected motion. As will be described in more detail later, signals provided by accelerometer 160 may be processed together with signals provided by EMG sensors 110 by processor 140.
Throughout this specification and the appended claims, the term “accelerometer” is used as a general example of an inertial sensor and is not intended to limit (nor exclude) the scope of any description or implementation to “linear acceleration.”
Throughout this specification and the appended claims, the term “provide” and variants such as “provided” and “providing” are frequently used in the context of signals. For example, an EMG sensor is described as “providing at least one signal” and an accelerometer is described as “providing at least one signal.” Unless the specific context requires otherwise, the term “provide” is used in a most general sense to cover any form of providing a signal, including but not limited to: relaying a signal, outputting a signal, generating a signal, routing a signal, creating a signal, transducing a signal, and so on. For example, a capacitive EMG sensor may include at least one electrode that capacitively couples to electrical signals from muscle activity. This capacitive coupling induces a change in a charge or electrical potential of the at least one electrode which is then relayed through the sensor circuitry and output, or “provided,” by the sensor. Thus, the capacitive EMG sensor may “provide” an electrical signal by relaying an electrical signal from a muscle (or muscles) to an output (or outputs). In contrast, an accelerometer may include components (e.g., piezoelectric, piezoresistive, capacitive, etc.) that are used to convert physical motion into electrical signals. The accelerometer may “provide” an electrical signal by detecting motion and generating an electrical signal in response to the motion.
As previously described, each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 may include electric circuitry.
Signals that are provided by EMG sensors 110 in device 100 are routed to processor pod 108 for processing by processor 140. To this end, device 100 employs a set of communicative pathways (e.g., 121 and 122) to route the signals that are provided by sensor pods 101, 102, 103, 104, 105, 106, and 107 to processor pod 108. Each respective pod structure 101, 102, 103, 104, 105, 106, 107, and 108 in device 100 is communicatively coupled to at least one other pod structure by at least one respective communicative pathway from the set of communicative pathways. Each communicative pathway (e.g., 121 and 122) may be realized in any communicative form, including but not limited to: electrically conductive wires or cables, ribbon cables, fiber-optic cables, optical/photonic waveguides, electrically conductive traces carried by a rigid printed circuit board, and/or electrically conductive traces carried by a flexible printed circuit board.
The present systems, articles, and methods describe a human-electronics interface in which a wearable EMG device (e.g., device 100) is used to control another electronic device. The human-electronics interface may be characterized as a system that enables electromyographic control of an electronic device.
Each pod structure 201 is electrically coupled to at least one adjacent pod structure by at least one respective communicative pathway 220 to route signals in between pod structures (e.g., to route signals from sensor pods to a processor pod). Each pod structure 201 is also physically coupled to two adjacent pod structures 201 by at least one adaptive coupler 260 and the set of pod structures forms a perimeter of an annular or closed loop configuration.
Each pod structure 201 includes respective electric circuitry 230 and at least one electric circuitry 230 includes a first processor 240 (e.g., akin to processor 140 in device 100 of
Unspecified electronic device 280 may be any electronic device, including but not limited to: a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and/or a vehicle. Electronic device 280 is denominated as “unspecified” herein to emphasize the fact that the gesture identification flags output by wearable EMG device 270 are generic to a variety of electronic devices and/or applications executed by the electronic devices. The electronic device 280, its operating characteristics and/or the operating characteristics of applications executed by the electronic device 280 may not be a priori known by the EMG device 270 during use, or even prior to use when a mapping between signals, gesture flags, and/or gestures is initially defined or established. As previously described, a data signal output by device 270 through output terminal 250 may include a gesture identification flag as a first portion thereof and may also include at least a second portion to implement known telecommunications protocols (e.g., Bluetooth®). Thus, electronic device 280 may remain “unspecified” with respect to the gesture identification flag portion(s) of signals output by EMG device 270 but electronic device 280 may be “specified” by the telecommunications portion(s) of signals output by EMG device 270 (if such specification is necessary for signal transfer, e.g., to communicatively “pair” device 270 and device 280 if required by the telecommunications protocol being implemented). For example, electronic device 280 may be and remain “unspecified” while muscle activity is detected by EMG device 270 and while the processor in EMG device 270 determines a gesture identification flag based, at least in part, on the detected muscle activity. After a gesture identification flag is determined by the processor in EMG device 270, electronic device 280 may become “specified” when the gesture identification flag is combined with telecommunication data and transmitted to electronic device 280. In this scenario, the gesture identification flag itself does not include any information that is specific to electronic device 280 and therefore electronic device 280 is “unspecified” in relation to the gesture identification flag.
Electronic device 280 includes an input terminal 281 to in use interface with wearable EMG device 270. For example, device 280 may receive gesture identification flags from device 270 through input terminal 281. Device 280 also includes a second processor 283 to in use process gesture identification flags received from device 270. Second processor 283 may include or be communicatively coupled to a non-transitory processor-readable storage medium or memory 284 that stores processor-executable instructions to be executed by second processor 283.
Wearable EMG device 270 and electronic device 280 are, in use, communicatively coupled by communicative link 290. More specifically, output terminal 250 of wearable EMG device 270 is, in use, communicatively coupled to input terminal 281 of electronic device 280 by communicative link 290. Communicative link 290 may be used to route gesture identification flags from wearable EMG device 270 to electronic device 280. Communicative link 290 may be established in variety of different ways. For example, output terminal 250 of wearable EMG device 270 may include a first tethered connector port (e.g., a USB port, or the like), input terminal 281 of electronic device 280 may include a second tethered connector port, and communicative link 290 may be established through a communicative pathway (e.g., an electrical or optical cable, wire, circuit board, or the like) that communicatively couples the first connector port to the second connector port to route gesture identification flags from output terminal 250 to input terminal 281. Alternatively, output terminal 250 of wearable EMG device 270 may include a wireless transmitter and communicative link 290 may be representative of wireless communication between wearable EMG device 270 and electronic device 280. In this case, input terminal 281 of electronic device 280 may include a wireless receiver to in use wirelessly receive gesture identification flags from the wireless transmitter of wearable EMG device 270 (using, for example, established wireless telecommunication protocols, such as Bluetooth®); or, input terminal 281 may be communicatively coupled to a wireless receiver 282 (such as a USB dongle communicatively coupled to a tethered connector port of input terminal 281) to in use wirelessly receive gesture identification flags from the wireless transmitter of wearable EMG device 270.
As previously described, known proposals for human-electronics interfaces that employ a wearable EMG device are limited in their versatility because they involve mapping gestures to functions on-board the wearable EMG device itself. Thus, in known proposals, the wearable EMG device outputs control signals (i.e., “commands”) that embody pre-defined instructions to effect pre-defined functions that are specific to a pre-defined receiving device. If a user wishes to use such a wearable EMG device for a different purpose (i.e., to control a different receiving device, or a different application within the same receiving device), then the definitions of the commands themselves must be re-programmed within the wearable EMG device. Conversely, the various embodiments described herein provide systems, articles, and methods for human-electronics interfaces that employ a wearable EMG device that controls functions of another electronic device by outputting generic gesture identification flags that are not specific to the particular electronic device being controlled. The electronic device being controlled may include or may access an Application Programming Interface (i.e., an “API” including instructions and/or data or information (e.g., library) stored in a non-transitory processor-readable storage medium or memory) through which a user may define how gesture identification flags are to be interpreted by the electronic device being controlled (i.e., where the user may define how the electronic device responds to gesture identification flags). The present systems, articles, and methods greatly enhance the versatility of human-electronics interfaces by employing a wearable EMG device that outputs the same gesture identification flags regardless of what it is being used to control, and may therefore be used to control virtually any electronic receiving device. The functions or operations that are controlled by the wearable EMG devices described herein are defined within the receiving device (or within the applications within the receiving device) rather than within the wearable EMG device.
At 301, muscle activity of a user (i.e., a wearer of the wearable EMG device) is sensed, measured, transduced or otherwise detected by at least one EMG sensor of the wearable EMG device. As previously described, the at least one EMG sensor may be, for example, a capacitive EMG sensor and sensing, measuring, transducing or otherwise detecting muscle activity of the user may include, for example, capacitively coupling to electrical signals generated by muscle activity of the user.
At 302, at least one signal is provided from the at least one EMG sensor to the processor of the wearable EMG device in response to the sensed, measured, transduced or otherwise detected muscle activity. The at least one signal may be an analog signal that is amplified, filtered, and converted to digital form by electric circuitry within the wearable EMG device. Providing the at least one signal from the at least one EMG sensor to the processor may include routing the at least one signal to the processor through one or more communicative pathway(s) as described previously.
At 303, a gesture identification flag is determined by the processor of the wearable EMG device, based at least in part on the at least one signal provided from the at least one EMG sensor to the processor. The gesture identification flag is substantially independent of the downstream electronic device. As will be described in more detail later (e.g., with reference to
At 304, the gesture identification flag is transmitted to the electronic device by the output terminal of the wearable EMG device. As previously described, the output terminal of the wearable EMG device may include a wireless transmitter, and transmitting the gesture identification flag to the electronic device may include wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
As an example, the at least one EMG sensor may include a first EMG sensor and at least a second EMG sensor, and muscle activity of the user may be sensed, measured, transduced or otherwise detected by the first EMG sensor and by at least the second EMG sensor (at 301). In this case at least a first signal is provided from the first EMG sensor to the processor of the wearable EMG device in response to the detected muscle activity (at 302) and at least a second signal is provided from at least the second EMG sensor to the processor of the wearable EMG device in response to the detected muscle activity (at 302). The processer of the wearable EMG device may then determine (at 303) a gesture identification flag based at least in part on both the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from at least the second EMG sensor to the processor.
As previously described, in some applications it may be advantageous to combine or otherwise make use of both EMG signals and motion signals sensed, measured or otherwise detected, for example, by an accelerometer. To this end, the wearable EMG device may include at least one accelerometer, and an additional method employing further acts may be combined with acts 301-304 of method 300 to detect and process motion signals.
At 401, motion effected by the user of the wearable EMG device is sensed, measured, transduced or otherwise detected by at least one accelerometer in the wearable EMG device. The at least once accelerometer may be part of an IMU that includes multiple accelerometers (such as an MPU-9150 Nine-Axis MEMS MotionTracking™ Device from InvenSense). The motion effected by the user that may be detected and/or measured may include, e.g., translation in one or multiple spatial directions and/or rotation about one or more axes in one or more spatial directions. The motion(s) may be detected in terms of a presence or absence of translation and/or rotation, and/or measured in terms of a speed of translation and/or rotation and/or acceleration of translation and/or rotation.
At 402, at least one signal is provided from the at least one accelerometer to the processor in response to the sensed, measured, transduced or otherwise detected motion. The at least one signal may be an analog signal that is amplified, filtered, and converted to digital form by electric circuitry within the wearable EMG device. The at least one signal may be routed to the processor in the wearable EMG device via one or more communicative pathway(s) as described previously.
As previously described, act 303 of method 300 involves determining, by a processor of the wearable EMG device, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor in response to detected muscle activity. In applications where the wearable EMG device further includes at least one accelerometer and acts 401 and 402 of method 400 are performed, act 303 of method 300 may be replaced by act 403 of method 400.
At 403, a gesture identification flag is determined by the processor, based at least in part on the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor. The wearable EMG device may include a non-transitory processor-readable medium (e.g., memory 284 of device 280 from
In some implementations, the at least one signal provided from the at least one accelerometer to the processor (i.e., at act 402) may be combined with at least one signal provided from at least one EMG sensor to the processor (i.e., at act 302 of method 300 from
After act 403, the gesture identification flag may be transmitted or output by an output terminal of the wearable EMG device (i.e., according to act 304 of method 300) to any downstream electronic device and interpreted or otherwise processed by the downstream electronic device to cause the downstream electronic device to perform some function(s) or operation(s), or otherwise effect an interaction with or response from the downstream electronic device, in response to the gesture identification flag.
In accordance with the present systems, articles, and methods, at least one signal provided by at least one EMG sensor (either alone or together with one or more signals provided by one or more transducers such as an accelerometer or other motion or acceleration responsive transducers) may represent or be indicative of a gesture performed by a user of a wearable EMG device. Determining a gesture identification flag corresponding to that at least one signal may involve identifying, by a processor, the gesture performed by the user based at least in part on the at least one signal(s) from the EMG and/or other sensors or transducers, and determining, by the processor, a gesture identification flag that corresponds to that determined gesture. Unless the specific context requires otherwise, throughout this specification and the appended claims “a” gesture identification flag should be interpreted in a general, inclusive sense as “at least one” gesture identification flag with the understanding that determining any number of gesture identification flags (e.g., determining one gesture identification flag, or determining multiple gesture identification flags) includes determining “a” gesture identification flag. Each gesture identification flag may include, or be represented by, one or more bits of information. Furthermore, “determining” a gesture identification flag by a processor may be achieved through a wide variety of different techniques. For example, a processor may determine a gesture identification flag by performing or otherwise effecting a mapping between gestures (e.g., between EMG and/or accelerometer signals representative of gestures) and gesture identification flags (e.g., by invoking a stored look-up table or other form of stored processor-executable instructions providing and/or effecting mappings between gestures and gesture identification flags), or a processor may determine a gesture identification flag by performing an algorithm or sequence of data processing acts (e.g., by executing stored processor-executable instructions dictating how to determine a gesture identification flag based at least in part on one or more signal(s) provided by at least one EMG sensor and/or at least one accelerometer).
As shown in mapping 500, each gesture identification flag may, for example, comprise a bit string (e.g., an 8-bit data byte as illustrated) that uniquely maps to a corresponding gesture performed by a user. For example, a “gun” or “point” hand gesture may correspond/map to gesture identification flag 00000001 as illustrated, a “thumbs up” gesture may correspond/map to gesture identification flag 00000010 as illustrated, a “fist” gesture may correspond/map to gesture identification flag 00000011 as illustrated, and a “rock on” gesture may correspond/map to gesture identification flag 00000100 as illustrated. A person of skill in the art will appreciate that an 8-bit data byte can be used to represent 256 unique gesture identification flags (corresponding to 256 unique gestures). In practice, gesture identification flags having any number of bits may be used, and if desired, multiple gestures may map to the same gesture identification flag and/or the same gesture may map to multiple gesture identification flags. In accordance with the present systems, articles, and methods, a gesture identification flag contains only information that identifies (i.e., maps to) a gesture performed by a user of a wearable EMG device. A gesture identification flag does not contain any information about a function or operation that the corresponding gesture may be used to control. A gesture identification flag does not contain any information about any downstream electronic device and/or application that the corresponding gesture may be used to control. A gesture identification flag may be appended, adjoined, supplemented, or otherwise combined with additional data bits as needed for, e.g., the purposes of telecommunications.
Mapping 500 represents gestures with actual illustrations of hands solely for ease of illustration and description. In practice, a gesture may be represented by any corresponding configuration of signals provided by at least one EMG sensor and/or at least one accelerometer. For example, a gesture may be represented by a particular signal waveform, a particular signal value, or a particular configuration/arrangement/permutation/combination of signal waveforms/values.
The present systems, articles, and methods describe human-electronics interfaces. Methods 300 and 400 provide methods of operating a wearable EMG device to control an unspecified electronic device (e.g., methods of operating device 100 from
Acts 601, 602, 603, and 604 are substantially similar to acts 301, 302, 303, and 304 (respectively) of method 300 from
At 611, the gesture identification flag that is transmitted or output by the output terminal of the wearable EMG device at 604 is received by the input terminal of the electronic device. As previously described, transmission of gesture identification flags between the wearable EMG device and the electronic device may be through a wired or wireless communicative link (e.g. communicative link 290 from
At 612, a second processor on-board the electronic device determines a function of the electronic device based at least in part on the gesture identification flag received by the input terminal of the electronic device at 611. As described previously, the electronic device may include a non-transitory processor-readable storage medium or memory that stores an API or other information or data structures (e.g., implemented as one or library(ies)) through which a user may define mappings (i.e., processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the electronic device, and/or the non-transitory processor-readable storage medium may store processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
At 613, the function determined at 612 is performed by the electronic device. The function may be any function or operation of the electronic device. For example, if the electronic device is an audio and/or video player (or a computer running an application that performs audio and/or video playback), then the corresponding function may be a PLAY function that causes the audio/video to play, a STOP function that causes the audio/video to stop, a REWIND function that causes the audio/video to rewind, a FAST FORWARD function that causes the audio/video to fast forward, and so on.
Throughout this specification and the appended claims, reference is often made to “determining a function of an electronic device based at least in part on a gesture identification flag.” Unless the specific context requires otherwise, throughout this specification and the appended claims “a” function should be interpreted in a general, inclusive sense as “at least one” function with the understanding that determining any number of functions (e.g., determining one function, or determining multiple functions) includes determining “a” function. Furthermore, “determining” a function by a processor may be achieved through a wide variety of different techniques. For example, a processor may determine a function by employing a defined mapping between gesture identification flags and functions (e.g., by invoking a stored look-up table or other form of stored processor-executable instructions providing defined mappings between gesture identification flags and functions), or a processor may determine a function by performing an algorithm or sequence of data processing steps (e.g., by executing stored processor-executable instructions dictating how to determine a function based at least in part on one or more gesture identification flag(s)).
For the illustrative example of
In accordance with the present systems, articles, and methods, processor-executable instructions that embody and/or produce/effect a mapping from gestures to gesture identification flags (e.g., mapping 500 from
In accordance with the present systems, articles, and methods, an electronic device may store multiple mappings (e.g., multiple sets of processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the electronic device, and when the electronic device receives a gesture identification flag it may perform a corresponding function based on the implementation of one of the multiple stored mappings (e.g., one or more of the multiple sets of processor-executable instructions). For example, the electronic device may be a computer such as a desktop computer, a laptop computer, a tablet computer, or the like. The computer may include a non-transitory processor-readable storage medium or memory that stores multiple mappings (e.g., multiple sets of processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the computer (e.g., multiple variants of mapping 700 from
In accordance with the present systems, articles, and methods, a wearable EMG device may be used to control multiple electronic devices, or multiple applications within a single electronic device. Such is distinct from known proposals for human-electronics interfaces that employ a wearable EMG device, at least because the known proposals typically store a direct mapping from gestures to functions within the wearable EMG device itself, whereas the present systems, articles, and methods describe an intermediate mapping from gestures (e.g., from EMG and/or accelerometer signals representative of gestures) to gesture identification flags that are stored and executed by the wearable EMG device and then mappings from gesture identification flags to functions that are stored and executed by the downstream electronic device. In accordance with the present systems, articles, and methods, the mapping from gestures to gesture identification flags stored and executed by the wearable EMG device is independent of the downstream electronic device and the same mapping from gestures to gesture identification flags may be stored and executed by the wearable EMG device regardless of the nature and/or function(s) of the downstream electronic device.
The implementation of gesture identification flags as described herein enables users to employ the same wearable EMG device to control a wide range of electronic devices and/or a wide range of applications within a single electronic device. Since the gesture identification flags output by the wearable EMG device are not tied to any specific functions or commands, a user may define their own mappings (including their own techniques for performing mappings) between gesture identification flags and electronic device functions. For example, a user may adapt the human-electronics interfaces described herein to control virtually any functions of virtually any electronic device (e.g., to control virtually any application executed by a computer) by defining processor-executable instructions that embody and/or produce a corresponding mapping between gesture identification flags and electronic device functions (such as mapping 700 from
The various embodiments described herein provide human-electronics interfaces in which a wearable EMG device (i.e., a controller) provides generic signal “flags” and a downstream receiving device interprets and responds to the generic flags. The flags provided by the wearable EMG device are substantially independent of any downstream receiving device. In accordance with the present systems, articles, and methods, other forms of controllers (i.e., controllers that are not wearable and/or controllers that do not employ EMG sensors) may similarly be configured to provide generic flags in this way. For example, instead of or in addition to employing EMG sensors and/or accelerometers providing gesture control, a controller that operates in accordance with the present systems, articles, and methods may employ, for example, tactile sensors (e.g., buttons, switches, touchpads, or keys) providing manual control, acoustic sensors providing voice-control, optical/photonic sensors providing gesture control, or any other type(s) of user-activated sensors providing any other type(s) of user-activated control. Thus, the teachings of the present systems, articles, and methods may be applied using virtually any type of controller employing sensors (including gesture-based control devices that do not make use of electromyography or EMG sensors), with the acts described herein as being performed by “at least one EMG sensor” and/or “at least one accelerometer” being more generally performed by “at least one sensor.”
The above description of illustrated embodiments, including what is described in the Abstract, is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Although specific embodiments of and examples are described herein for illustrative purposes, various equivalent modifications can be made without departing from the spirit and scope of the disclosure, as will be recognized by those skilled in the relevant art. The teachings provided herein of the various embodiments can be applied to other portable and/or wearable electronic devices, not necessarily the exemplary wearable electronic devices generally described above.
For instance, the foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, schematics, and examples. Insofar as such block diagrams, schematics, and examples contain one or more functions and/or operations, it will be understood by those skilled in the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, the present subject matter may be implemented via Application Specific Integrated Circuits (ASICs). However, those skilled in the art will recognize that the embodiments disclosed herein, in whole or in part, can be equivalently implemented in standard integrated circuits, as one or more computer programs executed by one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs executed by on one or more controllers (e.g., microcontrollers) as one or more programs executed by one or more processors (e.g., microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of ordinary skill in the art in light of the teachings of this disclosure.
When logic is implemented as software and stored in memory, logic or information can be stored on any computer-readable medium for use by or in connection with any processor-related system or method. In the context of this disclosure, a memory is a computer-readable medium that is an electronic, magnetic, optical, or other physical device or means that contains or stores a computer and/or processor program. Logic and/or the information can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions associated with logic and/or information.
In the context of this specification, a “non-transitory computer-readable medium” can be any element that can store the program associated with logic and/or information for use by or in connection with the instruction execution system, apparatus, and/or device. The computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. More specific examples (a non-exhaustive list) of the computer readable medium would include the following: a portable computer diskette (magnetic, compact flash card, secure digital, or the like), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), a portable compact disc read-only memory (CDROM), digital tape, and other non-transitory media.
The various embodiments described above can be combined to provide further embodiments. To the extent that they are not inconsistent with the specific teachings and definitions herein, all of the U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the Application Data Sheet, including but not limited to: U.S. Provisional Patent Application Ser. No. 61/869,526; U.S. Provisional Patent Application Ser. No. 61/857,105 (now U.S. Non-Provisional patent application Ser. No. 14/335,668); U.S. Provisional Patent Application Ser. No. 61/752,226 (now U.S. Non-Provisional patent application Ser. No. 14/155,107); U.S. Provisional Patent Application Ser. No. 61/768,322 (now U.S. Non-Provisional patent application Ser. No. 14/186,889); U.S. Provisional Patent Application Ser. No. 61/771,500 (now U.S. Non-Provisional patent application Ser. No. 14/194,252); U.S. Provisional Application Ser. No. 61/860,063 (now U.S. Non-Provisional patent application Ser. No. 14/276,575), and U.S. Provisional Application Ser. No. 61/866,960 (now U.S. Non-Provisional patent application Ser. No. 14/461,044), are incorporated herein by reference, in their entirety. Aspects of the embodiments can be modified, if necessary, to employ systems, circuits and concepts of the various patents, applications and publications to provide yet further embodiments.
These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.
Claims
1. A wearable electromyography (“EMG”) device comprising:
- at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity;
- a processor communicatively coupled to the at least one EMG sensor, the processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor; and
- an output terminal communicatively coupled to the processor to in use transmit the gesture identification flag.
2. The wearable EMG device of claim 1 wherein the gesture identification flag is independent of any downstream processor-based device and generic to a variety of end user applications executable by a variety of downstream processor-based devices useable with the wearable EMG device.
3. The wearable EMG device of claim 1, further comprising:
- a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags.
4. The wearable EMG device of claim 3 wherein the non-transitory processor-readable storage medium stores processor-executable instructions that embody and/or produce a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the processor, the processor-executable instructions cause the processor to determine a gesture identification flag in accordance with the mapping.
5. The wearable EMG device of claim 3 wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the processor, cause the processor to determine a gesture identification flag based at least in part on at least one signal provided by the at least one EMG sensor.
6. The wearable EMG device of claim 1, further comprising:
- at least one accelerometer responsive to motion effected by the user of the wearable EMG device and communicatively coupled to the processor to provide at least one signal in response to the detected motion, and wherein the processor determines the gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least one signal provided by the at least one accelerometer.
7. The wearable EMG device of claim 1 wherein the processor is selected from the group consisting of: a digital microprocessor, a digital microcontroller, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a programmable gate array, and a programmable logic unit.
8. The wearable EMG device of claim 1 wherein the at least one EMG sensor includes a plurality of EMG sensors, and wherein the wearable EMG device further comprises:
- a set of communicative pathways to route signals provided by the plurality of EMG sensors to the processor, wherein each EMG sensor in the plurality of EMG sensors is communicatively coupled to the processor by at least one communicative pathway from the set of communicative pathways.
9. The wearable EMG device of claim 8, further comprising:
- a set of pod structures that form physically coupled links of the wearable EMG device, wherein each pod structure in the set of pod structures is positioned adjacent and physically coupled to at least one other pod structure in the set of pod structures, and wherein the set of pod structures comprises at least two sensor pods and a processor pod, each of the at least two sensor pods comprising a respective EMG sensor from the plurality of EMG sensors and the processor pod comprising the processor.
10. The wearable EMG device of claim 9 wherein each pod structure in the set of pod structures is positioned adjacent and in between two other pod structures in the set of pod structures and physically coupled to the two other pod structures in the set of pod structures, and wherein the set of pod structures forms a perimeter of an annular configuration.
11. The wearable EMG device of claim 9, further comprising:
- at least one adaptive coupler, wherein each respective pod structure in the set of pod structures is adaptively physically coupled to at least one adjacent pod structure in the set of pod structures by at least one adaptive coupler.
12. The wearable EMG device of claim 1 wherein the output terminal includes at least one of a wireless transmitter and/or a tethered connector port.
13. The wearable EMG device of claim 1 wherein the at least one EMG sensor includes at least one capacitive EMG sensor.
14. A method of operating a wearable electromyography (“EMG”) device to provide electromyographic control of an electronic device, wherein the wearable EMG device includes at least one EMG sensor, a processor, and an output terminal, the method comprising:
- detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor;
- providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity;
- determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor, wherein the gesture identification flag is independent of the electronic device; and
- transmitting the gesture identification flag to the electronic device by the output terminal.
15. The method of claim 14 wherein:
- detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor includes detecting muscle activity of the user of the wearable EMG device by a first EMG sensor and by at least a second EMG sensor,
- providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity includes providing at least a first signal from the first EMG sensor to the processor in response to the detected muscle activity and providing at least a second signal from the second EMG sensor to the processor in response to the detected muscle activity, and
- determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes determining, by the processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from the at least a second EMG sensor to the processor.
16. The method of claim 14 wherein the wearable EMG device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes executing the processor-executable instructions by the processor to cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor.
17. The method of claim 14 wherein the wearable EMG device further comprises at least one accelerometer, and wherein the method further comprises:
- detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and
- providing at least one signal from the at least one accelerometer to the processor in response to the detected motion,
- and wherein determining a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes:
- determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
18. The method of claim 17 wherein the wearable EMG device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor includes executing the processor-executable instructions by the processor to cause the processor to determine the gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
19. The method of claim 14 wherein the output terminal includes a wireless transmitter, and wherein transmitting the gesture identification flag to the electronic device by the output terminal includes wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
20. A system that enables electromyographic control of an electronic device, the system comprising:
- a wearable electromyography (“EMG”) device comprising: at least one EMG sensor responsive to muscle activity of a user of the wearable EMG device and provide at least one signal in response to a detected muscle activity, a first processor communicatively coupled to the at least one EMG sensor, the first processor which in use determines a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor, and an output terminal communicatively coupled to the first processor to transmit the gesture identification flag;
- and
- an electronic device comprising: an input terminal to receive the gesture identification flag, and a second processor communicatively coupled to the input terminal, the second processor which in use determines a function of the electronic device based at least in part on the gesture identification flag.
21. The system of claim 20 wherein the gesture identification flag is independent of the electronic device and generic to a variety of end user applications executable by the electronic device.
22. The system of claim 20 wherein the wearable EMG device of the system further comprises:
- a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags.
23. The system of claim 22 wherein the non-transitory processor-readable storage medium of the wearable EMG device stores processor-executable instructions that embody and/or produce a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the first processor, the processor-executable instructions cause the first processor to determine a gesture identification flag in accordance with the mapping.
24. The system of claim 20 wherein the wearable EMG device of the system further comprises:
- a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the first processor, cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor.
25. The system of claim 20 wherein the wearable EMG device of the system further comprises:
- at least one accelerometer communicatively coupled to the first processor, the at least one accelerometer responsive to motion effected by the user of the wearable EMG device and which provides at least one signal in response to a detected motion, and wherein the first processor determines a gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least on signal provided by the at least one accelerometer.
26. The system of claim 20 wherein the electronic device of the system further comprises:
- a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores at least a set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
27. The system of claim 20 wherein the electronic device of the system further comprises:
- a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores:
- a first application executable by the electronic device;
- at least a second application executable by the electronic device;
- a first set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the first application based at least in part on a gesture identification flag; and
- a second set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the second application based at least in part on a gesture identification flag.
28. The system of claim 20 wherein the output terminal of the wearable EMG device includes a first tethered connector port, the input terminal of the electronic device includes a second tethered connector port, and further comprising:
- a communicative pathway that in use communicatively couples the first tethered connector port to the second tethered connector port and to route the gesture identification flag from the output terminal of the wearable EMG device to the input terminal of the electronic device.
29. The system of claim 20 wherein the output terminal of the wearable EMG device includes a wireless transmitter that in use wirelessly transmits the gesture identification flag and the input terminal of the electronic device includes a tethered connector port, and wherein the system further comprises:
- a wireless receiver that in use communicatively couples to the tethered connector port of the electronic device and to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
30. The system of claim 20 wherein the output terminal of the wearable EMG device includes a wireless transmitter to wirelessly transmit the gesture identification flag, and wherein the input terminal of the electronic device includes a wireless receiver to wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
31. The system of claim 20 wherein the electronic device is selected from the group consisting of: a computer, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and a vehicle.
32. A method of electromyographically controlling at least one function of an electronic device by a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes at least one EMG sensor, a first processor, and an output terminal and the electronic device includes an input terminal and a second processor, the method comprising:
- detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor;
- providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity;
- determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor, wherein the gesture identification flag is independent of the electronic device;
- transmitting the gesture identification flag by the output terminal of the wearable EMG device;
- receiving the gesture identification flag by the input terminal of the electronic device;
- determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag; and
- performing the function by the electronic device.
33. The method of claim 32 wherein:
- detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor includes detecting muscle activity of the user of the wearable EMG device by a first EMG sensor of the wearable EMG device and by at least a second EMG sensor of the wearable EMG device,
- providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity includes providing at least a first signal from the first EMG sensor to the first processor in response to the detected muscle activity and providing at least a second signal from the send EMG sensor to the first processor in response to the detected muscle activity, and
- determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes determining, by the first processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the first processor and the at least a second signal provided from the at least a second EMG sensor to the first processor.
34. The method of claim 32 wherein the wearable EMG device further comprises a non-transitory processor-readable medium that stores processor-executable instructions, and wherein determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes executing the processor-executable instructions by the first processor to cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor.
35. The method of claim 32 wherein the wearable EMG device further comprises at least one accelerometer, and wherein the method further comprises:
- detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and
- providing at least one signal from the at least one accelerometer to the first processor in response to the detected motion,
- and wherein determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes:
- determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor and the at least one signal provided by the at least one accelerometer to the first processor.
36. The method of claim 32 wherein the output terminal of the wearable EMG device includes a wireless transmitter and the input terminal of the electronic device includes a wireless receiver, and wherein:
- transmitting the gesture identification flag by the output terminal of the wearable EMG device includes wirelessly transmitting the gesture identification flag by the wireless transmitter of the wearable EMG device, and
- receiving the gesture identification flag by the input terminal of the electronic device includes wirelessly receiving the gesture identification flag by the wireless receiver of the electronic device.
37. The method of claim 32 wherein the electronic device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag includes executing the processor-executable instructions by the second processor to cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
Type: Application
Filed: Aug 21, 2014
Publication Date: Feb 26, 2015
Inventors: Matthew Bailey (Kitchener), Stephen Lake (Kitchener), Aaron Grant (Kitchener)
Application Number: 14/465,194
International Classification: G06F 3/01 (20060101); A61B 5/0488 (20060101);
