Audio feedback for motor control training

An audio feedback signal for a subject engaged in an exercise includes a reference note (of a musical scale) that provides a marker of the goal or target position of the exercise for the subject, and a sequence of real-time feedback notes that define progress towards the goal while the exercise is underway. By providing the reference note with the feedback note, the listener can instantly understand whether they have presently reached the goal. The progress is defined by having at least 3 distinct feedback notes mapped to respective sensed ranges of positions within the exercise, and this provides multiple opportunities for the exerciser to compare the feedback and reference notes, and provides cognitive feedback to the exerciser.

Skip to: Description  ·  Claims  · Patent History  ·  Patent History
Description
FIELD OF THE INVENTION

The present invention relates, in general, to equipment for training of motor control using audio feedback and, in particular, relates to audio feedback for motor control training, and methods and apparatus for providing it.

BACKGROUND OF THE INVENTION

There are many situations where it is desirable for a subject to learn or relearn how to control muscles. For example, there are medical therapies and interventions, such as physiotherapy, as well as professional, athletic, and recreational training. In many instances, motor control exercises are needed to encourage subjects to achieve a goal that can be determined prior to commencing the exercise, and for which achievement of the goal is gradual such that, in real time, feedback can be provided to the subject to guide the subject throughout the exercise.

Music has been shown to have a strong pain-reducing effect; it also has demonstrated therapeutic qualities associated with distraction and stress reduction (relaxation response). Additionally, mapping sound to movement has been shown to provide patients with an attention goal that enhances their focus on, and motivation during, physiotherapy activities. Similarly the ability for sound to reduce stress and to provide focus-enhancing attention on a goal is useful in training a wide variety of subjects in various tasks. There is a need for audio feedback that is effective and simple to interpret for the public in general.

In a paper entitled Audio-Biofeedback For Balance Improvement: An Accelerometry-Based System by Chiari, L., Dozza, M., et al. (2005) published in IEEE Transactions on Biomedical Engineering, 52(12), (pp. 2108-2111), Chiari et al. comment on an audio-biofeedback system described in US 2007/0249466. The described audio-biofeedback system for conditioning the balance and motor co-ordination of a user comprises a system for acquisition of information relative to the kinematics of at least one part of the body of the user, a processing interface connected to the acquisition system to encode the information in a signal, and a pair of earphones operating between the processing interface and the user. The earphones emit a signal suitably defined by a stereophonic sound which can be transmitted in an audio channel. The audio-biofeedback according to Chiari et al. is a stereophonic signal modulated by volume, frequency, and balance. The audio signals sent to respective ears are preferably sinusoidal, and the frequency is modulated according to a linear sectional law, for example that ranges over 150-1000 Hz.

The paper elaborates on the audio signal provided, and notes one way to take advantage of human sound recognition. Humans are better able to recognize differences in sound if a reference sound is given for comparison. This system uses a reference region (RR); when the user's sway is restricted to within this region, the system emits a stereo, low volume (a few dB above hearing threshold) tone (400 Hz)—equivalent to middle G—which they call their “reference tone/sound”. When the user sways outside of the RR, the audio feedback changes in one or both ears to indicate direction and speed of sway. Although they refer to a reference tone or sound, that particular tone is only audible when the user is inside the safe range of sway. When he/she is outside of the RR, the reference tone is replaced with dynamically changing real-time audio feedback which relates directly to his/her direction and speed of movement.

As the reference tone disappears when the user sways out of the RR, no reference sound is, in fact, available for comparison. People are not generally very good at retaining a pitch in their head and making comparisons between heard and remembered sounds. Difficulty for a wide audience to discern the reference tone from the dynamic audio feedback is made all the more difficult because of the small differences in frequencies that are associated with small displacements in sway. It may be very difficult for many users to discern small changes in frequencies hearing only one at a time.

U.S. Pat. No. 7,033,281 to Carnahan et al. teaches an Augmented Kinematic Feedback Device and Method according to which a biomechanical angle of a body joint of a trainee during a given physical activity is continuously monitored, and a corresponding audible tracking signal is provided, in real time, to the trainee. According to Carnahan et al., this audible tracking signal augments kinematic feedback to the trainee regarding the physical activity and thereby facilitates ease or rapidity of training. In various embodiments, multiple body joints can be monitored, sensitivity of the tracking signal with respect to biomechanical angle excursions can be modified, tracking signals can be recorded for later playback, and a metronome beat can be provided in conjunction with the tracking signal. The biomechanical angle of the user corresponds to user-discernable audible signals. In the examples a frequency range from about 400 Hz to about 2 kHz is chosen.

It is noted that the movements to be trained with the system of Carnahan et al. are exemplified by fast actions for which no time is provided for cognitive feedback. As such Carnahan et al. is directed to a continuum of tones that are heard concurrent with or during a golf stroke (for example) to serve as an enhancement of native kinematic feedback, to accelerate training. Even if the system of Carnahan et al. were applicable to slower motions such as motions during which cognitive feedback can be provided, just like Chiari et al. the continuum of tones provided to the user (see FIGS. 3,4) according to Carnahan et al. are of the continuous frequency modulation type, and no comparative tone is taught or suggested. Small frequency differences will be difficult to discern for many subjects. Furthermore, it would not be helpful for Carnahan's stated purpose to provide a single comparative tone as this would complicate the signal unnecessarily, and not assist the listener in hearing the soundscape that is associated with desired and undesired strokes. Carnahan et al. relies on a subject's ability to relate a soundscape that consists of continuously varied frequencies heard during an activity to the post-activity evaluation of the result of his/her action in order to map one to the other.

In a paper by Huang, H., et al. entitled: Recent Developments in BioFeedback for Neuromotor Rehabilitation, (2006), Journal of Neurolmaging and Rehabilitation, 3(11), an immersive multimedia environment for biofeedback therapy is proposed. The focus is on task-oriented, repeated, reaching and grasping tasks. Musical audio feedback is provided to reflect the smoothness and temporal-spatial parameters of the endpoint of the reach trajectory in order to improve multi-joint coordination. Music notes within a musical phrase are distributed spatially along a trajectory path—the notes indicate the distance the hand has moved toward the target, with each note corresponding to a short distance along the path; the duration of any given note depends on the speed at which the patient's arm is moving—hence, patients can essentially compose melodies based on movement pattern.

The authors point out that studies have shown that music can synchronize motor outputs, improve the motor coordination of Parkinson patients, and enhance motor learning in a patient with large-fiber sensory neuropathy.

There remains a need for a simple and efficient audio feedback for training motor skills to a wide audience, especially one that is useful for guiding a subject through a cognitive exercise in which the feedback is interpreted and acted on in real time.

SUMMARY OF THE INVENTION

Applicant has invented a feedback technique that guides a subject during an exercise in moving towards a goal by providing audio feedback that permits intuitive distinction between non-goal and goal points within the exercise. In accordance with the invention, a persistent note (called herein a reference note) taken from a musical scale is provided for comparison with real-time feedback notes that each represent respective positions of the subject in the exercise as measured by a sensor. The reference note is related to the goal of the exercise, in that it matches a feedback note associated with the goal in a distinctive manner. For example, the feedback note of the goal and reference note may be the same, one or more octave(s) apart, or simply much more consonant with each other than at least the feedback notes associated with the positions nearing the goal. During the course of the exercise, the reference note is emitted with the appropriate (based on the position of a part of the subject) real-time feedback note; together they provide feedback that can be instantly discerned by most subjects who are not hearing impaired, permitting direct determination as to whether the goal is reached.

In accordance with the invention an audio feedback signal for a subject engaged in an exercise is provided. The audio feedback signal includes a reference note taken from a musical scale that provides a relatively persistent marker of a goal for the subject, and a sequence of real-time feedback notes of the scale. Each feedback note is played with the reference note for a duration sufficient to permit a subject to compare the notes. The specific feedback note played at an instant depends on where the subject is within the exercise, as detected by a sensor. One of the feedback notes corresponds to the subject reaching the goal, and this feedback note matches the reference note in a distinctive manner.

Preferably there are at least 3 distinct feedback notes corresponding to respective positions within the exercise, whereby progress towards the goal is indicated to the subject by the changing of the audio feedback signal in real-time, and the subject repeatedly compares the feedback note with the reference note during the exercise. More preferably there are at least 4 distinct feedback notes or at least 5 distinct feedback notes.

The association of the feedback notes to the positions within the exercise may be defined by a fixed map. The association may be pre-established in any number of ways (as a map, as a method of computing the mapping in dependence on history, etc.), may be dynamically computed (e.g., to change in response to exercise execution parameters), or may be instantiated as a neural network that changes in response to execution of the present or previous repetitions of the exercise, for example. In some embodiments a fixed map provides a corresponding note in the scale for each of a predefined set of sensed ranges of positions that the subject could assume during the exercise. For example, from an exercise range (i.e., a range of positions within the exercise over which the sensor is adapted to detect the position), a plurality of the sensed ranges may be chosen. Each of the sensed ranges is a proper subset of the exercise range. The mapping may provide one scale note mapped to each sensed range. At each point in time in the exercise, the last sensed position may lie in one or more of these sensed ranges, and the real-time feedback note may include the scale notes associated with these one or more sensed ranges.

In preferred embodiments the sensed ranges partition the exercise range (i.e., the sensed ranges are mutually exclusive and collectively exhaustive). As such, exactly one feedback note is played at a time throughout at least a phase of the exercise. For example, the partitioning may lead to substantially equal time for each feedback note for an expected execution of the exercise, may have equal divisions of the exercise range, or may lead to a shorter time for feedback notes during an expected difficult segment of the exercise, or one that requires greater attention from the subject. While it may be easiest to distinguish a single feedback note from a single reference note at a time, the sensed ranges do not have to be disjoint. For example, a small amount of overlap during which the previous note fades out while the next feedback note fades in provides for a blending of the notes and an overlap of the sensed ranges according to one of several possible schemes for transitions between feedback notes/ranges. Furthermore, while it may be preferred to provide feedback continuously, or substantially continuously, the sensed ranges do not have to collectively exhaust the exercise range, as another scheme for transition would be to have them fade in and out of silence. Such transition schemes may be preferable if it is desirable to avoid noise from rapid transitions between sensed ranges, for a particular application.

Preferably, the exercise range is large enough, the sensed ranges are sufficiently distributed within the exercise range, and the exercise is expected to be slow enough that the subject can hear the emitted reference and feedback signal as musical feedback indicating a proximity to the goal during the exercise.

In some embodiments the mapping is constant for multiple repetitions of the exercise, and so the subject may associate specific feedback notes with specific ranges of positions. In some embodiments, the association of feedback notes with positions is in frequency order with respect to proximity to the goal. In some embodiments the feedback notes are chosen in relation to a consonance/dissonance of the scale note with respect to the reference note.

The feedback note mapped to the goal may distinctly match the reference note as the two notes may be consonant to a higher degree than any of the other feedback notes, or to any feedback note other than a feedback note associated with a starting position of the exercise. For example the feedback note mapped to the goal and the reference note may be the same, or one or more octave(s) apart, and at least some of the other sensed ranges (especially those that are proximate to the goal) may be mapped to scale notes that are substantially dissonant, or markedly less consonant with the reference note.

In accordance with the invention a method for motor control training is provided. The method involves providing a sensor for detecting a position of a part of a subject during an exercise in relation to a goal; and generating an audio feedback signal including a reference note of a musical scale that provides a marker of a goal for the subject, and a sequence of real-time feedback notes of the scale. Each feedback note is played with the reference note for a duration sufficient to permit the subject to compare the notes, and is played when the sensor detects that the subject is in a corresponding position within the exercise. One of the feedback notes corresponds to the subject reaching the goal, and this note matches the reference note in a distinctive manner.

Generating the sequence of feedback notes may involve generating at least 3 distinct feedback notes corresponding to respective positions within the exercise from a starting position of the exercise to the goal. In this manner, progress towards the goal is indicated to the subject by the changing of the audio feedback signal in real-time, and the subject repeatedly compares the feedback note with the reference note during the exercise.

The sensor may be provided for continuously detecting a position of the subject in the exercise over a range of positions that includes a plurality of sensed ranges that are mapped to respective feedback notes to uniquely associate a note with each of the sensed ranges. The plurality of sensed ranges may be distributed within the range of positions so that for the expected duration of the exercise, a subject can register the audio signal as musical feedback. The feedback notes may be associated with positions as described above.

The sensor provided may be adapted to sense a state of flexure of a muscle set directly or indirectly, in any manner known in the art. Preferably the method is non-invasive or minimally invasive. Preferably it is readily configured and set up, and is accurate.

An apparatus for motor control training is also provided. The apparatus includes a sensor for outputting a position of a part of a subject during an exercise, a signal processor for receiving position output by the sensor and controlling an emitter to emit an audio feedback signal, and the emitter for playing the audio feedback signal to the subject. The audio feedback signal being that defined above.

The emitter provided may include two emitters or two channels of a stereophonic emitter, for example. If two channels are used, and the reference note is the same as a note associated with the goal state of the exercise, the subject will be understood to be seeking to match the notes heard in the two ears.

Further features of the invention will be described or will become apparent in the course of the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the invention may be more clearly understood, embodiments thereof will now be described in detail by way of example, with reference to the accompanying drawings, in which:

FIG. 1 is a schematic illustration of an apparatus in accordance with an embodiment of the invention;

FIG. 2 is a schematic illustration of an example of an apparatus in accordance with an embodiment of the invention applicable to conditioning muscles controlling a joint;

FIG. 3 is a schematic illustration of a mapping of joint angles to feedback notes used in the apparatus of FIG. 2; and

FIGS. 4a-c are screen-shots of principal graphical user interfaces of a system using the apparatus of FIG. 2.

 DESCRIPTION OF PREFERRED EMBODIMENTS

Herein a form of audio feedback is disclosed that is useful in motor control training exercises, including physiotherapy of all kinds, when a fixed goal is known, an ordered set of steps towards that goal are defined, and a detector is available for measuring a subject's position within the exercise in relation to the ordered set of steps. While this can be used for recreational or athletics training, in certain circumstances, or in training of professional manual skills, the particular advantages of pain and stress reduction and muscle relaxation associated with music can be leveraged in physio-therapeutic exercises. The invention is applicable to all areas of physiotherapy (including orthopedics, prosthetics, occupational, neurology, cardiopulmonary, geriatric, rheumatology, etc.) and may be for conditioning, habilitation, or rehabilitation, such as after surgery, accident, or trauma. The invention applies to exercises for pressure or weight distribution training (e.g., in physiotherapy boots or in a seat/wheelchair), standing postural training, sit-to-stand postural training, passive manipulation, head positioning training or seated postural training for paraplegics or those with other disabilities.

In sports/athletics training, the invention may have particular application for muscle flexion training, flexibility training, or in static exercise training such as may be desired in yoga. Furthermore, there are numerous examples where someone is required to reach a body position or joint angle before starting an activity (a dive, a move, an action, etc.). For example, to execute a back handspring, it is useful for the gymnast to lean back so that an ankle is bent more than about 80° before jumping. Similarly, before a serve (racket sports, volleyball, etc.), a swing (golf, baseball, etc.), or other serve, a desired posture of the athlete may be corrected or verified with an apparatus in accordance with the invention.

In recreation, the invention may apply to train a user in a manner of holding or a posture while holding a musical instrument, a manner of holding a paint brush or other applicator, or a specific position in dance, especially when training a limb out of a field of view of the dancer, or when the person has no frame of reference to determine whether a part of their body is in a desired position.

Professional skills training may also require exercises that can be performed using the present invention. For example, a device for ergonomics training such as postural alignment when sitting at a desk, repetitive strain injury (RSI) prevention, etc., may make use of audio feedback in accordance with the present invention.

It should be noted that the term ‘exercise’ herein is intended to cover motions of a body or part towards or away from an objective, but could also apply to postural conditioning that is performed, for example at a desk or at a workstation, with a view to minimizing risks of injury. In some cases the user might not consider the feedback to be provided during an ‘exercise’, and might consider the actions not for practice, but the feedback is nonetheless provided with a view to ameliorating or monitoring cognitive control over respective muscles and to this extent is an exercise.

Other examples of simple tasks that are independently taught in construction, mechanics training, surgery, precious stone cutting, or other precision cutting training where a single angle, force, or constrained position of a device in relation to a user's state of muscle flexure is of great importance. For example, a lifting task requiring pinching an object without crushing it, surgically actuating a stent or other device with only haptic feedback as to a state of the device, etc. Another example is in the military application of firearm training, for example, in terms of the holding position of a firearm relative to a target.

The audio feedback signal in accordance with the present invention provides a reference note and a feedback note together. The reference note is constant throughout (a part of) the exercise and the feedback note varies with a proximity of the subject to a goal state of the exercise. In general the reference note and feedback notes define a scale, such as the chromatic scale, or selected parts of such a scale.

The feedback notes and/or reference note may be continuously played throughout the whole exercise (assuming the whole motion of the muscle is mapped to respective notes), may be continuously played during a phase of the exercise, or may be played intermittently, for example with fixed duration pulses. If intermittent, the pulse duration is long enough for clear discernment by the subject, such as a few seconds. The reference note has a greater persistence than the feedback notes which vary throughout the exercise. In accordance with the invention there is substantial overlap between the feedback notes and the reference note at least throughout an active phase of the exercise.

While the reference note and real-time feedback notes may be provided on a single channel, it may be preferred to provide them on different audio channels (for stereophonic output) or otherwise by separate emitters. For example, by supplying the reference note to one ear and the feedback notes to the other, a mental activity is provided for correlating the signals that may have advantages for distracting a subject from a pain response, and focusing the subject on the music and associated motion. Having the signals on separate channels delivered with appropriate stereo panning may also make it more immediately obvious to a subject when the real-time feedback notes approach and then match the reference note because spatial separation of the tone information changes aural perception, and because two notes are typically “blurred” to some degree when delivered together on the same mono signal.

The reference note provides a continuous marker of a goal for the subject in that it does not change throughout the exercise, and preferably does not change throughout multiple repetitions of the exercise. If the mapping is constant for multiple repetitions of the exercise, the subject may associate specific feedback notes to specific positions within the exercise, which may be desirable. If a number of repetitions of the exercise is desired, the scale may be transposed in successive repetitions to vary the note pattern in a somewhat predictable manner while providing some variation for the subject. For example, this may be done to inform the subject that they have passed to another level of difficulty, or that some other parameter of the exercise is changing. If greater variation is desired, the notes can be randomly (typically according to some pattern) selected in each repetition, or after a preset event.

Preferably the feedback notes of the scale define progress towards the goal while the exercise is underway by way of a mapping between the feedback notes to respective ranges of positions within the exercise. A sensor is provided for sensing a position of a (part of a) subject within the exercise. When the sensed position is within one of the respective ranges of positions, the scale note associated with the range is played. The mapping of the feedback notes may be in (descending or ascending) frequency order as this may match expectations of the subject. In general the selection of notes is an aesthetic choice, that provides important qualities to the system, in terms of the subject's enjoyment, and likelihood of adopting the system. Frustration may be experienced when it is not clear to the subject whether the notes match or not, and advantageously dissonant scale notes (with respect to the reference note) can be used to facilitate distinctiveness of the matching (goal feedback note) and non-matching feedback notes. Careful selection of the notes is part of creating a useful system that is likely to be adopted.

While the exercise generally has a goal, it will be noted that this goal may not be unique, as there may be multiple goal states. It is possible that multiple goal states are all contiguous, in which case the goal states may be viewed as sub-states of a single goal. In other embodiments, the goal states may not be continguous. For example, and exercise may consist in a cycle between two goal states, as opposed to a linear motion from a rest state to a goal state.

Preferably, for an exercise range, the range of positions within the exercise range detected by the sensor is large enough, the sensed ranges are large enough and sufficiently distributed within the exercise range, and the exercise is expected to be slow enough that the subject can hear the emitted reference and feedback signal as musical feedback indicating a proximity to the goal during the exercise. In this way the feedback serves as cognitive feedback for the subject of the exercise.

Preferably at most one feedback note is played at a time, and the sensed ranges are disjoint. Preferably the ranges span a continuous part of the exercise range, so that every physical position within an exercise is associated with one real-time feedback note within the continuous part of the exercise range. A feedback note associated with the initial range may be the same note as the goal note in some embodiments. This may give the subject an indication of what the goal note is to listen for.

The feedback notes may be chosen from the scale principally for easy discernment of the feedback notes from the reference note. The scale notes chosen for the feedback may be chosen in relation to a consonance of the scale note with respect to the reference note. For example, at points in the exercise where the subject would be expected to have difficulty, the feedback notes may be more consonant with the reference note and to discourage lingering at a point a more dissonant note may be chosen. The sensed ranges may provide substantially equal time for each feedback note in an expected execution of the exercise, or may provide smaller ranges in the neighbourhood of points in the exercise that are expected to require more focus, or would be expected to be more painful, or otherwise difficult.

In some embodiments, one of the sensed ranges is a goal range, which corresponds to achievement of the goal of the exercise. The feedback note mapped to the goal range is preferably more consonant with the reference note than any of the intermediate ranges (from an initial range to the goal range). The goal range feedback note may be the same scale note as the reference note, one or more scales above or below the reference note, or may simply be more consonant than any of the intermediate range feedback notes, for example. Applicant has found that the judicious use of at least some scale notes that are dissonant with the reference note can be useful, at least for facilitating discernment of the notes.

FIG. 1 is a schematic illustration of an embodiment of an apparatus for providing an audio feedback signal in accordance with an aspect of the invention. The apparatus includes a sensor 12 for detecting (directly or indirectly) a position of a muscle of a subject during an exercise. A wide variety of sensors 12 are possible. Some sensors 12 are invasive, some require different degrees of surface contact with the subject, and others can provide adequate sensing from a distance. Some sensors 12 detect gross changes such as angles or positions of joints with respect to other joints, or any applied frame of reference, some may detect muscle response from secondary indicators such as blood flow, or temperature. Pressure on a tensile sensor wrapped around a muscle group may be useful for determining a state of flexure of a muscle. Examples of devices are: goniometers, displacement actuators, force sensors, and optical image analysis software with a video image, etc., although thermometers or IR cameras, and other sensors could equally be used. Indirectly muscle motion can be monitored by devices connected to a harness of various kinds or by a user's touching or moving an object to which the sensor is coupled. The sensor may be coupled to an implement or device held or manipulated by the subject, and may register a force, an angle, a distance, a time, etc., in which case the state of the muscle may be inferred from the state of the implement.

An output from the sensor 12 may be output to a feedback control processor 14 in any manner known in the art, but may preferably be by electrical, or electromagnetic signaling. The feedback control processor 14 receives the sensor output, uses the sensor output to select a note on the scale corresponding to feedback associated with the sensor output. For example, the note may be provided by a fixed, predetermined mapping as described above, or by a process for computing the map, and may be changed in one way or another by the subject, an administrator of the exercise, or others. The mapping can be varied in terms of the number of sensor ranges, size of the sensor ranges, or notes associated with the sensor ranges, for example, in response to previous repetitions of the exercise, or sensor information regarding how the present exercise is being performed, for example.

The feedback control processor 14 controls an emitter 16 to output an audio feedback signal 18 that includes the reference note and the real-time feedback note which are supplied together at least intermittently, and for a duration sufficient for cognitive comparison of the notes. The audio output signal 16 is emitted by an emitter 18. The audio output signal 16 may be a stereophonic signal sent to headphones worn by the subject, may be separate signals sent to different speakers, or may further be mixed onto a monophonic audio signal output. Well known electrical equipment and devices are suitable for use in the invention and adaptation of such devices is well within the scope of those skilled in the respective arts.

Example

Applicant has produced a joint angle rehabilitation/conditioning system (AudioPhys) for providing audio feedback during physiotherapy that meets the following needs:

    • ability to collect, store, and present data generated during patient exercises (progress data) since it can be used to conduct quantitative progress analysis and to allow for quantitative progress measures;
    • allows therapists to vary, monitor, and evaluate patients' exercises by adjusting difficulty levels during therapy sessions, and for difficulty levels to be set for a patient based on the patient's previous performance, stage of rehabilitation, and general capabilities—for example, permitting measures of success to be based on an unaffected limb of the patient, when suitable;
    • audio feedback parameters and delivery can be customized to the individual preferences of patients and the individual requirements of each patient given the extent of variance across patients in capability and stage of rehabilitation;
    • able to be set up for a patient within 3 minutes;
    • leverages the effect music has been shown to have on pain reduction, the demonstrated therapeutic qualities associated with distraction and stress reduction (relaxation response), and the focus enhancing and motivation provided by mapping of sound to movement during physiotherapy activities;
    • provides a patient with configuration control (such that the technology is habituated as an extension of the user's own body), is comfortable, and encourages participation, fostering adoption by patients; and
    • provides a fun, interesting, and mentally stimulating exercise format.

AudioPhys has been designed to support physiotherapeutic motor rehabilitation of clients using musical-based audio cues relative to a client's specific joint angle. AudioPhys has been designed to be mobile/portable and sympathetic to current physiotherapeutic processes. It has been designed to support in-session activities and for out-patient use (i.e., for patients to use it in their own home between physiotherapy sessions). AudioPhys is a mobile system and practice exercises can be set by the physiotherapist with a view to providing additional feedback outside of sessions with clients, and accelerating rehabilitation. Additionally AudioPhys can record the inter-session exercises.

FIG. 2 is a schematic illustration of the AudioPhys system for conditioning the muscles of a user's 20 joints. While the knee joint sensors have been used and tested, it will be appreciated that skeletal muscles for other joints (hip, elbow, etc.) could be conditioned with the same, similar, or dissimilar equipment, and furthermore that motions of multiple joints concurrently can be measured by a single device or by multiple devices in variants of the present embodiment.

A system 22 was provided for acquiring information about an angle of the joint. System 22 is embodied as a joint angle sensor 24 (typically a goniometer) which senses the angle of the joint, and transmits angle data to a datalog 26 which wirelessly transmits the angle data to a processing interface 28 in a prescribed form. In the present embodiment, the datalog 26 is secured to the joint angle sensor 24, and the processor 28 is a mobile device adapted to be worn by the user 20.

Processor 28 includes a system management and feedback mapping software system 30, a wireless communication interface 32, and a sound card 34. The system management and feedback mapping software system 30 receives the angle data from the angle sensor 24 via a communication channel provided by the wireless communication interface 32, the angle data encoding the positional information of the joint. The system management and feedback mapping software system 30 includes program instructions for mapping the angle data to a unique scale note, as is further described below in relation to FIG. 3. The sound card 34 outputs, in a prescribed form, an acoustic analog signal including of the reference note and the feedback note. The feedback note is selected based on the angle data, in accordance with the mapping, at least for a period of the exercise, in accordance with a configuration of the system. In the present embodiment, the reference note and feedback note are modulated as a stereophonic sound signal. The acoustic analog signal is transferred to a pair of headphones 36 worn by the user 20 as a means for communicating feedback in the form of a stereophonic sound.

A sampling rate of the joint angle sensor 24, signaling rates of the wireless communication interface 32, and processing rate of processor 28 are chosen so that the stereophonic audio feedback is provided to the user 20 in substantially real-time, to provide the user 20 with a dynamically updated indication of his/her current/actual joint angle relative to a fixed, continuous indication of his/her target joint angle (the reference note).

The AudioPhys system is designed to be set up as follows: The joint angle sensor 24 is appropriately (non-invasively) attached to the joint of interest above and below the joint. In most cases, the affected joint is located on one side of the user (the affected side). The joint angle sensor is preconfigured to measure the angle of the affected joint, and is turned on such that the datalog 26 begins receiving the angle data in a prescribed form, and wirelessly transmitting it to the processor 28 located on a mobile device which is worn/strapped to the user 20. The form of the angle data received by the datalog 26 may not be the same as that of the angle data sent to the processor 28, and various data processing transformations could be applied, as will be understood by those of skill in the art. The angle data is then processed by the system management and feedback mapping system 30 in order to generate the appropriate stereophonic audio feedback which is fed to the user via a pair of headphones 36. The stereophonic audio feedback signal on the affected side of the user 20 plays the feedback notes while the opposite side plays the reference note so that the user hears the feedback notes as associated with the affected side. Thus, if the left knee is being conditioned, the feedback notes will be stereo-panned to the user's left ear.

The premise of the AudioPhys feedback mechanism is that the user will want to equalize the audio feedback across both ears—that is, to move his/her joint into a position such that the dynamically changing feedback note that corresponds to his/her actual joint position (as fed to one ear) is the same as the fixed, continuous reference note corresponding to his/her target position (as fed to the other ear). By having a continuous, pre-determined goal note delivered to one ear at the same time as receiving the dynamically changing feedback corresponding to his/her actual joint angle in the other ear, the user can gauge his/her progress towards the target. When the target is reached, the client will hear the same note in both ears simultaneously.

There are many possible arrangements of parameters for the system and these may be configured by the therapist or the user, depending on the system specifications. The specifications chosen for the AudioPhys system are described below in relation to FIG. 4. In general, the AudioPhys system is designed to be configurable such that: (a) a physiotherapist/physical therapist can configure settings such as target joint position, feedback granularity, comfortable hearing (frequency) range, and volume on a per-user, per-session basis; (b) the system is easy to use and wear, with minimal set up time per use; and (c) the system can be set up for users (i.e., clients of physiotherapists/physical therapists) to take home and use in between scheduled physical therapy sessions, thereby providing users with feedback and target information that would otherwise be unavailable outside of a scheduled therapy session.

The joint angle to audio feedback mapping used by the system is described with reference to FIG. 3. It should be noted that, for ease of description, the following figures represent 1D joint motions (i.e., do not attempt to show or describe rotation or off plane pivoting); the system could, however, handle rotation about a joint as well using any mapping from the 2D space to a line.

For the purpose of AudioPhys, the joint angle sensor 22 reports a single angle. While in other embodiments, pairs of angles may be measured to determine whether a given angle of a joint is relative to another, or relative to a fixed frame of reference, there is a safe range of joint angles created between bones A and B which reflect the variety of clinically safe positions for a joint. If user 20 exceeds the safe range, he/she is considered to have moved into a clinically unsafe position, and AudioPhys uses a simple alarm sound to alert users to the fact that their affected joint is not in a safe position. In one mode of operation, musical-based stereophonic audio feedback is always played while the user 20 is within the safe range. In other modes silence indicates that the user 20 is not operating within the expected range of the exercise, but is still within the safe range. This may be because the user 20 has surpassed the goal state, or because the user 20 has not yet moved into a starting position for the exercise.

AudioPhys could be developed to provide a plurality of goal states having different feedback notes associated with different sub-ranges of the goal range, if it is desirable that the user plays with or varies the angles within the goal range. If so all of the goal range states may be generally more consonant with the reference note than the other feedback notes.

The exercise range (a sub-range of the safe range) is partitioned into a sequence of sensed ranges. Limiting angles of 5 sensed ranges are shown at the bottom of FIG. 3. The number of sensed ranges will depend on a granularity of feedback appropriate to the user 20. For example, if a physiotherapist/physical therapist deems that a given client is only likely to make very small improvements in terms of joint position in any given session, he/she might set the sub-ranges to correspond to 2° differences in angles such that the client receives a dynamic change in audio feedback for even a small change in joint position; but if the user 20 is deemed capable of making more substantial changes to his/her joint angle, the physiotherapist/physical therapist might set the sub-ranges to correspond to 5° differences in angles such that the client needs to make larger changes in joint positions to effect a change in audio feedback. In FIG. 3 only 5 angles are shown mapped to respective notes, however it will be appreciated that any number of sensed ranges may be defined. According to AudioPhys these ranges are chosen by the physiotherapist/physical therapist. Furthermore, the physiotherapist/physical therapist may determine that it is appropriate to have a contiguous set of sensed ranges at one level of granularity and the remaining set of sub-ranges at other levels of granularity to reflect the areas of focus or difficulty for the user 20. Finally, the physiotherapist/physical therapist 20 may choose to switch off the audio feedback for a given, contiguous set of sub-ranges, such that the feedback only becomes apparent at a given physical joint position. All of these configurable options are aimed at ensuring that the most appropriate feedback is provided to each client as an individual, based on the client's current state of physical rehabilitation.

Once the physiotherapist/physical therapist has determined the granularity and on/off points for the audio feedback, he/she can set the comfortable hearing range appropriate/preferable to the user 20. Thereafter, the system management and feedback mapping system 30 assigns selected notes from within the identified comfortable hearing range to each of the sensed ranges with a one-to-one mapping appropriate to the joint being measured. If the user moves the affected joint within the exercise range where feedback-generation is selected, he/she hears a rising or falling sequence of discrete notes. The feedback notes would be delivered by a headphone, to an ear on his/her affected side. Additionally and concurrently, the user would hear the reference note delivered by a headphone, to an ear on his/her unaffected side.

In accordance with AudioPhys, the following is an example of a mapping that is used:

Segment Size (of 180° Midi range) Note Note Frequency 2 84 C6 1046.5 Hz Reference or target/goal 2 81 A5 880.00 Hz 2 77 F5 698.46 Hz 2 74 D5 587.33 Hz 2 71 B4 493.88 Hz 5 70 B4b 466.16 Hz 5 69 A4 440.00 Hz 5 68 G4# 415.30 Hz 5 66 F4# 369.99 Hz 5 65 F4 349.23 Hz 5 63 E4b 311.13 Hz 10 62 D4 293.67 Hz 10 61 C4# 277.18 Hz 10 60 C4 261.63 Hz 10 59 B3 246.94 Hz 25 58 B3b 233.08 Hz 25 57 A3 220.00 Hz 25 56 G3# 207.65 Hz 25 54 F3# 185.00 Hz Max feedback note

The feedback notes were selected from the chromatic scale such that they avoid the most harmonious pairings with the reference note. The dissonance of the note pairs (feedback to reference) renders them more distinguishable by a wide range of users and makes the point where the feedback note matches the reference note most obvious. Furthermore, the dissonance of the note pairings (intervals) at particular angle positions may provide (in some users) a motivation to pass the corresponding angular sub-range.

The physiotherapist/physical therapist is also expected to determine the target joint angle (or range of angles) for which the user should aim during a therapy session, and may project targets for subsequent sessions.

FIGS. 4a-c are screen-shots of 3 graphical user interfaces (GUIs) showing configuration and use of AudioPhys. The screen-shots are presented on a display screen of the processor 28, which is a touch screen interface.

FIGS. 4a,b show two different screen-shots presented to a physiotherapist/physical therapist permitting the configuration of the AudioPhys system for the user 20. The configuration tab first permits the specification of the joint in question. A knee joint is selected in the illustrated case, and hence the indication of a side of the body is permitted (with radio buttons). In the present case the left knee is being conditioned. The cued option indicates whether the user is being assisted by a physiotherapist or physical therapist during the present exercise, which is mainly used for reporting and analysis. The audio option permits the audio feedback to be on or off during the present exercise. It may be preferred to keep the audio off for preliminary assessment of mobility, for example. The pitch option permits subjects with better hearing (or simply hearing preferences) in higher, lower, or mid range (comfortable hearing range) to have appropriate feedback. Finally the reference or target/goal note can be set to be the highest pitch or lowest pitch in the feedback range (i.e., the user will hear feedback notes rising or falling to the reference or target/goal note, respectively), depending on the desires of the user 20.

A slider along a scale of angles is provided for setting of a range of angles for which audio feedback is or is not provided. The GUI further requires the identification of a current target angle, either in terms of a fixed number, or taken from a snapshot of the user's actual angle, which may be a position set by the physiotherapist/physical therapist. The (ultimate) end goal and best to date angles shown are for assisting the physiotherapist/physical therapist in entering a numerical target and, in the case of the latter, in tracking or quickly referencing progress to date.

A green section of the current target shows an active phase of the exercise over which the measured joint angles are expected to range, and over which the stereophonic audio feedback will be played. It will be noted that in FIG. 4a the exercise is bending the left knee from a straight position to 90° bent, and that the stereophonic audio feedback is to be played throughout the exercise, whereas in FIG. 4b the exercise is to straighten the left knee that is initially bent, and the stereophonic audio feedback is only played when the knee reaches a certain angle (110°).

FIG. 4c illustrates an active tab of the AudioPhys system that is presented on the touch screen of the mobile device during an exercise. The user may play, pause, or stop the session (including feedback and recording) using the buttons, as if the device were a media player. There is also a volume adjustment slider provided for adjusting the volume. The user 20 may also visually follow the angles registered by the joint angle sensor 24, and compare this with the target for the present session, that was set previously using the configuration tab. The angle data received by the system management and feedback mapping system 30 is dynamically updated in the actual box of the touch screen.

Other advantages that are inherent to the structure are obvious to one skilled in the art. The embodiments are described herein illustratively and are not meant to limit the scope of the invention as claimed. Variations of the foregoing embodiments will be evident to a person of ordinary skill and are intended by the inventor to be encompassed by the following claims.

Claims

1. An audio feedback signal for a subject engaged in an exercise comprising:

a reference note of a musical scale that provides a marker of a goal for the subject; and
a sequence of real-time feedback notes of the scale, each feedback note being played with the reference note for a duration sufficient to permit the subject to compare the notes, and each feedback note being played when a sensor detects that the subject is in a corresponding position within the exercise,
wherein one of the feedback notes corresponds to the subject reaching the goal, and this feedback note matches the reference note in a distinctive manner.

2. The audio feedback signal of claim 1 wherein the sequence includes at least 3 distinct feedback notes corresponding to respective positions within the exercise, whereby progress towards the goal is indicated to the subject by the changing of the audio feedback signal in real-time, and the subject repeatedly compares the feedback note with the reference note during the exercise.

3. The audio feedback signal of claim 2 wherein a mapping of each of a plurality of sensed ranges of positions within the exercise to respective feedback notes uniquely associates a note with each of the sensed ranges, and the sensor continuously detects a position of the subject in the exercise.

4. The audio feedback signal of claim 3 wherein the exercise includes enough positions, the sensed ranges are sufficiently distributed, and the exercise is expected to be performed slowly enough that the subject can hear the emitted reference and feedback signal as musical feedback indicating a proximity to the goal during the exercise.

5. The audio feedback signal of claim 1 wherein:

the corresponding positions within the exercise and associated feedback notes are constant for multiple repetitions of the exercise whereby the subject may associate specific feedback notes to specific ranges of positions;
the corresponding positions within the exercise are associated with feedback notes in frequency order with respect to proximity to the goal;
the scale is a chromatic scale;
exactly one feedback note is played at a time throughout at least a phase of the exercise;
exactly one feedback note is played at a time throughout at least a phase of the exercise, and each feedback note is played for a substantially equal time in an expected execution of the exercise;
exactly one feedback note is played at a time throughout at least a phase of the exercise, and feedback notes played during an expected difficult segment of the exercise are played for a shorter time in an expected execution of the exercise;
exactly one feedback note is played at a time throughout at least a phase of the exercise, and feedback notes played during a segment of the exercise that requires greater attention are played for a shorter time in an expected execution of the exercise;
the feedback note that corresponds to the subject reaching the goal matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes except a feedback note associated with a starting position of the exercise;
the feedback note that corresponds to the subject reaching the goal matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes;
the feedback note that corresponds to the subject reaching the goal matches the reference note in a distinctive manner by being the only feedback note that is exactly an octave apart;
the feedback note that corresponds to the subject reaching the goal matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note; or
the feedback note that corresponds to the subject reaching the goal matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note, and at least some of the rest of the sensed ranges are mapped to scale notes that are dissonant with the reference note.

6. The audio feedback signal of claim 1 wherein the reference note and feedback notes are emitted to the subject from separate sound sources, whereby a directional component to the feedback is provided.

7. A method for motor control training, comprising:

providing a sensor for detecting a position of a part of a subject during an exercise; and
generating an audio feedback signal comprising a reference note of a musical scale that provides a marker of a goal for the subject, and a sequence of real-time feedback notes of the scale, each feedback note being played with the reference note for a duration sufficient to permit the subject to compare the notes, and each feedback note being played when the sensor detects that the subject is in a corresponding position within the exercise, wherein one of the feedback notes corresponds to the subject reaching the goal, and this note matches the reference note in a distinctive manner.

8. The method of claim 7 wherein generating the sequence of feedback notes includes generating at least 3 distinct feedback notes corresponding to respective positions within the exercise, whereby progress towards the goal is indicated to the subject by the changing of the audio feedback signal in real-time, and the subject repeatedly compares the feedback note with the reference note during the exercise.

9. The method of claim 7 wherein the sensor is provided for continuously detecting a position of the subject in the exercise over a range of positions that includes a plurality of sensed ranges that are mapped to respective feedback notes to uniquely associate a note with each of the sensed ranges.

10. The method of claim 7 wherein the sensor is provided for detecting a range of positions within the exercise, the range of positions including a plurality of sensed ranges that are distributed so that for the expected duration of the exercise, a subject can register the audio signal as musical feedback.

11. The method of claim 7 wherein generating the audio feedback signal comprises:

playing associated feedback notes when the corresponding positions within the exercise are reached, wherein the association of the feedback notes to the positions is constant for multiple repetitions of the exercise, whereby the subject may associate specific feedback notes to specific ranges of positions;
playing associated feedback notes when the corresponding positions within the exercise are reached, wherein the association of the feedback notes to the positions is in frequency order with respect to proximity to the goal;
playing associated feedback notes selected from a chromatic scale;
playing exactly one feedback note at a time throughout at least a phase of the exercise;
playing exactly one feedback note at a time throughout at least a phase of the exercise, wherein each feedback note is played for a substantially equal time in an expected execution of the exercise;
playing exactly one feedback note at a time throughout at least a phase of the exercise, wherein feedback notes played during an expected difficult segment of the exercise are played for a shorter time in an expected execution of the exercise;
playing exactly one feedback note at a time throughout at least a phase of the exercise, wherein feedback notes played during a segment of the exercise that requires greater attention are played for a shorter time in an expected execution of the exercise;
playing the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes except a feedback note associated with a starting position of the exercise;
playing the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes;
playing the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note that is exactly one or more octaves apart from the reference note;
playing the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note;
playing the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note, and at least some of the rest of the feedback notes are more dissonant with the reference note
playing the feedback note by emitting an acoustic signal from a separate emitter distinct from an emitter that emits the reference note, whereby the subject hears the two signals being emitted from different directions;
playing the feedback note and reference note is performed by emitting acoustic signals on respective channels of a stereophonic emitter; or
playing the feedback note and reference note is performed by emitting acoustic signals on respective channels of a stereophonic emitter, the exercise is performed by the part which is on one side of the subject, the feedback notes are played on the one side of the subject, and the reference note is played on the other side of the subject.

12. The method of claim 7 wherein providing the sensor comprises:

providing a sensor for sensing a state of flexure of a muscle set; or
providing a sensor for sensing a state of flexure of a muscle set as determined by an angle at a joint with a goniometer.

13. An apparatus for motor control training, comprising:

a sensor for outputting a position of a part of a subject during an exercise;
a signal processor for receiving position output by the sensor and controlling an emitter to emit an audio feedback signal comprising a reference note of a musical scale that provides a marker of a goal for the subject, and a sequence of real-time feedback notes of the scale, each feedback note being played with the reference note for a duration sufficient to permit the subject to compare the notes, and each feedback note being played in dependence upon the output position; and
the emitter coupled to the signal processor for playing the audio feedback signal to the subject,
wherein one of the feedback notes corresponds to the subject reaching the goal, and this note matches the reference note in a distinctive manner.

14. The apparatus of claim 13 wherein the signal processor controls the emitter to emit the sequence of real-time feedback notes that includes at least 3 distinct feedback notes corresponding to respective positions within the exercise, whereby progress towards the goal is indicated to the subject by the changing of the audio feedback signal in real-time, and the subject repeatedly compares the feedback note during the exercise.

15. The apparatus of claim 13 wherein the sensor is adapted to detect a range of positions within the exercise and signal processor controls the emitter to play the feedback notes in dependence on the position output so that for the expected duration of the exercise, a subject can register the audio signal as musical feedback.

16. The apparatus of claim 13 wherein the signal processor controls the emitter to:

play associated feedback notes when the corresponding positions within the exercise are reached, wherein the association of the feedback notes to the positions is constant for multiple repetitions of the exercise, whereby the subject may associate specific feedback notes to specific ranges of positions;
play associated feedback notes when the corresponding positions within the exercise are reached, wherein the association of the feedback notes to the positions is in frequency order with respect to proximity to the goal;
play associated feedback notes selected from a chromatic scale;
play exactly one feedback note at a time throughout at least a phase of the exercise;
play exactly one feedback note at a time throughout at least a phase of the exercise, wherein each feedback note is played for a substantially equal time in an expected execution of the exercise;
play exactly one feedback note at a time throughout at least a phase of the exercise, wherein feedback notes played during an expected difficult segment of the exercise are played for a shorter time in an expected execution of the exercise;
play exactly one feedback note at a time throughout at least a phase of the exercise, wherein feedback notes played during a segment of the exercise that requires greater attention are played for a shorter time in an expected execution of the exercise;
play the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes except a feedback note associated with a starting position of the exercise;
play the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being more consonant with the reference note than any of the other feedback notes;
play the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note exactly one or more octaves apart from the reference note;
play the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note;
play the feedback note that corresponds to the subject reaching the goal, comprises playing a feedback note that matches the reference note in a distinctive manner by being the only feedback note that is the same note as the reference note, and at least some of the rest of the feedback notes that are dissonant with the reference note
play the feedback note by emitting an acoustic signal from a separate emitter distinct from an emitter that emits the reference note, whereby the subject hears the two signals being emitted from different directions;
play the feedback note and reference note is performed by emitting acoustic signals on respective channels of a stereophonic emitter; or
play the feedback note and reference note is performed by emitting acoustic signals on respective channels of a stereophonic emitter, the exercise is performed by the part which is on one side of the subject, the feedback notes are played on the one side of the subject, and the reference note is played on the other side of the subject.

17. The apparatus of claim 14 wherein the emitter:

comprises a first emitter for playing the reference note to the subject, and a second emitter for playing the real-time feedback notes to the subject;
comprises a stereophonic emitter having a first channel for playing the reference note to the subject, and a second channel for playing the real-time feedback notes to the subject; or
comprises a stereophonic emitter having a first channel for playing the reference note and a second channel for playing the real-time feedback notes to the subject, wherein the exercise is performed by the part that is on one side of the subject, and the second channel is played to an ear on the one side of the subject.

18. The apparatus of claim 13 wherein the sensor is adapted to:

detect a state of flexure of a muscle;
detect a state of flexure of a muscle by determined by an angle at a joint; or
detect a state of flexure of a muscle by determined by an angle at a joint with a goniometer.
Patent History
Publication number: 20110021318
Type: Application
Filed: Jul 19, 2010
Publication Date: Jan 27, 2011
Inventors: Joanna Lumsden (Lichfield), Danny D'Amours (Fredericton), Rene Richard (Fredericton)
Application Number: 12/804,290
Classifications
Current U.S. Class: Monitors Exercise Parameter (482/8)
International Classification: A63B 71/00 (20060101);