Human-motion-training system

Abstract

Certain embodiments of the present invention are directed to automated human-motion-training systems. These automated human-motion-training systems include a hardware platform that provides for stored-instruction processing and that includes memory, an I/O interface, and an audio-signal generation and output component, and an operating system or control program that executes on the hardware platform and that provides a program-execution environment. The automated human-motion-training systems further include components of the human-motion-training system that provide for automated proctoring of human-motion exercises and training regimes, including monitoring of a user's body position, producing feedback corresponding to a user's body position, and determining a user's performance in order to modify the exercises and training regimes so that they provide optimal-challenge-point-based training over the course of multiple repetitions and training sessions.

Description

TECHNICAL FIELD

The present invention is related to computationally-assisted training systems and, in particular, to a computationally-assisted human-motion-training system that can run on a variety of different hardware and operating-system platforms and that uses feedback to provide optimal-challenge-point-based training to human users.

BACKGROUND

A wide variety of different types of training systems have been developed, over the past century, to assist training of human users to carry out various types of physical tasks, including physical-therapy-related exercises and movements, sports-related exercises and movements, dance routines, and a variety of other such tasks. However, to date, these training systems have met with mixed success. It is difficult to design training systems sufficiently flexible to provide detailed training and analysis of user execution and feedback to effectively assist a diverse range of users with varying capabilities, physical constraints, limb dimensions, and other such characteristics and parameters. In many cases, the training systems are meant to be partially or fully accompanied by human-trainer intervention, but, in the absence of a human trainer, effective monitoring of users' interactions with automated training systems is lacking or deficient. Moreover, in the absence of a human trainer, a user often lacks sufficient direction, control, and meaningful feedback to properly carry out exercises. Furthermore, even with a human trainer present, currently available training systems and human trainers often fail to optimize training sessions to provide maximum benefit to users. For all of these reasons, researchers and developers, manufacturers, and users of partially or fully automated training systems continue to seek training systems with new and improved functionality that better serve the of training-system users.

SUMMARY

Certain embodiments of the present invention are directed to automated human-motion-training systems that are cost-effectively implemented on many different hardware, operating-system, and control-program platforms. Human-motion-training systems that represent embodiments of the present invention employ optimal-challenge-point-based training and employ feedback and computational representations of exercises to provide optimal or near-optimal learning and capability-development through training.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-B illustrate a simple, example human-motion-training goal.

FIGS. 2A-D illustrate an initial training session, or physical-therapy session, according to one example of the present invention, in which initial exercise parameters are determined.

FIG. 3 illustrates a time sequence and exercise envelope computationally generated and stored by a training system that represents one example of the present invention during the initial training session.

FIG. 4 illustrates a computationally generated envelope and time sequence of the positions, similar to those shown in FIG. 3, which represent a training goal according to one embodiment of the present invention.

FIGS. 5A-B illustrate training feedback employed by human-motion-training systems that represent examples of the present invention.

FIG. 6 is a plot that illustrates optimal-challenge-point learning, a concept incorporated into embodiments of the present invention.

FIG. 7 illustrates a desired increase in functional difficulty of an exercise with respect to skill for performing the exercise acquired, over multiple training systems, by a user of a human-motion-training system that represents an embodiment of the present invention.

FIG. 8 illustrates various components, functional relationships between components, data flow, and control strategy for a human-motion-training system that represents an embodiment of the present invention.

FIG. 9 illustrates certain of the data objects stored within the memory system and/or file system of a hardware device on which a human-motion-training system that represents an embodiment of the present invention is implemented.

FIG. 10 provides a control-flow diagram for the scheduler component of a human-motion-training-system that represents one embodiment of the present invention.

FIG. 11 shows a generalized finite-state-machine state diagram for the finite-state machines in the human-motion-training system that represents an embodiment of the present invention.

FIGS. 12-15 provide simple state-transition diagrams for the four finite-state machines of a human-motion-training system that represents an embodiment of the present invention discussed above with reference to FIG. 8.

FIG. 16 is a control-flow diagram that represents launching of command execution by a user-interface finite-state-machine.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 1A-B illustrate a simple, example human-motion-training goal. As shown in FIG. 1A, a training or exercise session begins with a human user or patient 102 sitting in a chair 104 with the bottoms of both feet resting on the ground. As shown in FIG. 1B, a goal for the patient may be to raise his or her left leg 110 to the position shown in 1B, with the leg moderately bent at the knee and the knee raised nearly to chest height. The patient, for example, may have injured the left knee, as a result of which the patient cannot currently lift his or her left leg to the position shown in FIG. 1B.

FIGS. 2A-D illustrate an initial training session, or physical-therapy session, according to one example of the present invention, in which initial exercise parameters are determined. With the patient initially in the sitting position shown in FIG. 1A, a trainer may initiate a automatically recorded, initial number of exercises in which the trainer instructs the user to move his or her leg upward, towards the desired position, or, when the patient is incapable of doing so, lifts and manipulates the user's leg to illustrate the desire motion, as shown in FIGS. 2A-D. Perhaps, because of pain associated with the movement, the leg position shown in FIG. 2D is the highest and most bent position that the user is capable of tolerating. During the automated recording of a number of initial exercises, any of various types of motion and/or location sensors mounted to the user are computationally monitored by a human-motion-training system that represents an embodiment of the present invention in order to generate both a time sequence of limb movement and an envelope surface that describes the volume of space occupied by the user's limb over the course of the initial attempts by the user to perform the exercise under the guidance of a trainer.

FIG. 3 illustrates a time sequence and exercise envelope computationally generated and stored by a training system that represents one example of the present invention during the initial training session. The envelope 302 is a computationally defined surface that encloses the maximum excursions of the surface of the user's limb during the exercise. The envelope can be extrapolated from known sensor positions on the user's leg and the sensor data collected during the initial exercises. Additionally, a time sequence of limb positions, represented in FIG. 3 by dashed arrows, such as dashed arrow 304, may be generated to represent a time sequence of positions and orientations of the limb to represent limb motion during the exercise. FIG. 4 illustrates a computationally generated envelope and time sequence of the positions, similar to those shown in FIG. 3, which represent a training goal according to one embodiment of the present invention. Note that the upper surface of the envelope corresponds to the desired position of the user's left leg shown in FIG. 1B. Thus, the envelope 402 and time sequence of positions 404-408, represent the end goal of the training program, while the initial envelope and time sequence of positions, shown in FIG. 3, represents the initial capabilities of the patient. The goal of the human-motion-training system that represents an embodiment of the present invention is to then provide a sequence of monitored training sessions that efficiently and optimally, or near-optimally, through repeated training, steer and coach the user to be able to execute the motion represented by the envelope and time sequence of leg positions shown in FIG. 4. In human-motion-training systems that represent examples of the present invention, the training system accomplishes a training program by monitoring, through sensors attached to a patent's or user's body, the user's motion during exercise sessions in order to ensure that the user carries out predetermined motions correctly and in order to ensure that, over time, the user is able to increase the user's capabilities and proficiency in order to achieve a desired motion or series of motions.

FIGS. 5A-B illustrate training feedback employed by human-motion-training systems that represent examples of the present invention. As shown in FIG. 5A, when a user is executing an exercise in conformance with the desired motion or series of motions, then, in the example illustrated in FIGS. 1A-4, the user's left leg is positioned with the computationally defined envelope that represents the extent of the motion or motions carried out in the exercise. Moreover, the current position of the user's left leg corresponds to a position within a range of positions expected at the current time point within the exercise relative to the start of the exercise, when timing is a consideration. In this case, the user may receive any of various positive feedbacks from the human-motion-training system. As one example, the user may wear headphones 502 and may receive pleasing, soothing, encouraging, or other positive audio signals to indicate that the user is moving his or her body in conformance with a pre-planned exercise. By contrast, as shown in FIG. 5B, when the user moves his or her body to a position that does not conform with the desired time sequence of positions that represent the exercise, or when the user moves a limb into an undesired position, detected by the human-motion-training system by comparing sensor output to the computational envelope, a warning signal 504 is generated by the human-motion-training system and provided to the user as a discordant or unpleasant audio signal, or by some other means. For example, in FIG. 5B, the user's toes have been moved outside the computational envelope 506 and the user is provided with an audio signal indicating that the user has moved in a way that does not conform to the desired movement for the exercise.

Any of various different types of feedback signals may be provided to a user during an exercise. Audio signals represent one effective type of feedback signal, for which musical-instrument-digital-interface (“MIDI”) devices and interfaces are widely available. Alternative feedback signals may also be used alone or in various combinations. These may include mechanical vibrations, visually displayed signals on a computer monitor, mobile-phone screen, and/or peripheral lighting system, or other types of physical signals or events that can be quickly and intuitively perceived by a user. In addition, because the user's body positions are monitored continuously by the human-motion-training system, the user may received continuous feedback to more effectively steer the user's motions. For example, the user may receive pleasing or encouraging signals when the user's body position conforms to the desired position at a particular point in the training session, but may receive increasingly discordant or unpleasant feedback signals as the user's body position strays further and further from the desired position. As the user attempts to correct the body position to conform to the exercise, as the body position begins to approach a more desired position, the discordant or unpleasant signals may diminish and be replaced by increasingly encouraging signals. Thus, by continuously providing feedback signals, the human-motion-training system that represents one embodiment of the present invention can steer a user's movements to assist the user in avoiding body-position excursions beyond the computational envelope that describes the current exercise or training movement.

While use of one or more computational envelopes and time sequences of body positions to represent an exercise and training feedback are important components of embodiments of the present invention, providing only a static exercise that does not change, over time, does not lead to an efficient, let alone optimal or near-optimal, training regime. For effective training, a particular exercise should evolve, over time, as the user learns and acquires proficiency in the body movements that the user is being trained to perform. In the example of FIGS. 1A-5B, the physical-therapy patient may not, initially, be able to raise his or her left leg much further than the position shown in FIG. 2B, without assistance from a physical therapist. However, the purpose of the physical-therapy exercise is for the user to be able to achieve, over time, an ability to raise his or her left leg to the leg position illustrated in FIG. 1B. Therefore, the human-motion-training systems that represent embodiments of the present invention employ on-going monitoring and exercise-modification that evolve an exercise, over the course of multiple training sessions, to appropriately challenge the user so that the user obtains maximum learning benefit from the training sessions.

FIG. 6 is a plot that illustrates optimal-challenge-point learning, a concept incorporated into embodiments of the present invention. In FIG. 6, the horizontal axis 602 represents increasing functional difficulty of an exercise proctored by a human-motion-training system that represents an embodiment of the present invention. The functional difficulty of exercise is the difficulty of the exercise with respect to the current capabilities and skill of a user performing the exercise as well as the various temporal and environmental conditions under which the exercise is currently being performed. The functional difficulty of an exercise differs from the nominal difficulty of an exercise, which is a more objective assessment of the difficulty of the exercise independent of the skill of the performer of the exercise and the conditions under which the exercise is performed. The left-hand vertical axis 604 represents the level of performance of the user of the human-motion-training system.

The level of performance can be, as one example, the accuracy by which the user can move in conformance with the time sequence of positions that represent the exercise in the human-motion-training system. The more frequently the user violates the computational envelope for the exercise and the user's body position differs from the body position expected at a particular point in time during the exercise, the lower the performance level. The right-hand vertical axis 606 represents the potential learning benefit of an exercise to the user. In the graph shown in FIG. 6, performance curves that represent functions of performance with respect to functional difficulty are plotted as solid curves, such as solid curve 606, and learning curves, which represent the functional relationship between the potential learning benefit of an exercise and the functional difficulty, are represented as dashed curves, such as dashed curve 608.

Curve 607 represents the performance of a skilled or experienced user with respect to a particular exercise in relation to the functional difficulty of the exercise, and curve 610 represents the performance of a novice or unskilled user for the exercise with respect to the functional difficulty of the exercise. As can be seen in the different shapes of the two curves, an experienced user generally achieves a relatively high level of performance over a wide range of functional difficulty until the functional difficulty approaches a threshold difficulty 616 after which a user's performance rapidly deteriorates with even modest further increases in functional difficulty. By contrast, the performance of a novice or unskilled user, represented by curve 610, rapidly deteriorates with even modestly increased levels of functional difficulty. In the case of the inexperienced user, the deterioration in performance is initially sharp, but reaches a threshold functional difficulty 618 after which the deterioration in performance flattens with increased functional difficulty. As can be seen in both learning curve 608, for the experienced user, and learning curve 620, for the inexperienced user, the potential learning benefit of an exercise steeply increases up to the respective threshold points for the learning curves of experienced and inexperienced users and then quickly decreases as the functional difficulty increases past those threshold values. For any given user, there is a region, referred to as the optimal challenge point and represented in FIG. 6 as disk-like regions 622 and 624, at which a given level of functional difficulty results in optimal or near-optimal learning.

When the functional difficulty of an exercise or training regime is significantly lower than the optimal challenge point for a particular user under particular conditions, the user achieves relatively little or modest benefit from performing the exercise. Similarly, when the functional difficulty of an exercise significantly exceeds the abilities of a user, the learning benefit is also relatively poor. For example, were a normal, average, healthy human adult asked to jump up and down, in place, three times, the normal adult would obtain little learning benefit from undertaking the jumping exercise, since it is well within the capability of almost all normal adults to jump in place three times. Similarly, were a normal adult not trained in ballet asked to perform a ten-minute sequence of complex ballet moves from a famous ballet, the normal but untrained adult would almost certainly fail to perform even a handful of initial movements in conformance with the desired movements for the ballet, and would likely learn very little from embarking on the exercise other than the fact that performing ballet is harder than it appears from watching other perform ballet. However, were an average adult shown three or four body motions that represent an initial sequence of a ballet move, it is likely that, by attempting the sequence of movements, the normal adult might well achieve a degree of learning that could be rapidly improved, with practice, and that could serve as a reasonable approach to acquiring cumulative skill in ballet.

FIG. 7 illustrates a desired increase in functional difficulty of an exercise with respect to skill for performing the exercise acquired, over multiple training systems, by a user of a human-motion-training system that represents an embodiment of the present invention. As shown in FIG. 7, the human-motion-training system that represents an embodiment of the present invention needs to begin with an exercise of relatively low functional difficulty, in order to provide reasonable learning benefit to a user first undertaking the exercise, and then needs to monitor the user's performance, over time, in order to steadily increase the functional difficulty of the exercise so that the exercise presents an optimal challenge point to the user with each repetition and, desirably, even over the time course of a single repetition. The functional difficulty can be increased by changing the computational envelope and sequence of body positions that define the exercise and/or by changing the conditions under which the exercise is performed by, for example, intermixing the exercise with other exercises, changing the tempo or pace of the exercise using performance queues, or by many other means.

The human-motion-training systems that represent embodiments of the present invention are intended to be implemented on a wide variety of different available hardware and software platforms in order to provide convenient, flexible, and economic solutions to human-motion-training problems. This, in turn, motivates the next-described implementation strategy and architecture for a human-motion-training system that incorporates computational exercise representations, feedback training, and optimal-challenge-point-based learning and that represents one embodiment of the present invention.

FIG. 8 illustrates various components, functional relationships between components, data flow, and control strategy for a human-motion-training system that represents an embodiment of the present invention. The components shown in FIG. 8 are implemented within the program-execution environment provided by any of various different types of operating system running above any of various different types of hardware platforms, from cell phones to high-end work stations. In order to be implemented on such a diverse range of operating systems and hardware platforms, the human-motion-training systems that represent embodiments of the present invention generally employs an implementation that relies only on basic and common system services provided through well-known and standardized interfaces. A core functional element and control strategy is represented by a system scheduler 802 that schedules execution of a number of finite state machines (“FSMs”) 804, 806, 808, and 810. In certain embodiments of the present invention, the system scheduler provides a strict cyclical time-division multiplexing of the computational bandwidth of an underlying hardware device, executing each of the FSMs once during each cycle. The cyclical nature of the scheduler is illustrated by curved arrow 811 in FIG. 8. The scheduling is generally non-preemptive, but the execution engine which comprises the scheduler of additional services includes event detection and event delivery service that allows various types of events to be detected and delivered asynchronously to the FSMs. In FIG. 8, only four FSMs are shown, but implementations of the human-motion-training systems that represent embodiments of the present invention are flexible and dynamic, in nature, and can accommodate a range of numbers of FSMs. The FSMs are implemented as plug-ins, or dynamically loaded modules, the number of which can be dynamically altered to implement a given exercise or training regime. In general, the scheduler can be stopped and restarted, with a new or differ complement of active FSMs, to begin proctoring of a different exercise.

As briefly mentioned above, the position of a user's limbs and other relevant body parts are determined by one or more sensors that are attached to the user's limbs and other body parts prior to undertaking training. These sensors may include three-axis accelerometers, relative location sensors to determine position based on preferential signal return, and other types of electro-mechanical-optical sensing devices. In FIG. 8, the four sensors 820-823 are shown intercommunicating with a single sensor FSM. In alternative embodiments, each sensor may be associated with its own sensor FSM, with the system supporting multiple sensor FSMs. The sensor FSM 804 interfaces to each of sensors 820-823 through a generally simple electronic interface that allows the sensor FSM to access current sensor data or sensor readouts from the sensor and, in certain cases, additionally provides the ability to power on, power off, initialize, and test the operability of the sensor. The sensor FSM 804 acquires sensor data at regular intervals or, in certain cases, when notified of certain sensor-related events, such as detection of onset of rapid movements or other such events. The sensor FSM prepares normalized, standard sensor-information data packets from sensor readings accumulated over finite periods of time and queues the sensor-information data packets to a first-in-first-out (“FIFO”) queue (“FIFO Q”) 826. The sensor-information data packets are dequeued from the FIFO Q and received by an analysis FSM 806 which accumulates sensor data, over time, and analyzes the data in order to prepare information messages that indicate user motion and body position with respect to the timed sequence of positions and the computational exercise envelope for an on-going exercise. For example, the analysis

FSM may compute, from current sensor readings, the current position of a limb or other body part and then compare that position with the desired position, at the current point of time within the exercise, and with the envelope for the exercise, generating information messages such as: (1) on-target, or in conformance with the currently conducted exercise; (2) flagging target position; (3) ahead of current target position; (4) approaching envelope surface; (5) retreating away from envelope surface; (6) violating envelope surface, and other such types of information for which appropriate feedback can be prepared. The analysis FSM thus interprets sensor output to form conclusions with respect to the user's body position relative to the currently proctored exercise. The synthesis FSM 808 receives the information packets prepared by the analysis FSM and queued by the analysis FSM to a second FIFO Q 830 and processes these packets by launching a synthesis script 812 which processes the information packets into a control signals or commands that are passed to the interface of an output device, such as a MIDI interface to an output-audio-signal-producing application or operating-system component that generates audio signals transmitted to headphones worn by the user. In addition, the synthesis script 812 may prepare output for display on a display device 816. A user-interface FSM 810 receives input, through an I/O-display device, from a user and/or trainer, launches execution of user commands, and displays output to users and/or trainers via the I/O-display device 816. The user-interface component may also be implemented as a set of library routines or as a component of the execution engine that includes the scheduler. The implementation also includes various services and auxiliary components, including a data store 836 that allows FSMs to store data in, and retrieve data from, an electronic memory and/or mass-storage devices included within, or accessible from, the hardware platform on which the human-motion-training system is implemented.

FIG. 9 illustrates certain of the data objects stored within the memory system and/or file system of a hardware device on which a human-motion-training system that represents an embodiment of the present invention is implemented. Data objects may include user profiles 902, exercise descriptions 904, a configuration file 906, a log file 908, and a data dictionary 910. User profiles include various types of information about individual users. In particular, the user profiles may contain indications of the current proficiency or skill level of a particular user with respect to each of the different exercises, so that when the user next initiates a training session, the human-motion-training system can select or generate a particular exercise repetition with the functional difficulty appropriate for the user's current skill or proficiency in order to maximize the learning benefit from the exercise. Exercise descriptions include, in certain cases, a time sequence of body positions, initial, final, and, in certain cases, one or more intermediate computational envelopes that describe an acceptable range of body positions over the course of the exercise, and other information needed to proctor a next exercise of a particular user.

A configuration file may include indications of all of the plug-ins and other dynamic components currently needed by the human-motion-training system, and may include additional information, such as the location of FIFO Qs, various operational parameters and characteristics to control operation of the FSMs, scheduler, and other components of the human-motion-training system, and any other information needed at run time by the human-motion-training system. A log file 908 stores continuously received output from the human-motion-training system and a data-dictionary file 910 stores values of named variables that can be accessed by the FSMs and various other implementation components at run time.

FIG. 10 provides a control-flow diagram for the scheduler component of a human-motion-training-system that represents one embodiment of the present invention. In step 1002, the scheduler receives a list of the plug-ins to load for conducting or proctoring a next exercise or training regime. This list of plug-ins may be extracted from a configuration file, from memory, or may transmit into the scheduler by other means. In step 1004, the scheduler unloads any currently loaded but unneeded plug-ins, loads any plug-ins in the received list that are not currently loaded, may initialize or partially initialize the plug-ins, the temporary data store, the FIFO Qs, event detection and event delivery mechanisms, and the stored FSM states, or FSM-state-functions to be called in the next scheduler cycle. Then the scheduler enters a continuous loop of steps 1006-1010 in which the scheduler first tests, in step 1007, whether or not a termination condition has arisen, as a result of which the scheduler terminates execution or carries out a next scheduler cycle represented by the for-loop in steps 1008-1010. In each scheduler cycle, the scheduler considers, in turn, each of the FSMs that currently reside within the human-motion-training system. In step 1009, the scheduler calls, through a stored function pointer or reference, a function of the FSM that was returned by a previous call to the FSM or loaded initially in step 1004. Thus, each FSM is responsible, upon being called through any of various FSM functions, for returning a next function to call in a next scheduler cycle. In general, these functions map directly to the states of the FSM. In certain cases, an FSM may continue to execute following termination of the scheduler in order to continue to receive and respond to input and other events and prepare for re-launching of the scheduler to begin execution of a new exercise-proctoring session.

FIG. 11 shows a generalized finite-state-machine state diagram for the finite-state machines in the human-motion-training system that represents an embodiment of the present invention. In general, there is an initial state 1102 entered when the FSM is launched, in which the FSM may carry out certain initialization tasks. The FSM inhabits a running state 1104 during various normal operations. In certain cases, the FSM may transition to a waiting state 1106 when the FSM needs to wait for the occurrence of some event in order to proceed. In general, the states are represented as circles or disks that are labeled with the name of the state, and transitions between the states are shown as curved arrows, such as curved arrow 1112 representing the transition from the running state to the delaying state in FIG. 11. Similarly, the FSM may transition to a delaying state 1108 when the FSM needs to delay execution for some period of time or until execution is resumed by an external entity or process. Finally, the FSM may transition to a stopped state 1110 under certain conditions, such as the occurrence of unrecoverable error, the occurrence of an error or other condition that is handled by another entity, or upon occurrence of an event or condition that signals conclusion of FSM operation. Of course, any particular state transition diagram for a particular FSM may include many additional states and state transitions.

FIGS. 12-15 provide simple state-transition diagrams for the four finite-state machines of a human-motion-training system that represents an embodiment of the present invention, discussed above with reference to FIG. 8. FIG. 12 shows a state transition diagram for the sensor FSM. Initially, the sensor is powered onto the initial state 1202, from which the sensor transitions to the running state 1204 following any needed initialization operations as well as initialization of the variable nxtT to indicate a next time at which a sensor monitored by the sensor FSM can be queried for sensor output. Initialization may involve a sensor-position-discovery operation in which the position of the sensor with respect to a user's body is determined. Operation of the sensor FSM may be suspended by the occurrence of a suspension event, resulting in transition 1206 to the delay state 1208, from which operation can be resumed by occurrence of a resume event 1210. Upon the occurrence of a timer event, which represents a next tick of a system clock, and when a current time reported by the timer event is greater than or equal to the time stored in variable nxtT, the sensor FSM requests a sensor reading from the sensor, sets a timer, and transitions to the waiting state 1212. When the sensor responds, the timer is disabled, the variable nxtT is updated to a next point in time at which a sensor reading should be taken, and the received sensor data is normalized and formatted and queued to the FIFO Q for output to the analysis FSM after transition back to the running state 1204. In the running state, when a timer event occurs, but when the current time reported by the system timer is less than that stored in nxtT, the sensor FSM remains in the running state, as indicated by transition 1214. When a timer expires while the sensor FSM is waiting for data from the sensor, then the sensor FSM reports the error and transitions to the stopped state 1216. Upon occurrence of an error-handled event, the sensor FSM transitions back to the running state 1218. Thus, in general, the sensor FSM monitors timer events and, at particular intervals in time, queries one or more sensors for sensor output, packages the sensor output and queues the sensor output to a FIFO Q, and returns to monitoring the system clock. Of course, this is a simplistic state-transition diagram of the sensor FSM, which may be significantly more complex in an actual implementation, and may include additional transitions initiated by asynchronous events generated by certain types of sensors.

FIG. 13 provides the state-transition diagram for the analysis FSM that is included in the human-motion-training system that represents one embodiment of the present invention. In this case, the analysis FSM generally waits, in the waiting state 1302, for sensor-information input from the sensor FSM. When a FIFO Q input event occurs, the analysis FSM transitions 1304 to the running state 1306. If the queued information, when added to cumulative sensor information already collected since a last information message was prepared and queued to the FIFO Q serviced by the synthesis FSM, is sufficient to justify generation and modification of feedback signals to the user, then the analysis FSM generates an output information message to the synthesis FSM, queues that message to the FIFO Q for the synthesis FSM, and transitions back to the waiting state 1308. Otherwise, when the newly received sensor information, when added to any sensor information already recently collected, does not justify generation or modification of the feedback signal, then the analysis FSM 1310 transitions back to the waiting state without preparing an information message and retaining the received sensor information as part of cumulative sensor information received since the last information message was prepared.

FIG. 14 provides a state-transition diagram for the synthesis FSM. The synthesis FSM generally waits, in the waiting state 1402, for input from the analysis FMS. When input is received, the synthesis FSM transitions back to the running state 1404, in which the synthesis FSM de-queues a message from the FIFO Q queued to the FIFO Q by the analysis FSM, launches a synthesis script to process the information message and generate control input to a feedback device, and returns to the waiting state in transition 1406.

FIG. 15 is a state-transition diagram for the user-interface (“UI”) FSM within a human-motion-training system that represents one embodiment of the present invention. In general, the UI FSM monitors an input device for input, transitioning back and forth between the running state 1502 and the waiting-for-input state 1504 until sufficient input has been received to specify a command. The UI FSM then launches execution of the command and transitions to the delaying state 1506, while the command executes. Upon command completion, transitions back to the running state. During command execution, when output is generated for output to the user interface, the UI FSM transitions temporarily from the delaying state 1506 back to the running state 1502 for output generation before returning to the delaying state 1506.

FIG. 16 is a control-flow diagram that represents launching of command execution by a user-interface finite-state-machine. In step 1602, the UI FSM receives sufficient input to indicate a next exercise to conduct for a particular user. In step 1604, the user's profile is accessed to determine the functional difficulty from the selected exercise, as well as the current computational envelope and timing sequence for the exercise based on this functional difficulty. As discussed above, the functional difficulty is a function of the user's efficiency or ability for carrying out the exercise. In certain embodiments of the present invention, the user's proficiency or capability may be decreased, over time, from the ability or capability detected during the last-executed training session, rather than remaining static, in order to take into consideration the general trend of learned behavior to be forgotten or degraded over time. In step 1606, the exercise description for the exercise to be proctored is accessed to determine the plug-ins needed for the exercise. These plug-ins may be inserted into a configuration file or passed directly to the scheduler. Then, in step 1608, the scheduler is launched to carry out the exercise. When the scheduler terminates, due to passage of a sufficient amount of time for the exercise, user input, or other termination events, the output and log files may be analyzed in order to determine the user's proficiency or skill in carrying out the exercise and updating the user's profile with that information in step 1610.

FIG. 17 illustrates a general-purpose computer system that, when executing a portion of a human-motion training system that represents an embodiment of the present invention, comprises a system embodiment of the present invention. The computer system contains one or multiple central processing units (“CPUs”) 1702-1705, one or more electronic memories 1708 interconnected with the CPUs by a CPU/memory-subsystem bus 1710 or multiple busses, a first bridge 1712 that interconnects the CPU/memory-subsystem bus 1710 with additional busses 1714 and 1716, or other types of high-speed interconnection media, including multiple, high-speed serial interconnects. These busses or serial interconnections, in turn, connect the CPUs and memory with specialized processors, such as a graphics processor 1718, and with one or more additional bridges 1720, which are interconnected with high-speed serial links or with multiple controllers 1722-1727, such as controller 1727, that provide access to various different types of mass-storage devices 1728, electronic displays, input devices, and other such components, subcomponents, and computational resources. Embodiments of the present invention may also be implemented on distributed computer systems, can also be implemented partially in hardware logic circuitry, and can be implemented on many other types of microprocessor-controlled devices, from mobile phones to computer notebooks and laptops.

Although the present invention has been described in terms of particular embodiments, it is not intended that the invention be limited to these embodiments. Modifications will be apparent to those skilled in the art. For example, a human-motion-training system can be implemented, according to the current invention, in many different ways by varying any of many different design and implementation parameters, including programming language, control structures, modular organization, data structures, and other such parameters. Any of various different sensors may be employed for each of an arbitrary number of exercises or training regimes. Exercises and training regimes may be simple timed sequences of motion, or may be composed of multiple different motions. Human-motion-training systems can be employed in physical therapy, sports training, occupational training, and many other types of training in which users need to acquire body-motion skills or restore motion-related abilities. In certain embodiments of the present invention, a scripting interface and engine provides users with the ability to modify or incorporate new analysis and synthesis functionality into the human-motion-training systems that represent embodiments of the present invention.

The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that the specific details are not required in order to practice the invention. The foregoing descriptions of specific embodiments of the present invention are presented for purpose of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments are shown and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents:

Claims

1. A human-motion-training system comprising:

a hardware platform that provides for stored-instruction processing and that includes memory, an I/O interface, and an audio-signal generation and output component;

an operating system or control program that executes on the hardware platform and that provides a program-execution environment; and

components of the human-motion-training system that provide for automated proctoring of human-motion exercises and training regimes, including monitoring of a user's body position, producing feedback corresponding to a user's body position, and determining a user's performance in order to modify the exercises and training regimes so that they provide optimal-challenge-point-based training over the course of multiple repetitions and training sessions.

2. The human-motion-training system of claim 1

wherein the components of the human-motion-training system include a scheduler and a number of finite-state-machine-implementing plug-in modules or dynamic-library routines; and

wherein the scheduler calls a routine for each finite state machine during each scheduler cycle, the routine returning a function pointer or reference through which the scheduler calls the finite state machine in a subsequent scheduler cycle.

3. The human-motion-training system of claim 2 wherein the finite state machines include:

a sensor finite state machine;

an analysis finite state machine; and

a synthesis finite state machine.

4. The human-motion-training system of claim 3 wherein the sensor finite state machine receives sensor output from one or more sensors, normalizes the received sensor output, and packages the normalized sensor output into sensor-information messages that the sensor finite state machine queues to a first first-in-first-out queue.

5. The human-motion-training system of claim 4 wherein the analysis finite state machine:

dequeues sensor-information messages from the first-in-first-out queue;

uses the sensor information contained in the dequeued sensor-information messages to prepare information messages regarding feedback appropriate for the current sensor positions and relative time within an exercise; and

queues the prepared information messages to a second first-in-first-out queue.

6. The human-motion-training system of claim 5 wherein the synthesis finite state machine:

dequeues information messages from the second first-in-first-out queue;

uses the analysis information contained in the dequeued information messages to launch a script that generates commands to control the audio-signal generation and output component to produce feedback.

7. The human-motion-training system of claim 1 wherein the components of the human-motion-training system prepare and store one or more computational envelopes and one or more time sequences of sensor positions to describe each exercise.

8. A human-motion-training system comprising:

one or more sensors mounted to known locations on a trainee's body;

a sensor-data-acquisition-and-processing component that receives data from the one or more sensors to continuously monitor the trainee's body position and compare the trainee's body position to a computational envelope and time sequence of body positions, stored in an electronic memory, that define an exercise or training session and that generates one or more feedback signals that represent conformance of the trainee's current body position to a current range of body positions specified by the computational envelope and time sequence of body positions stored in the electronic memory; and

a feedback component that receives the one or more feedback signals from the sensor-data-acquisition-and-processing component and renders the received one or more feedback signals into signals that can be perceived by the trainee.

9. The human-motion-training system of claim 8 wherein the one or more sensors are one or more of:

an accelerometer;

a gyroscope; and

a relative-position-determining device.

10. The human-motion-training system of claim 8 wherein the sensor-data-acquisition-and-processing component includes:

a number of finite-state-machine components; and

a scheduler that repeatedly cycles through the finite-state-machine components, invoking, in a particular scheduling cycle, each finit-state-machine component once.

11. The human-motion-training system of claim 10 wherein the finite-state-machine components include:

a sensor finite-state-machine component;

an analysis finite-state-machine component; and

a synthesis finite-state-machine component.

12. The human-motion-training system of claim 11 wherein the sensor finite-state-machine component continuously monitors the one or more sensors and prepares normalized sensor-data messages that are queued to a first input queue for the analysis finite-state-machine component.

13. The human-motion-training system of claim 12 wherein the analysis finite-state-machine component continuously monitors the first input queue for sensor-data messages and analyzes sensor data received in the sensor-data messages to determine the trainee's body position, compare the trainee's body position to the computational envelope and time sequence of body positions stored in the electronic memory to determine whether a feedback-related body-positioning event has occurred, and, when feedback-related body-positioning event has occurred, prepares an information message that the analysis finite-state-machine queues to a second input queue.

14. The human-motion-training system of claim 13 wherein the synthesis finite-state-machine component continuously monitors the second input queue for information messages and launches a feedback-signal-generating script to generate an appropriate feedback signal for the feedback-related body-positioning event indicated by one or more received information messages.

15. The human-motion-training system of claim 14 wherein the feedback-signal-generating script inputs control parameters to a signal-generating interface, such as a MIDI interface.

16. The human-motion-training system of claim 8 wherein the computational envelope represents the maximal extent of body position expected during the exercise.

17. The human-motion-training system of claim 8 wherein the time sequence of body positions represent the body motion that comprises the exercise.