AUTOMATED MOTION OF INTEREST RECOGNITION, DETECTION AND SELF-LEARNING

Abstract

A motion monitoring system can detect and identify motions of interest in motion data. The motion monitoring system can identify instances of the motion of interest in the motion data and can train a machine-learned model for identifying or characterizing motions of interest in motion data based on extracted features from the motions of interest. The motion monitoring system can also monitor motion data from a user and can help the user alter the performance of the motions of interest. For example, the motion monitoring system can generate an improvement strategy that helps the user improve their performance of the motion of interest. The motion monitoring system's altering of a user's performance of a motion of interest can be applied to physical therapy, gunfire detection, workplace motion improvement, sports, and driving.

Description

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No. 62/211,513, filed on Aug. 28, 2015, U.S. Provisional Application No. 62/213,003, filed Sep. 1, 2015, U.S. Provisional Application No. 62/293,658, filed Feb. 10, 2016, U.S. Provisional Application No. 62/338,016, filed May 18, 2016, and U.S. Provisional Application No. 62/355,528, filed Jun. 28, 2016, each of which is herein incorporated by reference in its entirety.

This application also is related to and incorporates by reference in its entirety U.S. patent application Ser. No. ______, filed 26 Aug. 2016, titled “System and Method for Automatically Time Labelling Repetitive Data”.

BACKGROUND

Field of the Invention

This disclosure relates generally to movement identification and more specifically to identifying motions of interest in signal data.

Description of Art

Sensors, such as accelerometers, can be used to generate signals of physical motion, and these signals can describe the physical motion of a sensor coupled to a person. For example, an accelerometer attached to the wrist of a user can detect the acceleration and motion of the user's arm. Conventionally, it is difficult to identify a motion of interest (MOI) in sensor signals. A motion of interest is one or more particular actions performed by a user. For example, a motion of interest can be a single repetition of a set of exercises (e.g., a single push-up or a single bicep curl), an action for work (e.g., lifting a box or reaching for a document), or an action associated with the usage of a firearm (e.g., drawing or firing the weapon). Motions of interest can be difficult to detect because the sensor signals can often contain noise that masks the portions of the signal that represent the motion of interest.

Some systems attempt to identify motions of interest in sensor signals using machine-learned models. However, these models often require humans to hand label sensor signals, which is time and resource intensive. Thus, these models either require significant resources to develop and continue training, or are trained on small training data sets, which reduces the accuracy of the models.

SUMMARY

A motion monitoring system can detect and identify motions of interest in motion data. The motion monitoring system can receive motion data from movement measurement devices. A movement measurement device can be a wearable device worn by a user that can transmit motion data describing the user's movement to the motion monitoring system. The motion data can include sensor signals from one or more sensors in the movement measurement device. The motion data can also include a count of the instances of the motion of interest and a motion of interest type.

The motion monitoring system can determine time labeling parameters that are used to time label the instances of the motion of interest in the sensor signals. Example time labeling parameters can include a motion of interest duration estimate, a sensor signal combination, and a motion of interest center. The motion monitoring system can generate time labels that define the start and end of candidate for motions of interest within the sensor signals. Each time label can include a starting time stamp and an ending time stamp. The motion monitoring system can generate the time labels based on the time labeling parameters. For example, the motion of interest duration estimate may be used to select a cutoff frequency for a low pass filter that filters out high frequency noise, the determined signal combination may be used to emphasize the motion of interest within the sensor signals, and the motion of interest center may be used to identify peaks or valleys associated with the motion of interest.

The motion monitoring system can identify time labels that represent true instances of motions of interest and time labels that are just noise. The motion monitoring system may use a reference signal and a count of instances of the motion of interest in the motion data to identify the time labels that represent motions of interest. The motion monitoring system can extract features describing the motion of interest from the motion data. Example features of a motion of interest include the speed, angle, consistency, fatigue, high frequency resonance from muscle twitching, range of motion, force, work performed, device orientation, body part orientation, and form. The motion monitoring system can then train a machine-learned model for identifying or characterizing motions of interest in motion data based on the extracted features.

The motion monitoring system can additionally help a user alter the performance of motions of interest. For example, the motion monitoring system can help a user improve the performance of a motion of interest, or may encourage/discourage the performance of the motion of interest. The motion monitoring system can identify instances of motions of interest in initial motion data and can generate a user motion model based on the identified motions of interest. A user motion model describes the motive abilities and behavior of a user. The user motion model can be compared with a reference model to determine an improvement strategy for the user.

The motion monitoring system can monitor additional motion data that can describe motions of interest performed by the user recommended by the improvement strategy. The motion monitoring system can identify instances of motions of interest in the additional motion data and can update the user motion model based on the identified instances of the motion of interest. The motion monitoring system can compare the updated user motion model to a reference model and, based on whether the user has met success criteria, can update the reference model or can adjust the improvement strategy. The motion monitoring system's altering of a user's performance of a motion of interest can be applied to physical therapy, gunfire detection, workplace motion improvement, sports, and driving, for example.

In some embodiments, the motion monitoring system automatically generates motion of interest models based on motion data without hand labeling by a human, unlike conventional techniques. Thus, the motion data used to train motion of interest models can include data from multiple users performing the same motion of interest in order to quickly obtain a large training data set. In addition, the motion monitoring system can identify portions of the motion data that are noise and can generate the motion of interest models based on the portions of the motion data that actually represent the instances of the motion of interest. Thus, the motion of interest models generated by the motion monitoring system can be more accurate than conventional methods can provide.

BRIEF DESCRIPTION OF DRAWINGS

The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

Figure (FIG. 1 illustrates an example system environment for a motion monitoring system, in accordance with some embodiments.

FIG. 2 is a flowchart illustrating a method for detecting and identifying motions of interest in motion data, in accordance with some embodiments.

FIG. 3 shows sensor signals from a three dimensional accelerometer, including an x-axis acceleration signal, a y-axis acceleration signal, and a z-axis acceleration signal, in accordance with some embodiments

FIG. 4A illustrates local maxima detected in sensor signals and time intervals between the detected local maxima, in accordance with some embodiments.

FIG. 4B illustrates local minima detected in sensor signals and time intervals between the detected local minima, in accordance with some embodiments.

FIG. 5 illustrates a reference signal used to determine the motion of interest center, in accordance with some embodiments.

FIG. 6 illustrates time labels for sensor signals, in accordance with some embodiments.

FIG. 7 illustrates time labels that have been identified as motions of interest and time labels that have been identified as noise, in accordance with some embodiments.

FIG. 8 is a flowchart illustrating a method for improving the performance of a motion of interest, in accordance with some embodiments.

FIG. 9 illustrates time labels that have been generated for a reference signal, in accordance with some embodiments.

FIG. 10 illustrates time labels on the reference signal that have been identified as motions of interest and time labels that have been identified as noise, in accordance with some embodiments.

FIG. 11 illustrates a sliding window that is used to extract features from portions of the sensor signals that are identified as representing a motion of interest and ignoring portions of the sensor signals that are identified as noise, in accordance with some embodiments.

FIG. 12 is a flowchart for monitoring patient improvement from an injury and adjusting an recovery strategy based on the patient's improvement, in accordance with some embodiments.

DETAILED DESCRIPTION

The features and advantages described in the specification are not all inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings and specification. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter.

Example System Environment and Architecture

FIG. 1 illustrates an example system environment for a motion monitoring system, in accordance with some embodiments. FIG. 1 includes a movement measurement device 100, a network 110, and a motion monitoring system 120. Alternate embodiments can include additional, fewer, or different components than those illustrated in FIG. 1 and the functionality of the components may be divided up differently from the description below. For example, while only one movement measurement device 100 is illustrated in FIG. 1, alternate embodiments can include a plurality of movement measurement devices 100 in communication with the motion monitoring system 120 through the network 110.

The movement measurement device 100 collects motion data describing a user's movement and transmits the motion data to the motion monitoring system 120. The movement measurement device 100 can be a wearable device, such as a smart watch, a fitness bracelet/anklet, or a headset. In some embodiments, the movement measurement device 100 communicates with a personal computing device (e.g., a smart phone, a tablet, a personal computer) to transmit the motion data to the motion monitoring system 120. The movement measurement device 100 includes one or more sensors to generate the motion data, such as an inertial measurement unit, an accelerometer, a gyroscope, a GPS module, a magnetometer, an electromyograph, or an electronic compass. The one or more sensors of the movement measurement device 100 generate one or more sensor signals to be included in the motion data. For example, an accelerometer in the motion measurement device 100 may generate a sensor signal that describes the acceleration of the movement measurement device 100 over time. In some embodiments, the movement measurement device 100 processes the motion data before transmitting the motion data to the motion monitoring system 120. For example, the movement measurement device 100 may encrypt, compress, or reformat the motion data. In some embodiments, multiple movement measurement devices 100 can be used simultaneously to capture motion data. These movement measurement devices 100 may communicate with the motion monitoring system or with each other through the network 110.

The movement measurement device 100 can communicate with the motion monitoring system 120 via the network 110, which may comprise any combination of local area and wide area networks employing wired or wireless communication links. In one embodiment, the network 110 uses standard communications technologies and protocols. For example, the network 110 includes communication links using technologies such as Ethernet, 802.11, worldwide interoperability for microwave access (WiMAX), 3G, 4G, code division multiple access (CDMA), digital subscriber line (DSL), Bluetooth, etc. Examples of networking protocols used for communicating via the network 110 include multiprotocol label switching (MPLS), transmission control protocol/Internet protocol (TCP/IP), hypertext transport protocol (HTTP), simple mail transfer protocol (SMTP), and file transfer protocol (FTP). Data exchanged over the network 110 may be represented using any format, such as hypertext markup language (HTML) or extensible markup language (XML). In some embodiments, all or some of the communication links of the network 110 may be encrypted.

The motion monitoring system 120 identifies and analyzes motions of interest in motion data received from the movement measurement device 100. The identification and analysis information about a motion of interest may be provided by the motion monitoring system 120 to the user through the movement measurement device 100 or through a personal computing device, such as a computer, a laptop, a tablet, or a phone. The motion monitoring system 120 can identify where a motion of interest is represented in motion data from the movement measurement device and can identify the motion of interest based on features of the motion data. Additionally, the motion monitoring system 120 provides an analysis of the identified motions of interest to a user. For example, the motion monitoring system 120 may compare features of exercise motions of interest to stored reference models of the exercise to determine whether the user is performing the exercises correctly and whether the user is improving in physical therapy at an appropriate rate. Methods that can be used by the motion monitoring system 120 are described below. The motion monitoring system 120 may store motion data from the motion measurement device 100. The stored motion data may include motion data collected from the user of the motion measurement device 100, other users of the motion monitoring system 120, and motion data from third party systems. In some embodiments, the motion monitoring system 120 stores historical motion data.

Detecting and Identifying Motions of Interest

FIG. 2 is a flowchart illustrating a method for detecting and identifying motions of interest in motion data, in accordance with some embodiments. Alternate embodiments may include more, fewer, or different steps from the steps illustrated in FIG. 2, and the steps may be performed in a different order from the order described herein.

The motion monitoring system 120 receives 200 motion data from a movement measurement device 100 that includes one or more sensor signals from one or more sensors in the motion measurement device 100. The sensor signals describe motions of interest performed by the user, such as exercise repetitions in an exercise regimen. FIG. 3 illustrates example sensor signals from the motion measurement device 100, in accordance with some embodiments. FIG. 3 shows sensor signals from a three dimensional accelerometer, including an x-axis acceleration signal 300A, a y-axis acceleration signal 300B, and a z-axis acceleration signal 300C, in accordance with some embodiments. In addition to the sensor signals, the motion data can include a motion of interest count, which describes the number of motions of interest that are included in the motion data. In some embodiments, the motion of interest count is input by the user to specify the number of motions of interest that are represented by the motion data. In other embodiments, the motion of interest count is pre-defined by the motion monitoring system 120 and the motion data must include the motion of interest count of instances of the motion of interest. The motion data can also include a name or other identifier of the type of the motion of interest. In some embodiments, the motion data includes motion data from multiple movement measurement devices 100 associated with one or more users.

The motion monitoring system 120 determines 210 time labeling parameters that are used to time label the motions of interest in the sensor signals. In the embodiment illustrated in FIG. 2, three time labeling parameters are estimated: a MOI duration estimate, a sensor signal combination, and an MOI center, which are described in further detail below. In other embodiments, more, fewer, or different time labeling parameters can be determined by the motion monitoring system 120.

The motion monitoring system 120 determines 220 an MOI duration estimate by determining a raw estimate of the MOI duration. A raw estimate is obtained by dividing the total length of time covered by the sensor signals by the MOI count. In other words, the raw estimate is an estimate that assumes that the motions of interest are evenly distributed throughout the sensor signals, and provides an upper limit of the motion of interest duration. The raw period can be used as the window length of a local extremum detector. The local extremum detector may be used to detect local extrema in the one or more sensor signals and may detect local minima, local maxima, or both. In some embodiments, the local extremum detector detects a local extremum if it is a local extremum of its type within a neighborhood with a width that is substantially equal to the raw estimate. For example, a local maximum may only be detected as a local maximum if it is the local maximum within the neighborhood of the local maximum. In some embodiments, the local extremum detector detects a local extremum if it is a local extremum of its type within a symmetric neighborhood around the local extremum.

The motion monitoring system 120 applies the local extremum detector to each of the one or more sensor signals in the motion data to detect local extrema within each of the sensor signals. The motion monitoring system 120 can determine time intervals between adjacent local extrema in the one or more sensor signals. If the local extrema include local minima and local maxima, the motion monitoring system 120 may determine time intervals between local extrema of the same type (i.e., time intervals between local maxima and time intervals between local minima) and can determine the MOI duration estimate based on lengths of the time intervals. In some embodiments, the MOI duration estimate is the median of the lengths of the determined time intervals.

FIG. 4A illustrates local maxima 400 detected in sensor signals 300 and time intervals 410 between the detected local maxima 400, in accordance with some embodiments. FIG. 4B illustrates local minima 420 detected in sensor signals 300 and time intervals 430 between the detected local minima 420, in accordance with some embodiments. To determine a MOI duration estimate, the motion monitoring system can select the time interval with the median length as the MOI duration estimate.

The motion monitoring system 120 determines 230 a signal combination that emphasizes the motion of interest. The signal combination can be a combination of one or more sensor signals that accentuates the motion of interest in the sensor signals. For example, for sensor signals describing push-ups, the combination signal may accentuate movement or acceleration along an axis perpendicular to the ground more than an axis parallel to the ground. In some embodiments, the motion monitoring system 120 selects a combination from a set of combinations, and selects the combination that provides a combination signal with the greatest variance. Possible combinations, using accelerometer sensor signals of FIG. 3 as an example, include:

$\begin{array}{c}\pm A_x\\ \pm A_y\\ \pm A_z\\ \pm \left(A_x+A_y\right)\\ \pm \left(A_x+A_z\right)\\ \pm \left(A_y+A_z\right)\\ \pm \left(A_x-A_y\right)\\ \pm \left(A_x-A_z\right)\\ \pm \left(A_y-A_z\right)\\ \pm \sqrt{A_x^2+A_y^2+A_z^2}\end{array}$

where A_x, A_y and A_z denote the raw data from accelerometer the x-axis acceleration signal 300A, the y-axis acceleration signal 300B, and the z-axis acceleration signal 300C respectively. Similar combinations apply to other sensor signals. The potential combinations are not limited to the lists above. For example, Euclidean sum, Eigenvector decomposition, and principle component analysis can also be used. In some embodiments, the combination of sensor signals may be selected based on the computational intensity of the combination. For example, if a complex combination allows for better time labeling of repetitions but at a great computational cost, the motion monitoring system 120 may select a combination that provides for worse labeling accuracy, but that is simpler to compute.

The motion monitoring system 120 determines 240 a center of the motion of interest in a reference signal. FIG. 5 illustrates a reference signal 500 used to determine the motion of interest center, in accordance with some embodiments. The motion of interest can be considered to be one of a series of repetitive motions that transfer between two states A and B, such that a typical sequence is A, B, A, B, . . . . Thus, a single instance of a motion of interest in this case may be either A to B to A or B to A to B. The starting states of these two cases are A and B, respectively, and the motion of interest centers of these two cases are B and A, respectively. In some embodiments, the repetitive motions that transfer between the two states A and B periodically and an instance of a motion of interest can be a portion of a single cycle of the periodic transitions between the two states. For example, if the motion of interest is a push-up, the two states are the straight arm high body position and the bent arm low body position, and the transitions between the two states can occur regularly within the motion data as the user performs a set of push-ups. In some embodiments, the transitions between the two states that correspond to an instance of a motion of interest can occur irregularly within the motion data, and an instance of the motion of interest is a transition from one state to the other state and back again to the first state. For example, if the motion of interest is the user lifting a box, the two states can be when the user is standing and when the user is squatting, and an instance of the motion of interest can be when the user goes from standing to squatting and back to standing.

The reference signal is a single signal that is used to represent the motion data. In some embodiments, the reference signal is generated based on the combination of the one or more sensor signals determined by the motion monitoring system 120. The reference signal may also be filtered using a low-pass filter with a cutoff frequency that is selected based on the MOI duration estimate. In some embodiments, the reference signal is the labeling signal described in U.S. patent application Ser. No. ______, titled “System and Method for Automatically Time Labelling Repetitive Data”.

To determine the motion of interest center, a portion at the beginning of the reference signal may be considered the starting state of the motion of interest. In some embodiments, the motion monitoring system 120 can require that the beginning portion of the reference signal be the starting state for a designated amount of time. Local maxima 505 and local minima 510 can be detected over the reference signal, and the median local maximum value 515 and the median local minimum value 520 can be determined. The mean value 525 of the reference signal can also be determined. The motion monitoring system 120 calculates the mean-maximum difference 530 and the mean-minimum difference 535. The mean-maximum difference 530 is the difference between the median local maximum value 515 and the reference signal mean value 525. The mean-minimum difference 535 is the difference between the median local minimum value 520 and the reference signal mean value 525. The motion monitoring system 120 selects, as the motion of interest center, the local extremum whose corresponding difference is greater than the other difference. For example, if the mean-minimum difference 535 is greater than the mean-maximum difference 530, then the reference signal mean value 525 is closer to the median local maximum value 515 than the median local minimum value 520. Thus, the local minima 510 represent the motion of interest centers and the local maxima 505 represent the starting state.

The motion monitoring system 120 generates 250 time labels that define the start and end of candidates for motions of interest. Each time label can include a starting time stamp and an ending time stamp, which are the times within the one or more sensor signals the candidate motion of interest starts and ends, respectively. The motion monitoring system 120 can generate the time labels based on the time labeling parameters. For example, the MOI duration estimate may be used to select a cutoff frequency for a low pass filter that filters out high frequency noise. The determined signal combination may be used to emphasize the motion of interest within the sensor signals, and the motion of interest center may be used to identify peaks or valleys associated with the motion of interest. The time labeling of sensor signals is described in further detail in U.S. patent application Ser. No. ______ titled “System and Method for Automatically Time Labelling Repetitive Data”. FIG. 6 illustrates time labels 600 for sensor signals, in accordance with some embodiments.

Some of the generated time labels may label noise in the sensor signals that appears to be, but are not actually, motions of interest. Conventional techniques typically collect motion data when the subject is performing an assigned motion of interest so that the motion data is labeled. Features are generated for machine learning algorithms of a machine learning tool. The machine learning algorithms train a classification model. The trained model is used to classify feature values from sensor data collected from an unknown activity/exercise. Additionally, conventional techniques generate a set of feature values from a fixed duration portion of the raw sensor data using a sliding window. The sliding window goes through the entire motion data from the beginning to the end with a certain step size. In conventional methods, all of the features obtained in this step are considered to describe the motion of interest, indiscriminately across all of the windows. Even though some windows within the motion data set may be noise such as walking or other activity instead of the motion of interest. Therefore, conventional methods result in inaccurate training data and therefore inaccurate models.

Instead, the motion monitoring system 120 identifies 260 time labels that are actually associated with motions of interest rather than noise. The motion monitoring system 120 may use the reference signal described above and the motion of interest count to identify the motion of interest time labels. For example, the motion monitoring system 120 may identify the local extrema of the reference signal within the generated time labels, and then may select, as true motions of interest, the set of the N time labels with the most extreme local extrema, where N is the motion of interest count. For example, if the motion of interest count is 10 and local minima of the reference signal represent the motions of interest, the motion monitoring system 120 may identify the time labels with the 10 lowest local minima as the true motions of interest and the remaining time labels as noise. The motion monitoring system 120 may also use machine-learned classifiers to determine which time labels represent motions of interest based on features of the portions of the one or more sensor signals within the time labels. Examples of machine-learning algorithms that could be used by the motion monitoring system 120 includes random forest, k-means, linear regression, logistic regression, decision tree, support vector machine, Naïve Bayes, or k-nearest neighbors.

FIG. 9 illustrates time labels 900 that have been generated for a reference signal 500, in accordance with some embodiments. FIG. 10 illustrates time labels on the reference signal 500 that have been identified as motions of interest 1000 and time labels that have been identified as noise 1010, in accordance with some embodiments. In the embodiments illustrated in FIG. 10, the time labels containing the 10 lowest local minima have been identified as instances of a motion of interest.

FIG. 7 illustrates time labels on the sensor signals that have been identified as motions of interest 700 and time labels that have been identified as noise 710, in accordance with some embodiments.

The motion monitoring system 120 extracts 270 features describing the motions of interest from the one or more sensor signals. Example features of a motion of interest include the speed, angle, consistency, fatigue, high frequency resonance from muscle twitching, range of motion, force, work performed, device orientation, body part orientation, and form. To extract the features from the one or more sensor signals, the motion monitoring system 120 can perform an angle analysis, a time analysis, or a form analysis of the sensor signals. The angle analysis can determine the angle between the direction of an axis of a sensor in the movement measurement device 100 and the direction of gravity. The angle analysis can also determine the change in that angle. The time analysis can analyze the timing of features of an instance of a motion of interest (e.g., the start and end timestamp of the motion of interest) and can identify a pace or frequency of the motions of interest, for example. A form analysis analyzes the features of two instances of a motion of interest and can determine the similarity or differences between those two instances. In some embodiments, the motion monitoring system 120 only extracts features from time-labeled portions of the sensor signals that have been identified as representing motions of interest. For example, the motion monitoring system 120 may use a sliding window to extract features from the sensor signals, and may ignore portions of the sensor signals that are time labeled and have not been identified as representing a motion of interest. In this way the content within the motion data is further refined when compared to conventional techniques in order to discriminate motion data of the true motion of interest from the noise. The performance in terms of classification accuracy is improved since the features can be extracted from portions of the motion data that is more accurate.

FIG. 11 illustrates a sliding window 1100 that is used to extract features from portions of the sensor signals that are identified as representing a motion of interest 1110 and ignoring portions of the sensor signals that are identified as noise 1120, in accordance with some embodiments.

The motion monitoring system 120 trains 280 a machine-learned MOI model for identifying the motion of interest represented in the one or more sensor signals. The MOI model may be a classifier model, such as a random forest, a support vector machine, or a neural network, and can be trained based on the extracted features from the identified MOI. The MOI model can be used to identify instances of a motion of interest of a particular type. For example, a push-up MOI model can be used to identify motions of interest within the one or more sensor signals that represent push-ups. Additionally, the MOI model can be used to describe the features of a motion of interest. In some embodiments, the motion monitoring system 120 trains an existing machine-learned MOI model based on the extracted features to improve the accuracy of the existing MOI model. In some embodiments, this step is optional.

The motion monitoring system 120 detects 290 instances of the motion of interest using the MOI model. The motion monitoring system 120 can extract features from the additional motion data and, based on the features, can use the MOI model to identify the motion of interest in the additional motion data. For example, if the motion monitoring system 120 receives motion data that describes a user's performance of push-ups, the motion monitoring system 120 can detect the push-ups in the motion data. Detecting the instances of the motion of interest in the motion data can include identifying portions of the motion data that represent the motion of interest, determining a count of instances of the motion of interest, and determining features of the instances of the motion of interest.

Training Customized MOI Models

A user in communication with the motion monitoring system 120 may be able to use the methods described here to define their own motion of interest and build a customized MOI model for the particular MOI. To do so, the motion monitoring system 120 may make application programming interfaces (API's) available to users (e.g., software developers) to allow a user to generate a custom MOI model.

To generate a MOI model for a custom motion of interest, the motion monitoring system 120 can receive motion data describing the motions of interest. The motion monitoring system 120 can also receive a name for the type of the motion of interest and a count of the instances of motion of interest in the motion data. In some embodiments, the motion monitoring system 120 requires that a particular number of instances of the motion of interest be provided in the motion data. For example, the motion monitoring system 120 may require that the user provide at least 10 instances of the motion of interest in the motion data. The motion monitoring system 120 can store the motion data and tag the motion data with the motion of interest name and, in some embodiments, a user identifier, such as a developer API key. The user identifier is unique to the user, so custom motions of interest generated by different users can be distinguished even if the different users choose the same motion of interest name.

An MOI model can be generated based on the provided motion data in accordance with the method described with FIG. 2. The user can then perform a cross-validation test to estimate the accuracy of the MOI model. In some embodiments, cross-validating the MOI model includes partitioning the received motion data into a set of groups, each containing an approximately equal portion of the motion data. A MOI model is generated using the motion data from a subset of the groups, and the motion data in the remaining groups is used to test the resulting model by using the algorithm to estimate the number of instances of the motion of interest in the motion data in those groups. An accuracy value is obtained by comparing the estimated number of MOI instances to the number reported by the user when the motion data was originally received. In some embodiments, this accuracy estimate is used as the accuracy of the MOI model. However, in other embodiments, this process is may be repeated on multiple different subsets of the set of groups, and the accuracy values obtained are averaged together to obtain an overall accuracy estimate. In some embodiments, if a subset of the set of groups results in an accuracy value that is significantly lower than accuracy values for other subsets of the set of groups, the subset may be identified as a noisy subset, and the motion data in the subset may be disregarded in training the MOI model and in determining the accuracy of the MOI model.

In some embodiments, the MOI model is generated and tested by the motion monitoring system 120. In other embodiments, the motion monitoring system 120 provides the motion data to the user to generate or test the MOI model locally.

Improving Motion of Interest Performance

FIG. 8 is a flowchart illustrating a method for improving the performance of a motion of interest, in accordance with some embodiments. Alternate embodiments may include more, fewer, or different steps from the steps illustrated in FIG. 8, and the steps may be performed in a different order from the order described herein.

The motion monitoring system 120 receives 800 initial motion data from a user's movement measurement device 100. The initial motion data describes motions of interest performed by a user. In some embodiments, the motion data is specifically generated for the purpose of training an MOI model. For example, the user may take additional steps to ensure that the motion data is less noisy or may perform a specific number or type of motion of interest. Additionally, the motion data may only contain motion data generated by a single user, or may include motion data from multiple users, where the motion data from each user may be designated with a user identifier.

The motion monitoring system 120 identifies 805 the instances of the motion of interest represented by the motion data in accordance with the methods described by FIG. 2 and generates 810 a user motion model based on the identified motions. The user motion model is a model of the user's motive ability and behavior. The user motion model can include one or more MOI models that are trained based on a single user's motion data and thereby describe a motion of interest as performed by the user. The user motion model can be used to identify motions of interest performed by the user within motion data and can be used to describe features of motions of interest performed by the user. For example, the user motion model may describe the speed at which the user typically performs a motion of interest. In some embodiments, the user motion model can be used to perform the angle analysis, time analysis, or form analysis to describe the features of the user's performance of a motion of interest. The user motion model may use MOI models of a user's performance of motions of interest to describe features of the user's motive abilities. For example, the user motion model may use MOI models to determine the user's strength or range of motion. In some embodiments, the user motion model can describe the motive abilities of different parts of a user's body. For example, the user motion model may describe the strength or range of motion of the user's elbow, shoulder, or knee.

The motion monitoring system 120 compares 815 the user motion model to a reference model. The reference model can describe the motive abilities and behaviors of multiple users. The reference model may include MOI models generated based on motion data from multiple users and can be used to identify or characterize motions of interest performed by other users. In some embodiments, the reference model is generated with motion data from users that are similar to the user. For example, the reference model may be generated based on motion data from users of the same age, sex, physiology, movement behavior, or personality as the user. Additionally, the motion monitoring system 120 may compare the user motion model and the reference model based on historical behavior of the users. For example, the motion monitoring system 120 may compare the user's change in performance of a motion of interest with the change of other users' performances of the motion of interest. In some embodiments, the reference model can also be generated from motion data such that the reference model can characterize how a motion of interest can be performed ideally.

By comparing the user motion model to the reference model, the motion monitoring system 120 can determine differences between how the user performs a motion of interest to how the motion of interest would be performed ideally. In some embodiments, the motion monitoring system 120 determines the differences by comparing features of the motion of interest performed by the user to features of the ideally performed motion of interest described by the reference model. For example, the motion monitoring system 120 can determine the differences in form, frequency, amplitude, consistency, speed, acceleration, or velocity of motions of interest.

The motion monitoring system 120 generates 820 an improvement strategy for the user based on the comparison of the user motion model to the reference model. The improvement strategy can describe motions of interest the user can perform to improve their performance of a motion of interest. For example, if the user is not able to perform push-ups quickly or is not able to hold the push-up in the lower body position for a long enough time, the improvement strategy may recommend that the user perform more push-ups or may recommend that the user perform other motions of interest that can help improve the user's push-up performance (e.g., bicep curls or bench presses). In some embodiments, the improvement strategy includes recommendations for how the user can adjust or practice the motion of interest. For example, if the motion of interest is a push-up, the improvement strategy may recommend that the user hold the lower body position for a longer period of time or may designate a recommended number of push-ups to perform. Similarly, if a user is bending their back while performing a push-up, the improvement strategy may recommend that the user adjust their performance of a push-up to straighten their back.

The improvement strategy may also automatically change the recommendations provided to the user over time based on a predicted improvement by the user. For example, the improvement strategy may increase the push-up sets or number of push-ups per set over time as the user improves their push-up performance. In some embodiments, the improvement strategy may include a recommendation for the user to use additional or improved movement measurement devices 100 to gather additional motion data to be used to determine the improvement strategy. For example, if the user is performing push-ups and currently is only wearing a movement measurement device on their wrist, the improvement strategy may recommend that the user wear a movement measurement device on their chest or waist.

In some embodiments, the motion monitoring system 120 automatically generates the improvement strategy. In other embodiments, the motion monitoring system 120 sends information about the user's performance of the motion of interest (e.g., the user motion model, the reference model, the motion data, the identified motions of interest, features about the motion of interest) to another user, and the other user provides the motion monitoring system 120 with the improvement strategy. For example, the motion monitoring system 120 may provide a personal trainer with the information about the user's push-up performance and the personal trainer may provide the motion monitoring system 120 with an improvement strategy for the user.

The motion monitoring system 120 monitors the user's progress in implementing the improvement strategy. The motion monitoring system 120 receives 825 additional motion data from the movement measurement device 100. The additional motion data can include motion data describing motions of interest recommended by the improvement strategy, and can also include other motions of interest. The motion monitoring system 120 identifies 830 the instances of the motion of interest in the additional motion data and updates 835 the user motion model based on the additional motion data. The motion monitoring system 120 may update the user motion model to describe the user's historic progress as well as the user's current performance of the motion of interest. In some embodiments, the motion monitoring system 120 updates the reference model such that the user motion model is being compared to a reference model that is generated based on motion data from users similar to the user. These users similar to the user may be identified based on the additional motion data or the updated user motion model. For example, if the user motion model was originally compared to a reference model based on users that improve quickly and the additional motion data shows that the user is improving more slowly than originally predicted, the reference model may be adjusted to compare the updated user motion model with users that improve more similarly to the user.

The motion monitoring system 120 compares 840 the updated user motion model to the reference model and determines 845 if success criteria have been met. The success criteria describe the final motive ability the user seeks to accomplish through the improvement strategy. The success criteria can include a threshold value for one or more features of a motion of interest, and may be determined based on features of an ideally performed motion of interest. If the user's improvement meets the success criteria, the motion monitoring system may update 850 the reference model based on the user's motion data. The update to the reference model may include updating the reference model to describe how the user's performance changed over time based on the improvement strategy.

If the user's improvement does not meet the success criteria, the motion monitoring system 120 may adjust 855 the improvement strategy based on the comparison of the user motion model and the reference model. For example, if the user's performance of the motion of interest is significantly deficient compared to the reference model and the rate at which the user is improving is low, the motion monitoring system 120 may adjust the improvement strategy to recommend a different way to perform the motion of interest or a different motion of interest to perform. In some embodiments, the motion monitoring system 120 only adjusts the improvement strategy if the user's improvement is significantly below the success criteria.

While the description above uses improving a user's performance of a motion of interest as an example, the method described by FIG. 8 is not limited to improving a user's performance of a motion of interest. It is possible to adjust the methods described herein to generally alter a user's performance of a motion of interest. For example, the method above can be adjusted to encourage or discourage a user from performing a motion of interest. In these examples, the motion monitoring system 120 may monitor the additional motion data and identify if the user does not perform a motion of interest they are supposed to perform or if the user performs a motion of interest they are not supposed to perform.

Example Applications of MOI Model

The following are examples of how the system and methods herein can be applied to particular contexts. These examples are not meant to be limiting and are not meant to be an exhaustive list of all possible applications of the described systems and methods. One of skill in the art can appreciate additional applications of the systems and methods herein.

Injury Detection and Recovery

The systems and methods herein can be used to help a patient recover from an injury through physical therapy. The motion monitoring system 120 may instruct a patient to perform diagnostic movements and motion data from the diagnostic movements can be used to generate a user motion model. The user motion model can be used to determine a recovery improvement strategy for the user, which can include exercise motions of interest the user is recommended to perform regularly. The motion monitoring system 120 can monitor the patient's progress and, if the patient is not improving at a predicted rate, the motion monitoring system 120 can adjust the recovery improvement strategy to better the patient's improvement and avoid further injuring the patient. The success criteria can be set for the user to recover to some or all of their pre-injury ability. In some embodiments, the motion monitoring system 120 generates and adjusts the user's recovery improvement strategy. In other embodiments, a doctor or physical therapist is provided with information about the user's motions of interest and improvement, and provides the motion monitoring system with the recovery improvement strategy.

Examples for the use of embodiments include activity detection in everyday life or physical therapy such as classification of sitting, walking, running, etc. In addition embodiments can be used to discriminate exercise in training, such as pushups, crunches, weight training, etc.

FIG. 12 is a flowchart illustrating a methodology that can be used in accordance with some embodiments. In embodiments, a user of a movement measurement device 100, such as a fitness tracking device, wears the device or otherwise uses 1202 the device so that motion data of the user, e.g., a patient, is automatically tracked. This information is sent to the motion monitoring system, which can be one or more servers. In one embodiment the motion monitoring system 120 is remote and data is sent over a wide area network 110, e.g., the Internet. Data transmission can be based on any transmission technique, such as wired, wireless including Bluetooth, WiFi, near field communication techniques, etc. A user may be instructed to perform 1204 prescribed movements and the data from the prescribed movements is analyzed 1206. The analysis can be done on the movement measurement device 100, using a processor within close proximity to the user and/or using a processor at a remote location. In one example, a computer based algorithm analyzes the prescribed movement data and provides information about the analyzed data to a second person, for example, a physician or physical therapist, who further analyzes the information and can prescribe a recovery strategy 2108 that includes a variety of movements.

The patient then performs 1210 recovery strategy movements and the data from the recovery strategy movements is transmitted 1212 to a remote server, for example. The data may first be transmitted from one or more movement measurement devices to a computing device, e.g., an application being executed on a desktop computer, a laptop computer, a mobile phone running an application, a computer tablet, another fitness tracking device, etc. In other embodiments the movement measurement device (s) transmits data to the server directly.

The data from the recovery strategy movements (RSM) is analyzed 1214 and if the analysis indicates that the user/patient's recover is not complete 1216 then the recovery strategy can be modified 1218 and the process can continue with step 1210. Determining whether the recovery is complete can be accomplished with an algorithm executed on a computer and/or by analysis by a person, e.g., a physician/physical therapist/medical personnel or anyone else trained to interpret the data. The recovery strategy movement data (e.g., motion data) can be stored and used 1220 to assist in generating recovery strategies for future patients.

One benefit of the embodiments is the ability to collect long term motion data about an individual's movement behaviors in order to establish signatures that can later be used to identify specific changes in behavior, potential illnesses/diseases, conditions, and injuries/potential injuries. The long term motion data can be stored by the motion monitoring system 120 and can be used in generating/updating the user motion model, or reference models for identifying potential illnesses, diseases, conditions, or injuries.

The motion monitoring system 120 can identify asymmetrical user movements that might indicate a loss of motor skill, a loss of range of motion, or other injury. The motion monitoring system 120 can compare the patient's user motion model with the reference model to determine if an injury has occurred or if an injury is likely to occur. For instance, a user can begin with a baseline range of motion on a joint, such as an elbow for a baseball pitcher. Over time not only could the range of motion for the elbow be monitored via a movement measurement device 100, but the motion monitoring system 120 can continually monitor a range of motion on that elbow and shoulder as the patient performs motions of interest either while exercising or from a recovery improvement strategy. In other examples, a range of motion can also be determined for bicep curls, shoulder lateral raises, or rubber band rotation exercises. If the user's range of motion reaches a limit, the motion monitoring system 120 can determine that an injury is more likely and can notify the user of the potential for injury to ensure the user is less likely to be injured. Additionally, the motion monitoring system 120 can adjust a recovery improvement strategy to reduce the likelihood that further injury will occur. This type of assessment and monitoring can be performed on any limb of the user with any variation of one or multiple sensors. Additionally, the motion monitoring system 120 may determine a likelihood of injury or potential injury based on reductions in tempo or load of motions of interest in a workout.

The user motion model can be continually compared to the reference model to track deviations in the user's performance of a motion of interest over time. Deviations are a change in a feature of a motion of interest, such as pace, form, consistency, or range of motion. Deviations can be identified and sent to one or more users, including the patient, a doctor, a physical therapist, or an electronic medical record, for example. For example, motions of interest from a patient that is recovering from a knee replacement surgery can be monitored for speed of movement, range of motion progression, and overall knee-movement form during the recovery period. Based on the motion data from the patient, the motion monitoring system 120 may determine that the patient is not performing the exercises as quickly as they did previously during the recovery period. The motion monitoring system 120 may then modify the recovery improvement strategy to ensure recovery is safe and to reduce the likelihood that the patient will become reinjured or discouraged. Similarly, the motion monitoring system 120 may adjust the recovery improvement strategy for patients who are recovering more quickly than similar patients.

The motion monitoring system 120 may compare a user's MOI model to a reference model that has been generated from motion data from users with a similar injury history or recovery history as the user. In some embodiments, the motion monitoring system 120 determines users that have similar injury or recovery history to the patient based on pre-surgical, post-surgical, or post-injury motion data.

The motion monitoring system 120 can identify injuries in a user by determining if certain motion of interest features indicate that the user is compensating for an injury. For example, one common result of having weak hips is excess strain on the knee, which can be caused by tight hip flexors or weak gluteus medius. This can make the thigh rotate inward, which can cause the user to perform motions of interest with features that show that the user is putting undue strain on the patella. The motion monitoring system 120 can identify that the user may have weak hips, and thus may generate an improvement strategy that can improve the user's hip strength. In some embodiments, the motion monitoring system 120 determines if the user is compensating for an injury based on the comparison of the user motion model and the reference model.

The motion monitoring system may initially receive motion data from a single movement measurement device 100 for initial motion data. The motion monitoring systems 120 may generate an improvement strategy that recommends additional movement measurement devices 100. For example, the motion movement system 120 may recommend a post-operation patient perform a series of motions of interest so that the motion monitoring system 120 can monitor the patient's progress in real time. As the patient progresses, the motion monitoring system 120 can compare the user's MOI model to reference models and can detect motion of interest features that are associated with a specific form of post operation movement impairment. The motion monitoring system 120 can generate or adjust an improvement strategy that recommends that more or better sensors be used to more accurately monitor this patient's progress/regress. For example, an ACL injury might have a poor range of motion recovery, and thus the motion monitoring system may recommend that more or better sensors be used to understand this range of motion degradation.

In some embodiments, the motion monitoring system 120 can generate a user motion model based on motion data from one side of the user's body and use motion data from the other side of the user's body to generate a reference model. For example, if the motion monitoring system 120 determines that a patient has some potential loss of motor skill or range of motion in the left leg, the physician can ask the patient to wear a movement measurement device 100 on both their left leg and right leg. The user motion model may be generated based on motion data from the left leg, and a reference model may be generated based on motion data from the right leg. The comparison of the user motion model and the reference model can show if the user has experienced a loss of speed, rotational velocity, acceleration, or range of motion, or has different fatigue points. For example, a user may have recently had their left shoulder replaced, and during recovery, they may have a tempo on a motion of interest with their left arm that is slightly slower than their right arm. Using motion data from other users, the motion monitoring system 120 may identify that the left arm's tempo disparity from the right is correlated with a shoulder injury or a poor shoulder replacement surgery.

The motion monitoring system 120 can identify which motions of interest were performed, which were performed incorrectly, and whether certain motions of interest were not performed at all or were performed with fatigue. The motion monitoring system 120 can adjust an improvement strategy to reflect the user's current state. The motion monitoring system 120 can use past motion data to identify incorrect motions of interest and correlate them with fatigue points or as potentially inducing an injury.

In some embodiments, the motion monitoring system 120 adjusts the improvement strategy based on whether the user performs motions of interest recommended by the improvement strategy. For example, the user may deviate from the improvement strategy by changing the number of repetitions, number of sets, weight used, the time of day, or types of exercises. The motion monitoring system 120 may notify the user of the deviation from the improvement strategy and may provide the user with an adjusted improvement strategy based on the deviation. In some embodiments, the motion monitoring system 120 notifies another user if the user deviates from the improvement strategy, such as a doctor or a therapist. The motion monitoring system 120 can identify the deviation in the improvement strategy and can notify the other user via an application notification, email, text, or phone call. The motion monitoring system 120 may also allow the other user to contact the user directly to intervene with the user's performance of the improvement strategy. For example, a therapist may be allowed to call the user to inform them of the deviation from the improvement strategy or to discuss the user's performance of the improvement strategy.

The motion monitoring system 120 may dynamically adjust an improvement strategy based on the user's real time performance of the motions of interest. For example, the motion monitoring system 120 may generate an improvement strategy that recommends a user do regular arm curls, hammer curls, and pushups for therapy. If user A performs regular curls poorly, the motion monitoring system 120 can determine that the motion of interest was performed poorly and can adjust the improvement strategy in real time, recommending that user A should skip the hammer curls or maybe replace the hammer curls with a less strenuous exercise as any more curls may risk injuring the user.

In some embodiments, the motion monitoring system 120 uses electromyography (EMG) motion data to determine features about a user's performance of a motion of interest, such as the quality of the user's muscle firing. The motion monitoring system 120 can determine if the user has an injury or the type of the injury using the EMG data. For example, if the EMG data shows that the user's muscle is firing weakly or atypically, the motion monitoring system 120 may determine that the user has an injury, and may determine the type of the injury based on features of the EMG motion data. In some embodiments, the motion monitoring system 120 can determine whether the user has an in jury by comparing the user motion model with a reference model generated based on motion data from other users.

The systems and methods described herein can also be used for insurance companies to assist in identifying risk profiles. Examples include: (1) automated premium adjustments based on daily, weekly, monthly motion of interest routines and regimens; (2) remote observational testing and monitoring through a movement measurement device of an individual for premium and health care price adjustment; (3) altering a price charged to a consumer based on motion data; (4) confirming adherence to rules/laws; for example, if a user does X when they should have done Y, then data is transmitted to the insurer who may increase the user's premium or risk profile; similarly if the user does what they are supposed to do, such as adhering to an exercise program, then that information may be transmitted to an insurer who may decrease the user's premium or positively adjust the user's risk profile.

Gunfire Detection and Training

The motion monitoring system 120 can be used to detect gunfire and to train a user to handle and fire a firearm properly and effectively. The motion monitoring system 120 can receive motion data from movement measurement devices 100 on one or more law enforcement officers. The motion monitoring system 120 can identify motions of interest related to firearms, including drawing, loading, firing, holstering, or dropping the firearm. The motion monitoring system can also identify motions of interest that relate to where the user is positioning the firearm, including whether the firearm is in the user's holster, whether the firearm is being held at the user's hip in a defensive posture, if the firearm is being held in front of the user in an assertive posture, or if the firearm is being aimed in an aggressive posture.

Additionally, the motion monitoring system 120 can extract features of the motions of interest from the motion data, including the type of firearm being fired, the type of ammunition being used, the rate of fire, the consistency of fire, the angle of the user's hand while the firearm is in use, and the aiming of the firearm. In some embodiments, these features can be determined by the motion monitoring system 120 based on an analysis of the motion data or a comparison of a user MOI model to a reference model.

The motion monitoring system 120 can determine if a user is aiming the firearm correctly. The motion monitoring system 120 can use motion data of the user aiming and firing the firearm to generate a user motion model that can describe the user's aim. The motion monitoring system 120 can compare the user motion model to a reference model that describes ideal aim for the firearm and can generate an improvement strategy for how the user can improve their aim. For example, if the user is firing the firearm too rapidly, the motion monitoring system 120 can recommend the user slow their fire rate to improve aim.

Additionally, the motion monitoring system 120 can use motion data to determine if a law enforcement officer is following a proper procedure for use of a firearm. A law enforcement officer may be expected to perform certain actions before resorting to presenting or using a firearm. For example, the law enforcement officer may be expected to provide a verbal warning first or may be expected to present a more defensive posture before using the firearm. The motion monitoring system 120 can generate a user motion model for the law enforcement officer that describes the law enforcement officer's actions before presenting/using a firearm and can compare the user motion model to a reference model that describes a user performing the proper procedure for presentation/use of a firearm. The motion monitoring system 120 can generate an improvement strategy for the law enforcement officer to improve their adherence to proper firearm use procedure and can intervene directly when the law enforcement officer is not following the proper procedure. In some embodiments, the motion monitoring system 120 notifies other users, such as the law enforcement officer's supervisor or other individuals in the law enforcement office, if the law enforcement officer is not adhering to proper firearm use procedure so that those users can intervene directly to train or discipline the law enforcement officer. Additionally, the motion monitoring system 120 can recreate the law enforcement officer's actions in a situation where a firearm was used so that the events of the situation can be more thoroughly evaluated by other users afterwards.

Workplace Activity Monitoring and Improvement

The motion monitoring system 120 can monitor motion data from a worker to ensure that the worker is performing workplace tasks effectively and safely. The motion monitoring system 120 can receive motion data from movement measurement devices 100 on one or more workers at a business. The motion monitoring system 120 can identify motions of interest that relate to the worker's performance of their job, such as lifting a box, using a tool, or walking/running. The motion monitoring system 120 can also identify important motions of interest that are not directly related to the worker's tasks, such as if the worker falls or if the worker is asleep/unconscious. The motion monitoring system 120 can generate a user motion model for the worker's performance of the workplace motions of interest and can compare the user motion model to a reference model for ideal performance of the workplace motions of interest. The reference model may be generated based on motion data from one or more users performing the workplace motion of interest correctly. The motion monitoring system 120 can determine if the user is performing the motion of interest ineffectively or in a way that could lead to an injury. For example, the motion monitoring system can determine if a worker is lifting boxes in a way that could injure their back. The motion monitoring system 120 can generate an improvement strategy for how the worker can improve their performance of the motion of interest and can monitor the worker's improvement over time, adjusting the improvement strategy if necessary. In some embodiments, the motion monitoring system 120 notifies other users, such as the worker's supervisor or a trainer, that the user is performing a motion of interest incorrectly, allowing the other user can intervene to train the worker directly or whether to allow the worker to continue performing the motion of interest.

Sports Performance Monitoring and Improvement

The motion monitoring system 120 can monitor motion data from an athlete to improve the athlete's performance in a sport. The motion monitoring system 120 can receive motion data from movement measurement devices 100 on one or more athletes. The motion monitoring system 120 can identify motions of interest that relate to the sport the athlete is participating in, such as running, jumping, walking, throwing, catching, punching, kicking, swimming, biking, or hitting. The motion monitoring system 120 can generate a user motion model for the athlete's performance of the motions of interest and can compare the user motion model to a reference model for ideal performance of the motions of interest. For example, the motion monitoring system 120 may compare the user's form in throwing a ball to an ideal throw. The reference model may be generated based on motion data from one or more other athletes performing the motion of interest correctly. The motion monitoring system 120 can determine if the athlete is performing the motion of interest ineffectively or in a way that could lead to an injury. For example, the motion monitoring system can determine if the athlete's running form is ineffective and can be improved to allow the athlete to run more quickly and with a lower likelihood of injury. The motion monitoring system 120 can generate an improvement strategy for how the athlete can improve their performance of the motion of interest and can monitor the athlete's improvement over time, adjusting the improvement strategy if necessary. The improvement strategy can include exercises the athlete should perform or recommended adjustments to the motion of interest. In some embodiments, the motion monitoring system 120 notifies other users, such as the athlete's coach or trainer, that the athlete is performing a motion of interest incorrectly, allowing the other user to intervene.

Driving Monitoring and Improvement

The motion monitoring system 120 can monitor motion data from a driver to improve the driver's driving abilities and reduce the likelihood of a car accident. The motion monitoring system 120 can receive motion data from movement measurement devices 100 on one or more drivers performing motions of interest related to driving a car. The motion monitoring system 120 can identify motions of interest that relate to the driver's behavior when driving the car, such as pressing a gas or brake pedal, pressing a clutch, shifting gears, turning the steering wheel, looking at cross-traffic, checking a mirror, opening/closing the door/window, adjusting the side-view or rear-view mirrors, putting on a seat belt, adjusting the seat, or interacting with an audio system. The motion monitoring system 120 can also identify motions of interest performed by the driver that are not related to driving the car, but may impact the driver's driving ability, such as using a phone or putting on make-up. The motion monitoring system 120 can use the identified motions of interest to other aspects of the driver's driving behavior, such as their hand positioning on the steering wheel, their speed, or frequency of lane change.

The motion monitoring system 120 can generate a user motion model for the driver's driving performance and can compare the user motion model to a reference model for ideal driving performance. The reference model may be generated based on motion data from one or more other drivers driving cars ideally. The motion monitoring system 120 can determine if the driver is driving a car safely. For example, the motion monitoring system 120 can determine if the driver is speeding, checking their phone, driving too aggressively, changing lanes too quickly, or does not check cross traffic. Additionally, the motion monitoring system 120 can determine if the driver's performance of driving motions of interest suggests that the driver is not in a state to safely drive the car, such as if the driver is tired or intoxicated. The motion monitoring system 120 can generate an improvement strategy for how the driver can improve their driving and can monitor the driver's improvement over time, adjusting the improvement strategy if necessary. The improvement strategy can include recommendations of additions, removals, or adjustments to motions of interest related to driving for the user drive more safely. In some embodiments, the motion monitoring system 120 notifies other users, such as a relative of the driver or law enforcement, that the driver is performing unsafe motions of interest while driving the car, allowing the other user can intervene to train the athlete directly. For example, if the driver performs a motion of interest that suggests that the driver is breaking the law or driving intoxicated, the motion monitoring system 120 may automatically notify law enforcement.

Additional Considerations

Reference in the specification to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment. The appearances of the phrase “in one embodiment” or “an embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps (instructions) leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic or optical signals capable of being stored, transferred, combined, compared and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. Furthermore, it is also convenient at times, to refer to certain arrangements of steps requiring physical manipulations or transformation of physical quantities or representations of physical quantities as modules or code devices, without loss of generality.

However, all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or “determining” or the like, refer to the action and processes of a computer system, or similar electronic computing device (such as a specific computing machine), that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.

Certain aspects of the embodiments include process steps and instructions described herein in the form of an algorithm. It should be noted that the process steps and instructions of the embodiments can be embodied in software, firmware, or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by a variety of operating systems. The embodiments can also be in a computer program product which can be executed on a computing system.

The embodiments also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the purposes, e.g., a specific computer, or it may comprise a computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Memory can include any of the above and/or other devices that can store information/data/programs and can be transient or non-transient medium, where a non-transient or non-transitory medium can include memory/storage that stores information for more than a minimal duration. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the method steps. The structure for a variety of these systems will appear from the description herein. In addition, the embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the embodiments as described herein, and any references herein to specific languages are provided for disclosure of enablement and best mode.

In addition, the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, the disclosure of the embodiments is intended to be illustrative, but not limiting, of the scope of the embodiments

While particular embodiments and applications have been illustrated and described herein, it is to be understood that the embodiments are not limited to the precise construction and components disclosed herein and that various modifications, changes, and variations may be made in the arrangement, operation, and details of the methods and apparatuses of the embodiments without departing from the spirit and scope of the embodiments.

Claims

1. A method comprising:

receiving motion data comprising a plurality of sensor signals describing a plurality of instances of a motion of interest and a count of the instances of the motion of interest;

generating a set of time labels for the plurality of sensor signals based on the one or more time labeling parameters, each time label identifying a portion of the sensor signals that can represent an instance of the motion of interest of the plurality of instances of the motion of interest;

extracting features of the motion of interest based on the set of time labels; and

detecting a type of the motion of interest based on the extracted features of the instances of the motion of interest.