Intelligent Sensor System
A sensor system and method of using the system synergistically to improve the accuracy and usefulness of measured results is described. The system is comprised of electronically linked components that act as markers to trigger events, producers that gather data from sensors and aggregators that combine the data from a plurality of producers using triggers from marker devices to select the data of interest. The system is shown to be applicable to selection of data regions of interest and to analysis of the data to improve accuracy. The analysis of the data of any particular sensor within the system makes use of extrinsic data, being data generated by other sensors and intrinsic data, that is data or data limits that are known to be true from nature, laws of physics or just the particular information the user wants to acquire. The system is demonstrated on the analysis of Doppler radar measurements of a thrown object.
Latest XBAND TECHNOLOGY CORPORATION Patents:
This application claims priority to U.S. Provisional application 61/662,011, filed on 20 Jun. 2012, entitled Intelligent Sensor System, by the same inventors and currently pending.
BACKGROUND OF THE INVENTION1. Technical Field
The present invention relates to sensors used to capture a sporting or other event and improved analysis of the sensor data.
2. Related Background Art
The use of sensors in sports and other activities to make measurements of the athlete's performance are becoming ubiquitous. Radar guns have long been used to measure the velocity of a pitched baseball, sensors on bicycles now measure speed, power output, pedaling cadence and heart rate of the rider. Video is being used to capture the swing motion of batters, golfers and tennis players. Slow motion replay of a baseball pitcher's motion or a batter's swing has been used for entertainment, instruction and training. Sensors and analyses of sensor data are used in a wide variety of sports and activities including for example: baseball, golf, tennis and other racket sports, football, gymnastics, dance and for help in rehabilitation of the people who have lost limbs and are learning how to walk or perform other activities with prosthetics.
Virtually all athletic skill development is an iterative process. One must perform a task, measure the outcome of the task and then analyze one's technique in order to improve. If any of these steps are missing in a training environment, this at best hinders the development of the athlete and at worst, prevents it. Young athletes who strive to compete at the highest levels in their sport are generally very self-motivated. They are the ones who work hardest during practice, stay after practice for extra repetitions and often train alone. Measurement is one of the key feedback mechanisms for specific skill development. In basketball, one can compute their shooting percentage for example while training alone. For many athletes, the velocity with which they can propel the ball in their sport is a critical measurement. Standalone radar units have been created to allow an athlete to gain a measure of their performance without the benefit of a coach or other observer being present. Other devices capture the speed, acceleration, and other dynamic attributes of bats, clubs, or racquets.
Inaccuracies in measurements of single events are common. Often the inaccuracies result in outlier data that may mislead the coach or athlete and/or result in lost data. Sifting through the data to pick out accurate data from outliers is a difficult and time consuming task. Outlier data may result from interference, such as an extraneous object in the field of view of the sensor, from electronic noise in the sensor data, or, from analysis of sensor data that is outside of a time range of interest. A means is needed to identify outlier data and remove such data from reporting.
Automatically, capturing the time range of interest is an important missing attribute of current systems. Sensors are often gathering data continuously. Yet the event of interest in the performance of the athlete may be just a few seconds or even fractions of a second buried in a mountain of continuous data. If the sensor is an image sensor for example, a coach or the athlete may sort through the image file to edit down to the time of interest. However this editing may not be readily available if the sensor is that of a radar gun or a heart rate monitor or other such device. A means is needed to sort and select the data of interest that is relevant to performance.
Often there is information that if available to a system analyzing sensor data could improve results. For example a video sensor might be able to pick out when a pitch is made, an audio sensor might provide information when a ball is struck. A radar sensor can determine when an object is moving within the sensor's field of view. A means is needed to make use of multiple sensor input to improve measurement results.
There is also other information available that is intrinsic to the event being captured that may be used to improve measurement results. For example it is extremely unlikely that a pitcher will hurl a baseball at 150 mile per hour, or that a very young pitcher will hurl a baseball at a speed greater than 70 mph. Current radar sensors regularly report such data in measurement results. These outlier measurements might be due to a variety of reasons. For example the radar guns are frequently located behind a screen that might produce interfering signals. Regardless of the source an intelligent analysis system is needed to recognize outlier data and remove it from reporting. There is also more subtle intrinsic data that may be used to improve measurement results. For example a pitched baseball will naturally be decelerating during its transit from the pitcher to the catcher. An intelligent analysis system is needed that can take advantage of this intrinsic knowledge and eliminate measurements of objects that are gaining speed during the measurement interval of interest.
Systems are needed that can repeatedly capture instances of a sporting activity including video and other sensors, make measurements of the outcome of each instance of the activity, automatically synchronize the video with the measurement, edit and analyze each instance of the video so that the athlete can compare actions and results of multiple attempts or instances. Systems are needed that take advantage of extrinsic data from other sensors and intrinsic information regarding the measurement of interest to improve the reported results of the measurement of an athlete's performance.
DISCLOSURE OF THE INVENTIONA system is described that addresses the deficiencies described above. A sensor system is described that makes use of both extrinsic data from a secondary sensor as well as intrinsic data regarding the measurement of interest to provide improved measurement results. One embodiment includes a communication protocol designed to allow various sports measurement devices to use relatively low-frequency RF communications to coordinate recording and measurement activities. In one embodiment a radar gun makes use of a secondary sensor to define the time interval of interest. In one embodiment the secondary sensor is a video sensor. In another embodiment the secondary sensor is an audio sensor that hears the ball hit the catcher's mitt or hears the bat hit the ball when the radar sensor is used to measure the speed of a pitched ball in for example baseball. In another embodiment the secondary sensor is an accelerometer located in the bat of the batter, again in a baseball situation. In another embodiment the secondary sensor is an audio sensor that detects the time of contact of a club head with a golf ball and the primary sensor is a video sensor capturing the golfer's swing. In another embodiment the primary sensor in the golf situation is a radar sensor measuring the speed of the ball. In another embodiment the primary sensor is an array of video sensors measuring the trajectory of the struck golf balls.
In another embodiment, a Doppler radar device that measures speed, an instrumented ball that measures orientation and spin, and a video camera that is placed to record a viewpoint of the pitcher's motion are all electronically linked. The ball and the Doppler device are each in turn “paired” with the video device. The video device is triggered by a wireless signal from the ball and the Doppler device to allow selective recording of important events (i.e. a pitch) and exclusion of non-important movements: the pitcher retrieving the ball or taking a drink. The video device also receives the measurements from the radar and spin rate devices and annotates them into the video. Thereby allowing the user athlete or a coach or fan to see and understand exactly what elements of the pitching motion leads to improved results such as faster throws or increased rotation at the proper angle for the desired pitch movement. The information from each device is aggregated and edited by the aggregator. The edited information provides annotated video for a plurality of events of interest (i.e. pitches) while excluding events not of interest.
In another embodiment a sensor may be used as both a primary measurement device and a secondary extrinsic sensor for a different but simultaneous other measurement. An example in baseball is where a video sensor is used to provide an alert that a pitch has been made so that a radar sensor will make measurements during the time interval of interest. The radar sensor may then be used as a secondary sensor to the same video sensor to indicate the time interval of interest for editing the video to a time interval of interest.
More complex arrangements can be made, but an ease of use factor to the user is that adding devices is just a series of “pairings” between devices. In the baseball example, the ball and radar are each “paired” with the video device. With no further configuration, they will work together to bring the scenario outlined above to fruition.
Another embodiment includes sensors that measure physiological and other parameters related to the participants. Non-limiting examples include heart rate monitors, blood pressure monitors, body temperature monitors and accelerometers. In some embodiments the sensors send signals to the coach warning of conditions of over exertion. In some embodiments the sensors send signals to the coach alerting of under exertion or lack of effort.
In another embodiment intrinsic information regarding the measurement is used. In one embodiment radar sensor data is analyzed making use of the fact that a pitched ball must be decelerating. In another embodiment outlier data is eliminated from sensor data using the fact that a pitched baseball must have a certain minimum speed to be in flight and a certain maximum speed to have been pitched by a human.
In another embodiment a communication protocol that includes three classes of devices is shown: markers, producers and aggregators. A marker detects an event and produces an electronic signal of the event or trigger. A producer collects data during an event. An aggregator collects data from multiple producers. In one embodiment a marker, producer and aggregator are two or three physically separate devices. In another embodiment the marker, producer and aggregator are a physically container in a single device. In another embodiment a producer may function both as a producer and as a marker.
Referring to
In the example shown the communication protocol including markers, producers and aggregators is described. Referring to
In another embodiment the marker 110 can also be interrogated to retrieve information about the events prior to and following their “trigger”. The main purpose of a marker is to inform other devices (Producers and Aggregators) that a specific event they are designed to detect has occurred. If the device makes more data available automatically through its interface than triggers it is both a Marker and a Producer.
Producers 109 have sensors that acquire data related to an event. They may rely on Marker(s) to trigger or complete their actions. Producers may in turn act as Markers themselves in the sense that they can trigger other devices. In the embodiment shown the ball 102 is instrumented to include an accelerometer sensor that can measure position and rotation of the ball. The Producer also includes means to transmit data to the Aggregator 111. In the embodiment shown the Producer 109 captures spin data 115 of the ball, captures the speed of the ball 116 and transmits the data 117.
In another embodiment the Producer presents the data collected without any external triggers required. In the example shown the ball further includes a display 124 that can show data collected including the maximum rotation achieved over the measurement time interval since a last reset.
The Aggregator 111 acquires data from one or more Producer devices 109 and presents that data 114 as an integrated set of information to a user. It is also possible for the Aggregator itself to act as a Marker or a Producer. In the instant case the aggregator further includes a video camera and radar detector 113 that provides video data and speed data for the ball as thrown by the pitcher (not shown) to the catcher. In the instant example the Aggregator 111 includes the functionality of both a Producer and an Aggregator. In the example shown the aggregator includes the process of starting the acquisition of a video 120, receiving a trigger 121, ending the video 122 and appending data to the video 123 producing a video clip and information on the display 114. In the instant case the aggregator produces a video of the pitchers motion 125 as well as data 126 related to the speed, rotation and accuracy of the pitch.
The main goal of a Marker 110 is to produce the “trigger” at the exact moment an event has occurred. However, depending on the analysis required of the data that the Marker collects, this may not be possible. Thus, we introduce the first two types of triggers, the Exact Trigger and the Revisable/Revision trigger pair. For purposes of this document, an Exact Trigger is guaranteed to have occurred within 1/1000 second of the actual event. A Revisable Trigger is transmitted when a device has determined that the event either has or is about to occur but its exact time cannot be pinpointed. Though the goal is to make the Revisable Trigger as close as possible to the exact time, as some devices may not be able to produce optimal output if a trigger is revised in a negative direction, an example of which would be a still camera. The receiver of a Revisable Trigger is required to note the time that the trigger was received in its own time domain. A Revisable Trigger is always followed within five seconds by a Revision Trigger that notes the time delta (either positive or negative) between the actual event and the time the Revisable Trigger was transmitted.
It is expected that these time measurements, due to some of the uncertainties involved in RF transmission, will have small errors. The goal is +/− 1/1000 second accuracy, which represents good accuracy for most athletic timing systems as well as synchronization of most video/photographic systems, though efforts will be taken to not constrain the accuracy of the system to that number. Note that the trigger is used to synchronize timing. It is not always used to trigger an event at the time of the trigger. In the example the aggregator 111 begins recording video prior to receiving a trigger 121. The aggregator then edits the stored video to produce a clip around the time of the event of interest. Referring to
Referring now to
In another embodiment shown in
Referring to
The techniques discussed are applicable to a wide variety of sport and non-sport measurement situations. The techniques have been demonstrated thus far with respect to measurements of a thrown baseball. Referring to
Referring now to
Referring to
Referring now to
Referring to
Referring to
Once the filtering step 1505 is completed the data is selected and averaged as already described. In one embodiment just the prominent pulse and the data buckets to either side of the prominent pulse are averaged to produce a measured result.
In another embodiment the filtering step 1206 based upon an object not slowing down is based more generally upon pulses indicating acceleration outside of a pre-selected range. The method is generally applicable to both accelerating and decelerating objects. The specific example of
In another embodiment shown in
In another embodiment the system of
A sensor system and method of using the system synergistically to improve the accuracy and usefulness of measured results is described. The system is comprised of electronically linked components that act as markers to trigger events, producers that gather data from sensors and aggregators that combine the data from a plurality of producers using triggers from marker devices to select the data of interest. The system is shown to be applicable to selection of data regions of interest and to analysis of the data to improve accuracy. The analysis of the data of any particular sensor within the system makes use of extrinsic data, being data generated by other sensors and intrinsic data, that is data or data limits that are known to be true from nature, laws of physics or just the particular information the user wants to acquire. The system is demonstrated on the analysis of Doppler radar measurements of a thrown object.
Those skilled in the art will appreciate that various adaptations and modifications of the preferred embodiments can be configured without departing from the scope and spirit of the invention. Therefore, it is to be understood that the invention may be practiced other than as specifically described herein, within the scope of the appended claims.
Claims
1. A system for making measurements on moving objects said system comprising:
- a) a first sensor that can detect a movement of objects,
- b) a second sensor that can detect the movement of objects,
- c) a processor that acquires data from the sensors, calculates measurements related to the movement of the objects and reports the measurements
- d) where the processor filters the raw data used in its calculation, said filters based upon both intrinsic information and data acquired by both of the sensors,
- e) where intrinsic information is at least one selected from: i) a maximum speed for the objects, ii) a minimum speed for the objects, iii) the three dimensional location of the objects at a start of the movement, iv) the three dimensional location of the objects at an end of the movement, and, v) a time interval between the start of the movement and the end of the movement,
- f) where the measurements are at least one selected from: i) the average speed of the objects, ii) the maximum speed of the objects, iii) acceleration of the objects, and, iv) a video of the objects during their movement.
2. The system of claim 1 where the first sensor is a video camera and the second sensor is a Doppler radar gun.
3. The system of claim 2 further including a third sensor that is a video camera that is placed at a known location relative to the first sensor.
4. The system of claim 3 wherein the measurement includes the three dimensional position of the objects.
5. The method of claim 1 wherein the first sensor detects a movement of the object and based upon that detection sends an alert signal to the second sensor, said alert signal activating the second sensor to store data.
6. The system of claim 1 where the objects include a living animal and further including sensors of physiological parameters of the animal and measurements further include physiological measurements.
7. The method of claim 6 further including a means for alerting a user of the system when a physiological measurement is measured outside of a preselected range.
8. A method for analysis of Doppler radar data said method including:
- a) acquiring pulse data from a Doppler radar sensor, said pulse data indicative of a speed of an object within the field of view of the Doppler radar sensor
- b) preparing a histogram of the pulse data, said histogram comprising parsing the data into time buckets based upon the pulse width of individual pulse data points and the frequency of occurrence of data points within the time intervals,
- c) filtering the data to remove pulse data that is less than a preselected minimum speed,
- d) filtering the data to remove pulse data that indicates a speed greater than a preselected maximum speed,
- e) filtering the data to remove pulse data that occurs outside of a time interval said time interval defined by a second sensor,
- f) filtering the data to remove pulse data that indicates an acceleration outside of a preselected acceleration range,
- g) selecting a bucket from the histogram with the highest frequency of occurrence of pulse data,
- h) calculating a weighted average of the pulse data including the bucket with the highest frequency of occurrence of pulse data and a preselected number of neighboring buckets,
- i) reporting the speed of the object based upon the weighted average pulse width.
9. The method of claim 8 wherein the object is a pitched baseball and the preselected acceleration range indicates a deceleration of the baseball.
10. The method of claim 8 wherein the second sensor is a video camera.
11. The method of claim 8 wherein the second sensor is a sound sensor.
12. A communication system and protocol for acquiring data related to a sport activity comprising:
- a) a marker device that includes an electronic sensor that detects an external event related to the sport activity said marker device sending a trigger signal when the sensor detects an external event,
- b) a producer device that includes an electronic sensor that detects an external event related to the sport activity and said producer device records in an electronic memory a signal from the electronic sensor in the producer device and transmits the recorded signal,
- c) an aggregator device that receives the trigger signal from the marker device and receives the transmitted recorded signal from the producer device and filters the transmitted recorded signal to include only data that corresponds to data recorded in a pre-selected time interval around the time of the trigger signal and displays the filtered data on an electronic display included in the aggregator device.
13. The communication system and protocol of claim 12 further including a plurality of producer devices each including an electronic sensor that detects an external event related to the sport activity and each recording in an electronic memory a signal from the electronic sensor in each of the plurality of producer devices and each transmitting the recorded signal.
14. The communication system and protocol of claim 12 further including a plurality of marker devices each including an electronic sensor that detects an external event related to the sport activity and each sending a trigger signal when the sensor in each of the plurality of marker devices detects an external event.
15. The communication system and protocol of claim 13 wherein the aggregator combines the data from the plurality of producer devices and displays the combined data on a display included in the aggregator device.
16.
Type: Application
Filed: Jun 6, 2013
Publication Date: Dec 26, 2013
Patent Grant number: 9746353
Applicant: XBAND TECHNOLOGY CORPORATION (Escondido, CA)
Inventors: Kirt Alan Winter (San Diego, CA), Jose Julio Doval (Escondido, CA)
Application Number: 13/912,005
International Classification: G01D 11/00 (20060101); G01S 13/58 (20060101); G01S 13/86 (20060101); G01P 21/02 (20060101);