Real Time Feedback Swim Training System and Method Based on Instantaneous Speed

Abstract

A swim training system and method includes a signal delivery sub-system configured to deliver signals to the swimmer and a machine vision sub-system with a detector and a target marker to be worn by a swimmer. The detector is configured to detect the target marker. A controller is configured to operably control the machine vision sub-system. A data processing sub-system is configured to process data collected by the machine vision sub-system and to generate signals for the signal delivery system. During a swim training session, the detector tracks a location of the target marker and transmits speed data to the data processing sub-system. The data processing sub-system processes the speed data and generates signals that are transmitted to the swimmer via the signal delivery sub-system in real time during the swim session. The signals are representative of a real time speed characteristic of the swimmer.

Description

RELATED APPLICATION DATA

This patent is related to and claims priority benefit of U.S. provisional patent application Ser. No. 61/753,759, which was filed on Jan. 17, 2013 and also entitled “Real Time Feedback Swim Training System and Method Based on Instantaneous Speed.” The entire content of this prior filed provisional application is hereby incorporated herein by reference.

BACKGROUND

1. Field of the Disclosure

The present disclosure is generally directed to swim training devices, systems, and methods, and more particularly to a swim training system that provides real time feedback based on an instantaneous swimming speed during swimmer training.

2. Description of Related Art and Problems with Same

It is well known in many athletic training endeavors to employ devices and systems to assist the athletes with objective feedback about their performance. This is particularly true for competitive swimming because speed is the critical indicator of each individual swimmer's performance. Objective feedback regarding swimming speed is critical for both swimming skill development and competitive swim training.

Objective feedback devices, methods, and systems that assist swim training related to swimming speed can be classified into two categories: 1) feedback delayed until the end of a swim and 2) real-time feedback during a swim. There are many widely known and used methods and systems of delayed feedback, which include the following.

Stop watch. A stop watch allows coach to measure time and speed of a swim. The coach can communicate the measured time to the swimmer at the end of the swim. All fluctuations of the swimming speed within one swim are necessarily averaged. The stop watch does not allow a coach or the swimmer to measure swimming speed fluctuations occurring during any one swim.

Pace clock. A pace clock allows a swimmer to measure the time and speed of any one swim. A swimmer can view and determine the time of a previous split at the end of the swim. All fluctuations of the swimming speed within one swim are averaged. A pace clock also does not allow a swimmer or coach to measure swimming speed fluctuations occurring during any one swim.

AVIDA Sports. The AVIDA Sports system provides a swimmer with a message to the swimmer about the split time of the immediately previous lap. The AVIDA Sports system uses the term “Real-Time” to indicate that the split time information is provided to the swimmer on every lap about the split time of the previous lap. All swimming speed fluctuations occurring within one length or lap of the pool are not averaged and not available to the swimmer or coach.

Methods using a cord attached to swimmer's waste belt. The following four methods employ a cord with one end that attaches to the swimmer and another end that attaches to a deck mounted piece. The cord stretches during the swim between the deck-mounted piece and the swimmer.

1) “Swim Power Test”—see http://www.globsport.org

2) “Torrent E-Rack Swim Speed Analyzer”—see http://www.teamtermin.com/solutions.shtml

3) “Speedometer Swimming Measuring System”—see http://www.sportthieme.co.uk/Swimming/Training_Equipment/Swimming_Benches/art=22 36041

4) “Velocity Meter/Video Telemetry”—see http://www.teamtermin.com/

These four methods and systems that use a cord attached to swimmer can provide feedback about speed fluctuations, but only after the swim is completed, not during the swim where the cord is employed. The attached cord methods and systems also only allow the swimmer to perform measurements when the swimmer is traveling in one direction, away from the deck mounted piece. A regular training swim typically involves two-way lap swimming.

Methods based on ultrasound Doppler effect. Koigerov, 1979. Some have tried to develop improved swim training devices as well, to varying degrees of success. For example, Koigerov/Kolmogorov utilized Doppler effects and ultrasound to measure the speed of a swimmer in a pool. However, accuracy is poor at lower speeds and ultrasound reflects off the walls of the pool, further reducing accuracy. The device is positioned at the bottom of the pool. The wavy surface of the water can also limit the capability of the device to locate the swimmer on the surface. The device can also be used only with a single swimmer in the pool. Otherwise the data would be compromised by obtaining data from the wrong swimmer. This device is best used for obtaining data for a swimmer's starts. This device is also not designed for real time feedback. Analysis of the data points collected is done only after the swimmer's training session.

Video recording analysis. Analysis of swimming speed fluctuations based on video recording is also widely used. For example, Karpovich, 1930, Maglischo 2003, Barbosa, et al 2012; etc. have used video recording techniques. These methods provide in-depth analysis of swimming speed fluctuations. However, feedback is available to the swimmers only after the swim is completed, not real time.

Methods, devices, and systems that can provide “real time” feedback to the swimmer or coach regarding instantaneous swimming speed during swim training include the following.

Real-Time Swimming Monitor. US Published Application No. 2010/0030482 to Xipu Li discloses a complex micro-electro-mechanical system (MEMS) device that uses accelerometers and gyroscopes. Accuracy of the results produced by the device is questionable due to the cumulative nature of measurement errors, inherent to accelerometers. This error accumulation is known as measurement drift. Measurement error of a current measurement cycle is added to the measurement errors of all previous measurement cycles after the last calibration of the system. Calibrations are required frequently to correct measurement drift in order to provide more accurate speed measurements. Any measurement drift correction or sub-system of the disclosed Real-Time Swimming Monitor can make the system and components complex and quite expensive.

Swim Meter. U.S. Pat. No. 5,767,417 discloses a device used to measure swimming speed in open water or non-standard width swimming pools. The measurements are based on flow of the water around the swimmer's body. The disclosed Swim Meter is not intended for measuring swimming speed fluctuations that occur within time intervals of 100 milliseconds or less. The accuracy of the swimming speed measurements of the Swim Meter, which is based on either of a rotating element, a pitot tube and pressure sensor, a current generator responsive to salt ions in the water, or an electrical generator, is not sufficient for speed measurements within such short time intervals.

The foregoing and other known devices, systems, and methods for providing feedback based on swimmer's speed are capable of determining average speed per swim. However, these devices, systems, and methods are not capable of providing sufficiently accurate measurement rates required for analysis of speed fluctuations within a swimming stroke cycle. Video analysis systems are used mostly for research into swimming stroke technique. However, the results of video analysis are not available to the swimmer in real time while they swim. The time delay, required for video data processing, causes delay in feedback delivery to the coach or swimmer. Delayed feedback is less effective than real time feedback for stroke correction and training.

SUMMARY

The disclosed NATATOMETRY™ system automates analysis of video data in order to provide speed fluctuation feedback delivered in real time to the swimmer. The real time feedback during a swim training session increases the effectiveness of stroke corrections and training regimens in real time while a swimmer is training.

In one example according to the teachings of the present disclosure, a swim training system has a signal delivery sub-system configured to deliver signals to a swimmer and a machine vision sub-system having a detector and a target marker to be worn by the swimmer. The detector is configured to detect the target marker. A controller is configured to operably control the machine vision sub-system. A data processing sub-system is configured to process data collected by the machine vision sub-system and to generate signals for the signal delivery system. During a swim training session of the swimmer, the detector tracks a location of the target marker and transmits speed data to the data processing sub-system. The data processing sub-system processes the speed data and generates signals that are transmitted to the swimmer via the signal delivery sub-system in real time during the swim session. The signals are representative of a real time speed characteristic of the swimmer.

In one example, the target marker can be a light belt, which can carry one or more lights visible to the machine vision sub-system.

In one example, the target marker can be a light belt with a plurality of omni-directional LED lights visible to the machine vision sub-system.

In one example, the signal delivery sub-system can include an in-ear audio transmitter worn by the swimmer.

In one example, the signal delivery sub-system can include a remote FM transmitter and can include an FM receiver worn by the swimmer. The FM transmitter and/or receiver can be tunable to a desired FM channel.

In one example, the data processing sub-system can include an electronic device with a data processor.

In one example, the controller can be a computer, a laptop computer, a smart phone, a tablet, or the like.

In one example, the detector of the machine vision sub-system can include one or more video cameras.

In one example, the detector of the machine vision sub-system can include one or more video cameras that can be configured for underwater use and to detect the target marker under water.

In one example, the real time speed characteristic can include a desired instantaneous swim speed range between a low threshold speed and a high threshold speed.

In one example, the signal delivery sub-system can deliver a first signal when an instantaneous swim speed of the swimmer is slower than a low threshold speed, a second signal when the instantaneous swim speed of the swimmer is faster than a high threshold speed, and a third signal when the instantaneous swim speed of the swimmer is within a desired instantaneous swim speed range between the low and high threshold speeds.

In one example, a first signal delivered to the swimmer can be a first auditory signal, a second signal delivered to the swimmer can be a second auditory signal different from the first auditory signal, and a third signal delivered to the swimmer can be the absence of an auditory signal.

In one example according to the teachings of the present disclosure, a swim training method includes providing a signal delivery sub-system, a data processing sub-system, and providing a machine vision sub-system that has a detector and a target marker. The target marker is attached to a swimmer and the detector of the machine vision sub-system is placed so that it can detect the target marker on the swimmer. A controller is operated to control the machine vision sub-system. The detector tracks the target marker as the swimmer moves during a swim session. Data is sent from the machine vision sub-system to the data processing sub-system during the swim session. The data processing sub-system processes the data. Signals are transmitted to the swimmer via the signal delivery sub-system in real time during the swim session. The signals are representative of a real time speed characteristic of the swimmer.

In one example, the method can include providing one or more cameras as the detector that can detect the target marker.

In one example, the method can include providing a light belt that carries one or more lights detectable by the detector of the machine vision sub-system.

In one example, the method can include placing the detector under water.

In one example, the method can include providing a programmable electronic device as the controller and configuring the electronic device to operate the machine vision sub-system.

In one example, the method can include transmitting an audible signal to the swimmer.

In one example, the method can include transmitting two or more different audible signals to the swimmer via an ear piece of the signal delivery sub-system.

In one example, the method can include the controller sub-system comparing a desired instantaneous swim speed range between a low threshold speed and a high threshold speed to an instantaneous swim speed of the swimmer during the swim session.

In one example, the method can include transmitting a first signal when an instantaneous swim speed of the swimmer is slower than a low threshold speed, a second signal when the instantaneous swim speed of the swimmer is faster than a high threshold speed, and a third signal when the instantaneous swim speed of the swimmer is within a desired instantaneous swim speed range between the low and high threshold speeds.

BRIEF DESCRIPTION OF THE DRAWINGS

Objects, features, and advantages of the present invention will become apparent upon reading the following description in conjunction with the drawing figures, in which:

FIG. 1 shows a simplified schematic of one example of a NATATOMETRY system constructed in accordance with the teachings of the present invention.

FIG. 2 shows a simplified plot of one example of real time swim speed data over time obtained, processed, and delivered by the NATATOMETRY system FIG. 1.

DETAILED DESCRIPTION OF THE DISCLOSURE

The disclosed real time feedback swim training system (hereinafter the “NATATOMETRY system”) solves or improves upon one or more of the above-noted and/or other problems and disadvantages with prior known swim training systems, methods, and devices. As used herein, the phrase “real time” is defined as a time interval of 100 milliseconds or less between the moment of instantaneous speed and the beginning of a signal or indicator, such as an audio alarm or a visual signal, being delivered to the swimmer, such as via an ear piece or swim goggles worn by the swimmer.

In one application, real time feedback, based on instantaneous speed, can be used by swimmers who are correcting swimming technique to do so while swimming. Using the disclosed NATATOMETRY system and method, a swimmer can achieve high instantaneous speed within a stroke cycle, or avoid a low instant speed within a stroke cycle, or stay within the range of desired instant speed fluctuations within a stroke cycle.

In one application, real time feedback, based on instantaneous speed, can be utilized by swimmers who are performing drills that involve or require intentional change of speed between stroke cycles during, for example, build up and build down drills. During such drills, the swimmer intentionally accelerates or decelerates through one swim. Real time feedback to the swimmer during the swim can help them achieve the desired speed changes.

In one example, the NATATOMETRY system can be programmed not to deliver any signal to notify the swimmer that his or her instant speed has fallen below any low speed threshold but can deliver a signal a signal to notify the swimmer that their instantaneous speed is above a high speed threshold. In one example, the NATATOMETRY system can be programmed not to deliver any signal to notify the swimmer that his or her instant speed is above a high speed threshold but can deliver a signal that their speed has fallen below a low speed threshold. In one example, the NATATOMETRY system can be programmed to deliver different signals to notify the swimmer when their instantaneous speed falls below or increases above such low and high speed thresholds, respectively. The NATATOMETRY system can be programmed to alter the thresholds during a swim to achieve a specific training regimen that requires speed variation during the training swim.

In one example, the disclosed NATATOMETRY system can provide a swimmer with real time feedback regarding their instantaneous speed in the form of audio alarm signals. In one example, the disclosed NATATOMETRY system can deliver an audible signal such as a high-pitch (900 Hz audio signal for example) to an ear piece worn by the swimmer that the swimmer can hear while swimming. In one example, the disclosed NATATOMETRY system can deliver a visual signal that is visible to the swimmer such as a green light illuminated through the goggles, for example, while swimming. In one example, the disclosed NATATOMETRY system can deliver both audio signals such as the high-pitch audio signal and visual signals such as the green light to the swimmer when their instantaneous speed is above a high speed threshold. These and/or other signal types can also be used to notify the swimmer that he or she has successfully reached a pre-set speed threshold, such as a high threshold for instant speed in real time.

In other examples, the NATATOMETRY system can also deliver any type of signal to notify the swimmer that his or her instant speed has fallen below a predetermined low speed threshold. In one example, the disclosed NATATOMETRY system can deliver a low-pitch (400 Hz) audio signal and/or a visual signal (red light through the goggle) to notify the swimmer that he or she has failed to avoid a pre-set low threshold of instant speed during the previous measurement cycle. In this particular example the NATATOMETRY system does not deliver any signal to notify the swimmer that his or her instant speed raised above any high speed threshold.

In one example, the disclosed NATATOMETRY system can be configured to deliver in real time:

1) a high-pitch (900 Hz) audio signal to notify the swimmer that he or she has reached a pre-set high threshold of instantaneous speed during the previous measurement cycle, or

2) a low-pitch (400 Hz) audio signal to notify the swimmer that he or she has failed to avoid a pre-set low threshold of instantaneous speed during the previous measurement cycle, or

3) the absence of a high-pitch or a low-pitch signal can indicate to the swimmer that his or her intra-cycle speed stays within the range of the pre-set high and low speed thresholds during the previous measurement cycle.

FIG. 1 shows a simplified schematic of one example of a NATATOMETRY system 10 according to the invention. The disclosed NATATOMETRY system 10 generally can be defined as having three sub-systems including a Machine Vision sub-system 12 for data collection, a Data Processing sub-system 14 for signal generation, and a Signal Delivery sub-system 16.

The Machine Vision sub-system 12 is used to measure the location of the swimmer in a current measurement cycle and to communicate this location to the Data Processing sub-system 14. The Machine Vision sub-system 12 in one example includes a target marker 18 such as a light emitting marker belt, worn by a swimmer S, around the body near the hip joint level. The Machine Vision sub-system 12 can also include a detector to detect the target marker 18. In one example, the detector is in the form of one or more video cameras 20. The NATATOMETRY system 10 can also include a controller 22, which can be a part of the Machine Vision sub-system 12 or part of the Data Processing sub-system 14. The controller 22 can be configured to provide camera synchronization, receive video signals from the cameras, and perform image processing in real time, to determine the swimmer's position, based on (in one example) Binary Large Object (BLOB) detection. A special controller, a desktop computer, a laptop computer, a tablet computer, a smart phone device, or the like can serve as the controller for controlling the Machine Vision sub-system 12 and can include controller hardware for the purpose of the disclosed NATATOMETRY system.

The Data Processing sub-system 14 is used to calculate the instantaneous speed values, based on a comparison of the locations of the swimmer in a current and a previous measurement cycle. The Data Processing sub-system 14 can be implemented on the controller 22 as in the example of FIG. 1, at least in part on the detector 18 of the of Machine Vision sub-system 12 itself, or on a separate computer (not shown). The instantaneous speed value for a giving measurement cycle is compared, in one example, to the pre-set thresholds. An appropriate signal, such as an audio alarm signal, can then be generated if the high or low speed the thresholds are reached. In one example, a high-pitch signal can be generated if the instantaneous speed is greater than the pre-set high speed threshold. A low-pitch alarm signal can be generated if the instantaneous speed is less than the pre-set low speed threshold. No signal can be generated if the calculated current instantaneous speed is within the range between high and low speed thresholds. If a signal is generated during the current inspection cycle, the Data Processing sub-system can then pass a high-pitch or low-pitch signal on to the Signal Delivery sub-system 16.

The Signal Delivery sub-system 16 can include an out of water, on-deck transmitter and a waterproof receiver worn by a swimmer. The sub-system 16 can also include a receiver worn by a coach or an on-deck speaker configured for the coach to hear. The Signal Delivery sub-system 16 can, in one example, convert the received high-pitch or low-pitch signal into an FM radio signal and transmit the radio signal through the airwaves. An FM radio receiver can be worn by a swimmer and be configured to receive the FM signal and deliver it to a bone conduction ear-piece, which can be mounted on or near the swimmer's ear. The coach can use, for example, a wearable receiver, which can be identical to the swimmer's receiver, or can carry or stand near an on-deck speaker. The Signal Delivery sub-system 16 can utilize a separate FM radio frequency for each of multiple swimmers to provide real time feedback for more than one swimmer training in the water at the same time. The controller 22 and Data Processing sub-system 14 can be configured to accommodate multiple swimmers by detecting multiple target markers 18 and/or receiving data collected by the detector or detectors. Each swimmer can have their own dedicated detector or the controller 22 can be configured to distinguish among the different target markers 18 of multiple swimmers detected by the same detector.

Machine Vision Sub-System Details

Target marker—In the disclosed example, the swimmer wears a target marker 18 device near their hip joints. The target marker includes a waist belt with Visual Identification Projectors Beacons (see http://www.nitevis.com/VIPIR_Legacy_detail.htm) that are attached to the waist belt. Multiple beacons (nine in one example) are used to compensate for a swimmer's body rotation along the lengthwise axis in direction of swimming. In another example, a low voltage waterproof Omni Directional LED light Bulb can be used as a target marker. The light from the target marker 18 forms a group of adjacent pixels in the image created by the cameras 20. This group of pixels is identified by the controller as a BLOB. The location of the swimmer in each image corresponds to the location of the center of the BLOB. Displacement of the BLOB from the edge of the inspection region corresponds to the location of the swimmer at the moment when the camera's electronic shutter is opened.

Video Camera—In this example, the Machine Vision system includes a video camera 20 (in one example an IR camera by SONY, Model XCEI50), or multiple such cameras, placed along the bottom of a pool or along the underwater windows located along the bottom of the pool. Optical characteristics of the water in the swimming pool and pool lighting conditions can play an important role in camera selection for the NATATOMETRY system. Water serves as an optical filter, which filters out most of the light energy emitted in the red and infra-red segments of the optical spectrum present in typical pool lighting and deck heating conditions. Infra-red energy emitted by deck heaters is effectively filtered out by water. For above-the-water surface camera solutions, green and blue optical filters are likely required.

Cable—In this example, a cable (not shown) or cables can electrically connect the video cameras 20 and the controller 22. In one example, the cable or cables can be a CA-CN17 cable or a CV-5000/XG cable Series. The cables are used to communicate and exchange control and video signals between the controller 22 and the one or more video cameras 20. A separate cable can be used to connect the controller with each of the cameras.

Machine Vision Controller—In this example, the functions of the controller 22 can be implemented as an algorithm on a programmable processor such as a portable computer, a smart phone, a tablet computer, a desktop or laptop personal computer, for storing and sending electronic signals and data. In one example, the controller is a XG-7000 series Vision System (XG-7002A—Flexible Image Processing System Analog camera controller by Keyence Corporation www.keyence.com). In this example, image collection, BLOB detection with pre-processing filters, and data transmission can take from 35 to 45 milliseconds depending on the complexities of the real images. The controller 22 can be configured to trigger a new measurement cycle once every 50 milliseconds.

The controller 22 can use an internal clock to synchronize and control multiple cameras 20. The controller 22 can adjust the camera's level of sensitivity, control opening and closing of the electronic shutters of the cameras 20, and the like. The controller 22 receives the video signals from the cameras 20 and can process the video signals.

The controller 22 can use, for example, two pre-processing filters to minimize false alarms in BLOB detection. The first of the two pre-processing filters can convert brightness of the pixels from grey scale into binary (black or white) category, using an adjustable threshold. The black/white threshold can be raised when the external background light is high. The binary threshold can be lowered when the level of external light is low. This first filter may not be needed if there is no external light in the pool. The second of the two pre-processing filters can convert a group of adjacent pixels of the same brightness into one object. This can help to minimize random fluctuations caused by the shape of the light signal projected onto the camera 20 by the target marker 18. The pre-processing filters may be needed to lower the rate of false alarms caused by small random particles in the water and by light reflections from the wavy surface of the water.

The controller 22 detects the BLOB in the inspection area. The inspection area is configured to correspond to the area between lane line dividers. In this manner, each of the cameras 20 can independently process the location of the targets 18 in multiple swimming lanes. The location of the target marker 18 is initially determined in pixels from the edge of the camera's Field of View (FOV).

At the end of a measurement cycle the controller transmits strings of values to the Data Processing sub-system 14. In one instance, the transmission can be configured to use a conventional Ethernet E5-I cable, which connect the controller 22 with a laptop computer used to host the Data Processing sub-system 14. One string can be transmitted per BLOB.

The string of values can consist of comma separated fields including: the number of the current inspection or measurement cycle, the number of the current camera 20, the number of the inspection area, the location of the BLOB on the X-axis, and the like. The location measurement is converted from pixels into linear values (yards for example) by using an arc tangent function within the Data Processing sub-system 14, which can function as follows:

L=D*tg(α),

where

• L—is the linear position of the swimmer in meters,
• D—is the distance from camera to the swimmer,
• α—is the angular displacement of the swimmer from the center of the FOV, and
• tg (α)—is the tangent of the angular displacement of the swimmer in the FOV of the camera.

Data Processing Sub-System Details

The Data Processing sub-system 14 generates the alarm signals of appropriate tone and delivers the signals to the Signal Delivery sub-system 16. Details of the processing algorithm in one example are found in the attached listing of the script written in Perl programming language.

#!/usr/bin/perl -w
use Audio::Beep;
use Math::Interpolate qw(linear_interpolate);
my $FALSE = 0;
my $TRUE = 1;
# my $DEBUG = $FALSE;
my $DEBUG = $TRUE;
my $BEEPED = 0; # FALSE
my $pevious_x = undef;
my $Delay_OK = 0; #counter of the delayed OK lines
my $Delay_NG = 0; #counter of the delayed NG lines
my $T_OK = 10;      # threshold of the delayed OK lines before starting a new swim
my $T_NG = 10;      # threshold of the delayed NG lines before starting a new idle
my $SwimInProgress = $FALSE;
my $IdleInProgress = $TRUE;
my $Swim_number = 0;
my @Delay_OK = ( ); # stack for the delayed OK lines.
my @Delay_NG = ( ); # stack for the delayed NG lines.
my $OK = 0; #couner of the “OK” lines in the current idle series.
my $NG = 0; #couner of the “NG” lines in the current idle series.
my $last_measured_x = undef; # last non-interpolated, but actully measured x
my $last_OK_counter = undef; # last counter with non-interpolated, but actully measured x
while (<>){
                    my $line = $_;
                    chomp $line;
                    $DEBUG && print “$line\n”;
                    my @inspection_data = split (/,/, $line); # split data by comma from the input line
                    my $x = $inspection_data[0];
                    my $counter = $inspection_data[2];
                    my $judgement = $inspection_data[5];
                    if ($judgement =~ /\+0$/){# judgement is “OK”: There is a BLOB if Swim is in Progress - process
this $line right away
                    if ($SwimInProgress){# Swim In Progress IS TRUE
                    foreach my $ng (@Delay_NG){
                    my @inspections = ($last_OK_counter, $counter);
                    my @positions = ($last_measured_x, $x);
                    $DEBUG && print “INTERPOLATE line $ng from \@Delay_NG
with non-measured_x\n”;
                    $DEBUG && print “LIPL x1 = $last_OK_counter y1 =
$last_measured_x, x2 = $counter y2 = $x\n”;
                    @delayed_ng = split (/,/, $ng); # split data by comma from the input
line
                    $delayed_counter = $delayed_ng[2];
                    my $delayed_x = linear_interpolate($delayed_counter, \@inspections,
\@positions);
                    $DEBUG && print “LIPL $delayed_x for $delayed_counter, from
@inspections, @positions\n”;
                    if (defined ($previous_x)){
                    $delta = $delayed_x − $previous_x;
                    printf SWIMLOG “%d\t%6.1f\t%6.2f\t*\n”,
$delayed_counter, $delayed_x, $delta;
                    printf   “%d\t%6.1f\t%6.2f\t*\n”, $delayed_counter,
$delayed_x, $delta;
                    }else{
                    printf SWIMLOG “%d\t%6.1f\t*\n”, $delayed_counter,
$delayed_x;
                    printf   “%d\t%6.1f\t*\n”, $delayed_counter, $delayed_x;
                    }
                    $previous_x = $delayed_x;
                    }
                    @Delay_NG = ( );
                    $DEBUG && print “Swim Is in Progress processing $line, increment \$OK\n”;
                    $delta = $x − $previous_x;
                    # printf SWIMLOG “%d\t%6.1f\t%6.2f\n”, $counter, $x, $delta;
                    # printf “%d\t%6.1f\t%6.2f\n”, $counter, $x, $delta;
                    printf SWIMLOG “%d\t%6.1f\t%6.2f\t”, $counter, $x, $delta;
                    printf   “%d\t%6.1f\t%6.2f\t”, $counter, $x, $delta;
                    if ($BEEPED == 0){
                    print “beep was not sent yet \$BEEPED == $BEEPED\n”;
                    if (abs($delta) >= 5){
                    # beep (559.255, 100);
                    beep (859.255, 250);
                    print “BEEP SLOW”;
                    print SWIMLOG “BEEP SLOW”;
                    # print “vvvvvvvv $swimlog SLOW vvvvvvvv\n”;
                    $BEEPED = 5;
                    }
                    }else{
                    print “beep should be quiet for $BEEPED inspections\n”;
                    $BEEPED−−;
                    }
                    printf SWIMLOG “\n”;
                    printf   “\n”;
                    $previous_x = $x;
                    $OK++;
                    $last_measured_x = $x;
                    $last_OK_counter = $counter;
                    }else{# Swim In Progress IS NOT TRUE
                    $DEBUG && print “Swim Is NOT in Progress yet\n”;
                    $DEBUG && print “push $line\nfor later processing\n”;
                    push @Delay_OK, $line;
                    if (@Delay_OK < $T_OK){ # Delay_OK is small we have to wait before
starting a new swim
                    $DEBUG && print “waiting to start a swim\n”;
                    $Delay_OK++;
                    }else{# $Delay_OK >= $T_OK we have enough OKs we need to start a new
swim
                    $Swim_number++;
                    print “*************\nstarting a new swim $Swim_number\n”;
                    $swimlog = “swim_” . $Swim_number . “.log”;
                    print “NEW swim log\t$swimlog\n”;
                    $BEEPED = $FALSE; # same as 0
                    open (SWIMLOG, “>$swimlog”) or die “can't open $swimlog for
appending: $!”;
                    printf SWIMLOG “Number\tPixels\tDelta \tINTPLD\n”;
                    printf   “Number\tPixels\tDelta \tINTPLD\n”;
                    $DEBUG && print “process lines from @Delay_OK stack\n”;
                    foreach my $ok (@Delay_OK){
                    $DEBUG && print “$ok\n”;
                    my @inspection_data = split (/,/, $ok); # split data by comma
from the input line
                    my $x = $inspection_data[0];
                    my $counter = $inspection_data[2];
                    if (defined ($previous_x)){
                    $delta = $x − $previous_x;
                    printf SWIMLOG “%d\t%6.1f\t%6.2f\n”, $counter,
$x, $delta;
                    printf   “%d\t%6.1f\t%6.2f\n”, $counter, $x,
$delta;
                    }else{
                    printf SWIMLOG “%d\t%6.1f\n”, $counter, $x;
                    printf   “%d\t%6.1f\n”, $counter, $x;
                    }
                    $previous_x = $x;
                    }
                    $last_measured_x = $x;
                    $last_OK_counter = $counter;
                    @Delay_OK = ( );
                    $SwimInProgress = $TRUE;
                    $IdleInProgress = $FALSE;
                    }
                    }
                    }elsif(($judgement =~ /\+1$/)){# NG No BLOB
                    if ($IdleInProgress){# Idle series is in progress
                    # print “since No Swim is in Progress and this new line is NG, clear accidental
OKs from Delay_OK list @Delay_OK\n”;
                    @Delay_OK = ( );
                    # print “Idle Is in Progress skipping this line, increment \$NG\n”;
                    $NG++;
                    }else{
                    $DEBUG && print “Idle Is NOT in Progress yet\n”;
                    $DEBUG && print “push $line\nfor later processing\n”;
                    push @Delay_NG, $line;
                    if (@Delay_NG < $T_NG){# Delay_NG is small We have to wait before
starting new idle print “Wait for before terminating the swim\n”;
                    $DEBUG && print “waiting to start a idle\n”;
                    $Delay_NG++;
                    }else{# $Delay_NG >= $T_NG. We have to terminate this swim and start a
new idle
                    $Idle_number++;
                    $DEBUG && print “*************\nstarting a new idle
$Idle_number\n”;
                    print “closing log $swimlog\n”;
                    close SWIMLOG;
                    $previous_x = undef;
                    $DEBUG && print “process lines from @Delay_NG stack\n”;
                    @Delay_NG = ( );
                    $IdleInProgress = $TRUE;
                    $SwimInProgress = $FALSE;
                    }
                    }
                    }else{
                    $DEBUG && print “$line\n$judgement is neither +0 nor +1\n”;
                    }

The foregoing Data Processing sub-system 14, and the processing program or software, can be implemented as part of the controller, such as if the controller 22 is a computer or a device with a data processor. The Data Processing sub-system 14 can also be implemented separately on a different device or within a component of the Machine Vision 12 sub-system that has a data processor.

A computer program product can be devised for operating the NATATOMETRY system 10 and for processing the date from the Machine Vision sub-system 12. The product can include one or more computer-readable storage media having stored thereon computer-executable instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the methods and functions of the NATATOMETRY system 10 disclosed and described herein

Signal Delivery Sub-System Details

The Signal Delivery sub-system 16 receives the signals from the Data Processing sub-system 14 and uses the on-deck FM transmitter and wearable waterproof FM receiver to deliver alarm signals to the swimmer and, optionally, the coach. A separate FM channel can be used for each swimming lane. One FM channel can be allocated to a swimmer in each swimming lane. In one example, the Signal Delivery sub-system 16 can be implemented as a Sennheiser EW 300 IEM G3, in-ear monitoring system with transmitter and receiver tuned onto one of the FM channels available in the Ultra High Frequency UF spectrum. In one example, the FM receiver can be placed in a water proof case (Sporti Water proof MP3 Player Case for example, see https://www.swimoutlet.com/ProductDetails.asp?ProductCode=32813&CartID). In one example, the FM receiver in the water proof case can be attached to the waist belt or the target marker 18 belt worn by the swimmer. In one example, bone conduction head phones (Audio Bone 1.0 for example, see http://www.audioboneheadphones.com/) can be worn by the swimmer as the ear piece to provide comfortable volume of audio alarm.

Novelty of the Disclosed NATATOMETRY System

Novelty of the disclosed NATATOMETRY system as compared to U.S. Pat. No. 5,767,417 lies at least in the use of a Machine Vision sub-system 12 to collect real time data about a swimmer's location. Machine vision is utilized in the disclosed NATATOMETRY system as a new and useful improvement over that disclosed in the '417 patent due to its higher spatial and temporal resolution.

Novelty of the disclosed NATATOMETRY system as compared to US Published Application 2010/0030482 lies at least in the use of a Machine Vision sub-system 12 to collect real time data about a swimmer's location. A machine vision system does not have the measurement drift of an accelerometer-based system as disclosed in the '482 publication.

Novelty of the disclosed NATATOMETRY system as compared to the real time heart rate monitors (stroke counters, distance per stroke monitors) lies at least in that the feedback is given to the swimmer based on an analysis of instantaneous swimming speed, not heart rate (stroke counters, distance per stroke monitors).

Novelty of the disclosed NATATOMETRY system as compared to the AVIDA Swim Training System lies at least in the duration of the real time measurement cycle. The duration of AVIDA Swim Training System measurement cycle is one lap of swimming, which takes 9 seconds or more at a world record pace. That prior art system averages speed per lap and does not show fluctuations of swimming speed occurring during or within any particular lap. The disclosed NATATOMETRY system is capable of providing duration of speed measurement cycles of 100 milliseconds or less and is capable of providing feedback instantaneously to the swimmer during individual stroke cycles.

Novelty of the disclosed NATATOMETRY system as compared to known methods of video analysis in swimming lies at least in that the feedback signals are delivered to the swimmer and/or coach in real time as defined herein.

In one example, the disclosed NATATOMETRY system can provide a swimmer with real time inter-cycle or inter-stroke feedback regarding their swimming speed. In one example, the disclosed NATATOMETRY system can provide an audio signal to the swimmer during a swim that is relevant to their speed. In one example, the disclosed NATATOMETRY system utilizes a machine vision device that can detect the location of a swimmer over a length of a pool. In one example, the disclosed NATATOMETRY system includes a plurality of visible lights worn by the swimmer that are detected by a machine vision device while in the pool. In one example, the disclosed NATATOMETRY system can detect and measure a swimmer's speed many times during a single stroke cycle and provide real time feedback of same to the swimmer. These and other objects, features, and advantages of the present invention will become apparent to those having ordinary skill in the art upon reading this disclosure.

The Machine Vision sub-system can be programmable to accommodate for different water depths and can be designed to be configured on site. The Machine Vision sub-system can also be configured to accommodate for the number of pixels in a frame. One pixel at the center of the unit will represent a different distance than one pixel at the edge of the field of view. Thus, as the swimmer swims across the field of view of the unit, the unit must be configured to compensate. A swimmer may move 70 pixels for one data point and 75 pixels for the next date point, but be swimming at the same speed. The unit can be configurable to accommodate.

The NATATOMETRY system can be configured to automatically and instantaneously download all data points to the Machine Vision sub-system controller and to the Data Processing sub-system. Alternatively or in addition, the controller and/or the Data Processing sub-system can store the data for later analysis. The goal of the disclosed NATATOMETRY system is to produce real time, accurate speed data and deliver useful information to the swimmer in the midst of a training session relating to such real time data.

FIG. 2 illustrates one representative example of a plot of average speed vs. actual real time speed is shown. As can be seen, an audible signal is delivered to the swimmer each time their velocity or speed is either too high or too low compared to the high and low threshold speeds.

The natatometrical data collected by the disclosed system are sequential and obtained in constant time intervals. Depending on the training objectives, various signals can be derived from raw natatometrical data. Any number of results and outcomes can be derived from the disclosed NATATOMETRY system and delivered to the swimmer and/or coach. These can be monitored for accuracy using any known quality control methods, such as, for example, checking a) a threshold of minimal instant speed per period of time, b) a threshold of maximum instant speed per period of time, c) a running range of instant speed per period of time, or d) a running standard deviation of instant speed per period of time, or performing a regression analysis, i.e., slope of the linear regression and/or performing an autocorrelation analysis to determine stroke cycle periods.

In another example, the Machine Vision sub-system FM signals disclosed above can be replaced using radio frequency identification (RFID). RFID receivers can be placed along the lane ropes. The swimmer can wear or carry an RFID tag or mark. As they swim by the RFID receivers, the receivers can provide information about their current location for the particular moment in time. Then the system can derive the speed and communicate it to the swimmer like in the earlier described NATATOMETRY system.

Although certain real time swim training systems and methods have been described herein in accordance with the teachings of the present disclosure, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all embodiments of the teachings of the disclosure that fairly fall within the scope of permissible equivalents.

Claims

1. A swim training system comprising:

a signal delivery sub-system configured to deliver signals to a swimmer;

a machine vision sub-system having a detector and a target marker to be worn by the swimmer, the detector configured to detect the target marker;

a controller configured to operably control the machine vision sub-system; and

a data processing sub-system configured to process data collected by the machine vision sub-system and to generate signals for the signal delivery system,

wherein, during a swim training session of the swimmer, the detector tracks a location of the target marker and transmits speed data to the data processing sub-system, wherein the data processing sub-system processes the speed data and generates signals that are transmitted to the swimmer via the signal delivery sub-system in real time during the swim session, the signals representative of a real time speed characteristic of the swimmer.

2. The swim training system according to claim 1, wherein the target marker is a light belt, which carries one or more lights visible to the machine vision sub-system.

3. The swim training system according to claim 2, wherein the one or more lights include a plurality of omni-directional LED lights.

4. The swim training system according to claim 1, wherein the signal delivery sub-system includes an in-ear audio transmitter worn by the swimmer.

5. The swim training system according to claim 1, wherein the signal delivery sub-system includes a remote FM transmitter and an FM receiver worn by the swimmer and tunable to a desired FM channel.

6. The swim training system according to claim 1, wherein the data processing sub-system includes an electronic device with a data processor.

7. The swim training system according to claim 1, wherein the detector of the machine vision sub-system includes one or more video cameras.

8. The swim training system according to claim 7, wherein the one or more video cameras are configured for underwater use and to detect the target marker under water.

9. The swim training system according to claim 1, wherein the real time speed characteristic includes a desired instantaneous swim speed range between a low threshold speed and a high threshold speed.

10. The swim training system according to claim 9, wherein the signal delivery sub-system can deliver a first signal when an instantaneous swim speed of the swimmer is slower than the low threshold speed, a second signal when the instantaneous swim speed of the swimmer is faster than the high threshold speed, and a third signal when the instantaneous swim speed of the swimmer is within the desired instantaneous swim speed range.

11. The swim training system according to claim 10, wherein the first signal is a first auditory signal, the second signal is a second auditory signal different from the first auditory signal, and the third signal is the absence of an auditory signal.

12. A swim training method comprising:

providing a signal delivery sub-system, a data processing sub-system, and a machine vision sub-system having a detector and a target marker;

attaching the target marker to a swimmer;

placing the detector of the machine vision sub-system so as to detect the target marker on the swimmer;

operating a controller to control the machine vision sub-system;

tracking the target marker via the detector as the swimmer moves during a swim session;

sending data from the machine vision sub-system to the data processing sub-system during the swim session;

processing the data at the data processing sub-system; and

transmitting signals to the swimmer via the signal delivery sub-system in real time during the swim session, the signals representative of a real time speed characteristic of the swimmer.

13. The swim training method according to claim 12, wherein the step of providing includes providing one or more cameras that can detect the target marker.

14. The swim training method according to claim 12, wherein the step of providing includes providing a light belt that carries one or more lights detectable by the detector of the machine vision sub-system.

15. The swim training method according to claim 12, wherein the step of placing includes placing the detector under water.

16. The swim training method according to claim 12, wherein the step of operating a controller includes configuring a programmable electronic device to operate the machine vision sub-system.

17. The swim training method according to claim 12, wherein the step of transmitting includes transmitting an audible signal to the swimmer.

18. The swim training method according to claim 12, wherein the step of transmitting includes transmitting two or more different audible signals to the swimmer via an ear piece of the signal delivery sub-system.

19. The swim training method according to claim 12, wherein the step of processing includes comparing a desired instantaneous swim speed range between a low threshold speed and a high threshold speed to an instantaneous swim speed of the swimmer during the swim session.

20. The swim training method according to claim 19, wherein the step of transmitting includes transmitting a first signal when an instantaneous swim speed of the swimmer is slower than the low threshold speed, a second signal when the instantaneous swim speed of the swimmer is faster than the high threshold speed, and a third signal when the instantaneous swim speed of the swimmer is within the desired instantaneous swim speed range.