SWING POSITION RECOGNITION AND REINFORCEMENT
A method and device is used to teach and simulate swing mechanics. A camera is coupled to a computer such that frames including an image of a golfer swinging a golf club may be captured. A key part of the image is identified by the computer in sequential frames captured from the camera, and the image is compared to swing mechanics of a known standard golfer. The position of the known standard golfer is capable of following the natural swing of a student golfer, and the method and device are capable of providing real-time feedback to the student golfer during the golfer's swing. An instant replay may be used to reinforce good swing mechanics. A marker attachment may be used to assist in tracking and analyzing the swing mechanics of the images. Parameters relating to club angle and apparent club length may be compared to a known standard for a plurality of reference swing positions in order to determine the swing mechanics of the student golfer during practice instruction and within a golf simulation game to determine ball trajectory. A ball launch monitor may be integrated with the image analysis, which improves the effectiveness of the method and device synergistically.
Latest Recognition Insight, LLC Patents:
CROSS REFERENCE TO RELATED APPLICATIONS
This application is a continuation-in-part application of U.S. appl. Ser. No. 11/198,570, filed Aug. 4, 2005, which is a continuation-in-part application of U.S. appl. Ser. No. 10/345,651, filed Jan. 17, 2003, now abandoned in favor of this application, which is incorporated herein by reference in its entirety, which is a continuation-in-part application of U.S. appl. Ser. No. 09/878,447 filed on Jun. 11, 2001, now issued as U.S. Pat. No. 6,533,675.
FIELD OF THE INVENTION
The field relates to golf instruction and golf simulators.
BACKGROUND AND DESCRIPTION OF PRIOR ART
A golf swing requires the synchronized actions of more than 80 different muscles in a way that is not particularly intuitive or consistent with the human framework. Focusing on the totality of these various muscle movements surpasses the ability of the conscious mind. This requirement for complex and unnatural muscle movements is the principle reason why golf is considered to be a difficult game to master. Even if one is successful in correctly emulating the precise body movements of a given professional, it is unclear that the club movements that are actually produced are correct for a particular person given his or her physical differences.
Correct movement of the golf club is the main objective of a golf swing. If the club moves correctly then learning the associated body movements becomes a far easier goal. There are as many swing styles as there are body sizes and shapes, even among professionals, but the objective of each is to move the club in a very similar way.
Training tools exist that focus on body movements during the golf swing, but none describes or teaches correct club movement in terms of geometric position of the golf club, club head rotation and three dimensional shaft direction at every point of the golf swing. And none performs computer image recognition of the golf club and body position to guide the club movements, analyze the swing and predict the ball flight.
U.S. Pat. No. 6,159,016 compares video recordings of the body motions of a student to that of an instructor or professional. But these recordings lack interactivity because it is only after the student has completed the exercise that he or she can view comparisons. The computers involved receive and display images but do not recognize or analyze club or body movements. That task is left to the student and/or his or her instructor.
U.S. Pat. No. 6,126,449 allows a student to swing a club within a body template image of a chosen professional instructor. The computer receives and displays images but does not recognize the student's actual club or body movements, leaving the entire burden of swing analysis to the student and his or her instructor. Because the golf swing analyzer has no awareness of what the student is actually doing once the template is set in motion, there can be no responsive interaction between the student and the device. No effort is made by the above cited patent to systematically describe correct body position as a function of correct golf club movement. Since the motion template of U.S. Pat. No. 6,126,449 progresses through the swing unconditionally in a constant predetermined motion without the benefit of position recognition, the student has no opportunity to initiate motion with position reinforcement being provided by an intelligent process. This invention can neither analyze a golfer's swing nor predict his or her ball flight, nor display a cutout of the golfer's image as he swings on a golf course background. Instead, the golfer merely follows the silhouette of a professional golfer, imitating the movements of the silhouette, to “interactively emulate . . . an instructor performing a selected motion,” as recited in the Abstract of the issued patent.
U.S. Pat. No. 6,126,449 also requires the student to synchronize his or her activities to the timing provided by the pre-recorded template. The student cannot test or exercise his or her club movements swinging freely back and forth using variable timing as he or she can with a system of swing position recognition and reinforcement.
In the above noted patents, choosing a particular template is an uncertain task, because the body type and swing type must be matched to the golfer. Then, the golfer must attempt to emulate the complex body movements executed by a professional, which can be very difficult, frustrating for beginning and advanced golfers, alike, and perhaps of no particular advantage in achieving the club and body movements that are correct for a particular golfer. Furthermore, physical differences between the student and the professional such as differences in anatomy, physiology, flexibility, and strength.
U.S. Pat. No. 6,059,668 teaches the effects of club movement by allowing the student to observe a light that shines in both directions along the club shaft generated by a device attached to the golf club shaft. This method does not help the student to know if the geometric location of the club is correct at any point. This method fails to detect very common problems, such as that of the golfer bending his forward arm during the back swing producing a swing whose arc is too narrow yet able to satisfy criteria indicating that the swing is correct.
With nothing other than a light beam to communicate to the student, the above cited patent lacks the effectiveness and accuracy of image analysis and video presentation. It suffers from not being able to review the analysis of a given swing many times at different speeds. This invention becomes the least effective during the forward swing before impact while the club is making its most rapid movement. During this time the student must interpret the light signals and switch his or her view from the light pointing toward the grip end of the club to that pointing toward the head of the club. This invention is unusable outdoors under bright sunlight and physical adjustments are necessary to the platform that reflects the beam of light when using clubs of different lengths.
U.S. Pat. No. 5,772,449 discloses a method for simulating a golfer's swing by a two step process that first collects data about a golfer's club and body movements and then feeds the data to a commercially available mechanical simulation package. In the second step, an android computer model attempts to reproduce the golfer's swing for the primary purpose of determining what kinds of clubs are best suited to that golfer. This method is not interactive with the golfer and does not teach or include a pattern of an ideal golf swing as is explained below.
Prior inventions have used video recordings of the golf swings of various professional golfers for comparison to a student as though they were magic formulas. In fact, many professionals owe their success more to extensive practice than to excellent techniques. Since golf tournaments can be seen on national television every weekend, most of the techniques that amateur golfers would like to learn have become common knowledge and are practiced by most professionals. What is needed is a systematic and effective way to learn and practice these well-known club and body movements. These techniques can be interactively taught by a software program that is capable of running on most personal computers equipped with an inexpensive video camera such as a USB web cam.
Thus a longstanding and unresolved need exists for an easy to use and inexpensive training system that allows a student to focus precisely on club position and club movement with real-time interactive assistance. A further need exists for analysis while practicing back and forth swings with no predetermined timing. The student should be able to spontaneously perform a sequence of swings without prior planning or setup. For a proficient and experienced golfer, this capability provides effective audio and video feedback during warm-up exercises.
Previous inventions do not have the benefit of the computer instantaneously recognizing, analyzing, understanding and responding to the motion of the student's golf club and reinforcing the club and body positions. While simulation based on analyzing the flight of the ball is effective in determining the ball flight, such simulation is ineffective in correcting a student golfer's swing mechanics. Even experience golf pros have great difficulty in determining and explaining the exact cause of a student golfer's poor swing mechanics. A video camera may be used to analyze the golf swing, but a student golfer has a hard time understanding what a correct swing should look like. Known swing analyzers require the student golfer to pace his or her swing to a moving image of a golf pro. This artificial and unnatural pacing of the student golfer causes subtle changes in the swing mechanics, which are not repeated when the student golfer swings at his or her own pace. Thus, there is a longstanding need for an image analysis system that follows the student golfer rather than the student golfer following the pace of the swing analyzer.
SUMMARY AND OBJECTS OF THE INVENTION
A golf swing training method and apparatus analyzes the image of a student golfer and matches certain parameters of the swing to determine the swing mechanics of the student golfer. The parameters may be used to provide reinforcing feedback, such as visual and audible feedback to the student golfer, as the student golfer is practicing the student golfer's swing mechanics at the student golfer's own pace. The parameters may be used to accurately predict the flight path of the ball by determining club head speed, direction and rotation during the swing and at impact with the ball. In one example, the parameters are coupled with a simple launch monitor to determine the flight path of the ball, which allows a small, inexpensive camera to be attached to the computer directly.
In another example, a short club may be used in a limited area with one or more simulated golf balls during instruction. The device is capable of teaching good swing mechanics for a club of any length. In one example, swing position recognition is accomplished by an attachment of one or more colored spheres just below the grip on the club shaft. Swing position reinforcement based upon the recognized step-by-step progress of the golf swing verifies correct body and club position during a computer-provided lesson. Swing position recognition also-allows the golfer to watch his or her cutout images swinging at various locations on a background golf course during a simulated golf game. When the computer has recognized and analyzed the nature of a particular golf swing problem and its solution, then the presentation to the student is more effective using computer generated graphics and audio methods. Since the computer's recognition software can correlate the club position with the position of the student's body parts, a more focused and systematic approach can be taken by the student to learn to swing correctly. Receiving a quantitative rating based upon the quality of the swing movements and viewing the predicted ball flight makes learning and practicing more enjoyable, taking the mystery out of the analysis of golf swing mechanics.
When the tracking process is applied to a simulated golf game, the golfer can watch his or her own image swinging on the simulated golf course and then observe the ball in flight and as it lands. He or she can then view a replay or an in depth step-by-step analysis of the previous swing before proceeding. In one example, the flight path of the golf ball is predicted based on the swing mechanics and/or a launch monitor and is compared to the flight path of a virtual pro.
In one example, the parameters measure a student golfer's swing mechanics against a semi-circular back swing plane orbit that is followed by a circular forward swing plane orbit. The locations within these orbits serve as reference points for the computer to recognize the correct club movements and body positions that comprise the ideal golf swing. These swing plane orbits can be further differentiated to describe swing planes with correct three dimensional angles that correspond to the effective club lengths of a short, medium or long club.
The derivation of these swing plane orbits is based upon analysis of a database of swing mechanics of many professional golfers. The database allows the correct swing to be superimposed on the student golfer's swing at the student golfer's own pace. Since the student golfer's swing mechanics are recognized in real time, the virtual pro follows the student golfer rather than the other way around. This greatly improves the effectiveness of the analysis, because the student golfer takes a normal swing along with all of the normal mistakes.
In a synchronized, slow motion process that revealed a pattern of an ideal swing mechanics common to substantially all successful professional golfers, a pattern developed. The pattern showed a semi-circular swing plane orbit for the golfer's hands during the back swing and a separate circular swing plane orbit for the golfer's hands during the forward swing. These swing plane orbits were easily extendable to an area of the club shaft near the golfer's hands. The choice of the location of the club shaft near the golfer's hands as a reference point was also valuable to keep the swing plane orbits as small as possible so the video camera's recognition capabilities are optimized by having the golfer as close to the camera as possible.
The swing plane orbits may be automatically determined while the golfer assumes his or her normal stance by considering the height and other physical characteristics of the golfer. In one example, the image analysis system instructs the student golfer to position the camera according to these same metrics, eliminating the need for the student golfer to measure the distances with a tape measure or other measuring device. For example, a golfer may position himself at a location for practicing swing mechanics, and a camera may be positioned to fit a shadow profile to the golfer or verbal instructions may be given to position the golfer in relation to the camera or a velocity detector may be positioned that the golfer will approach.
The proper golf swing mechanics may be further reinforced by several calibrating movements. The swing plane orbital locations form a basis for evaluating club shaft direction, clubface rotation and overall body position during a golf swing. They form the basis for swing position recognition and reinforcement that displays a humanoid pattern that represents the ideal club and body positions.
In one example, a marker attachment is placed on the shaft below the golf club grip, such as an orb or a set of orbs. Preferably, the material and/or the external coating on the orbs is selected to have an anti-reflective surface, such that substantially no glare is perceived by the camera from the orb or orbs. In one example, the orbs are identifiable by different colors. The image analysis algorithms may distinguish one orb from the other or one portion of an orb from another portion of the orb by the difference in color. Furthermore, blurring of the color may be used by the algorithms to analyze the golf swing mechanics.
The attachment may be tracked in real time by a software program that receives video camera images. The use of the attachment provide more information with a low resolution camera and lower frame rate than is possible with the same equipment without the attachment. For instance, a two-colored orb having one hemisphere in one color and the other hemisphere in another color may be used to determine club rotation. This allows the analyzer to determine whether the club face is open or closed at impact with the ball, which is an important parameter that is not measured by simple launch monitors or expensive infrared golf ball flight monitors, which cannot accurately determine ball rotation. The software program continuously monitors the position of the spherical attachment to determine if the club is in a correct swing plane orbital location and compares the student golfer's swing mechanics to a virtual pro. Any recognizable patterns on the attachment may be used to analyze the rotation of the club shaft that corresponds to the position of the clubface at a given swing plane orbit location and the direction of the club shaft. If camera resolution and frame rate are adequate, the system is capable of monitoring the rotation of the club head and/or the hands of the golfer. In this case, an attachment is not required, but image analysis techniques are used to recognize the outline of the club head or the hands and are compared with the appearance as the head and/or hands are rotated.
In one preferred embodiment, four distinct spheres are used to achieve quality recognition when slow camera speed causes the swinging club image to streak. In the downline view that is often used by actual professional trainers, the position of the solid-color balls is used to determine the club shaft's two-dimensional angle. A ball that has two dissimilarly colored hemispheres is divided in the same direction as the club shaft and is viewed to determine club's face rotation. Another ball may have two dissimilarly colored hemispheres divided perpendicularly to the club shaft. This ball, the measured club length, and attachment location determine the club shaft's three-dimensional angle for adherence to the ideal swing plane as the camera sees the golfer along the downline view.
Although the downline view is commonly used by golf pros, it is difficult to see the body movements and swing arcs from the downline view. In one example, a student golfer may position the camera for a face view of the student golfer. This usually feels more natural for the student golfer. In this case, the image analyzer is capable of determining the swing plane position merely by the location of key points of the arms and apparent club shaft length. This provides for a very rapid determination of swing mechanics errors and real time feedback to the student golfer even using very inexpensive equipment. A camera and image analyzer with a frame capture rate of less than 30 frames per second may be used to analyze such problems as a flat back swing and other swing mechanics. Indeed, the one-dimensional, apparent shaft length and position of the arms is able to accurately determine the two-dimensional swing plane and three-dimensional club position. This surprising and unexpected capability allows the system resources to be used to provide improved graphics and/or better determine rotation of the club in real time. Image analysis of apparent club length is easily determined even from grainy camera shots by measuring the length between the easily recognized club head and the golfer's hands. The position of the golfer's hands may also be used as a key point in determining the position of the swing. An attachment device, as discussed previously, further improves the capability of the system to analyze the swing mechanics with marginal hardware.
A conventional personal computer control interface such as a mouse or keyboard may be used to select the analysis activities, options and displays. In an alternative embodiment, the golfer controls the sequence of activities and choices during his computer lesson or golf game by moving the club attachment as a pointer onto a selection button. This feature permits the golfer to make various choices while remaining in his or her normal stance that may be about five feet from the camera and some distance from the display screen, keyboard and mouse. The student can use a separate monitor for each display or use multiple windows of a monitor capable of displaying combinations of views simultaneously that may show different views based upon the concurrent use of multiple video cameras.
An initial calibration may be used to adjust the camera position, lighting and position of the limited area platform for consistent ongoing positioning. During and after the calibration, a light meter feature is capable of telling the user whether the light level is dark, dim, normal or bright. The light meter reading is accomplished by examining the brightness and gray levels of known colors of the attachment following recognition, for example, without the need for a separate light meter attachment.
One display shows the student's image and swing plane orbits plus the correct club shaft direction and clubface rotation at various intervals along the swing plane orbital path. Another display shows a blow-up of the region of the spherical attachment to precisely view the student's actual club position at that moment and the relation to the display of the correct club position with respect to the swing plane orbital location, club shaft direction and club head rotation. Body movement errors are identified at each swing plane orbital location and illustrated suggestions for corrections are offered. Tempo of the swing is evaluated. A USGA handicap rating is assigned to the swing, the scale for which was determined statistically by a correlation of many golfer's actual handicaps versus their level of correctness of swing. Instant replay of the swing can be done at regular speed or in slow motion.
Another object and advantage is to display a computer generated humanoid image of the correct body position of the student relative to the current swing plane orbital location of the club attachment. This provides swing position reinforcement to help the golfer achieve correct club and body movement. This is an ideal learning situation since the student golfer takes the initiative rather than being led through each step and receives real-time reinforcement for his motion decisions.
Another analysis activity places the golfer on a golf course or practice range where the ball flight and distance are predicted based upon the real-time analysis of the swing leading to and following the ball impact position. During these activities, the golfer can see his ball in flight as it travels toward a simulated green. A replay or detailed analysis can be viewed subsequently for any swing. An embodiment of the golf course and practice range activities places a cutout camera image of the golfer onto a simulated golf course or practice range backgrounds. This cutout camera image corresponds to the current position of the club attachment with respect to the ideal swing pattern. As the current position proceeds near the correct positions of the ideal swing pattern, the golfer can see himself or herself swing on the golf course or practice range. In this way, the golfer can combine simulated play and swing analysis.
BRIEF DESCRIPTION OF THE DRAWINGS
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION
A system for improving golf swing mechanics includes a camera, a computer coupled to the camera, and a program that analyzes the swing of a golfer. The program may provide immediate feedback to the golfer, such as visual and audio feedback. Audio feedback may be provided by the computer or a speaker attached to the computer, which provides tips for improving the golf swing mechanics of the golfer while the golfer practices his golf swing at the golfer's own pace. Visual feedback may be displayed on a display screen, monitor or any other viewing system, such as virtual reality headsets and the like. The viewing system is coupled with and controlled by a computer. For example, visual feedback shows an image or outline of a known standard, such as a golf pro or a synthesis taken from a plurality of golf pros. The image of the known standard may be shown in a position that corresponds to the current reference swing position of the student golfer during the student golfer's swing. The current reference swing position is identified by the system and tracked, and the image of the known standard is paced to follow the student golfer, who merely swings at the student golfer's own pace.
In one embodiment, the student golfer may select a specific club to use in practicing the golfer's swing. The system matches the student golfer with a known standard using the same club. In one drill, the student golfer selects a plurality of clubs and practical instruction is provided with each club in sequence. By overlaying the image of a known standard using the same club as the student golfer over the image of the student golfer, without forcing the student golfer to change his or her own pace, the student golfer receives continuous feedback on swing mechanics during a practice swing. Being able to swing at the golfer's own pace is a significant advantage, allowing the golfer to concentrate on swing mechanics and not pace. The overlayed image follows the golfer and immediately shows the golfer if the golfer's swing mechanics are diverging from the known standard. In one embodiment, the student golfer selects the known standard from a menu of known standards. For example, the student golfer may choose a well known pro, an actual instructor, a longest drive champion or a composite known standard based on the combined swings of a plurality of professional golf instructors.
In one example, a camera is disposed such that a face-on view is captured of the golfer and the club, as shown in
The apparent distance from the grip to the club head may be determined by the number of pixels between the key part of the club head and the key part where the grip intersects the club shaft, for example. This apparent distance, normalized by the length of the club shaft in pixels, has been found to be the same for most professional golfers at about the same reference swing positions. By apparent distance it is meant that the distance is measured from the image of the club. A club disposed at an angle toward or away from a camera appears shorter than the same club disposed perpendicular to a line drawn between the camera and the club shaft. Thus, the apparent distance between any two points on the club normalized to the actual known distance between the same two points provides a measure of the three-dimensional, out-of-plane angle of the club shaft. Thus, a deviation between a known standard apparent distance and a measured apparent distance at the same reference swing position may be used to immediately instruct the student golfer of a defect in the student golfer's swing mechanics. For example, the distance in pixels of the shaft in
The method of determining the correctness of a swing using a camera to capture a face-on view of the golfer has never been used previously. It has been found that the system is capable of determining the correctness of the swing plane used by the golfer merely by analyzing the length of the shaft. The actual length of the club shaft may be entered by the golfer or may be imaged directly by the camera. The length of the club shaft during the swing may be computed as a ratio of the actual shaft length, for example. By comparing this ratio during the swing to the ratio of a correctly executed swing, the program is capable of detecting improper swing mechanics. Then, the program may provide visual or audio cues or instruction to the golfer to correct the improper swing mechanics. In one example, the apparent distance between two points on the shaft is determined by counting the number of pixels between the center of mass of two key parts of the club, such as the intersection of the grip and the shaft, the club head or two markers attached to the club shaft at a known spacing.
Markers are not required for analyzing the golf swing mechanics using this method. Even comparatively poor resolution video images, as shown in
In another example, such as illustrated in
Tracking involves image recognition in each frame of the golfer's current swing reference position coordinates. The swing reference position coordinates may be determined from one or more identifiable key parts, as discussed previously, such as the hands, a glove, the intersection between the grip and the shaft or a marker of a marker attachment. On preferred key part is the point of the golf club where the grip intersects the shaft. This key part is a pivot point for wrist cock and wrist release; therefore, discerning its coordinates serves multiple purposes: identifying the reference swing position and as a key coordinate in evaluating the student golfer's swing mechanics.
Wrist cock occurs when the shaft of the club continues clockwise motion during backswing. Wrist release occurs during the forward swing just prior to the head of the club striking the ball. Proper wrist cock and wrist release are essential for achieving a powerful drive from the tee, for example. Metrics that determine the angle of the club of a student golfer compared to a known standard may be used to discern any problems with wrist cock and wrist release, if monitored throughout the golfer's swing.
The reference swing position coordinates are a predetermined sequential set of image points derived from the swing of a selected virtual professional golfer. In one embodiment the student golfer or an instructor chooses the virtual professional golfer. In another embodiment, the computer matches a virtual professional golfer having similar body size, approach to the ball and stance with actual golfer. In yet another embodiment, a default standard virtual professional golfer is used.
In one example, the current swing position is the ordinal number corresponding to the reference swing position coordinates nearest the current swing position coordinates. Successful recognition of swing position coordinates may be accomplished by the identification of key parts, such as parts of the golf club and parts of the actual golfer's body. For example, key golf club parts may include the grip, the shaft and/or the club head, and key body parts may include the hands, the arms, the torso, the legs and/or the head of the student golfer. For example, the coordinates of these key parts may be determined based on the center of mass of pixels identified as a portion of these key parts.
In one specific example, a special club shaft attachment, as illustrated in
Additional orbs 64, 68 on the marker attachment device 61 are bi-colored blue and red. It will be apparent from
In the examples of
Identification of key golf club and body parts may begin during the “take your stance” phase of a swing or drill. When the student addresses the ball, the generic key part descriptors are compared to each of the actual key part stance areas of each incoming image. These key part areas are a function of the golfer's height. If a sufficiently probable set of key part matches is determined in any image, the stance position coordinates are recognized. The actual key part descriptors are formulated and may be saved for the student golfer.
Following recognition of stance position coordinates, the student is commanded to “begin your swing”. Each incoming image is then scanned to locate the current swing position coordinates. The actual key part descriptors from the last swing position coordinates match are compared in a restricted range of the image. The range is limited by the elapsed time since the last match and the specific part of the swing. Less scanning range is necessary during the back swing than the forward swing, because the club travels faster during the forward swing, usually three times faster.
In one example, as soon as a sufficiently probable set of key part matches is determined in an image, the current swing position coordinates are determined based on reference coordinates nearest the intersection of the grip and the shaft of the student golfer's club.
Reference Swing Position Coordinates
In one example, a reference swing position is displayed as a translucent and outlined image of a selected virtual professional golfer. The tracking system determines the reference swing position coordinates and overlays the correct reference swing over the image of the student golfer. Thus, the student golfer is provided immediate visual feedback that compares the student golfer's swing mechanics to the correct reference swing mechanics of the virtual pro. The back swing arc is about 180-degrees and the forward arc is approximately 360-degrees. For example, if the individual reference swing positions are chosen at 18-degree radial intervals, then a total of 36 swing position coordinates, as shown in
In another example, a reference swing position is displayed as a golf course scene with the outline of a selected virtual professional golfer cut-out from the scene. This cut-out scene is shown in
For example, each set of reference swing position coordinates may be associated with the correct reference swing. Values of a reference professional golfer's wrist cock angle, swing arc width, three-dimensional club angle of the two-dimensional swing plane, and spine angle may be compared to the actual student golfer's swing mechanics. For example, swing arc width may be determined by counting the number of pixels between the center of mass of one key body part, such as the torso or head, and another key part, such as the glove, intersection between the grip and the shaft or first orb of the marker attachment.
Key Parts Identifiers
Beginning at the initial stance, key part identifiers are formed by probable matches in the image. These identifiers may be dynamically created during the swing following each match for current swing position coordinates. The current key part identifier is used as the basis for subsequent comparisons.
For clubs with no special attachments, key parts identifiers may contain image data and other information about the coordinates of each key part, the actual pixels of the key parts, density of the key parts, an RGB classification of the key parts, various edge properties of the key parts, size of the key parts and rotation of the key parts. In addition, factors that describe the way the key parts are changing with time may be analyzed and recorded during the swing.
Use of special colored markers may allow less sophisticated image processing to be used to analyze golf swing mechanic values than is required for images captured without the use of the markers. When a marker attachment is used, the information desired is largely the same, but the complexity in identifying the key part identifiers is reduced, because the markers, which may be spherically shaped orbs, are uniform and may have an anti-reflective surface. For example, spherically-shaped orbs do not have rotational differences or interior edges. Also, attached markers may have a color scheme and size that makes it easy to identify and distinguish the markers from each other and from the background. In one example, the markers are made of a lightweight foam material that adds little weight to the shaft and provides an anti-reflective surface.
Real-time analysis allows a translucent and outlined image of a virtual pro to be overlayed on the image of the actual, student golfer. In one example, each successful match during the golf swing causes the image of the virtual pro to be advanced to the current position of the student golfer. For example, a large number of images of virtual pro golfers may be pre-recorded, and one may be selected for overlay on the student golfer's image captured by the camera.
In one example,
Unlike previously known golf swing imaging systems, the student golfer need not do anything different from a normal swing. No effort or awareness by the student is necessary, and the student can proceed at his or her own timing. The student can pause, back track and proceed during a swing motion as the virtual professional overlays continue to follow the golfer's own movements. This facilitates analysis for fault-correcting interactive drills, which are not possible with known systems.
A comparison is made between the student's key parts and those of the virtual pro at the current swing position. In one embodiment, the deviation limits may be predetermined by the student's choice of skill levels. If the deviation of the student's wrist-cock angle, swing plane angle or spine angle exceeds the limit, a specific swing fault may be announced immediately. This provides the ultimate interactivity for swings and drills. The student golfer may repeatedly exercise the golf swing, making corrections until the golf swing mechanics of the student golfer match, within the deviation limits selected, the golf swing mechanics of the virtual pro. A key difference between previous systems and this example of the present invention is that the system is capable of determining the position of the golf club. A shadow tracking mode is capable of matching a shadow of a pro or a previous image of a golf student with a current image of the student. As the student swings, at the student's own pace, the shadow tracks the student's swing matching the shadow tracker in real time to the actual position of the student's golf club (i.e. rather than the student having to pace the swing to match the a silhouette programmed to proceed at the pace of the professional golfer). This allows the student to swing naturally, providing a real time assessment of the student's own swing. The ability to locate the club in three dimensions allows advanced analysis of the golfer's swing.
In another embodiment, a replay may follow immediately after completion of the swing. In one example, the display hesitates at each swing position where an error was announced, which allows the student golfer to visualize the difference between the golfer's own image and that of the overlay of the virtual pro.
Fix Swing Faults
The system may provide a virtual tutor for the student golfer. For example, during or after the student golfer watches a replay of the student golfer's swing mechanics and overlay of the virtual pro, the shortcomings of the student golfer's golf swing mechanics may be critiqued both visually and audibly. Then, the program may recommend a swing mechanics drill or drills to correct the shortcomings. In one example, the drill or drills are shown to the student golfer in sequence, and the student golfer is given an opportunity to practice the drills. A replay of the drill followed by a still-view comparison of the original fault compared to the best progress at the swing position corresponding to the fault may be displayed. The student golfer may repeat the golf swing mechanics analysis as many times as necessary to correct all of the shortcomings. Then, the student golfer may change the deviation settings to continue improvement until the student golfer closely matches the desired golf swing mechanics of the selected virtual pro.
Simulation of a Golf Course Setting
In another embodiment, the student's image is displayed within the boundaries of the virtual professional's outline, bordered by a scene such as a golf course scene. In one example, each successful tracking match during the golf swing causes the virtual pro's scene cut-out image of the virtual pro, as shown in
The virtual pro's scene cut-out overlay follows the student golfer's image captured by the camera, while the student golfer makes a swing at the golfer's own pace based on tracking of the student golfer's image, identification of key parts and advancing of the reference swing position coordinates.
A replay may follow immediately after completion of the swing. In one example, the display hesitates at each swing position where an error was announced, which allows the student golfer to visualize the difference between the golfer's own image and that of the scene cut-out overlay of the virtual pro.]
Matching the Height of the Student to the Virtual Pro
It is important for a student to match height-wise to any selected virtual pro, both for tracking accuracy and visual comparisons. Based upon the specified height of the student, the student steps backward to reduce the height of his or her image or forward to increase the height of his or her image. The image of the virtual pro is raised or lowered accordingly, as shown in
In another example, the student golfer is able to play a simulated course, including driving, woods, irons, chipping, hitting out of bunkers and putting. The image analyzer provides information that is not available from known launch detection devices. Specifically, the effectiveness of launch monitors for short putts is extremely poor, but the image analyzer predicts the path for short putts very accurately. Also, the spin that is put on the ball by the club face is capable of causing the ball to hook or slice. If the club face is “closed” on contact with the ball, then the ball will tend to hook (i.e. curve to the golfer's left). If the club face is “open” on contact with the ball, then the ball will tend to slice (i.e. curve to the golfer's right). This curve depends on the spin put on the ball by rotation of the shaft during the golfer's swing and is not accurately measured by known launch detection devices, such as Q-Motions launch detection device. Launch detection devices are capable of measuring the velocity of the club head and direction of the club head on contact with the ball. This provides an initial speed and direction to the golf ball's trajectory. The spin on the ball, which may cause the ball to curve during its flight in one direction or the other, is independent of this speed and direction. The spin is important for determining the actual course of the trajectory, as any weekend golfer knows very well. Indeed, most professional golfers strike the ball in a direction that is a little right of their intended target and rely on a slight hook to curve the ball back to the target line.
Also, if the club head precedes the grip of the club, then the ball will tend to take a trajectory that is higher than normal for the club selected. Ordinarily, the grip of the club slightly precedes the club head to a line extending upwards through the ball. However, if the grip precedes the club head by too great of a distance, then the trajectory of the ball will be flat compared to the trajectory that is normal for the club selected. Known launch detection devices are not capable of determining the position of the grip at the time of impact; therefore, the trajectory is based only on the velocity and direction of the club head and the angle of the club selected.
In one embodiment, the image analysis system is used to detect the rotation of the club head and/or the position of the grip compared to the club head, such as at the time of impact with the ball. From the rotation of the club head, the rotation imparted on the ball may be accurately determined. From the position of the grip compared to the head, the divergence of the trajectory from the norm for the club selected may be determined. This improves the accuracy of the prediction of the actual flight of a golf ball and provides better feedback to the golfer. A golfer is capable of working on the golfer's hook, slice and grip location in a more realistic setting than during drills.
In one embodiment, the image analysis system also determines the velocity and direction of the golf head at impact with the ball. In this embodiment, a faster frame rate is desirable in order to have more information about the location of the key points during the highest velocity portion of the golf swing. For example, 30 frames per second may be adequate for estimating the golf club head velocity and direction at impact with the ball. More preferably, a higher frame capture rate is chosen to increase the number of reference swing positions. In one example, the image of the student golfer is shown against the setting of the golf course. In another example, the image of the student golfer is also overlayed with a virtual pro image. In yet another example, the student golfer may select to use a virtual pro's image as a sprite in the virtual play of the game.
For example, the student golfer's image may be displayed within the outline of a virtual pro within an imaginary or actual golf course setting, which may be selected by the student golfer. In one example, each successful tracking match during the golf swing causes the virtual pro's scene cut-out image of the virtual pro, as shown in
In one example, the flight of the student golfer's ball is compared to the trajectory of the virtual pro's ball. This can clearly show the affect of improper club rotation. For example, the trajectory of the virtual pro's ball may be shown as a colored arc, dashed arc or otherwise highlighted trajectory, compared to the flight of the student golfer's ball, In this example, the system may provide audible or visual tips to the student golfer before, during and after the swing based on the historical record maintained on the student golfer's golf swing mechanics. For example, the system may instruct the golfer to change the golfer's stance, to modify a grip, to avoid flat back or any other tip that might be provided by a live golf instructor.
One advantage of the image analysis system is that it captures video images of the student golfer that may be replayed, immediately after completion of the swing in order to provide a more detailed critique of the swing and the resulting impact on the trajectory of the golf ball. In one example, the display hesitates at each swing position where an error was announced, which allows the student golfer to visualize the difference between the student golfer's own image and that of the virtual pro.
An image recording device, such as a video camera or other charge coupled device, may be used to record the images. It is preferred to have a camera capturing at least 30 frames per second. A full speed golf swing requires about 1.5 seconds of continuous recording for an ordinary golfer; therefore, 30 frames per second is capable of yielding about 45 images during a golf swing. However, the rate of the swing increases as the club head approaches an impact with the ball. In this case, the number of images captured at 30 frames per second that are near the impact with the ball are limited in number by the speed of the club. Also, if the frame rate or shutter speed of the camera is too slow, then blurring of the images of the club may make accurate estimates of club angles difficult. A marker attachment allows for analysis of the key points even at impact with the ball using a camera that captures 30 frames per second.
An image should capture a full view of the student golfer and the club through the entire arc of the swing. In one embodiment, a wide-angle lens provides a field of view greater than 75 degrees, allowing the full view with a student golfer standing within 6 feet of the camera. Preferably the lighting is about 80 lux or brighter, as measured with a light meter, if a marker attachment is used. More preferably, the lighting is about 120 lux or brighter, especially if no marker attachment is being used. The camera system should include light balancing that eliminates glare spots.
Preferably, the camera contains at least 640×480 pixels to allow for key part identification and measurements. For example, if the camera uses a charge coupled device (CCD), then the device may have a dimension of ⅓″. Higher resolution cameras are preferred for image analysis without a marker attachment.
It is preferred to have system memory that is capable of storing all of the captured frames concurrently without saving to disk. For example, if 36 positions are to be captured and overlayed, then it is preferred to have adequate system memory to operate the program, capture the 36 images of the golfer and overlay the 36 images of the virtual pro. More memory is needed if the number of the images, resolution of the images or size of the images is increased. It is within the skill of a normal artisan to determine the amount of memory needed to operate the system described and claimed.
Preferably, a display device is viewable from the position of the student golfer to provide visual feedback. A speaker or audio device that is capable of being heard by the student golfer may be coupled with the computer to provide audio feedback.
Preferably, the system processor and/or graphic co-processor is capable of real-time image capture and processing. A graphics processor may couple the computer to the display such that the student golfer is capable of viewing the golfer's swing in real time during the swing.
In one embodiment, the student may select a specific club, and the system matches the known standard with the length of the club. In another embodiment, the student golfer uses the club as pointing to device to request the use of a specific club. Any club may be played, including putter, wedges, irons, woods and drivers. A database for known standards using different clubs is used to analyze the swing mechanics at the reference position coordintes previously described. In an alternative embodiment, the “club” may be a short club that does not reach the ground. In this embodiment, the marker attachment may be attached to the club or may be integrated with the club. For example, a short club according to this embodiment is shown in
In one embodiment, a light calibration is performed to improve lighting conditions based on the brightness and gray properties of known color patterns of the marker attachment, for example.
Other steps may be included in the setup and calibration of the system for a particular users preferences. For example the golfer may select a specific golf course to play, a specific voice and appearance of an instructor and many other customizable features of the system that have now become apparent to a practitioner in the field based on the description and the examples provided.
Many other embodiments of the system may be included, and the system may be used for swing dynamics in many other supports, such as tennis, racquetball, squash, baseball, and any other sport where control and analysis of swing dynamics is an important aspect of the game. In baseball, the position of the bat during a swing may be analyzed to correct errors in swing mechanics in a manner completely analogous to a golf swing. In tennis, the camera may preferably be aligned down the path of the ball in one example. In each of these examples, a system according to the present invention is capable of detecting and analyzing swing mechanics, and the system may be used to compare the student's swing mechanics with proper swing mechanics.
1. A device for analyzing swing mechanics of a student, the device comprising:
- a computer; and
- a camera coupled to the computer such that the computer is capable of capturing video frames from the camera of an image of the student and at least a portion of a marker attachment, a racquet, a bat or a club, such that when the student swings at the student's own pace, the computer matches the student's pace with an image or shadow of a known standard, the computer determines a position from at least one key part of the image, and the computer provides feedback to the student based on a divergence from the known standard of the position from the at least one key part of the image of the student.
2. The device of claim 1, wherein the student is a golfer and the camera is positioned face on to the golfer, and the computer measures a parameter related to a length of a club and compares the measured parameter related to the length to a known standard length for at least one position of the club.
3. The device of claim 2, further comprising a marker attachment having a first marker or a first marker section and a second marker or a second marker section spaced from the first marker or the first marker section, wherein the computer determines the length of the marker attachment by determining the distance between the first marker or the first marker section and the second marker or the second marker section.
4. The device of claim 3, wherein the distance is measured in pixels.
5. The device of claim 3, wherein the computer compares the length of the marker attachment to the known standard length for a plurality of club positions.
6. The device of claim 5, wherein the plurality of club positions is at least thirty club positions per second.
7. The device of claim 5, wherein the plurality of club positions is at least sixty club positions per second.
8. The device of claim 3, wherein the marker attachment has a plurality of markers and at least one of the markers comprises an orb, and the orb has at least two colors such that the computer is capable of determining rotation of the marker attachment from the changing fraction of the at least two colors that are visible in the image.
9. The device of claim 1, further comprising a marker attachment, wherein the marker attachment has a bi-colored marker or a bi-colored section such that rotation of the marker attachment may be detected and used to determine divergence from a known standard of rotation for proper swing mechanics.
10. The device of claim 9, wherein the divergence is used to calculate the effect on a trajectory of a ball in a golf simulator.
11. The device of claim 10, further comprising a ball launch monitor, wherein the launch monitor is capable of determining the speed and direction of a ball after impact with a head of the club, and the computer determines the trajectory of ball in a golf simulator based on the speed and direction of the ball and the rotation of the shaft of the club.
12. The device of claim 10, wherein the computer is capable of providing an instant replay that instructs the golfer how to correct improper golf swing mechanics to improve the trajectory of the golf ball.
13. The device of claim 9, wherein the marker attachment has a plurality of markers or marker sections and at least one of the markers or marker sections comprises a tube with a first color on one longitudinal side of the tube and a second color on the opposite longitudinal side of the tube, such that the computer is capable of determining rotation of the marker attachment from the changing fraction of the first color and the second color that are visible in the image.
14. The device of claim 13, wherein the rotation of the marker attachment is used to determine divergence from a known standard.
15. The device of claim 1, wherein the camera is positioned downline of a golfer, and the computer measures a parameter related to an angle of a shaft of the club and compares the measured parameter related to the angle to a known standard angle for at least one position of the club.
16. The device of claim 15, wherein the at least one key part includes the point where a grip of the club intersects the shaft of the club.
17. The device of claim 16, wherein the point where the grip intersects the shaft is marked by a first marker.
18. The device of claim 17, wherein the first marker is attached at one end of a spacer member having a second marker attached at an opposite end.
19. The device of claim 18, wherein the marker attachment has a plurality of markers and at least one of the markers comprises an orb, and the orb has at least two colors such that the computer is capable of determining rotation of the shaft from the changing fraction of the at least two colors that are visible in the image during rotation of the shaft of the club.
20. The device of claim 19, wherein the rotation of the shaft is used to determine divergence from rotation of a known standard for rotation of the shaft of the club.
21. The device of claim 20, wherein the divergence is used to calculate the effect on a trajectory of the ball in a golf simulator.
22. The device of claim 16, wherein a rotation of the shaft of the club is determined from the image of a head of the club.
23. The device of claim 22, further comprising a ball launch monitor, wherein the launch monitor is capable of determining the speed and direction of the ball after impact with a head of the club, and the computer determines the trajectory of the ball in a golf simulator based on the speed and direction of the ball and the rotation of the shaft of the club.
24. An interactive method of tracking and analyzing golf swing mechanics of a golfer using a golf club, the method comprising:
- viewing the golfer with a video camera;
- coupling the video camera to a computer such that the computer is capable of capturing images from the video camera;
- determining a key part from the images;
- tracking the position of the golf club during a swing using the key part;
- comparing the position of the golf swing to a reference position of a known standard swing;
- providing feedback to the golfer such that the golfer is capable of improving the swing mechanics.
25. The method of claim 24, wherein the step of providing feedback includes providing visual feedback or audible feedback or both visual feedback and audible feedback.
26. The method of claim 25, wherein the visual feedback includes:
- superimposing a pattern of a golfer having a known standard golf swing mechanics over the image of the golfer on a display screen based on the key position identified by the computer from the image.
27. The method of claim 26, wherein the audible feedback is capable of instructing the golfer about at least one improper aspect of swing mechanics in real time during a swing by the golfer.
28. The method of claim 27, wherein the audible feedback is capable of informing the golfer of a flat back swing based on image analysis of the image by the computer.
27. The method of claim 23, wherein the step of superimposing includes:
- advancing the image from one frame to the next in real time and superimposing the pattern of the golfer having a known standard golf swing mechanics to each of a plurality of sequential images of the golfer in each frame based on the key position identified by the computer from the image.
30. The method of claim 29, further comprising:
- attaching a marker attachment on a shaft of the club of the golfer.
31. The method of claim 30, further comprising:
- exhibiting a surface of the marker attachment having a color pattern thereon;
- analyzing the color pattern;
- comparing the color pattern of the marker of the golfer to the color pattern of a known standard golfer;
- determining divergence from the color pattern of the known standard golfer; and
- providing the golfer feedback concerning the divergence.
32. The method of claim 31, wherein the feedback is audible.
33. The method of claim 31, wherein the feedback is capable of informing the golfer of a slice or a hook.
34. The method of claim 33, wherein the feedback is a golf simulator capable of showing the trajectory of the golf ball of the golfer compared to the trajectory of the golf ball of the known standard golfer.
35. The method of claim 34, wherein the feedback includes an explanation of the defect in golf mechanics of the golfer and providing drills to practice correcting of the defect in golf mechanics.
36. The method of claim 35, wherein the feedback includes repeating the shot and showing a comparison of the original trajectory, the repeated trajectory and the known standard trajectory.
37. The method of claim 24, wherein the step of determining a key part from the images includes determining the coordinates of a first marker attached to a shaft of the club at the intersection of a grip and a shaft of the club.
38. The method of claim 37, wherein the step of tracking the position of the golf club includes determining the position of at least one additional marker spaced at known distance from the first marker, determining a linear fit from the first marker through the at least one additional marker, wherein an angle from a fixed direction from the first marker to the line of the linear fit is capable of being determined.
39. The method of claim 38, wherein the step of tracking the position of the golf club further comprises counting the number of pixels from the first marker to the at least one additional marker.
40. The method of claim 39, wherein the step of tracking the position of the golf club further comprises counting the number of pixels from a center of mass of the body to the position of the first marker.
41. The method of claim 40, wherein the step of comparing the position of the golf swing to a reference position of a known standard swing includes comparing at least one of the measurements made in the step of tracking the position to at least one related value measured from the known standard swing at the reference position associated with the actual position of the first marker of the golfer, the at least one of the measurements made in the step of tracking the position being selected from the group of measurements consisting of the number of pixels from a center of mass of the body to the position of the first marker, the number of pixels from the first marker to the at least one additional marker, the coordinates of the position of the center of mass of the golfer's head, and the angle from a fixed direction from the first marker to the line of the linear fit.
42. The method of claim 40, wherein the step of comparing the position of the golf swing to a reference position of a known standard swing includes comparing at least two of the measurements made in the step of tracking the position to at least two related values measured from the known standard swing at the reference position associated with the actual position of the first marker of the golfer.
43. The method of claim 24, wherein the step of viewing the golfer views the golfer from a plurality of cameras positioned for different views of the golfer.
44. The method of claim 24, further comprising a step of analyzing the brightness and gray properties of known color patterns of a marker attachment and reporting light conditions to the golfer.
45. The method of claim 24, further comprising a step of disposing the camera a proper distance from the golfer during setup by comparing the height of the golfer to the height of a known standard golfer.
46. The method of claim 24, further comprising using a marker attachment as a pointing device to control the computer.
47. The method of claim 46, wherein the step of using a marker attachment includes the step of mirroring the image of the golfer on the display.
48. The method of claim 24, further comprising:
- selecting a type of golf club;
- taking a practice swing with the type of golf club;
- viewing the practice swing in real-time during the practice swing;
- replaying the practice swing; and
- analyzing the practice swing step-by-step audibly commenting on the swing mechanics during each step having comments associated with the step.
49. The method of claim 24, further comprising:
- predicting the trajectory of a ball; and
- determining from the trajectory the lie of the ball in a simulated golf environment.
50. The method of claim 49, further comprising:
- using a launch monitor to determine ball speed and initial direction of the ball when using a putter for long putts, a wedge, an iron, a wood or a driver;
- attaching a marker having plurality of colors or patterns on the club;
- predicting the effect of ball rotation imparted by rotational position of the club at impact on the trajectory of the ball from image analysis of the marker having a plurality of colors using an image analyzer; and
- using the image analyzer to determine the ball speed and initial direction of the ball when using a putter for a short putt, when the image analyzer is capable of improving the accuracy of the simulation of the short putt.
51. The method of claim 50, further comprising a step of including a plurality of players in the golf simulation.
52. The method of claim 51, wherein the plurality of players are disposed in different locations and are playing over an electronic network.
53. A simulation system for a player, comprising:
- a device for practicing swing mechanics, the device comprising a computer, and a camera coupled to the computer such that the computer is capable of capturing video frames from the camera of a marker device on a shaft;
- the marker device having a plurality of colored sections, each having an anti-reflective surface, the plurality of colored sections including:
- a first section having a first color;
- a second section disposed below the first section and having a second color on a left side and a third color on a right side, opposite of the left side of the second section;
- a third section disposed below the second section, the third section having the third color on a left side and the second color on a right side, opposite of the left side, such that the left side of the third section and the right side of the second section are of the same color and the right side of the third section and the left side of the second section are of the same color, such that, when the player swings the marker device at the player's own pace, the system identifies swing mechanics information, including:
- determining coordinates of the marker device;
- fitting a linear fit from one of the first section or second section through another identifiable section of the marker device, providing an angle of the shaft;
- measuring the apparent distance between one of the first section, the second section or the third section and any other identifiable section of the marker device; and
- measuring a rotation of the shaft based on a count of the number of pixels of the second color and the third color; and
- the system compares the swing mechanics information to a known standard; and
- provides real-time visual and audible feedback to the player.
54. The system of claim 53, wherein the first section is a first orb, the second is a second orb and the third section is a third orb, wherein each of the first orb, the second orb and the third orb are spaced apart one from the other.
55. A marker attachment comprising a tube or a plurality of orbs, the tube or each of the plurality of orbs having an anti-reflective surface, the tube or plurality of orbs including:
- a first section or a first orb, respectively, having a uniform surface color;
- a second section or a second orb, respectively, of the same color as the first orb disposed at a distance from the first section or the first orb, respectively;
- a third section or a third orb, respectively, of the same color as the first orb disposed between the first section or the first orb and the second section or the second orb;
- a first bi-colored section or a first bi-colored orb of a second color and a third color; and
- a second bi-colored section or a second bi-colored orb of a second color and a third color;
- wherein the first bi-colored section or first bi-colored orb has the second color on a left half and the third color on a right half and the second bi-colored section or the section bi-colored orb has the second color on a right half and the third color on a left half.
56. The marker attachment of claim 55, wherein the first section or the first orb is a first section; the second section or the second orb is a second section; the third section or the third orb is a third section; the first bi-colored section or the first bi-colored orb is a first bi-colored section; and the second bi-colored section or the second bi-colored orb is a second bi-colored section.
International Classification: A63B 69/36 (20060101);