INERTIAL AND OPTICAL MOTION SENSING DEVICE INTEGRATED INTO SPORTS EQUIPMENT

Abstract

A motion sensing device with both inertial sensors and optical elements that integrates into a piece of sports equipment. Equipment with the device installed may look and feel like the original equipment. For example, for a baseball bat, a sensor package with a housing containing inertial sensors, cameras, and lights may be integrated into a knob that can be swapped with a standard bat knob; an additional sensor package may be integrated into a replacement bat cap. Sensor data and images from cameras in the device may be transmitted to a processor that calculates the equipment trajectory. External cameras may also capture images of lights in the device, which may flash or change in a pattern that is synchronized with the sensor data, and these images may also be used in trajectory calculations. Equipment with the motion sensing device may be used for virtual reality or for real sport activities.

Description

BACKGROUND OF THE INVENTION

Field of the Invention

One or more embodiments of the invention are related to the fields of motion sensing devices and sensor data analysis. More particularly, but not by way of limitation, one or more embodiments of the invention enable an inertial and optical motion sensing device integrated into sports equipment.

Description of the Related Art

Virtual reality headsets and controllers enable users to have virtual experiences such as games or to practice activities in a virtual environment. Virtual reality experiences may include simulations of sporting events or games, or sport practice or training sessions. A limitation of existing virtual reality systems is that generic virtual reality controllers (typically held in the hands) are used to simulate a piece of sports equipment, such as a baseball bat or a golf club. Although a virtual reality headset can display an image of the sports equipment, a controller does not feel at all like the equipment, since its weight, size, and shape do not match the real equipment. Some virtual reality applications have attempted to address this limitation by attaching a virtual reality controller to a piece of sports equipment; these attachments are typically clumsy and bulky, they change the feel of the equipment significantly, and they are not sufficiently rigid to provide good tracking of rapid motions of the sports equipment.

For at least the limitations described above there is a need for an inertial and optical motion sensing device integrated into sports equipment.

BRIEF SUMMARY OF THE INVENTION

One or more embodiments described in the specification are related to an inertial and optical motion sensing device integrated into sports equipment. Embodiments of the invention may include sensor packages that may be integrated into or attached to sports equipment to enable tracking of the equipment's motion.

In one or more embodiments of the invention, an inertial and optical motion sensing device may have one or more housings, each of which may be coupled to or integrated into a piece of sports equipment. It may have control electronics contained within at least one of the housings. The control electronics may include a microprocessor, a wireless communications interface coupled to the microprocessor, and a memory coupled to the microprocessor. It may have motion sensing electronics contained with at least one of the housings. The motion sensing electronics may include a three-axis accelerometer and a three-axis gyroscope, each coupled to the microprocessor by a wired or wireless communication link. It may have multiple optical elements contained within at least one of the housings. Each optical element may be either a light or a camera, and it may be proximal to a region of its corresponding housing that is open or is transparent to a wavelength associated with the optical element. Each optical element may be coupled to the microprocessor by a wired or wireless communication link. The microprocessor may be configured to capture sensor data from the three-axis accelerometer and the three-axis gyroscope during a motion of the piece of sports equipment. When the optical elements include one or more cameras, the microprocessor may obtain images from the cameras during this motion and add these images to the sensor data. When the optical elements include one or more lights, the microprocessor may command the lights to emit light patterns during the motion and to synchronize the light patterns with the sensor data. The microprocessor may transmit the sensor data (which may include images) over the wireless communications interface to a processor that is configured to analyze the sensor data to calculate a trajectory of the piece of sports equipment.

In one or more embodiments, each housing may be located at or near a corresponding end of the longitudinal axis of the piece of sports equipment.

In one or more embodiments, coupling or integrating the one or more housings into the piece of sports equipment may not change the length of the longitudinal axis by more than 10%, and may not change the maximum diameter of the piece of sports equipment around this longitudinal axis by more than 10%.

In one or more embodiments of the invention, the piece of sports equipment may be a bat, and a first housing of the one or more housings may be coupled to or integrated into a knob of the bat. In one or more embodiments, a second housing of the one or more housing may be coupled to or integrated into a cap of the bat at an opposite end of the longitudinal axis from the knob.

In one or more embodiments, the piece of sports equipment may be a golf club, and a first housing of the one or more housings may be coupled to or integrated into the grip of the golf club.

In one or more embodiments of the invention, the optical elements may include one or more lights, and the processor may be configured to obtain one or more scene images from one or more external cameras that view the piece of equipment during at least a portion of the motion of the equipment. The processor may analyze the scene images to determine the location of the lights in the scene images, synchronize these light locations with the sensor data, and analyze the light locations and the sensor data to calculate the trajectory of the piece of sports equipment.

In one or more embodiments, the processor and the external cameras may be integrated into a virtual reality headset. The virtual reality headset may be configured to display a representation of the piece of equipment on a display of the virtual reality headset, and to update this representation over time according to the trajectory of the equipment.

In one or more embodiments, the optical elements may include three or more lights located with at least one of the housings. In one or more embodiments, the optical elements may include three or more cameras located with at least one of the housings.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the invention will be more apparent from the following more particular description thereof, presented in conjunction with the following drawings wherein:

FIG. 1 shows an illustrative embodiment of the invention integrated into a baseball bat; motion sensing electronics are integrated into the bat knob and into the bat end cap.

FIG. 2 shows an illustrative embodiment of the invention integrated into a golf club; motion sensing electronics are integrated into the golf club grip.

FIG. 3 shows an illustrative baseball bat with a knob attachment and cap attachment that contain motion sensing electronics.

FIG. 4 shows the bat knob of FIG. 3, with an exploded view that shows cameras and LEDs behind an outer ring.

FIG. 5 shows an exploded view of the knob of FIG. 4 with the outer ring removed.

FIG. 6 shows an exploded view of the electronic components of the knob of FIG. 5.

FIG. 7 shows the bat end cap of FIG. 3, with an exploded view that shows the outer cap that attaches to the bat, and a contained inner ring with cameras and LEDs.

FIG. 8 illustrates calculation of a bat trajectory from sensor data captured by the electronics integrated into the bat knob and bat cap, and from images of the LEDs captured by external cameras.

DETAILED DESCRIPTION OF THE INVENTION

An inertial and optical motion sensing device integrated into sports equipment will now be described. In the following exemplary description, numerous specific details are set forth in order to provide a more thorough understanding of embodiments of the invention. It will be apparent, however, to an artisan of ordinary skill that the present invention may be practiced without incorporating all aspects of the specific details described herein. In other instances, specific features, quantities, or measurements well known to those of ordinary skill in the art have not been described in detail so as not to obscure the invention. Readers should note that although examples of the invention are set forth herein, the claims, and the full scope of any equivalents, are what define the metes and bounds of the invention.

One or more embodiments of the invention may include motion sensing electronics that may be integrated into or attached to a piece of sporting equipment. The motion sensing electronics may measure or enable measurement of any attribute of motion, such as position, orientation, speed, angular velocity, acceleration, or angular acceleration, either directly or indirectly. FIG. 1 shows an embodiment of the invention integrated into a baseball bat. Embodiments of the invention may be integrated into any type of equipment, used for sports activities or any other type of activity. Baseball bat 101 is a normal baseball bat without any electronics. This bat has a longitudinal axis 102 that extends from the bottom of the bat knob 104 to the top of the bat end cap 105. For many bats, the knob 104 and the end cap 105 are separate pieces that are attached to the main portion of the bat. These pieces may be removeable and replaceable. One or more embodiments of the invention may therefore use a bat knob and/or a bat cap that include motion sensing electronics; these components may be installed into a normal bat as replacements for the existing or standard bat knob and/or cap. In one or more embodiments the existing bat knob and/or cap may be retained, and electronic components may be integrated into or attached to the existing knob and/or cap.

In the example shown in FIG. 1, bat 101 is equipped with motion sensing electronics by replacing original bat end cap 105 with cap 115, and replacing original bat knob 104 with knob 114. In one or more embodiments only a portion of the knob or end cap may be replaced or altered. In one or more embodiments, either or both of the knob and the end cap may be replaced or altered. Either or both of the replaced (or altered) items may have a housing that contains electronic components that enable tracking of the motion of the bat. Illustrative components in bat knob 114 may include for example, without limitation, motion sensing electronics such as inertial motion sensors accelerometer 121 (which may have three axes in one or more embodiments) and gyroscope 122 (which may have three axes in one or more embodiments), optical elements such as camera or cameras 123 and light or lights 124. Some of the optical elements such as lights 124 may not track motion directly but may instead enable other elements such as external cameras to track the motion of the sports equipment. Knob 114 may also contain control electronics components such as a microprocessor 125 that is coupled to a memory 126 and to a wireless communications interface 127. Microprocessor 125 may also be coupled to the sensors 121 and 122, to cameras 123, and to lights 124, by wired or wireless communications links. The microprocess 125 may obtain data from the sensors 121, 122, and 123 during a motion of the bat, and it may transmit this sensor data over the wireless communications interface 127 to a processor that analyzes the data to determine the trajectory of the bat. Microprocessor 125 may also control the lights 124, and it may for example control the lights to emit light patterns during motion (such as flashes or changes in intensities or colors), so that the light patterns are synchronized with the sensor data samples. External cameras observing the lights may therefore by synchronized with the sensor data stream using the known timing of light patterns relative to the sensor data samples.

In one or more embodiments, cap 115 may also contain electronic components that may provide data that can be used to track the motion of bat 101. Illustrative components 131 through 137 may be identical to or similar to the components 121 through 127 contained within knob 114. In one or more embodiments a cap 115 may contain only a subset of these components. In embodiments with both a knob and a cap, the electronics in the knob and cap may communicate with one another, or they may act independently and may both transmit their own data to another processor for analysis. In one or more embodiments, only one of the knob and the cap may have a microprocessor and a communications interface, and the sensors in the other may be connected to this single microprocessor. This connection between the knob components and the cap components (when both exist) may be wireless or wired. For wired connections, in one or more embodiments one or more connections 140 may be routed for example through a hollow core of the bat. Other components may be placed in the hollow core of the bat, such as a power source (battery) 141, or vibration actuators that may be controlled to provide haptic feedback to a user.

The knob 114 and cap 115 may be configured to have similar size, shape, or appearance to the original knob 104 and cap 105. A benefit of one or more embodiments of the invention is that the bat 101 (or other equipment) with the installed motion sensing components looks and feels like a real bat, rather than looking and feeling like a separate virtual reality controller or a hybrid with a controller attached to a bat. The dimensions of the bat with the replaced components 114 and 115 may be identical to or similar to those of the bat 101 with original components 104 and 105. For example, the length 112 along the longitudinal axis of the bat with the additional motion sensing electronics may differ from the length of the original equipment along the longitudinal axis 102 by 10% or less. The maximum diameter 113 of the bat around (perpendicular to) the longitudinal axis with the additional motion sensing electronics may also differ from the maximum diameter 103 of the original equipment by 10% or less. Because the length 112 and maximum diameter 113 may be similar to (or identical to) the length and diameter of the original equipment, the user may largely not notice the added electronics, and may swing the bat with a very similar feel to that of the original bat. In one or more embodiments, added electronics and their housings may also be configured to have a minimal effect on other attributes of the bat, such as weight, center of mass, moment of inertia, or air resistance.

FIG. 1 illustrates an embodiment of the invention configured for a baseball bat. One or more embodiments of the invention may be configured to add motion sensing capabilities to any type of equipment, including for example, without limitation, any type of sports equipment such as a bat, club, racket, paddle, ball, or glove. One or more embodiments of the invention may add motion sensing electronics to equipment used for activities other than sports, such as medical or rehabilitation equipment. The motion sensing electronics may be integrated into or attached to the equipment in a manner that minimizes the effect on the appearance or feel of the equipment, so that a user of the equipment has an experience that does not change substantially when the motion sensing electronics is added. The electronics may be contained in one or more housings that are integrated into or attached to the equipment. Attachments may be relatively rigid, so that motion of the sensors in the housings is closely correlated with the motion of the equipment. Housings may also provide environment protection and shock isolation for the contained components.

FIG. 2 shows an illustrative embodiment of the invention integrated into a golf club 201. Like a bat, a golf club has a longitudinal axis 202, and it has a grip 204 at one end of this axis. Motion sensing electronics package 214 may be integrated into a replacement grip (or into a module that attaches to or integrates into an existing grip). This electronics package may have a housing that contains any or all of the components 221 through 227, similar to the components 121 through 127 of the baseball bat knob package of FIG. 1. The length 212 of the golf club with the grip motion sensing package 214 may be identical to or similar to the length of the original golf club 201 along its longitudinal axis 202. The maximum diameter 203 of the golf club around the longitudinal axis may not change, since the grip is small compared to the clubface. The user of the club therefore may notice very little or any difference when using the golf club with the motion sensing electronics package 214. In this illustrative example, the golf club has a single motion sensing package 214 at only one end of the longitudinal axis; in other embodiments there may be additional housings containing other components, such as another package at or near the clubhead, at the opposite end of the longitudinal axis from the motion sensing package 214.

FIGS. 3 through 7 show illustrative layouts for a baseball bat knob and end cap that contain motion sensing electronics. FIG. 3 shows bat 300 with the knob and cap electronics packages installed, and an exploded view with the bat body 301 separated from the knob package 314 and the cap package 315. The illustrative bat body 301 is for example a metal bat with a hollow core. The illustrative knob 314 slips over or into the bottom end 301b of the bat body; it may be attached using for example adhesive, a screw, or other fasteners. The illustrative cap 315 fits over or into the top end 301a of the bat body; it may be attached using for example adhesive, a screw, or other fasteners. Knob and cap components may attach to a bat body in any desired manner and may be secured to the body with any type of attachment.

FIG. 4 shows the bat knob package 314, with an exploded view that shows an outer ring 401 that covers electronics in the inner knob package 410. The inner knob package may for example contain a circular arrangement of optical elements such as cameras and/or lights, surrounded by the outer ring 401. The outer ring 401 may have holes or transparent regions near (in front of) the cameras or lights. For example, inner knob package 410 has three cameras 412a, 412b, and 412c (camera 412c is hidden from view in FIG. 4). Outer ring 401 has three corresponding holes 402a, 402b, and 402c through which the cameras can view the environment. Inner knob package 410 also has lights (such as LEDs) arrayed around the ring; embodiments may have any number of lights in any arrangement. For example, LEDs 413a, 413b, and 413c are visible in FIG. 4 (other LEDs are hidden from view). The lights may emit light of any wavelengths, including for example infrared. The portions of outer ring 401 in front of the lights may be open or may be transparent to the wavelengths emitted by the light. For example, region 403 of outer ring 401 may be made of a material through which light emitted by LEDs 413a and 413b can shine. Similarly cameras 412a, 412b, and 412c may capture images in any wavelengths, and the outer ring in front of these cameras may be open or may be transparent to these wavelengths.

In one or more embodiments, knob 314 may also have one or more connectors such as a USB connector 414. This connector may be used for battery charging, for transmission of data to or from the enclosed electronics, or both.

FIG. 5 shows an exploded view of inner knob package 410. A sleeve 501 may have an inner hole that slips onto (or into) the bottom end of the bat, and it may have one or more features onto or into which electronic components 511 may be mounted. FIG. 6 shows an exploded view of the electronic components 511 of the bat knob package. The electronics include a ring 602 that mounts on the outside of the sleeve 501, and inner components 601 that fit inside the sleeve 501. The inner components may include for example a battery 611 and a circuit board 612 onto which inertial sensors, the microprocessor, the memory, and the communications interface are mounted. The ring 602 may have a supporting structure 621 onto which are mounted various cameras and/or lights, of any number and in any geometrical arrangement. In this example the cameras 412a, 412b, and 412c are mounted approximately 120 degrees apart. Lights 413a through 413h are mounted at different locations around the ring 621. By mounting three or more lights around the ring, one or more lights may be visible to external cameras regardless of the orientation of the bat. One or more connections (not shown) may be made between the inner electronics 601 and the outer components 602, for example to supply power from the battery and to transmit data and commands to and from the microprocessor.

FIG. 7 shows an exploded view of an illustrative bat end cap package 315. In this illustrative embodiment, the end cap package contains only cameras and lights; it does not include control electronics such as a microprocessor, or inertial sensors such as an accelerometer or a gyroscope. In other embodiments, the end cap package may contain any additional components such as a microprocessor or inertial sensors. The package 315 has a cap shell 701, which fits around the end of the bat, and an enclosed inner ring 702 onto which cameras and/or lights are mounted. The configuration of the lights and cameras may be similar to that of the bat knob, although the number and arrangement of lights may be different so that views of the cap from external cameras can be distinguished from views of the knob. FIG. 7 shows illustrative cameras 712a, 712b, and 712c, and illustrative LEDs 713a, 713b, and 713c; other LEDs are hidden from view. The cap shell may have holes or transparent areas for the cameras and/or lights.

In one or more embodiments of the invention, data obtained from the motion sensors (such as accelerometer, gyroscope, and camera(s)) may be transmitted to an external processor for analysis. This processor may analyze the data to determine the trajectory of the piece of equipment in which the electronics package or packages are integrated or attached. This trajectory may for example include the position in three-dimensional space of any point of the equipment as a function of time. It may be of interest to calculate this trajectory for a movement of the equipment corresponding to a particular sports activity, such as the swing of a baseball bat or of a golf club. FIG. 8 illustrates this trajectory calculation for a swing of baseball bat 300 with attached bat knob 314 and bat end cap 315 that contain motion sensing electronics. Sensor data 801 is collected during the swing; this data may include acceleration values 802 and angular velocity values 803 at different points in time during the swing. Data may be sampled at any desired sampling rate, such as once per millisecond for example. This motion sensor data 801 may be transmitted to processor 810 for analysis, for example using the wireless communications interface in the knob and/or cap. The sensor data may also include images captured by one or more of the cameras in the knob and/or cap. These images may be sampled at any desired rates, and the sampling rate for images may not correspond to the sampling rate for inertial sensor data. Illustrative sensor data table 801 contains two images 804a and 804b captured at two different points in time from two different cameras. These images show views of the environment from the viewpoint of the bat knob or bat end cap. In one or more embodiments with lights in the knob and/or cap, additional tracking data may be available from external cameras oriented to view the scene of the equipment during its motion; these cameras may for example observe the lights of the knob and/or endcap during at least part of the equipment's motion. (For some motions, some of the lights may be occluded during some portions of the motion; however, the processor can use other sensor information during these periods of occlusion to calculate the trajectory throughout the motion.) Since patterns of lights such as flashing or intensity or color timing may be synchronized with the motion sensor samples, scene images of the lights may be correlated with the other sensor data to help determine the position and orientation of the bat at each point in time. Illustrative external camera 811a captures scene image 812a that shows the bat end cap with two active lights, and illustrative external camera 811b captures scene image 812b that shows the bat knob with one active light. The processor may analyze these images 812a and 812b to determine the locations of the lights in each image. These light locations may be synchronized with the sensor data. All of the data 801 (which includes inertial sensor data and camera images from cameras on the bat) and the synchronized light location data from external camera scene images 812a and 812b (possibly at different points in time) are transmitted to processor 810, which performs analyses 815 to calculate the equipment trajectory 816.

In one or more embodiments of the invention, equipment with one or more inertial and optical sensor packages may be used as virtual reality controllers, for example for games that simulate the sport that the equipment is used for. Processor 810 and external cameras 811a and 811b may be integrated into a virtual reality headset, for example, and the headset may display a representation of the equipment on the headset display, using the calculated trajectory 816 to update this equipment representation over time. The trajectory 816 may also be used to determine virtual reality actions and events, such as determining whether a swung bat hits a virtual ball in a virtual pitching and hitting game.

While the invention herein disclosed has been described by means of specific embodiments and applications thereof, numerous modifications and variations could be made thereto by those skilled in the art without departing from the scope of the invention set forth in the claims.

Claims

1. An inertial and optical motion sensing device integrated into a piece of sports equipment, comprising:

one or more housings, wherein each housing of said one or more housings is coupled to or integrated into a piece of sports equipment;

control electronics contained within at least one of said one or more housings, said control electronics comprising a microprocessor; a wireless communications interface coupled to said microprocessor; and a memory coupled to said microprocessor;

motion sensing electronics contained within at least one of said one or more housings, said motion sensing electronics comprising a three-axis accelerometer coupled to said microprocessor by a wired or wireless communication link; and a three-axis gyroscope coupled to said microprocessor by a wired or wireless communication link; and,

a plurality of optical elements contained within at least one of said one or more housings, wherein each optical element of said plurality of optical elements comprises a light or a camera; each optical element is located proximal to a region of its corresponding housing that is open or is transparent to a wavelength associated with said each optical element; and, each optical element is coupled to said microprocessor by a wired or wireless communication link;

wherein said microprocessor is configured to capture sensor data from said three-axis accelerometer and said three-axis gyroscope during a motion of said piece of sports equipment; when said plurality of optical elements comprises one or more cameras, obtain images from said one or more cameras during said motion of said piece of sports equipment and add said images to said sensor data; when said plurality of optical elements comprises one or more lights, command said one or more lights to emit light patterns during said motion of said piece of sports equipment and synchronize said light patterns with said sensor data; and, transmit said sensor data over said wireless communications interface to a processor configured to analyze said sensor data to calculate a trajectory of said piece of sports equipment.

2. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 1, wherein

each housing is located at or proximal to a corresponding end of a longitudinal axis of said piece of sports equipment.

3. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 2, wherein

coupling or integrating said one or more housings into said piece of sports equipment does not change a length of said longitudinal axis by more than 10%, and does not change a maximum diameter of said piece of sports equipment around said longitudinal axis by more than 10%.

4. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 3, wherein

said piece of sports equipment is a bat; and

a first housing of said one or more housings is coupled to or integrated into a knob of said bat.

5. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 4, wherein

said piece of sports equipment is a bat; and

a second housing of said one or more housings is coupled to or integrated into a cap of said bat at an opposite end of said longitudinal axis from said knob of said bat.

6. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 3, wherein

said piece of sports equipment is a golf club; and

a first housing of said one or more housings is coupled to or integrated into a grip of said golf club.

7. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 1, wherein

said plurality of optical elements comprises one or more lights; and,

said processor is further configured to obtain one or more scene images from one or more external cameras that view said piece of sports equipment during at least a portion of said motion of said piece of sports equipment; analyze said one or more scene images to determine locations of said one or more lights in said one or more scene images; and, synchronize said locations of said one or more lights with said sensor data; and, analyze said locations of said one or more lights and said sensor data to calculate said trajectory of said piece of sports equipment.

8. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 7, wherein

said processor and said one or more external cameras are integrated into a virtual reality headset; and,

said virtual reality headset is configured to display a representation of said piece of sports on a display of said virtual reality headset and to update said representation over time according to said trajectory of said piece of sports equipment.

9. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 1, wherein

said plurality of optical elements comprises three or more lights located within at least one of said one or more housings.

10. The inertial and optical motion sensing device integrated into a piece of sports equipment of claim 1, wherein

said plurality of optical elements comprises three or more cameras located within at least one of said one or more housings.