Apparatus and Method for Analyzing Sensor Data
An apparatus for detecting an event of interest, comprises a first primitive event detector (8b) for generating a stream of first primitive events (10b) using a sensor data stream (4), the sensor data stream (4) carrying raw data at a sensor data rate, the raw data being associated to one or more properties determined for multiple observed objects (72) using one or more sensors, wherein a data rate of the stream of first primitive events (10b) is lower than the sensor data rate as well as a second primitive event detector (8c) for generating a stream of different second primitive events (10c) using the sensor data stream (4), wherein a data rate of the stream of the second primitive events (10c) is lower than the sensor data rate. An event detector (16) is operable to determine the event of interest using the streams (10b, 10c) of the first and second primitive events.
Embodiments of the present invention generally relate to the analysis of sensor data and, in particular, to apparatuses and methods for analyzing sensor data being provided at a high sensor data rate.
BACKGROUNDWireless sensor networks of various technologies may be used for locating purposes, such as locating or tracking humans and/or other objects. Here, “locating” means the detection or determination of a geographical location or position, while tracking means locating at short time intervals such as to be able to follow the position and other kinematic variables of a moving observed object. Some specialized locating or position tracking systems may be used for locating players and other objects (e.g. a ball) in sport events, such as, for example, soccer, American football, rugby, tennis, etc.
When analyzing the gathered geographic location or positioning data of players and/or a ball it becomes possible to derive statistical information related to the whole sports event, for example a soccer match, or related to individual teams or players. Such derived statistical information may be interesting for various reasons. On the one hand, there are various commercial interests as certain statistics and their analysis may be of particular relevance for spectators in a stadium and/or in front of a television set at home. Hence, providing certain statistics may raise more interest in sport events. On the other hand, statistical data derived from the raw positioning data may as well be used for training purposes. Here, an opponent and/or the behavior of the own team may be analyzed as well as the performance and/or health condition of individual players.
The aforementioned locating or position tracking systems may be based on various technologies. For example, location information may be determined based on the evaluation of wireless radio signals and/or magnetic fields. For this purpose transmitters and/or receivers, generally also denoted as sensors, may be placed at the individual objects (e.g. players, ball, etc.) to be located by the system. Corresponding reception and/or transmission devices may also be mounted to predetermined locations around a geographical area of interest, as e.g. a soccer field. An evaluation of signal strengths, signal propagation times, and/or signal phases, just to name a few possible technical alternatives, may then lead to sensor data streams indicative of the geographic position or location of individual players or objects at different time instants. A location data sample may be associated with a time stamp indicating at which time an object was located at the observed or detected geographic position or location. As combined information, kinematic data, like velocity (speed), acceleration, etc. may as well be provided in addition to the location data comprising, for example, x-, y-, and z-coordinates. In the sequel of this specification the location and/or kinematic data delivered by the localization sensor system will also be referred to as raw (sensor) data.
In a particular example of a wireless tracking system people or objects may be equipped with tiny transmitters, which may be embedded in footwear, uniforms and balls and whose signals are picked up by a number of antennas placed around an area under observation as for example a soccer field. Receiver units process the collected signals and determine their Times of Arrival (ToA). Based on a calculation of the differences in propagation delay, each transmitter's position is determined continuously to track all objects under observation. In one particular example, the transmitters operate in the 2.4 GHz band to be globally license free, while the individual transmitters are distinguished using CDMA (Code Division Multiple Access) techniques.
However, tracking an object in a sports game with an appropriate accuracy easily produces a large amount of sensor data. For example, tracking a ball of a soccer game with an accuracy of roughly 3 cm may require to estimate the position of the ball up around 2000 times per second, assuming a maximum speed of the ball of about 150 km/h, which appears to be reasonable for soccer. However, other sports such as tennis or baseball certainly would require higher sampling or estimation rates. Moreover, not only the ball is to be tracked in a soccer game where tracking the 22 players is equally important. Since tracking with the above mentioned system is principally feasible with an accuracy of 1 cm, it may also be desirable to use more than one tracking sensor per player, to not only derive his position but also his posture in order to be able to decide on offside situations or the like. Equipping two teams and the necessary play equipment for a soccer match with a reasonable amount of sensors may result in up to 50000 position data samples or raw data samples per second that is the associated sensor data stream may have a sensor data rate of 50000 per second.
Analyzing such type of sensor data requires simultaneous knowledge on the raw data of multiple or all sensors at each decision making, since each incident to be analyzed or since each conclusion to be drawn requires knowledge on a different subset of the sensors raw data. That is, in a conventional analysis framework all data needs to be distributed to all analyzing entities in parallel, which is not feasible if the number of analyzing entities increases in order to analyze the data with respect to multiple aspects. To even accomplish this task in real time or close to real time is completely out of reach in those systems. However, this may be required in order to indicate offside positions in a soccer game or to answer similar time critical questions in the course of another sports game.
It is therefore required to provide a concept to analyze sensor data streams more efficiently in order to extract information of interest.
SUMMARYAccording to an embodiment of the invention, this is achieved by the use of an analysis framework for analyzing the raw data portions being transmitted in a sensor data stream at a sensor data rate. The analysis framework comprises an input layer comprising at least two or more primitive event detectors for generating at least a first and second stream of primitive events using exclusively the sensor data stream as an input and one or more event layers comprising one or more event detectors for detecting an event of interest, wherein any one of the one or more event detectors uses only the streams of the first and the second primitive events or a stream of other events as provided by another one of the event detectors as an input. That is, the analysis framework follows an event based hierarchical analysis approach, where interdependent events are determined or detected at different hierarchical levels of dependency.
An event may generally be defined to be an instantaneous occurrence of interest at a point of time. In general, an event is associated with a change of a related sensed quantity. An event may be a primitive event derived in the hierarchical base or input layer, such that a primitive event is directly based on sensor data (e.g. the kinematic data provided by a tracking system) and hence may indicate a change of the related sensed quantity. An event of interest is an event of a higher hierarchical order that is it is dependent on other events or primitive events, or more generally on previously detected other events. In another view, an event or a primitive event comprises information derived by applying a certain decision making criterion to another event or to the raw data.
An event based approach may provide for the possibility to analyze the underlying data in a parallelized manner with multiple processing nodes communicating over a network. This is particularly the case since the sensor data stream with its high data rate only needs to be provided to the primitive event detectors in the input layer, where they are processed to determine primitive events which are typically determined with a lower data rate. For the higher layers, primitive events are the basis for the analysis such that the sensor data stream does not have to be distributed to all decision making entities or event detectors. This makes the analysis of larger sets of position data at all possible.
Moreover, the reduced data rate of the input events for the event detectors in the higher hierarchy layers provides for the possibility to parallelize the processing, for example by distributing the event detectors and/or the primitive event detectors to different CPUs or computers within an analysis framework consisting of several CPUs or computing nodes. The reduced event data rates even provide for the possibility to communicate the occurrence of events via conventional networks, such as for example over an Ethernet based LAN (Local Area Network) without limiting the performance of the analysis framework even when a close to real time analysis is required. As compared to the raw data, only the fact that a decision making criterion of an event detector has been met (that is, that the event occurred) may be transmitted as an event information between interrelated event detectors. This may additionally help to keep the amount of data exchanged between event detectors low, since the length of each individual message is shorter than the length of a single raw data sample. That is, not only the number of data packets per time may be kept low but also the size of the transmitted data packets. In further embodiments, the exchanged event data may also comprise information about two, three or more possible states the event detector has determined. That is, a determined event or primitive event may contain more than a binary YES/NO information.
According to a further embodiment of the invention, the above object is achieved by an apparatus for detecting an event of interest, the apparatus comprising a first primitive event detector for generating a stream of first primitive events using the sensor data stream and a second primitive event detector for generating a stream of different second primitive events using the sensor data stream, wherein a data rate of each of the streams of the first primitive events and the second primitive events is lower than the sensor data rate. An event detector is operable to determine the event of interest using the streams of the first and second primitive events. An event detector can thus base its decisions on primitive events occurring at a lower data rate and thus perform its analysis without the need to access the high data rate sensor data stream directly.
With this respect the term data rate is to be understood not only as an instantaneous data rate but also as a mean data rate, which is as an average of the instantaneous data rate over a certain time interval. Depending on the chosen set of primitive events, it may occur that a certain primitive event is determined with the sensor data rate for a short time period. However, the mean data rate of the primitive event over the time of the observation, e.g. the time of a soccer match, may be much lower.
According to some embodiments of the invention, the primitive event detectors are operable to exclusively use kinematic data as the raw data in order to provide real time analysis of tracking data for sports events or the like. Kinematic data may be an arbitrary subset of position information, velocity information or acceleration information of an observed object. The data may be provided in a 1-, 2- or 3-dimensional space, such that the kinematic variables used herein shall be understood as being either vectors or scalars, depending on the chosen implementation. In further embodiments, the primitive event detectors may furthermore derive velocity- and/or acceleration information themselves from the provided position information, such that the sensors may be simple tracking or positioning devices or the like.
The hierarchical analysis is based on an appropriate choice of base events, such as to be able to derive any basic information needed for all higher order decisions.
According to one embodiment, a proximity-event is generated as a primitive event when position information of a first observed object and of a second observed object indicates that the second observed object is within a predetermined neighborhood of the first observed object. The neighborhood may, depending on the observed event or sports game, be a complex shaped volume or be defined by the distance between the first observed object and the second observed object. Proximity may be used as an input for numerous further event detections, as for example to answer the question whether a player/batter or other equipment was possibly hitting a ball or another observed object.
According to a further embodiment, a remote-event is generated as a primitive event when position information of a first observed object and of a second observed object indicate that the second observed object is outside of a predetermined neighborhood of the first observed object.
According to a further embodiment, a direction changed-event is generated as a primitive event when a change of a velocity (direction or quantity) or an acceleration of an observed object exceeds a predetermined threshold. This criterion may generally be useful in sports- or tracking applications. Depending on the observed sport, however, the height of the threshold may be different. The threshold may therefore be different for different kinds of sports or even varying during the course of a single game to account for changing environmental conditions, such as for example the occurrence of rain.
According to a further embodiment, an idle-event is generated as a primitive event, when the velocity of an observed object falls below a predetermined threshold to indicate that the object is passive or not moving. According to some embodiments, the idle event is only generated when the velocity of the observed object stays below the predetermined threshold for at least a predetermined minimum duration.
According to a further embodiment, an active-event is generated as a primitive event, when a velocity of an observed object exceeds a predetermined threshold. According to some embodiments, the active-event is only generated when the velocity of the observed object stays above the predetermined threshold for at least a predetermined minimum duration.
According to a further embodiment, a line crossing-event is generated as a primitive event, when at least one component of a position information of an observed object exceeds or falls below a predetermined threshold.
Some embodiments of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which
An input layer 6 of the framework 2 comprises several primitive event detectors 8a-8e for generating a stream of primitive events using exclusively the sensor data stream 4 as an input. That is, only the primitive event detectors 8a-8e of the input layer of the analysis framework 2 receive the raw data at the high sensor data rate as an input.
A first primitive event detector 8b and a second primitive event detector 8c generate a first stream of primitive events 10b and a second stream of primitive events 10c using exclusively the sensor data stream 4 as an input. The sensor data stream 4 may be provided using any suitable technique such as an Ethernet Network using copper or fiber links or any other wired or wireless technology, using standardized or custom communication protocols.
The analysis framework 2 comprises two event layers 12 and 14, each comprising event detectors for detecting events of interest, wherein the event detectors of the event layers 12 and 14 use only the streams provided by the primitive event detectors or of other event detectors of event layers 12 or 14 as an input. For example, event detector 16 determines an event of interest using the streams 10b and 10c of the first and second primitive event detectors 8b and 8c. Stream 10b is also provided to a further event detector 18. It goes without saying that further arbitrary interdependencies are possible. For the sake of clarity, however, only few dependencies or connections are illustrated here.
The links between the event detectors may be implemented using wired Ethernet connections or using any other network link based on an arbitrary technology known in the art. The individual event detectors may be running as different software processes on different computing nodes or some of them may be running on the same node. The event detectors may also be implemented as dedicated or programmable hardware or the like. With this respect, the dependencies between the event detectors shown in
In a first primitive event generation step 20, a stream of first primitive events 22 is generated, wherein a data rate of the stream of first primitive events 22 is lower than the sensor data rate. In a second primitive event generation step 24, a stream of second primitive events 26 is generated, wherein a data rate of the stream of the second primitive events 26 is lower than the sensor data rate.
In an event generation step 28 the event of interest 30 is determined, using the streams 22 and 26 of the first and second primitive events. It is to be noted that the first and second event generation steps 20 and 24 may be performed in any arbitrary order, that is the particular events of the of first second stream of primitive events which are used to determine an event of interest 30 do not necessarily have to be created or delivered at the same time.
To this end, event detectors may, for example, be implemented as state machines as it will be shown in
Based on the line crossing event 42, the ball is identified to be in a particular region, generating a region event 44. The identification of a particular region is elaborated on in more detail in the description of
Using a conventional approach to determine, whether the ball 72 is located inside region 74 would require to iterate over all regions of the playfield and to determine the relative position of the ball with respect to each edge of the region. That is, for example, a determinate had to be computed for each of the 4 edges of region 74 in order to conclude that the ball 72 is inside the region 74 when all determinates share the same sign. Performing this kind of analysis for, say, 2000 ball positions per second would hardly be feasible in real time.
Using an embodiment of the invention, however, the determination of the position of the ball may be based on a primitive “line crossing-event” determined for each of the lines used to subdivide a playfield into multiple regions, as for example lines 76 to 80, which provide the border of region 74. For each of the lines, a line crossing event or a relative position event is determined, when the ball crosses the line. As illustrated in
When, however, the primitive event detector associated to line 82 generates the line crossing element, an event generator receiving this event and in addition thereto the events determined by the primitive event detectors associated to lines 76, 78 and 80 will be able to generate an event indicating that ball 72 is now in region 74. The event detector could, for example, be implemented as a state machine, as shown in more detail in
In other words, a conventional approach would require, for each defined region:
-
- for each straight border segment of the region:
- calculate a determinate for the position of the observed object; compare the calculated determinates to conclude, whether the object is within the area or not.
- for each straight border segment of the region:
To the contrary, using an embodiment of the invention, one may define a number k of lines in order subdivide the playfield into a number n of regions, as, for example, illustrated in
-
- Start k primitive event detectors, each associated to one line, in order to determine, as a primitive event, whether the ball is on a first or a second side of the line.
- Start n Event detectors, to determine an event indicating that the ball is inside its associated region, wherein each event detector is connected with the primitive event detectors associated to the lines bordering the region associated with the event detector.
- Determine an event with the event detector, when all connected primitive event detectors provide corresponding information.
Although only described for a subset of events of interest, arbitrary further events of interest may be determined on the basis of the primitive events. Further Events of interest may, for example, be single passes from one player to another, double passes, shots on the goal, dribblings, free kicks, kick-offs or corner kicks.
It goes without saying that, although illustrated primarily with respect to a soccer match, further embodiments of the invention may be used in any other kind of sports game, such as for example basketball, handball, a game of volleyball, rugby, American Football, ice hockey, field hockey, polo, baseball, tennis, golf and the like.
The description and drawings merely illustrate the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the invention and are included within its spirit and scope. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass equivalents thereof.
Functional blocks denoted as “means for . . . ” (performing a certain function) shall be understood as functional blocks comprising circuitry that is adapted for performing a certain function, respectively. Hence, a “means for s.th.” may as well be understood as a “means being adapted or operable for s.th.”. A means being adapted for performing a certain function does, hence, not imply that such means necessarily is performing said function (at a given time instant).
Functions of various elements shown in the figures, including any functional blocks may be provided through the use of dedicated hardware, as e.g. a processor, as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included.
It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
Furthermore, the following claims are hereby incorporated into the Detailed Description, where each claim may stand on its own as a separate embodiment. While each claim may stand on its own as a separate embodiment, it is to be noted that—although a dependent claim may refer in the claims to a specific combination with one or more other claims—other embodiments may also include a combination of the dependent claim with the subject matter of each other dependent claim. Such combinations are proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim. Particularly, when a dependent claim is referring to an encoder or a sender, the corresponding feature of the related decoder or receiver shall herewith also be included and part of the disclosure of the description.
It is further to be noted that methods disclosed in the specification or in the claims may be implemented by a device having means for performing each of the respective steps of these methods.
Further, it is to be understood that the disclosure of multiple steps or functions disclosed in the specification or claims may not be construed as to be within the specific order. Therefore, the disclosure of multiple steps or functions will not limit these to a particular order unless such steps or functions are not interchangeable for technical reasons.
Furthermore, in some embodiments a single step may include or may be broken into multiple substeps. Such substeps may be included and part of the disclosure of this single step unless explicitly excluded.
Claims
1-26. (canceled)
27. An apparatus for detecting an event of interest in a sports game, comprising:
- a first primitive event detector for generating a stream of first primitive events using a sensor data stream, the sensor data stream carrying kinematic data and an associated time stamp as a raw data at a sensor data rate, the raw data being associated to one or more kinematic properties determined for multiple observed objects at the time indicated by the time stamp using one or more sensors, wherein a data rate of the stream of first primitive events is lower than the sensor data rate;
- a second primitive event detector for generating a stream of different second primitive events using the sensor data stream, wherein a data rate of the stream of the second primitive events is lower than the sensor data rate; and
- an event detector operable to determine the event of interest using the streams of the first and second primitive events.
28. The apparatus of claim 27, wherein the kinematic data is based on location data indicating a geographic position of an observed object at the time indicated by the time stamp, wherein the location data is derived from location information of sensors placed at the observed objects.
29. The apparatus of claim 28, wherein the kinematic data comprises position information, velocity information and acceleration information of an observed object.
30. The apparatus of claim 27, wherein the first and second primitive event detectors are operable to exclusively use the sensor data stream as an input.
31. The apparatus of claim 27, wherein a proximity event is generated as the first or the second primitive event when position information of a first observed object and of a second observed object indicates that the second observed object is within a predetermined neighborhood of the first observed object.
32. The apparatus of claim 31, wherein the predetermined neighborhood is defined by a predetermined maximum distance between the first observed object and the second observed object.
33. The apparatus of claim 27, wherein a remote event is generated as the first or the second primitive event when position information of a first observed object and of a second observed object indicates that the second observed object is outside of a predetermined neighborhood of the first observed object.
34. The apparatus of claim 27, wherein a direction changed event is generated as the first or the second primitive event when a change of a velocity or an acceleration of an observed object exceeds a predetermined threshold.
35. The apparatus of claim 27, wherein an idle event is generated as the first or the second primitive event, when a velocity of an observed object falls below a predetermined threshold.
36. The apparatus of claim 35, wherein the idle event is only generated when the velocity of the observed object falls below the predetermined threshold for at least a predetermined minimum duration.
37. The apparatus of claim 27, wherein an active event is generated as the first or the second primitive event, when a velocity of an observed object exceeds a predetermined threshold.
38. The apparatus of claim 37, wherein the active event is only generated when the velocity of the observed object exceeds the predetermined threshold for at least a predetermined minimum duration.
39. The apparatus of claim 27, wherein a line crossing event is generated as the first or the second primitive event, when a component of a position information of the first or the second observed object exceeds or falls below a predetermined threshold.
40. The apparatus of claim 27, wherein a hit event is detected as the event of interest when the first primitive event detector detects a proximity event for the first object and the second object and when the second primitive event detector detects a direction change event for the second object, the hit event indicating that the second object was hit by the first object.
41. The apparatus of claim 27, further comprising a further event detector for detecting a further event of interest using the first primitive event and the event of interest.
42. The apparatus of claim 41, wherein an object possession event is generated as the further event of interest when a proximity event is detected by the first primitive event detector and when a hit event is detected by the event detector.
43. The apparatus of claim 41, wherein an object loss event is generated as the further event of interest when a remote event is detected by the first primitive event detector and when a hit event is detected by the event detector.
44. The apparatus of claim 27, wherein the event detector is implemented as a state machine.
45. A method for detecting an event of interest in a sports game, the method comprising:
- generating a stream of first primitive events using a sensor data stream, the sensor data stream carrying kinematic data and an associated time stamp as a raw data at a sensor data rate, the raw data being associated to one or more kinematic properties determined for multiple observed objects at the time indicated by the time stamp using one or more sensors, wherein a data rate of the stream of first primitive events is lower than the sensor data rate;
- generating a stream of different second primitive events using the sensor data stream, wherein a data rate of the stream of the second primitive events is lower than the sensor data rate; and
- determining the event of interest using the streams of the first and second primitive events.
46. An analysis framework for analyzing raw data portions being transmitted in a sensor data stream at a sensor data rate to identify at least one event of interest in a sports game, each raw data portion being associated to one or more kinematic properties and an associated time stamp determined for an observed object at the time indicated by the time stamp using one or more sensors, the analysis framework comprising:
- an input layer comprising at least two or more primitive event detectors for generating at least a first and second stream of primitive events using exclusively the sensor data stream as an input wherein a data rate of the stream of first and second primitive events is lower than the sensor data rate; and
- one or more event layers comprising one or more event detectors for detecting an event of interest,
- wherein any one of the one or more event detectors uses only the streams of the first and the second primitive events or a stream of other events as provided by another one of the event detectors as an input.
47. The analysis framework of claim 46, wherein the one or more event detectors of the one or more event layers are connected with the at least two or more primitive event detectors of the input layer via a network.
48. The analysis framework of claim 47, wherein the network is a wired network.
49. The analysis framework of claim 47, wherein a communication in the network is based on the Internet Protocol.
50. The analysis framework of claim 46, wherein first and second primitive event detectors are operable to exclusively use kinematic data of an observed object as the raw data.
51. A computer program for performing, when running on a computer, a method for detecting an event of interest in a sports game, the method comprising:
- generating a stream of first primitive events using a sensor data stream, the sensor data stream carrying kinematic data and an associated time stamp as a raw data at a sensor data rate, the raw data being associated to one or more kinematic properties determined for multiple observed objects at the time indicated by the time stamp using one or more sensors, wherein a data rate of the stream of first primitive events is lower than the sensor data rate;
- generating a stream of different second primitive events using the sensor data stream, wherein a data rate of the stream of the second primitive events is lower than the sensor data rate; and
- determining the event of interest using the streams of the first and second primitive events.
Type: Application
Filed: Oct 31, 2011
Publication Date: Feb 19, 2015
Applicant: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. (Muenchen)
Inventors: Christopher Mutschler (Erlangen), Norbert Franke (Erlangen), Daniel Wolf (Berlin), Nicolas Witt (Nuernberg)
Application Number: 14/355,109
International Classification: A63F 9/24 (20060101);