REAL-TIME LARGE VOLUME DATA CORRELATION
A system and method uses a data correlator system to associate large volumes of related data. The data correlator system receives data packets with user content from a mobile network. The data correlator system creates a hash value for data associated with the user content, such as user account information or the location of the user device. The data correlator system uses the hash value to combine the user content with the associated data. The data correlator system provides the combined data to an optimization engine for optimizing the performance of the mobile network.
This application is a continuation of U.S. patent application Ser. No. 16/552,729, filed Aug. 27, 2019, now U.S. Pat. No. 11,122,464, which is herein incorporated by reference in its entirety.
TECHNICAL FIELDThe present disclosure relates generally to systems and methods for correlating large volumes of data from a communication network.
BACKGROUNDAs mobile devices such as cellular telephones, smart phones, and tablet computers continue to proliferate, the demands on communication networks continue to grow. For example, mobile network customers frequently access a variety of content, including text, images, and video using their mobile devices. This increased use of mobile devices has greatly increased the volume of data traffic handled by mobile networks. Techniques for optimizing the performance of the mobile network can assist the mobile network operator in handling increased volumes of data traffic. Optimizing the performance of the mobile network often requires quickly evaluating data associated with a particular content that a user is accessing over the mobile network. However, existing techniques for correlating data are not well-suited to handle large volumes of data. Additionally, existing techniques for correlating large volumes of data can be time consuming and can require substantial memory and processor resources. Accordingly, improved techniques for correlating large volumes of data associated with the data traffic on the mobile network can assist in improving the performance of the mobile network.
SUMMARYOne aspect of the present disclosure provides a system for correlating large volumes of data in connection with a communication network. The system comprises a communication network and a computing system, the computing system comprising the data correlator system. The computing system comprises a data correlator and a first data feed queue that supplies a first data packet from the communication network to the data correlator. The first data packet comprises a key data element, such as a device network identifier, and a user plane data element, such as a portion of one of an image, a video frame, or text, or a control plane data element, such as a network performance measurement, a key performance indicator, or a location of a user device. The computing system also comprises a data mapper and a second data feed queue that supplies a second data packet to the data mapper. The second data packet can comprise the key data element and a user identifier data element, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number. The data mapper creates a hash value for the second data packet. The data correlator uses the hash value and the key data element to identify the second data packet and create a combined data packet by combining the first data packet and the second data packet. The data correlator can provide the combined data packet to an optimization engine to optimize the functioning of the communication network.
Another aspect of the present disclosure provides a method for correlating large volumes of data in connection with a communication network. The method uses a data correlator executing on a computing system and the method comprises receiving a first data packet at the data correlator from a communication network, wherein the first data packet comprises a key data element and a user plane data element or a control plane data element. A data mapper executing on the computing system receives a second data packet which can comprise the key data element and a user identifier data element. The data mapper can create a hash value for the second data packet. The data correlator uses the hash value and the key data element to identify the second data packet and to combine the first data packet and second data packet to create a combined data packet. The data correlator can provide the combined data packet to an optimization engine to optimized the functioning of the communication network.
The foregoing aspects are example embodiments and are not meant to limit the scope of the present disclosure. These and other aspects will be described in connection with the following description and the accompanying drawings.
The present disclosure relates generally to systems and methods for correlating large volumes of data in connection with optimizing the performance of a communication network. As one example, a user device receiving user content, such as video content, over the communication network may require transmission of a large number of data packets. In order to optimize the performance of the communication network, it may be desirable to hand off the user device from one cell to a neighboring cell within the communication network. In order to hand off the user device from one cell to another cell, the communication network may require certain information associated with the user device such as the device network identifier, the signal strength associated with the user device, the location of the user device, and the telephone number associated with the user device. These different pieces of information typically come from different sources. For example, the signal strength and the location of the user may be determined by the communication network. However, the telephone number associated with the user device may be retrieved from a separate database of user information.
In order to correlate the different pieces of information for network optimization decisions, the data correlator system can use common information among the different pieces of information. For example, data packets associated with each different piece of information include a key data element such as the device network identifier, an evolved Universal Terrestrial Radio Access Network (eUTRAN) device identifier, or a mobility management entity (MME) device identifier. The foregoing examples of identifiers are generally referred to herein as device network identifiers for simplicity. The data correlator can use the key data element associated with the data packets of each different piece of information to correlate and combine the related data.
As a further optimization, the data correlator system can rely on data mappers that create a hash value associated with the different pieces of information. It is common for communication networks today to have millions of simultaneous users and each of those users can be receiving large volumes of data over the communication network. In order to quickly correlate millions of data packets being transmitted over the communication network and to minimize the data processing and memory requirements for such correlation, the data correlation system relies on the data mappers and hash values to operate quickly and efficiently.
The foregoing example relating to handing off a user device from one cell to a neighboring cell of a communication network is only one example of the types of network optimization that can be performed. In other examples, decisions relating to optimization of the communication network can be based on the type of user device or the signal strength of the user device. For instance, if a user is watching a video with the user device in a region of the communication network that has limited bandwidth, the network can suggest that the user switch to a lower resolution of the video in order to improve delivery of the video content to the user device. Moreover, the data correlation techniques described herein can be used in other types of networks such as financial system networks.
Referring now to
The communication network 100 may include an user device 102 that may be used by an end-user to communicate with the communication network using wireless services offered by the communication network 100. The end-user may be an individual user, a group of users, an enterprise or organizational entity, etc. As one example, in an LTE network, the user device 102 may be referred to as user equipment. However, in other networks, such as a 3G network, the user device 102 may be referred to as a mobile station. The user device 102 may include all user equipment and software needed for communication with the communication network 100. Examples of the user device 102 may include, but are not limited to, a smart phone, a cellular phone, a computer or laptop, a computing tablet, or any endpoint communication devices equipped with wireless capabilities. In some example embodiments, the user device 102 of the end-user may include a server, e.g., when the user is transmitting data for storage in a cloud server over the cellular network, or when the user is receiving streaming video content over the cellular network.
The user device 102 may access the wireless services offered by the communication network 100 via the radio access network 104. In particular, the user device 102 may wirelessly connect with the radio access network 104 over a wireless link 120, and the radio access network 104 may couple the user device 102 to other parts of the communication network, such as the core network 108. As an example, the radio access network of an LTE network may be an evolved Universal Terrestrial Radio Access Network (eUTRAN) that includes one or more evolved base stations. However, in other example communication networks, such as 3G, 4G, or 5G networks, the radio access network 104 may include base stations and/or base station controllers.
The radio access network 104 may transmit all the communication from the endpoint device 102 to the core network 108. The core network 108 of the communication network 100 may be responsible for the overall control of the actions requested by user device 102 that is wirelessly connected to the radio access network 104. For example, the core network 108 may be responsible for processing and routing the traffic (voice and/or data) from the user device 102 to other endpoint devices and/or external networks, such as Internet Protocol (IP) network 112 or public switched telephone network (PSTN) 110. Further, the core network 108 may also handle all the signaling in the control plane. For example, the core network 108 of an LTE network may be an Evolved Packet Core (EPC) that provides various functions that support wireless services in the LTE environment, e.g., as specified by the 3GPP standards. The EPC network may include at least a serving gateway (S-GW), a packet data network gateway (P-GW), and/or a mobility management entity (MME).
The MME is the control node that processes the signaling between the user device 102 and the core network 108. In particular, the functions of the MME may be categorized as a bearer function, such as, establishment, maintenance, and release of the bearers; and/or connection management functions, such as, establishment of the connection and security between the EPC and the user device 102. The S-GW may serve as a gateway that routes and forwards user data packets between the eUTRAN and the P-GW, while also acting as the mobility anchor for the user plane during handovers and as the anchor for mobility between LTE and other 3GPP technologies. The P-GW serves as a gateway that provides connectivity between the EPC and other external packet data networks. The P-GW may route data traffic to the IP network 112 and/or the PSTN 110 via an IMS network that includes a home subscriber server (HSS) database that is a central database that contains user-related and subscription-related information about all the network operator's subscribers (also referred to as users). The foregoing network components are merely examples and other communication networks can have other components.
Also illustrated in the example communication network 110 are data correlator systems 114a and 114b (referred to generally as 114). Although only two data correlator systems are shown in the example of
Referring now to
The data correlator system 114 of
The data mapper 206 receives data set D and creates a hash table enabling the data correlator 205 to more quickly identify and correlate data packets. The hash table will be explained further below in connection with
Once data packets are correlated, the data correlator can combine the data packets and provide them to external systems for use in optimizing the communication network 100. For example, an optional data writer 218 can write the combined data to files that can be used by a variety of external clients 220 that optimize the network. Alternatively, the data writer 218 and its associated files can be omitted and the data correlator 205 can provide correlated and combined data packets directly to an optimization engine. The combined data packets provide more complete information associated with the user device 102 thereby enabling the optimization engine to make optimization decisions.
One benefit of using the techniques of the data correlator and data mapper of
The data correlator system 114 illustrated in
Referring now to
While components of the data correlator systems 114 illustrated in
Referring now to
Referring to
Referring now to
As illustrated in
In contrast to the parallel method illustrated in
The components of the example communication network 100 have been described with reference to the example systems and methods illustrated in
The example embodiments described herein can be used with computer hardware and software that perform the methods and processing functions described previously. The systems, methods, and procedures described herein can be embodied in a programmable computer, computer-executable software, or digital circuitry. The software can be stored on tangible computer-readable media. For example, computer-readable media can include a floppy disk, RAM, ROM, hard disk, removable media, flash memory, memory stick, optical media, magneto-optical media, CD-ROM, etc. Digital circuitry can include integrated circuits, gate arrays, building block logic, field programmable gate arrays (FPGA), etc.
The processor 504 may comprise one or more specific or general purpose processors configured to execute instructions stored on the memory 510 that, when executed, control the processor 504 to execute various applications and perform various functions associated with the computing device 500. Additionally or alternatively, the processor 504 may include a programmable gate array and operate, at least in part, based on firmware. As an example, the processor 504 may execute instructions stored on the memory 510 including instructions for an operating system of the computing device 500 and instructions for applications. The applications that may be executed by the computing device 500 include mobile applications and engines, data mapping and data correlation processing applications, network optimization applications, an e-mail application, a map viewer application, and a web-browser application, among others. The input commands control applications executing on the computing device 500 to perform various tasks or functions associated with the applications.
The display/speaker/microphone 506 is configured to display applications executing on the processor 504 and provide visual and audible feedback to a user of the computing device 500. To that end, the display/speaker/microphone 506 may comprise one or more display devices such as LCD, LED, OLED, and Electronic Ink displays, among others. The display/speaker/microphone 506 may also comprise one or more speakers and one or more microphones.
The wireless communication module 508 is configured to provide wireless communication of data to and from the computing device 500. As a non-limiting group of examples, the wireless communication module 508 may be configured for cellular communications using one or more of GSM, CDMA, TDMA, OFDM and other cellular communications protocols, wireless area network communications using one or more of the family of 802.11x protocols and other wireless area network communications protocols, and Bluetooth communications protocols.
The memory 510 may comprise a Random Access Memory (RAM), Read Only Memory (ROM), or any other tangible storage memory configured to store software programs for execution by the processor 504. As a non-limiting example group, the memory 510 may comprise one or more of dynamic, persistent, and semi-persistent solid state memories, magnetic memories, removable memories, or any other known memories suitable for the application of storing data and software programs for the computing device 500.
The I/O 512 includes inputs and outputs of the computing device 500 such as power connectors, data connectors, and other input and output devices. The I/O 512 may comprise, for example, wired data communication input and output interfaces, power charging interfaces, infra-red interfaces, light and proximity sensors, capacitive sensors, “soft” and “hard” buttons, switches, and other input/output interfaces of the computing device 500. The bus 514 is configured to electrically and communicatively connect the processor 504, the display/speaker/microphone 506, the wireless module 508, the memory 510, and the I/O 512 for transfer of data and instructions between elements of the computing device 500.
The example methods and acts described in the embodiments presented previously are illustrative, and, in alternative embodiments, certain acts can be performed in a different order, in parallel with one another, omitted entirely, and/or combined between different example embodiments, and/or certain additional acts can be performed, without departing from the scope and spirit of the invention. Accordingly, such alternative embodiments are included in the inventions described herein.
Although specific embodiments have been described above in detail, the description is merely for purposes of illustration. It should be appreciated, therefore, that many aspects described above are not intended as required or essential elements unless explicitly stated otherwise. Modifications of, and equivalent acts corresponding to, the disclosed aspects of the example embodiments, in addition to those described above, can be made by a person of ordinary skill in the art, having the benefit of the present disclosure, without departing from the spirit and scope of the invention defined in the following claims, the scope of which is to be accorded the broadest interpretation so as to encompass such modifications and equivalent structures.
Claims
1. A computing system comprising:
- a data correlator;
- a first data feed queue that supplies a first data packet from a network connection to the data correlator, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element;
- a data mapper; and
- a second data feed queue that supplies a second data packet to the data mapper, wherein the second data packet comprises the key data element and a user identifier data element,
- wherein the data mapper creates a hash value for the second data packet;
- wherein the data correlator uses the hash value and the key data element to identify the second data packet and combines the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and at least one of: the user plane data element or the control plane data element; and
- wherein the data correlator provides the combined data packet to an external client.
2. The computing system of claim 1, wherein the key data element comprises a device network identifier.
3. The computing system of claim 1, wherein the user plane data element comprises a portion of one of: an image, a video frame, or text.
4. The computing system of claim 1, wherein the user identifier data element comprises one of: a telephone number, an international mobile equipment identity code, an international mobile subscriber identity code, a name, or an account number.
5. The computing system of claim 1, wherein the user identifier data element is retrieved from a user database.
6. The computing system of claim 1, wherein the external client comprises an optimization engine, wherein the optimization engine uses the control plane data element and the key data element to identify a first cell location of a device and to transfer the device to a second cell location within a communication network.
7. The computing system of claim 1, wherein the computing system further comprises:
- a second data correlator; and
- a third data feed queue that supplies a third data packet to the second data correlator, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
- wherein the second data correlator uses the hash value and the key data element to identify the third data packet and to combine the third data packet and the second data packet to create a combined second data packet comprising the user identifier data element and the one of: the user plane data element or the control plane data element that is absent from the first data packet; and
- wherein the second data correlator provides the combined second data packet to the external client.
8. The computing system of claim 1, further comprising:
- a second data correlator; and
- a second data mapper;
- wherein the second data correlator receives the combined data packet from the data correlator;
- wherein the second data mapper receives a third data packet, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
- wherein the second data mapper creates a second hash value for the third data packet;
- wherein the second data correlator uses the second hash value and the key data element to identify the third data packet and to combine the combined data packet and the third data packet to create a second combined data packet comprising the user identifier data element, the user plane data element, and the control plane data element; and
- wherein the second data correlator provides the second combined data packet to the external client.
9. The computing system of claim 8, wherein the control plane data element comprises one of: a network performance measurement or location data.
10. The computing system of claim 1, further comprising a delayed data correlator, wherein the delayed data correlator:
- receives a third data packet from the data correlator when the data correlator is unable to combine the third data packet and a fourth data packet;
- receives a second hash value from the data mapper;
- uses the second hash value and the key data element to identify the third data packet and to combine the third data packet received from the data correlator and the fourth data packet received at the data mapper to create a delayed combined data packet comprising the user identifier data element and one of: the user plane data element or the control plane data element; and
- provides the delayed combined data packet to the external client.
11. A method comprising:
- receiving, from a network connection, a first data packet at a data correlator, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element, and wherein the data correlator executes on at least one processor; and
- receiving a second data packet at a data mapper, the data mapper executing on the at least one processor, wherein the second data packet comprises the key data element and a user identifier data element;
- creating, by the data mapper, a hash value for the second data packet;
- identifying, by the data correlator, the second data packet by using the hash value and the key data element;
- combining, by the data correlator, the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and the user plane data element or the control plane data element; and
- providing the combined data packet to an external client.
12. The method of claim 11, wherein the key data element comprises a device network identifier.
13. The method of claim 11, wherein the user plane data element comprises a portion of one of: an image, a video frame, or text.
14. The method of claim 11, wherein the user identifier data element comprises one of: a telephone number, an international mobile equipment identity code, an international mobile subscriber identity code, a name, or an account number.
15. The method of claim 11, wherein the user identifier data element is retrieved from a user database.
16. The method of claim 11, wherein the external client comprises an optimization engine, wherein the optimization engine uses the control plane data element and the key data element to identify a first cell location of a device and to transfer the device to a second cell location within a communication network.
17. The method of claim 11, further comprising:
- receiving a third data packet at a second data correlator, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
- identifying, by the second data correlator, the third data packet by using the hash value and the key data element;
- combining, by the second data correlator, the second data packet and the third data packet to create a combined second data packet comprising the user identifier data element and the one of: the user plane data element or the control plane data element that is absent from the first data packet; and
- providing, by the second data correlator, the combined second data packet to the external client.
18. The method of claim 11, further comprising:
- receiving, at a second data correlator, the combined data packet from the data correlator;
- receiving, at a second data mapper, a third data packet, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
- creating, by the second data mapper, a second hash value for the third data packet;
- identifying, by the second data correlator, the third data packet by using the second hash value and the key data element;
- combining, by the second data correlator, the combined data packet and the third data packet to create a second combined data packet comprising the user identifier data element, the user plane data element, and the control plane data element; and
- providing, by the second data correlator, the second combined data packet to the external client.
19. The method of claim 18, wherein the control plane data element comprises one of:
- a network performance measurement or location data.
20. A non-transitory computer-readable medium storing instructions which, when executed by a computing system including at least one processor, cause the computing system to perform operations, the operations comprising:
- receiving, from a network connection, a first data packet, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element; and
- receiving a second data packet, wherein the second data packet comprises the key data element and a user identifier data element;
- creating a hash value for the second data packet;
- identifying the second data packet by using the hash value and the key data element;
- combining the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and the user plane data element or the control plane data element; and
- providing the combined data packet to an external client.
Type: Application
Filed: Sep 13, 2021
Publication Date: Dec 30, 2021
Inventors: Peter Aung Htut Wong (Newark, CA), Baofeng JIANG (Pleasanton, CA)
Application Number: 17/447,572