Systems and methods for parallel transport of data between computer systems
Embodiments of the present disclosure include techniques for moving data. In one embodiment, computer systems import and/or export data. The computer systems may be application servers coupled to a content management system accessible over a plurality of tenant frontends. The data movement may use a plurality of jobs executing in parallel to move packages of content between the systems. In one embodiment, before a job is executed, a check is made of other jobs to ensure any overlap does not cause data conflicts.
Latest SAP SE Patents:
- REFERENCE-FREE EVALUATION OF COLOR CONSISTENCY IN TEXT-TO-IMAGE GENERATION MODELS
- Estimation of microservice system capacity
- Differentially private variational autoencoders for data obfuscation
- Dynamic, automated fulfillment of computer-based resource request provisioning using deep reinforcement learning
- Large hierarchy node dynamically generated data framework
This application is related to, and concurrently filed with, U.S. patent application Ser. No. 18/531,384, filed Dec. 6, 2023, now issued as patent No.: U.S. Pat. No. 12,438,869, the disclosure of which is hereby incorporated herein by reference.
This application is related to, and concurrently filed with, U.S. patent application Ser. No. 18/531,332, filed on Dec. 6, 2023, now issued as patent No. 12,603,934, the disclosure of which is hereby incorporated herein by reference.
This application is related to, and concurrently filed with, U.S. patent application Ser. No. 18/531,354, filed on Dec. 6, 2023, now issued as U.S. Pat. No. 12,578,893, the disclosure of which is hereby incorporated herein by reference.
This application is related to, and concurrently filed with, U.S. patent application Ser. No. 18/531,369, filed on Dec. 6, 2023, now issued as U.S. Pat. No. 12,639,463, the disclosure of which is hereby incorporated herein by reference.
BACKGROUNDThe present disclosure relates generally to software systems, and in particular, to systems and methods for parallel transport of data between computer systems.
Computer systems require data to produce useful and meaningful results. Data preparation and analytics can involve complex, time consuming preparation of relations, visualizations, and compilations of data. When data is prepared in such a way, it may be beneficial to share the data across multiple computer systems. However, moving complex data structures across different systems can be a challenge. One particular challenge is to ensure that the latest version of data is transported. In some cases, data being transported may be an older version. Accordingly, new data can be overwritten with old data and/or old data may be retrieved from a system when newer data was available.
The present disclosure addresses these and other challenges and is directed to techniques for moving data between computer systems.
Described herein are techniques for backing up data. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of some embodiments. Various embodiments as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below and may further include modifications and equivalents of the features and concepts described herein.
In various embodiments, the user may use the techniques described herein to move data from content management system 102 to server 103 (e.g., data import to server) or to move data from server 103 to content management system 102 (e.g., data export from server). Content management system 102 may similarly import data from a server or export data to a server, for example. In some embodiments, users of multiple different application servers 103 may store data generated locally on server 103 in a content management system 102 to share data with other users on other applications servers (not shown). Accordingly, it may be desirable to import and export data between content management system 102 and server 103. In some embodiments, the data may comprise data packages. Data packages passed between computer systems may include a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package. Example objects in object data store that may be shared include objects corresponding to complex analytics content (e.g., models, stories, visualizations, dimensions, connections, etc. of data) that may be used to discover the unseen data patterns to boost the business productivity, for example.
To facilitate movement of data between systems, content management system 102 includes an automated move process 110. In various embodiments, data movement between the systems may include initiating automated process 110 to move data between a first computer system and a second computer system. Advantageously, data may be moved in parallel. For example, automated process 110 may instantiate a plurality of jobs 111a-n to move data units between the first computer system and the second computer system. In some embodiments, the data units are data packages comprising a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package. Each job 111a-n may assemble and move one data package for example. However, because the data is shared, in various scenarios it may be desirable to ensure that the data being moved is a latest version and/or that other users are not modifying the data in the process of being moved, for example. Thus, prior to execution of each job 111a-n, the system may determine if an overlap exists between data units of the jobs. For instance, if one job is moving data off the content management system while another job is moving a modified version of the same data off the content management system, the overlap may be detected. A job may be executed when no overlap exists between data units moved by any of the currently executing jobs 111a-n and a particular job to be executed, for example. Further examples and advantages of the present techniques are described further below.
The present disclosure may be used in the context of an analytics content network in a cloud computer system that combines business intelligence (BI) and planning and predictive capabilities, for example. In any business intelligence application, the analytics content (model, story, visualizations, etc.) plays the central role in discovering the unseen patterns to boost the business productivity. Hence sharing of the analytics content across users is very helpful for better collaboration. Also, a standard content template can be reused by all user by plugging their corresponding data. This infrastructure for sharing the analytics content is sometimes referred to as an “Analytical Content Network” (ACN). The content entity that contains content to be shared is called “package.”
ACN may be arranged landscapes as a central component, all of which are connected. An application landscape is a coherent set of interconnected applications often within an enterprise, business, or organization, which are often associated with different geographical regions, for example. Logically ACN is “one global content network” which can provision or share any content with servers and users across landscapes. ACN may supports the following end-user workflows. A content creator creates content in the form of stories, models, dimensions, connections, Value-Driver Trees (VDT) etc. If authorized, the user can then export this content from a tenant (a portion of system resources securely assigned to a particular group) to ACN by creating a “content package,” which can contain any number of these content items and share this with multiple other tenants, for example. Another SAC Content user can view all available content packages in their listing and import those packages relevant for their analytic workflows. This includes Public content (Templates or Demo content) and Private Content (shared privately with them). To achieve sharing across tenants, the content is bundled in what is referred to as a “package.” A package may contain the details of each object present in the package, the dependency information between those objects, and an overview which summarizes the content details, for example. Data objects are stored in datastore 312 and data describing the data objects and packages may be stored in database 313, for example. An example package is shown in
Embodiments of the present disclosure may include three steps for transportation of data content and objects across tenants as shown in
One issue with important packages from a CMS to a server is that packages may contain common objects. If Tenant1 has packages:
-
- PackageA—Story1, Model1, Model2, Dimension1, Connection1; and
- PackageB—Story1, Model1, Currency3,
then Story1, Model1 are common objects.
Therefore, if more than one user triggers import of packages containing the same set of objects at same time, there are chances of data overwrite. For example, the following scenario may occur. UserC creates (exports) Story1, adds a table view, and creates PackageA containing Story1 and its dependencies, and if UserC imports PackageA (e.g., at 4 PM and it takes an hour to complete) with an import order of: Dimension1, Connection1, Model1, Model2, Story1. UserD modifies Story1, adds a pie chart, a line chart (in addition to the above added table view), and creates (exports) PackageB containing Story1 and its dependencies, where UserD imports PackageB at 4:30 PM with an import order of: Currency3, Model1, Story1. For the above scenario, if PackageA and PackageB execute at non-overlapping times, there is no conflict. However, if PackageA import is ongoing when PackageB import is initiated, there is chance to override Story1, Modell objects. PackageB imports Story1 (table view, pie chart, line chart). PackageA imports Story1 (table view), and may overwriting changes made by UserD, for example. Note, generally, exported object structure is maintained so it can be recreated in another system. If new elements are added/removed from a package (story), the package in CMS will not be affected.
Features and advantages of the present disclosure allow parallel transport post analysis of already executing jobs (package and/or content) for faster movement of data between systems without overlap or conflict to avoid data loss as described above.
Using the techniques described in the above example, an ACN supported package transport may be performed in parallel to reduce the time taken to export or import multiple packages, improve usage of resources (e.g., DB connection pool, number of instances and threads), and reduce time taken during tenant migration, which may involve bulk package transport.
In some systems, computer system 510 may be coupled via bus 505 to a display 512 for displaying information to a computer user. An input device 511 such as a keyboard, touchscreen, and/or mouse is coupled to bus 505 for communicating information and command selections from the user to processor 501. The combination of these components allows the user to communicate with the system. In some systems, bus 505 represents multiple specialized buses for coupling various components of the computer together, for example.
Computer system 510 also includes a network interface 504 coupled with bus 505. Network interface 504 may provide two-way data communication between computer system 510 and a local network 520. Network 520 may represent one or multiple networking technologies, such as Ethernet, local wireless networks (e.g., WiFi), or cellular networks, for example. The network interface 504 may be a wireless or wired connection, for example. Computer system 510 can send and receive information through the network interface 504 across a wired or wireless local area network, an Intranet, or a cellular network to the Internet 530, for example. In some embodiments, a frontend (e.g., a browser), for example, may access data and features on backend software systems that may reside on multiple different hardware servers on-prem 531 or across the network 530 (e.g., an Extranet or the Internet) on servers 532-534. One or more of servers 532-534 may also reside in a cloud computing environment, for example.
FURTHER EXAMPLESEach of the following non-limiting features in the following examples may stand on its own or may be combined in various permutations or combinations with one or more of the other features in the examples below. In various embodiments, the present disclosure may be implemented as a system, method, or computer readable medium.
Embodiments of the present disclosure may include systems, methods, or computer readable media. In one embodiment, the present disclosure includes computer system comprising: at least one processor and at least one non-transitory computer readable medium (e.g., memory) storing computer executable instructions that, when executed by the at least one processor, cause the computer system to perform a method as described herein and in the following examples. In another embodiment, the present disclosure includes a non-transitory computer-readable medium storing computer-executable instructions that, when executed by at least one processor, perform a method as described herein and in the following examples.
In some embodiments, the method of moving data comprising: initiating an automated process to move data between a first computer system and a second computer system; instantiating a plurality of jobs to move data units between the first computer system and the second computer system; prior to execution of each job, determining if an overlap exists between data units of the jobs; and executing each particular job when no overlap exists between data units moved by a currently executing job and each particular job.
In some embodiments, the first computer system is an application server and the second computer system is a content management system.
In some embodiments, the content management system instantiates the plurality of jobs and wherein the data units are data packages comprising a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package.
In some embodiments, the automated process to move data is a data import operation.
In some embodiments, determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job share a same set of objects, wherein when the currently executing job and the new job share the same set of objects, the new job is not executed.
In some embodiments, the automated process to move data is a data export operation.
In some embodiments, determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job operate on a same data package, wherein when the currently executing job and the new job operate on a same data package, the new job is not executed.
The above description illustrates various embodiments along with examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of some embodiments as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations, and equivalents may be employed without departing from the scope hereof as defined by the claims.
Claims
1. A method of moving data comprising:
- initiating an automated process to move data between a first computer system and a second computer system;
- instantiating a plurality of jobs to move data units between the first computer system and the second computer system;
- placing the plurality of jobs in a job queue and selecting, by a content management system, a job from the job queue for execution;
- prior to execution of each job, determining if an overlap exists between data units of the jobs, wherein the overlap comprises a shared data object dependency across data packages, wherein determining if the overlap exists comprises: querying a database to identify currently executing jobs associated with a same tenant; and comparing data objects or data packages of the selected job with data objects or data packages of the currently executing jobs to identify conflicts; when an overlap is identified, returning the selected job to the job queue without execution; and
- executing each particular job when no overlap exists between data units moved by a currently executing job and each particular job.
2. The method of claim 1, wherein the first computer system is an application server and the second computer system is a content management system.
3. The method of claim 2, wherein the content management system instantiates the plurality of jobs and wherein the data units are data packages comprising a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data packages.
4. The method of claim 3, wherein the automated process to move data is a data import operation.
5. The method of claim 4, wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job share a same set of objects, wherein when the currently executing job and the new job share the same set of objects, the new job is not executed.
6. The method of claim 3, wherein the automated process to move data is a data export operation.
7. The method of claim 6, wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job operate on a same data package, wherein when the currently executing job and the new job operate on a same data package, the new job is not executed.
8. A computer system comprising:
- at least one processor;
- at least one non-transitory computer readable medium storing computer executable instructions that, when executed by the at least one processor, cause the computer system to perform a method of moving data comprising:
- initiating an automated process to move data between a first computer system and a second computer system;
- instantiating a plurality of jobs to move data units between the first computer system and the second computer system;
- placing the plurality of jobs in a job queue and selecting, by a content management system, a job from the job queue for execution;
- prior to execution of each job, determining if an overlap exists between data units of the jobs, wherein the overlap comprises a shared data object dependency across data packages, wherein determining if the overlap exists comprises: querying a database to identify currently executing jobs associated with a same tenant; and comparing data objects or data packages of the selected job with data objects or data packages of the currently executing jobs to identify conflicts; when an overlap is identified, returning the selected job to the job queue without execution; and
- executing each particular job when no overlap exists between data units moved by a currently executing job and each particular job.
9. The computer system of claim 8, wherein the first computer system is an application server and the second computer system is a content management system.
10. The computer system of claim 9, wherein the content management system instantiates the plurality of jobs and wherein the data units are data packages comprising a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data packages.
11. The computer system of claim 10, wherein the automated process to move data is a data import operation.
12. The computer system of claim 11, wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job share a same set of objects, wherein when the currently executing job and the new job share the same set of objects, the new job is not executed.
13. The computer system of claim 10, wherein the automated process to move data is a data export operation.
14. The computer system of claim 13, wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job operate on a same data package, wherein when the currently executing job and the new job operate on a same data package, the new job is not executed.
15. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by at least one processor, perform a method of moving data, the method comprising:
- initiating an automated process to move data between a first computer system and a second computer system;
- instantiating a plurality of jobs to move data units between the first computer system and the second computer system;
- placing the plurality of jobs in a job queue and selecting, by a content management system, a job from the job queue for execution;
- prior to execution of each job, determining if an overlap exists between data units of the jobs, wherein the overlap comprises a shared data object dependency across data packages, wherein determining if the overlap exists comprises: querying a database to identify currently executing jobs associated with a same tenant; and comparing data objects or data packages of the selected job with data objects or data packages of the currently executing jobs to identify conflicts; when an overlap is identified, returning the selected job to the job queue without execution; and
- executing each particular job when no overlap exists between data units moved by a currently executing job and each particular job.
16. The non-transitory computer-readable medium of claim 15, wherein the first computer system is an application server and the second computer system is a content management system.
17. The non-transitory computer-readable medium of claim 16, wherein the content management system instantiates the plurality of jobs and wherein the data units are data packages comprising a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data packages.
18. The non-transitory computer-readable medium of claim 17, wherein the automated process to move data is a data import operation.
19. The non-transitory computer-readable medium of claim 18, wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job share a same set of objects, wherein when the currently executing job and the new job share the same set of objects, the new job is not executed.
20. The non-transitory computer-readable medium of claim 17, wherein the automated process to move data is a data export operation, and wherein determining if an overlap exists between data units of the jobs comprises determining if a currently executing job and a new job operate on a same data package, wherein when the currently executing job and the new job operate on a same data package, the new job is not executed.
| 6564261 | May 13, 2003 | Gudjonsson et al. |
| 10664451 | May 26, 2020 | Liao et al. |
| 20020083124 | June 27, 2002 | Knox et al. |
| 20050203887 | September 15, 2005 | Joshi et al. |
| 20070156633 | July 5, 2007 | Sugita |
| 20090037373 | February 5, 2009 | Gilbert |
| 20090254572 | October 8, 2009 | Redlich et al. |
| 20100250497 | September 30, 2010 | Redlich et al. |
| 20120030167 | February 2, 2012 | Appiah |
| 20120185448 | July 19, 2012 | Mensch et al. |
| 20130139241 | May 30, 2013 | Leeder |
| 20130159062 | June 20, 2013 | Stiehl |
| 20130191519 | July 25, 2013 | Chang et al. |
| 20130262392 | October 3, 2013 | Vibhor et al. |
| 20140359576 | December 4, 2014 | Rath et al. |
| 20150074286 | March 12, 2015 | Anschutz |
| 20150120649 | April 30, 2015 | Nakadai |
| 20150236907 | August 20, 2015 | Popli |
| 20160110828 | April 21, 2016 | Master et al. |
| 20170195373 | July 6, 2017 | Toh et al. |
| 20180081905 | March 22, 2018 | Kamath et al. |
| 20190303445 | October 3, 2019 | Padmanabhan |
| 20200092177 | March 19, 2020 | Marvin et al. |
| 20200118330 | April 16, 2020 | Connelly et al. |
| 20200242481 | July 30, 2020 | Kim et al. |
| 20210058447 | February 25, 2021 | Trufasiu et al. |
| 20210109862 | April 15, 2021 | Yang et al. |
| 20210134407 | May 6, 2021 | Vera-Ciro et al. |
| 20210168128 | June 3, 2021 | Carru et al. |
| 20210385252 | December 9, 2021 | Lebin et al. |
| 20210399877 | December 23, 2021 | Ohashi et al. |
| 20210406397 | December 30, 2021 | Brazeau et al. |
| 20220027509 | January 27, 2022 | Amico |
| 20220076240 | March 10, 2022 | Ghani et al. |
| 20220086139 | March 17, 2022 | Venkatesh et al. |
| 20220245290 | August 4, 2022 | Luciani |
| 20230007483 | January 5, 2023 | Mueck et al. |
| 20230098596 | March 30, 2023 | Karri et al. |
| 20240069234 | February 29, 2024 | Meenakshi Sundaram Gandhi et al. |
| 20240333822 | October 3, 2024 | Parla et al. |
| 20240419456 | December 19, 2024 | Eberlein |
| 115063026 | September 2022 | CN |
| 2375006 | October 2002 | GB |
| 2008-084182 | April 2008 | JP |
| 4444518 | March 2010 | JP |
- The extended European search report of Application No. 24210027.9 dated May 12, 2025. 13 pages.
- Bigstar: “Site to site transfers (FXP)—1-15 INV. Flash FXP Forums”, Apr. 22, 2014 (Apr. 22, 2014), pp. 1-2, XP093246533, The Way back machine Retrieved from the Internet: URL:https://web.archive.org/web/20140718041834/https://www.flashfxp.com/forum/flashf xp/frequently-asked-questions-faq-/14656-site-site-transfers-fxp.html [retrieved on Feb. 4, 2025].
- Cppcis231: “How to FTP and FXP”, 1-15, Mar. 5, 2012 (Mar. 5, 2012), XP093246544, Retrieved from the Internet: URL:https://www.youtube.com/watch?v=6XXQge RWWRw [retrieved on Feb. 4, 2025].
- The extended European search report of Application No. 24206608.2 dated Mar. 10, 2025. 6 pages.
- Stefan Momma, Enterprise Content Management for SAP, Redbook,50 pages, Sep. 15, 2015. (Year: 2015).
- Lomotey, Richard K., and Ralph Deters. “Reliable consumption of web services in a mobile-cloud ecosystem using 1 REST.” In 2013 IEEE Seventh International Symposium on Service-Oriented System Engineering, pp. 13-24. IEEE, 2013. (Year: 2013).
Type: Grant
Filed: Dec 6, 2023
Date of Patent: Oct 20, 2026
Patent Publication Number: 20250190130
Assignee: SAP SE (Walldorf)
Inventors: Sahana Durgam Udaya (Bangalore), Suneel Sristi (Bangalore)
Primary Examiner: Gregory A Kessler
Application Number: 18/531,289