Abstract: A graphical mapping interface and method are provided for creating and displaying a schema map, which is used by a data transformation system to perform a data transformation between a primary source schema and at least one target schema. The graphical mapping interface includes a main map window and a mini-map window. The main map window is adapted for displaying a graphical representation of a primary source schema. The mini-map window is adapted for creating a mapping between one or more nodes of the primary source schema and one or more nodes of an intermediate target schema, and displaying a graphical representation of the mapping within the mini-map window. Upon closing the mini-map window, a graphical representation of the intermediate target schema is displayed along with the primary source schema within a source schema region of the main map window and is accessible for mapping within the main map window.

Abstract: Embodiments of a graphical mapping interface and method are provided herein for creating and displaying a schema map, which may be used by a data transformation system to perform a data transformation between at least one source schema and at least one target schema. According to one embodiment, the graphical mapping interface may generally comprise a main map window and a mini-map window. The main map window comprises a source schema region, which is adapted for displaying a graphical representation of a primary source schema defining a structure of a primary data source. The mini-map window is adapted for creating a mapping between one or more nodes of the primary source schema and one or more nodes of an intermediate target schema, and displaying a graphical representation of the mapping within the mini-map window.

Abstract: Graphical mapping interface embodiments and method are provided for creating and displaying a schema map, which may be used by a data transformation system to perform a data transformation between at least one source schema and at least one target schema. According to one embodiment, the graphical mapping interface may comprise a source schema region for displaying a graphical representation of at least one source schema, a target schema region for displaying a graphical representation of at least one target schema, and a mapping region for displaying graphical representations of a plurality of links connecting the source nodes displayed in the source schema region to the target nodes displayed in the target schema region. The plurality of links may comprise at least one control link having at least one ECA rule associated therewith and at least one data link having at least one textual-based target field expression associated therewith.

Abstract: Graphical mapping interface embodiments and method are provided for creating and displaying a schema map, which may be used by a data transformation system to perform a data transformation between at least one source schema and at least one target schema. According to one embodiment, the graphical mapping interface may comprise a source schema region for displaying a graphical representation of at least one source schema, a target schema region for displaying a graphical representation of at least one target schema, and a mapping region for displaying graphical representations of a plurality of links connecting the source nodes displayed in the source schema region to the target nodes displayed in the target schema region. The plurality of links may comprise at least one control link having at least one ECA rule associated therewith and at least one data link having at least one textual-based target field expression associated therewith.

Abstract: The execution environment provides for scalability where components will execute in parallel and exploit various patterns of parallelism. Dataflow applications are represented by reusable dataflow graphs called map components, while the executable version is called a prepared map. Using runtime properties the prepared map is executed in parallel with a thread allocated to each map process. The execution environment not only monitors threads, detects and corrects deadlocks, logs and controls program exceptions, but also data input and output ports of the map components are processed in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions. Data tokens are batched to minimize synchronization and transportation overhead and thread contention.

Abstract: A system and method for managing data, such as in a data warehousing, analysis, or similar applications, where dataflow graphs are expressed as reusable map components, at least some of which are selected from a library of components, and map components are assembled to create an integrated dataflow application. Composite map components encapsulate a dataflow pattern using other maps as subcomponents. Ports are used as link points to assemble map components and are hierarchical and composite allowing ports to contain other ports. The dataflow application may be executed in a parallel processing environment by recognizing the linked data processes within the map components and assigning threads to the linked data processes.

Abstract: A system and method for managing data, such as in a data warehousing, analysis, or similar applications, where dataflow graphs are expressed as reusable map components, at least some of which are selected from a library of components, and map components are assembled to create an integrated dataflow application. Composite map components encapsulate a dataflow pattern using other maps as subcomponents. Ports are used as link points to assemble map components and are hierarchical and composite allowing ports to contain other ports. The dataflow application may be executed in a parallel processing environment by recognizing the linked data processes within the map components and assigning threads to the linked data processes.

Abstract: A system and method for managing data, such as in a data warehousing, analysis, or similar applications, where dataflow graphs are expressed as reusable map components, at least some of which are selected from a library of components, and map components are assembled to create an integrated dataflow application. Composite map components encapsulate a dataflow pattern using other maps as subcomponents. Ports are used as link points to assemble map components and are hierarchical and composite allowing ports to contain other ports. The dataflow application may be executed in a parallel processing environment by recognizing the linked data processes within the map components and assigning threads to the linked data processes.

Abstract: The execution environment provides for scalability where components will execute in parallel and exploit various patterns of parallelism. Dataflow applications are represented by reusable dataflow graphs called map components, while the executable version is called a prepared map. Using runtime properties the prepared map is executed in parallel with a thread allocated to each map process. The execution environment not only monitors threads, detects and corrects deadlocks, logs and controls program exceptions, but also data input and output ports of the map components are processed in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions. Data tokens are batched to minimize synchronization and transportation overhead and thread contention.

Abstract: The execution environment provides for scalability where components will execute in parallel and exploit various patterns of parallelism. Dataflow applications are represented by reusable dataflow graphs called map components, while the executable version is called a prepared map. Using runtime properties the prepared map is executed in parallel with a thread allocated to each map process. The execution environment not only monitors threads, detects and corrects deadlocks, logs and controls program exceptions, but also data input and output ports of the map components are processed in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions. Data tokens are batched to minimize synchronization and transportation overhead and thread contention.

Abstract: An environment and method for developing dataflow applications where a dataflow transformation is encapsulated into a dataflow graph expressed as a reusable map component using a host development language such as Java®, and the dataflow graphs are linked together via ports. A scalar map component performs a specific data transformation and a composite map component is built from a number of other map components typically representing a family of data transformations. Map components have interface and implementation properties which assist in developing customized, hierarchical data transformation logic. The ports are polymorphic and can be specific, generic, or composite data types, to allow linking of map components which compile to generate a dataflow pattern. A map component may employ dynamic logic where internal sub-graphs or links are added or properties changed based on information received from the compiler, such as port types, design properties, or static data transformation.

Abstract: A system and method is described for event-driven data transformation. Generally, the system and method is directed to a transformation engine that iterates through one or more data sources, transforms data received from the data sources, and stores the output to one or more data targets. More specifically, the transformation engine is driven by executing specified event actions upon occurrence of specified triggering events. Thus, flexible, adaptable, highly tailored transformations can be implemented without incurring the often substantial expense of developing customized point-to-point solutions from scratch. The present invention supports one-to-one mappings, many-to-one mappings, one-to-many mappings, and many-to-many mappings. In addition, the present invention supports both hierarchical and flat data sources and targets.

Abstract: A system and method is described for event-driven data transformation. Generally, the system and method is directed to a transformation engine that iterates through one or more data sources, transforms data received from the data sources, and stores the output to one or more data targets. More specifically, the transformation engine is driven by executing specified event actions upon occurrence of specified triggering events. Thus, flexible, adaptable, highly tailored transformations can be implemented without incurring the often substantial expense of developing customized point-to-point solutions from scratch. The present invention supports one-to-one, mappings, many-to-one mappings, one-to-many mappings, and many-to-many mappings. In addition, the present invention supports both hierarchical and flat data sources and targets.