Abstract: In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user's infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.

Abstract: A data integration system is disclosed that incorporates one or more techniques for simplifying the design and maintenance of a mapping. As components are added or removed to an existing design, the data integration system removes the need to specify all input and output attributes. In one aspect, components types are implemented that allow assignment expressions to reference all or part of upstream components. Therefore, attributes of certain types of components can be propagated to downstream components or otherwise inherited from upstream components with minimal effort on the part of a map designer. During code generation the attributes required to be projected by any component can be derived based on the needs of the downstream components.

Abstract: In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user's infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.

Abstract: A data integration system is disclosed that incorporates one or more techniques for simplifying the design and maintenance of a mapping. As components are added or removed to an existing design, the data integration system removes the need to specify all input and output attributes. In one aspect, components types are implemented that allow assignment expressions to reference all or part of upstream components. Therefore, attributes of certain types of components can be propagated to downstream components or otherwise inherited from upstream components with minimal effort on the part of a map designer. During code generation the attributes required to be projected by any component can be derived based on the needs of the downstream components.

Abstract: One embodiment of the present invention provides a system that supports cross-component references in an object-oriented programming system. During operation, the system makes a first attempt to obtain a name for a referenced object through a cross-component reference between a referencing object and the referenced object. This first attempt involves attempting to obtain the name for the referenced object from the referenced object, which involves attempting access the referenced object through an identifier for the referenced object within the cross-component reference. If this first attempt fails, the system makes a second attempt to obtain the name for the referenced object, which involves obtaining a pathname for the referenced object from the cross-component reference.