Abstract: Data processing approaches are disclosed that include receiving a configuration indicating a plurality of parameters for performing a data processing job, identifying available compute resources from a plurality of public cloud infrastructures, where each public cloud infrastructure of the plurality of public cloud infrastructures supports one or more computing applications, one or more job schedulers, and one or more utilization rates, selecting one or more compute clusters from one or more of the plurality of public cloud infrastructures based on a matching process between the parameters for performing the data processing job and a combination of the one or more computing applications, the one or more job schedulers, and the one or more utilization rates, and initiating the one or more compute clusters for processing the data processing job based on the selecting.

Abstract: A method that includes receiving a first configuration and a second configuration that define a set of rules for matching and merging a set of source data objects that are associated with a tenant and that are received from a plurality of data sources. The method may further include generating a set of merged data objects from the set of source data objects based on an identification of matching values from fields of the set of source data objects and selecting a value for each field of each merged data object having multiple values. The method may further include generating a mapping between primary keys associated with each merged data object and corresponding primary keys of the source data objects. The method may further include storing the merged data objects and the mappings in a first datastore and a second datastore that is different from the first datastore.

Abstract: A multi-tenant system processes tenant data stored in storage systems of the multi-tenant system as well as external data sources. The multi-tenant system creates objects including objects based on tenant data stored in the storage system of the multi-tenant system and objects based on data stored in the external data sources. The multi-tenant system provides access to the objects to an external system distinct from the multi-tenant system. The multi-tenant system receives requests to perform operations on objects from the external system. An operation performed on an object based on data stored in external data source results in one or more of accessing data required for the operation from the external data source or storing data modified by the operation in the external data source.

Abstract: A relational database system may receive from a client a query that is supported by the relational database system, the relational database system being configured to store a plurality of data objects such that each data object is associated with a respective data table of a plurality of data tables. The system may determine that the query is indicative of a data object type that is associated with data stored in a data system separate from the relational database system. The system may identify a schema of the data object type using a schema record maintained by the relational database system, transmit to the separate data system a request for data associated with the query, receive requested data, and return a query response including the requested data.

Abstract: A method that includes receiving a first configuration and a second configuration that define a set of rules for matching and merging a set of source data objects that are associated with a tenant and that are received from a plurality of data sources. The method may further include generating a set of merged data objects from the set of source data objects based on an identification of matching values from fields of the set of source data objects and selecting a value for each field of each merged data object having multiple values. The method may further include generating a mapping between primary keys associated with each merged data object and corresponding primary keys of the source data objects. The method may further include storing the merged data objects and the mappings in a first datastore and a second datastore that is different from the first datastore.

Abstract: A relational database system may receive, from a client, a query for a data object of a first type. The system may be configured to store a plurality of data objects of a second type such that each data object of the second type is associated with a respective data table and a respective value of a key prefix of an object identifier field. The key prefix may result in an upper limit of data objects supportable by the system. The system may determine that the query is indicative of the data object of the first type of a plurality of data objects of the first type that share a key prefix value such that the upper limit is exceeded. The system may identify a result based on determining that the query indicates data object of the first type and return a query response that indicates the result.

Abstract: A relational database system may receive from a client a query that is supported by the relational database system, the relational database system being configured to store a plurality of data objects such that each data object is associated with a respective data table of a plurality of data tables. The system may determine that the query is indicative of a data object type that is associated with data stored in a data system separate from the relational database system. The system may identify a schema of the data object type using a schema record maintained by the relational database system, transmit to the separate data system a request for data associated with the query, receive requested data, and return a query response including the requested data.

Abstract: A relational database system may receive, from a client, a query for a data object of a first type. The system may be configured to store a plurality of data objects of a second type such that each data object of the second type is associated with a respective data table and a respective value of a key prefix of an object identifier field. The key prefix may result in an upper limit of data objects supportable by the system. The system may determine that the query is indicative of the data object of the first type of a plurality of data objects of the first type that share a key prefix value such that the upper limit is exceeded. The system may identify a result based on determining that the query indicates data object of the first type and return a query response that indicates the result.

Abstract: Data processing approaches are disclosed that include receiving a configuration indicating a plurality of parameters for performing a data processing job, identifying available compute resources from a plurality of public cloud infrastructures, where each public cloud infrastructure of the plurality of public cloud infrastructures supports one or more computing applications, one or more job schedulers, and one or more utilization rates, selecting one or more compute clusters from one or more of the plurality of public cloud infrastructures based on a matching process between the parameters for performing the data processing job and a combination of the one or more computing applications, the one or more job schedulers, and the one or more utilization rates, and initiating the one or more compute clusters for processing the data processing job based on the selecting.

Abstract: A method that includes receiving a first configuration and a second configuration that define a set of rules for matching and merging a set of source data objects that are associated with a tenant and that are received from a plurality of data sources. The method may further include generating a set of merged data objects from the set of source data objects based on an identification of matching values from fields of the set of source data objects and selecting a value for each field of each merged data object having multiple values. The method may further include generating a mapping between primary keys associated with each merged data object and corresponding primary keys of the source data objects. The method may further include storing the merged data objects and the mappings in a first datastore and a second datastore that is different from the first datastore.

Abstract: An application may support ingestion of data to a data server. A user may identify the data source, and the server may determine a connector corresponding to the identified data source. The server may support a plurality of connectors for data ingestion. Based on the selected source, the user may select a data channel type, where the data channel type is associated with one or more packaged data sets that include different data set types. The server may then ingest the data set using the determined connector. The application may further support mapping of source and target attributes, custom mapping, and transformation functions.

Abstract: Extensible architecture and data flow in a web based diagram visualization service is accomplished by enabling a diagram update engine to launch custom data modules. Data modules for data sources that are not inherently supported are created by third parties through a specially defined interface (e.g. .NET® assemblies) and made visible for the web based diagram services and callable from a diagram update engine. Thus, a number of data sources from which diagram data can be retrieved and the way incoming data is manipulated and aggregated is extensible through code.

Abstract: Technologies are described herein for refreshing data-linked diagrams on a server computer and viewing and consuming the refreshed diagrams via a Web browser. A drawing program allows equations within a diagram definition to define how external data is utilized to modify the attributes of a diagram element. When the diagram is published to a server computer, the definition is converted to server-legible definition. A published diagram is generated that includes a diagram representation defined by the server-legible definition. Upon a request for the published diagram, the external data is refreshed and the diagram definition is updated. The equations are recalculated to generate new element attributes. The diagram representation is then updated with the new attributes and returned for display by a client Web browser. An interface provides exploration tools and a client API exposes methods for surfacing external data and annotating the diagram.

Abstract: Extensible architecture and data flow in a web based diagram visualization service is accomplished by enabling a diagram update engine to launch custom data modules. Data modules for data sources that are not inherently supported are created by third parties through a specially defined interface (e.g. .NET® assemblies) and made visible for the web based diagram services and callable from a diagram update engine. Thus, a number of data sources from which diagram data can be retrieved and the way incoming data is manipulated and aggregated is extensible through code.

Abstract: Technologies are described herein for refreshing data-linked diagrams on a server computer and viewing and consuming the refreshed diagrams via a Web browser. A drawing program allows equations within a diagram definition to define how external data is utilized to modify the attributes of a diagram element. When the diagram is published to a server computer, the definition is converted to server-legible definition. A published diagram is generated that includes a diagram representation defined by the server-legible definition. Upon a request for the published diagram, the external data is refreshed and the diagram definition is updated. The equations are recalculated to generate new element attributes. The diagram representation is then updated with the new attributes and returned for display by a client Web browser. An interface provides exploration tools and a client API exposes methods for surfacing external data and annotating the diagram.