Abstract: A system and method are provided for managing creation of data objects, for example in a network or big data environment. A method comprises: receiving, at a processor, a data object creation command for creating the data object; comparing, at the processor, at least one data object creation parameter with a stored data object creation standard; and initiating creation of the data object in response to the data object creation parameter matching a corresponding parameter in the stored data object creation standard. The data object creation command can be modified to meet the stored data object creation standard, and to change an associated permission level. Embodiments of the present disclosure encapsulate standards defining how to create a data object, such that it is no longer necessary to have a person to make sure that these standards are followed, and automate management of object creation while ensuring conformity to organizational standards.

Abstract: A system and method are provided for coordinating data ingestion and workflow. In an implementation, the method includes: obtaining, at a processor, a plurality of data ingestion jobs; identifying, based on a stored batching factor, a subset of the plurality of data ingestion jobs to be grouped together; performing batch processing of the subset of data ingestion jobs together in a single shell action; and creating a workflow schedule based on the single shell action comprising the batched data ingestion jobs. The present disclosure advantageously provides batch processing of data ingestion jobs themselves, in contrast to existing approaches which may use data ingestion jobs to perform batch processing on underlying data. The data ingestion jobs can be Sqoop jobs, or in other formats or using other approaches such as through Kafka, Flume or Spark.

Abstract: A system and method are provided for managing creation of data objects, for example in a network or big data environment. A method comprises: receiving, at a processor, a data object creation command for creating the data object; comparing, at the processor, at least one data object creation parameter with a stored data object creation standard; and initiating creation of the data object in response to the data object creation parameter matching a corresponding parameter in the stored data object creation standard. The data object creation command can be modified to meet the stored data object creation standard, and to change an associated permission level. Embodiments of the present disclosure encapsulate standards defining how to create a data object, such that it is no longer necessary to have a person to make sure that these standards are followed, and automate management of object creation while ensuring conformity to organizational standards.