Abstract: A package manager used with a containerization platform can organize code portions into immutable layers. Collections of layers can be organized and saved together as an executable unit. Disclosed solutions recognize that because layers do not change, they can be reused by the same user and can also serve as shared building blocks for multiple environments running simultaneously. To facilitate sharing layers, a system can analyze which ones are common to multiple environments and allow multiple simultaneous environments to share common layers. Layer compression and dominator algorithms can be used to address inherent layer constraints. To facilitate use of existing layers for efficient start-up, code packages can be organized into base layers and additional layers, and commonly-used layers can be cached. A just-in-time approach can combine layers into new images on the fly and cache the new images for later use.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A package manager used with a containerization platform can organize code portions into immutable layers. Collections of layers can be organized and saved together as an executable unit. Disclosed solutions recognize that because layers do not change, they can be reused by the same user and can also serve as shared building blocks for multiple environments running simultaneously. To facilitate sharing layers, a system can analyze which ones are common to multiple environments and allow multiple simultaneous environments to share common layers. Layer compression and dominator algorithms can be used to address inherent layer constraints. To facilitate use of existing layers for efficient start-up, code packages can be organized into base layers and additional layers, and commonly-used layers can be cached. New or unique environment requests can combine previously-cached base layers and additional layers to avoid redundant calculations.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A package manager used with a containerization platform can organize code portions into immutable layers. Collections of layers can be organized and saved together as an executable unit. Disclosed solutions recognize that because layers do not change, they can be reused by the same user and can also serve as shared building blocks for multiple environments running simultaneously. To facilitate sharing layers, a system can analyze which ones are common to multiple environments and allow multiple simultaneous environments to share common layers. Layer compression and dominator algorithms can be used to address inherent layer constraints. To facilitate use of existing layers for efficient start-up, code packages can be organized into base layers and additional layers, and commonly-used layers can be cached. New or unique environment requests can combine previously-cached base layers and additional layers to avoid redundant calculations.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A package manager used with a containerization platform can organize code portions into immutable layers. Collections of layers can be organized and saved together as an executable unit. Disclosed solutions recognize that because layers do not change, they can be reused by the same user and can also serve as shared building blocks for multiple environments running simultaneously. To facilitate sharing layers, a system can analyze which ones are common to multiple environments and allow multiple simultaneous environments to share common layers. Layer compression and dominator algorithms can be used to address inherent layer constraints. To facilitate use of existing layers for efficient start-up, code packages can be organized into base layers and additional layers, and commonly-used layers can be cached. New or unique environment requests can combine previously-cached base layers and additional layers to avoid redundant calculations.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.

Abstract: A package manager used with a containerization platform can organize code portions into immutable layers. Collections of layers can be organized and saved together as an executable unit. Disclosed solutions recognize that because layers do not change, they can be reused by the same user and can also serve as shared building blocks for multiple environments running simultaneously. To facilitate sharing layers, a system can analyze which ones are common to multiple environments and allow multiple simultaneous environments to share common layers. Layer compression and dominator algorithms can be used to address inherent layer constraints. To facilitate use of existing layers for efficient start-up, code packages can be organized into base layers and additional layers, and commonly-used layers can be cached. New or unique environment requests can combine previously-cached base layers and additional layers to avoid redundant calculations.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.