Abstract: Methods, systems and computer program products are provided for managing protection domains (PDs) for files at a file-level or a page-level. PDs may be allocated for multiple purposes, e.g., to protect processes, files, buffers, etc. Files stored in nonvolatile memory (NVM) subject to direct access (DAX) may be protected by file-level or page-level PDs. PDs may comprise protection keys (PKEYs) with user-configurable read and write access control registers (PKRUs). NVM files may be protected from corruption (e.g. by stray writes) by leaving write access disabled except for temporary windows of time for valid writes. File PDs may be managed by a file manager while buffer PDs may be managed by a buffer pool manager. File associations between PDs, files and file address space may be maintained in a file object. Buffer associations between PDs, buffers and buffer address space may be maintained in a buffer descriptor.

Abstract: Described herein is a system and method of performing flow control for multi-threaded access to contentious resource(s) (e.g., shared memory). A request to enter a critical section of code by a particular thread of a plurality of concurrent threads is received. A determination is made as to whether or not to allow the particular thread to enter the critical section of code based, at least in part, upon a CPU core associated with the particular thread, a state associated with the particular thread, and/or a processing rate in the critical session of code associated with the particular thread. When it is determined to allow the particular thread to enter the critical section of code, the particular thread is allowed to enter the critical section of code.

Abstract: Methods, systems and computer program products provide protection domains for processes in shared address space. Multiple processes may share address space, for example, in a software isolated process running on top of a library operating system (OS). A protection domain (PD), such as a Protection Key (PKEY), may be assigned to a process to protect its allocated address spaces from access by other processes. PDs may be acquired from a host OS. A library OS may manage PDs to protect processes and/or data. A PD may be freed and reassigned to a different process or may be concurrently assigned to multiple processes, for example, when the number of processes exceeds the number of protection domains. Threads spawned by a process may inherit protection provided by a PD assigned to the process. Process PDs may be disassociated with address spaces as they are deallocated for a process or its threads.

Abstract: Performing a distributed query across a data pool includes receiving a database query at a master node or a compute pool within a database system. Based on receiving the database query, a data pool within the database system is identified. The data pool comprises a plurality of data nodes. Each data node includes a relational engine and relational storage. Each node in the data pool caches a different partition of data from an external data source in its relational storage. The database query is processed across the plurality of data nodes. Query processing includes requesting that data node perform a filter operation against its cached partition of the external data source stored in its relational storage and return any data from the partition that matches the filter operation.

Abstract: Methods, systems and computer program products provide protection domains for processes in shared address space. Multiple processes may share address space, for example, in a software isolated process running on top of a library operating system (OS). A protection domain (PD), such as a Protection Key (PKEY), may be assigned to a process to protect its allocated address spaces from access by other processes. PDs may be acquired from a host OS. A library OS may manage PDs to protect processes and/or data. A PD may be freed and reassigned to a different process or may be concurrently assigned to multiple processes, for example, when the number of processes exceeds the number of protection domains. Threads spawned by a process may inherit protection provided by a PD assigned to the process. Process PDs may be disassociated with address spaces as they are deallocated for a process or its threads.

Abstract: Methods, systems and computer program products are provided for managing protection domains (PDs) for files at a file-level or a page-level. PDs may be allocated for multiple purposes, e.g., to protect processes, files, buffers, etc. Files stored in nonvolatile memory (NVM) subject to direct access (DAX) may be protected by file-level or page-level PDs. PDs may comprise protection keys (PKEYs) with user-configurable read and write access control registers (PKRUs). NVM files may be protected from corruption (e.g. by stray writes) by leaving write access disabled except for temporary windows of time for valid writes. File PDs may be managed by a file manager while buffer PDs may be managed by a buffer pool manager. File associations between PDs, files and file address space may be maintained in a file object. Buffer associations between PDs, buffers and buffer address space may be maintained in a buffer descriptor.

Abstract: Methods for differencing and conversion of software application containers are performed by systems and devices. An initiation instruction that identifies an application container image associated with an operating system is received, and a request for the container image is provided to a repository. The container image is received and a converter extracts a portion of the container image and generates another container image of the application associated with a different operating system and that includes the extracted container image portion. Difference operations are performed on layers of different container images to isolate base layers, and application artifacts are identified for conversions in layers above the operating system. Similarly, container images are converted on the basis of processor instruction sets via binary translations, and applications native to different operating systems are generated based on original container images.

Abstract: Methods for differencing and conversion of software application containers are performed by systems and devices. An initiation instruction that identifies an application container image associated with an operating system is received, and a request for the container image is provided to a repository. The container image is received and a converter extracts a portion of the container image and generates another container image of the application associated with a different operating system and that includes the extracted container image portion. Difference operations are performed on layers of different container images to isolate base layers, and application artifacts are identified for conversions in layers above the operating system. Similarly, container images are converted on the basis of processor instruction sets via binary translations, and applications native to different operating systems are generated based on original container images.

Abstract: Described herein is a system and method of performing flow control for multi-threaded access to contentious resource(s) (e.g., shared memory). A request to enter a critical section of code by a particular thread of a plurality of concurrent threads is received. A determination is made as to whether or not to allow the particular thread to enter the critical section of code based, at least in part, upon a CPU core associated with the particular thread, a state associated with the particular thread, and/or a processing rate in the critical session of code associated with the particular thread. When it is determined to allow the particular thread to enter the critical section of code, the particular thread is allowed to enter the critical section of code.

Abstract: Automatically provisioning resources within a database system includes receiving, at a master service of the database system, a declarative statement for performing a database operation. Based on receiving the declarative statement, a control plane is instructed that additional hardware resources are needed for performing the database operation. Based on instructing the control plane, a provisioning fabric provisions computer system hardware resources for one or more of (i) a storage pool that includes at least one storage node that comprises a first database engine, a big data engine, and big data storage; (ii) a data pool that includes at least one data node that comprises a second database engine and database storage; or (iii) a compute pool that includes a compute node that comprises a compute engine that processes queries at one or both of the storage pool or the data pool.

Abstract: Processing a database query over a combination of relational and big data includes receiving the database query at a master node or a compute node within a database system. Based on receiving the database query, a storage pool within the database system is identified. The storage pool comprises a plurality of storage nodes, each storage node including a relational engine, a big data engine, and big data storage. The database query is processed over a combination of relational data stored within the database system and big data stored at the big data storage of at least one of the plurality of storage nodes. The relational data could be stored at the master node and/or at one or more data nodes. An artificial intelligence model and/or machine learning model might also be trained and/or scored using a combination of relational data and big data.

Abstract: Performing a distributed query across a storage pool includes receiving a database query at a master node or a compute pool within a database system. Based on receiving the database query, a storage pool within the database system is identified. The storage pool comprises a plurality of storage nodes. Each storage node includes a relational engine, a big data engine, and big data storage. The storage pool stores at least a portion of a data set using the plurality of storage nodes by storing a different partition of the data set within the big data storage at each storage node. The database query is processed across the plurality of storage nodes. Query processing includes requesting that each storage node perform a query operation against the partition of the data set stored in its big data storage and return any data from the partition that is produced by the query operation.

Abstract: Performing a distributed query across a data pool includes receiving a database query at a master node or a compute pool within a database system. Based on receiving the database query, a data pool within the database system is identified. The data pool comprises a plurality of data nodes. Each data node includes a relational engine and relational storage. Each node in the data pool caches a different partition of data from an external data source in its relational storage. The database query is processed across the plurality of data nodes. Query processing includes requesting that data node perform a filter operation against its cached partition of the external data source stored in its relational storage and return any data from the partition that matches the filter operation.

Abstract: Implementing a safe driver that can support high throughput and low latency devices. The method includes receiving a hardware message from a hardware device. The method further includes delivering the hardware message to one or more driver processes executing in user mode using a zero-copy to allow the one or more driver processes to support high throughput and low latency hardware devices.

Abstract: The present invention extends to methods, systems, and computer program products for protected mode scheduling of operations. Protected mode (e.g., user mode) scheduling can facilitate the development of programming frameworks that better reflect the requirements of the workloads through the use of workload-specific execution abstractions. In addition, the ability to define scheduling policies tuned to the characteristics of the hardware resources available and the workload requirements has the potential of better system scaling characteristics. Further, protected mode scheduling decentralizes the scheduling responsibility by moving significant portions of scheduling functionality from supervisor mode (e.g., kernel mode) to an application.

Abstract: Implementations of various technologies for compiling a query. In one implementation, the memory consumption of the query may be monitored. If the amount of memory consumed by the query exceeds one or more gateway thresholds, then the query may be processed to acquire one or more gateways. The query may then be compiled as having acquired the one or more gateways.

Abstract: The present invention extends to methods, systems, and computer program products for protected mode scheduling of operations. Protected mode (e.g., user mode) scheduling can facilitate the development of programming frameworks that better reflect the requirements of the workloads through the use of workload-specific execution abstractions. In addition, the ability to define scheduling policies tuned to the characteristics of the hardware resources available and the workload requirements has the potential of better system scaling characteristics. Further, protected mode scheduling decentralizes the scheduling responsibility by moving significant portions of scheduling functionality from supervisor mode (e.g., kernel mode) to an application.

Abstract: Scheduling system resources. A system resource scheduling policy for scheduling operations within a workload is accessed. The policy is specified on a workload basis such that the policy is specific to the workload. System resources are reserved for the workload as specified by the policy. Reservations may be hierarchical in nature where workloads are also hierarchically arranged. Further, dispatching mechanisms for dispatching workloads to system resources may be implemented independent from policies. Feedback regarding system resource use may be used to determine policy selection for controlling dispatch mechanisms.

Abstract: Caching architecture that facilitates translation between schema data and relational structures. A schema translation component consumes schema data (e.g., XML schema data) having a schema structure. The schema structure is shredded into tables. A validation component interfaces to a memory management interface to facilitate loading only the necessary components to perform instance validation. During validation, only parts of the schema that are used are loaded and cached. A schema cache stores the in-memory representation of the schema optimized for instance validation. The schema components are loaded from metadata into the cache memory as read-only objects such that multiple users can use the in-memory objects for validation.

Abstract: Implementations of various technologies for compiling a query. In one implementation, the memory consumption of the query may be monitored. If the amount of memory consumed by the query exceeds one or more gateway thresholds, then the query may be processed to acquire one or more gateways. The query may then be compiled as having acquired the one or more gateways.