Abstract: Development of machine learning models and applications tends to be iterative and complex, made even harder because most of the necessary tools are not built for the entire machine learning lifecycle. Introduced here is a data platform that is able to accelerate time-to-value by enabling users to utilize applied machine learning prototypes (“AMPs”) made by others. These AMPs may be extendable, by the data platform, to new datasets, allowing machine learning to be developed and deployed more rapidly.

Abstract: A machine has a bus and a network interface circuit to receive different data streams from a network. The network interface circuit is connected to the network and the bus. A processor is connected to the bus. A memory is connected to the bus. The memory stores instructions executed by the processor to continuously increment aggregate functions associated with data parameters within the different data streams. Visualizations of the different data streams are periodically updated on different client devices connected to the network.

Abstract: A system has a distributed database with database partitions distributed across worker nodes connected by a network. An analytical view recommendation engine defines an analytical view comprising attributes and measures defined prior to the receipt of a query. The analytical view is maintained as a data unit separate from the distributed database. The analytical view recommendation engine includes instructions executed by a processor to identify a poorly performing report, evaluate queries associated with the poorly performing report, create a recommended analytical view to enhance the performance of the poorly performing report, and deploy the recommended analytical view.

Abstract: A system includes a distributed data storage system disseminated across worker machines connected by a network. A distributed data storage management module has instructions executed by a processor to utilize data block identifiers to track data block accesses to the distributed data storage system. A sampling module with instructions executed by the processor receives a new sample request from a client machine connected to the network. Initial data block samples are gathered from the distributed data storage system during a first time period. A revised sample request is received from the client machine during the first time period. The initial data block samples are gathered. New data block samples are collected from the distributed data storage system. The initial data block samples and the new data block samples are combined to form cumulative data block sample results. The cumulative data block sample results are supplied to the client machine.

Abstract: A computer readable storage medium with executable instructions specifies the execution of a state machine operating across a set of computing nodes in a distributed computing system. The executable instructions execute a set of operators, where the execution of each operator is under the control of a state machine that periodically invokes pause control states to pause the execution of an operator in response to a violation of a service level agreement specifying an operating condition threshold within the distributed computing system. Partitions of input data are formed that are worked on independently within the distributed computing system. A set of data batches associated with the input data is processed. Data partition control states to process the partitions associated with the set of data batches are specified. Key control states to process a set of keys associated with a data partition of the partitions are defined.

Abstract: A computer readable storage medium includes executable instructions to receive a request for data from a multi-dimensional cube divided into partitions distributed across nodes supporting a shared nothing distributed multi-dimensional database. The request is transformed into physical access layer operators in a tree structure. The tree structure is reduced into an index scan operator.

Abstract: A computer readable storage medium includes executable instructions to define an interface to a multi-dimensional cube divided into partitions distributed across nodes supporting a shared nothing multi-dimensional distributed database. The interface includes a metadata module characterizing access paths to data within the multi-dimensional cube and an access path module to convert a data request into an access path protocol to retrieve data from the multi-dimensional cube to satisfy the data request.

Abstract: A computer readable storage medium with executable instructions specifies the execution of a state machine operating across a set of computing nodes in a distributed computing system. The executable instructions execute a set of operators, where the execution of each operator is under the control of a state machine that periodically invokes pause control states to pause the execution of an operator in response to a violation of a service level agreement specifying an operating condition threshold within the distributed computing system. Partitions of input data are formed that are worked on independently within the distributed computing system. A set of data batches associated with the input data is processed. Data partition control states to process the partitions associated with the set of data batches are specified. Key control states to process a set of keys associated with a data partition of the partitions are defined.

Abstract: A computer readable storage medium includes executable instructions to receive a request for data from a multi-dimensional cube divided into partitions distributed across nodes supporting a shared nothing distributed multi-dimensional database. The request is transformed into physical access layer operators in a tree structure. The tree structure is reduced into an index scan operator.

Abstract: A computer readable storage medium with executable instructions specifies the execution of a state machine operating across a set of computing nodes in a distributed computing system. The executable instructions execute a set of operators, where the execution of each operator is under the control of a state machine that periodically invokes pause control states to pause the execution of an operator in response to a violation of a service level agreement specifying an operating condition threshold within the distributed computing system. Partitions of input data are formed that are worked on independently within the distributed computing system. A set of data batches associated with the input data is processed. Data partition control states to process the partitions associated with the set of data batches are specified. Key control states to process a set of keys associated with a data partition of the partitions are defined.

Abstract: A computer readable storage medium with executable instructions specifies the execution of a state machine operating across a set of computing nodes in a distributed computing system. The executable instructions execute a set of operators, where the execution of each operator is under the control of a state machine that periodically invokes pause control states to pause the execution of an operator in response to a violation of a service level agreement specifying an operating condition threshold within the distributed computing system. Partitions of input data are formed that are worked on independently within the distributed computing system. A set of data batches associated with the input data is processed. Data partition control states to process the partitions associated with the set of data batches are specified. Key control states to process a set of keys associated with a data partition of the partitions are defined.

Abstract: A distributed computing system includes a memory storing execution state collected prior to an operator pause state. A controller modifies, during the operator pause state, characteristics of the distributed computing system. The controller invokes execution of the operator after the pause state such that the operator accesses the execution state to complete an operation.

Abstract: A method for reserving a shared volume in a multiple node data storage system is disclosed. The method ensures that only one host at a time has access to the shared volume. As a result, the data/information stored in the shared volume cannot become corrupted due to multiple hosts accessing the shared volume at the same time through different nodes.

Abstract: A computer readable storage medium with executable instructions specifies the execution of a state machine operating across a set of computing nodes in a distributed computing system. The executable instructions execute a set of operators, where the execution of each operator is under the control of a state machine that periodically invokes pause control states to pause the execution of an operator in response to a violation of a service level agreement specifying an operating condition threshold within the distributed computing system. Partitions of input data are formed that are worked on independently within the distributed computing system. A set of data batches associated with the input data is processed. Data partition control states to process the partitions associated with the set of data batches are specified. Key control states to process a set of keys associated with a data partition of the partitions are defined.

Abstract: A method is provided for a data storage system to move data from a source logical disk (LD) region to a target LD region while the data storage system remains online to a host. The method includes determining if a region move will create excessive load so the data storage system appears offline to the host. If not, the method includes causing writes to the source LD region to be mirrored to the target LD region, causing data in the source LD region to be copied to the target LD region, blocking reads and writes to the data storage system, and flushing dirty cache in the data storage system. If flushing the dirty cache is fast so the data storage system appears online to the host, the method includes updating mappings of the virtual volume to the LD regions and resuming the reads and writes to the data storage system.

Abstract: A method is provided for a data storage system to change the RAID type, the layout characteristics, and the performance characteristics of a virtual volume mapped to logical disk regions in one or more logical disks while the data storage system remains online to a host. Another method is provided for a data storage system to consolidate space in one or more logical disks mapped to a virtual volume while the data storage system remains online to a host. The one or more logical disks can be consolidated to free unused chunklet regions for use in other logical disks.

Abstract: One embodiment of the present invention provides a distributed file system that is able to use direct connections between clients and disks to perform file system operations. Upon receiving a request at a client to access a file, the client performs a lookup in a local cache to determine what physical disk blocks are associated with the request. If the lookup cannot be satisfied from the local cache, the client forwards the request to a server. In response to the forwarded request, the client receives a block map for the file from the server. This block map includes location information specifying physical disk blocks containing the file. The client uses this block map to determine which physical disk blocks are involved in the request and then accesses the file directly from the disk without going through the server.

Abstract: A method is provided for a data storage system to change the RAID type, the layout characteristics, and the performance characteristics of a virtual volume mapped to logical disk regions in one or more logical disks while the data storage system remains online to a host. Another method is provided for a data storage system to consolidate space in one or more logical disks mapped to a virtual volume while the data storage system remains online to a host. The one or more logical disks can be consolidated to free unused chunklet regions for use in other logical disks.

Abstract: A method is provided for a data storage system to move data from a source logical disk (LD) region to a target LD region while the data storage system remains online to a host. The method includes determining if a region move will create excessive load so the data storage system appears offline to the host. If not, the method includes causing writes to the source LD region to be mirrored to the target LD region, causing data in the source LD region to be copied to the target LD region, blocking reads and writes to the data storage system, and flushing dirty cache in the data storage system. If flushing the dirty cache is fast so the data storage system appears online to the host, the method includes updating mappings of the virtual volume to the LD regions and resuming the reads and writes to the data storage system.

Abstract: One embodiment of the present invention provides a distributed file system that is able to use direct connections between clients and disks to perform file system operations. Upon receiving a request at a client to access a file, the client performs a lookup in a local cache to determine what physical disk blocks are associated with the request. If the lookup cannot be satisfied from the local cache, the client forwards the request to a server. In response to the forwarded request, the client receives a block map for the file from the server. This block map includes location information specifying physical disk blocks containing the file. The client uses this block map to determine which physical disk blocks are involved in the request and then accesses the file directly from the disk without going through the server.