Abstract: A machine learning service implements programmatic interfaces for a variety of operations on several entity types, such as data sources, statistics, feature processing recipes, models, and aliases. A first request to perform an operation on an instance of a particular entity type is received, and a first job corresponding to the requested operation is inserted in a job queue. Prior to the completion of the first job, a second request to perform another operation is received, where the second operation depends on a result of the operation represented by the first job. A second job, indicating a dependency on the first job, is stored in the job queue. The second job is initiated when the first job completes.

Abstract: A system that maps files into multiple parts and initiates storage of each part within a respective storage provider. The mapping facilitates this by correlating each of the multiple parts to a corresponding storage provider which may be different than the storage provider that stores a different part of the same file. The system then initiates storage of the multiple parts of the file in each of their corresponding storage provider in accordance with the mapping, and using a storage-provider-facing API. Similarly, retrieval and/or editing of a partial file may be accomplished by referencing the mapping, and using the same API issued to the appropriate storage provider. Each storage provider is capable of interfacing with the system using this same API.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Abstract: Mechanisms to modify a dataflow execution graph that processes a data stream. An intermediate dataflow execution graph is used during modification of the dataflow execution graph from one configuration (the old dataflow execution graph) to the next (the new dataflow execution graph). Data messages of the data stream may continue to feed into the intermediate dataflow execution graph, thereby reducing latency and maintaining throughput during reconfiguration of the dataflow execution graph. Control message(s) that are structured to accomplish the reconfiguration is/are also passed into the intermediate dataflow execution graph during reconfiguration. As the control message(s) are all processed by the intermediate dataflow execution graph, the intermediate dataflow execution graph assumes the topology of the new dataflow execution graph.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Abstract: A system that maps files into multiple parts and initiates storage of each part within a respective storage provider. The mapping facilitates this by correlating each of the multiple parts to a corresponding storage provider which may be different than the storage provider that stores a different part of the same file. The system then initiates storage of the multiple parts of the file in each of their corresponding storage provider in accordance with the mapping, and using a storage-provider-facing API. Similarly, retrieval and/or editing of a partial file may be accomplished by referencing the mapping, and using the same API issued to the appropriate storage provider. Each storage provider is capable of interfacing with the system using this same API.

Abstract: Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Abstract: Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Abstract: Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Abstract: Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Abstract: Various methods and apparatuses are provided which may be implemented using one or more computing devices within a networked computing environment to support a computing grid having selective storage of shared data files within certain distributed the systems provided by dusters of computing devices. The selective storage may represent limited duplicative storage of a shared file.

Abstract: A multi-stage technique invalidates and replaces loadable physical volume block numbers (pvbns) stored in indirect blocks of a dual vbn (“flexible”) virtual volume (vvol) of a storage system to enable efficient image transfers and/or fragmentation handling of the flexible vvol. Each loadable pvbn of a pvbn/virtual vbn (vvbn) block pointer pair is converted into a special block pointer having a predefined reserved value that provides a temporary “pvbn_unknown” placeholder until replaced by a real (actual) pvbn. The technique further allows the storage system to serve data from the flexible vvol using the placeholders while the actual pvbns are computed, thereby eliminating latencies associated with completion of actual pvbn replacement for the pvbn_unknown placeholders.

Abstract: A file system migrates a traditional volume to a virtual volume without data copying. In an embodiment, a traditional volume index node is selected for migration. The traditional volume index node is converted to a virtual volume index node. In one embodiment, the virtual volume index node provides both physical address information and virtual address information.

Abstract: A method and apparatus for reducing performance overhead of a media server sending packetized audio/video data to an end-player or user is provided. The main sources of performance overhead associated with a media server transmitting packetized audio/video data are (1) copying data from the user space buffer to the kernel buffer and (2) context switching from user level to kernel level. There are several techniques that can be used to address overhead. One technique involves creating an aggregate packet, which includes a policy, at the user level and transmitting this aggregate packet from the user level to the OS level with a system call. A second technique involves manipulating packets based on a policy in the OS level rather than in the user level. The manipulation of packets may include redirecting packets or splitting packets.

Abstract: A file system migrates a traditional volume to a virtual volume without data copying. In an embodiment, a traditional volume index node is selected for migration. The traditional volume index node is converted to a virtual volume index node. In one embodiment, the virtual volume index node provides both physical address information and virtual address information.

Abstract: A method and apparatus for reducing performance overhead of a media server sending packetized audio/video data to an end-player or user is provided. The main sources of performance overhead associated with a media server transmitting packetized audio/video data are (1) copying data from the user space buffer to the kernel buffer and (2) context switching from user level to kernel level. There are several techniques that can be used to address overhead. One technique involves creating an aggregate packet, which includes a policy, at the user level and transmitting this aggregate packet from the user level to the OS level with a system call. A second technique involves manipulating packets based on a policy in the OS level rather than in the user level. The manipulation of packets may include redirecting packets or splitting packets.