Abstract: Systems, methods, and computer program products for distributed validation of credentials are described. Upon receiving a request to perform an action by a user, a system performs a multi-part authentication where in each part, only a portion of authentication information is passed. In a first stage, an application manager of the system receives a first token than specifies partial access rights. In a second stage, a cloud controller of the system requests and receives privileges of the user separately from the first token. An API is presented with a token that only contains the authorities that the API needs, while still allowing validation of cloud controller permissions without having to escalate the user's privileges.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for instrumentation and control of workloads in a massively parallel processing database. Deployment is in a cluster which mirrors the cluster of the database to be controlled. The system includes data publishing modules, action agents, rules processing modules, deployment managers, rule compilation and management tools. Together these provide a distributed, fault tolerant system for the automated rules-based control of work in a database cluster. For example, in deploying an update, a deployment manager pushes the update to one or more nodes and instructs each of the one or more nodes to restart in a bootstrap mode. The deployment manager generates a respective configuration package for each of the one or more nodes, and restarts each of the one or more nodes in a production mode.

Abstract: This specification describes methods, systems, and computer-readable media for sidecar-backed services for a cloud computing platform. An application is bound to a service. A transformer component of the cloud computing platform transforms credential information for accessing the service by replacing an access address to the service with a localhost address to a sidecar program associated with the application. When the application uses the service, the application communicates with the sidecar program rather than with the service directly. The sidecar program then forwards the communication to the service.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a task centric resource scheduling framework. A scheduler executing on a master node computer of a distributed parallel computing system allocates computing resources of the parallel computing system to a program according to one or more policies associated with the program. Each policy includes a set of pre-determined computing resource constraints. Allocation of the computing resources includes performing multiple iterations of negotiation between the scheduler and a resource manager of the parallel computing system. In each iteration, a policy engine of the scheduler submits requests to get more resources from, or requests to release already acquired resources to, the resource manager. The policy engine generates the requests by balancing suggestions provided by analyzer components of the policy engine and a corresponding policy.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a distributed parallel computing system to adapt a foreign program to execute on the distributed parallel computing system. The foreign program is a program written for a computing framework that is different from a computing framework of the parallel computing system. The distributed parallel computing system includes a master node computer and one or more worker node computers. A scheduler executing on the master node computer acts as an intermediary between the foreign program and the parallel computing system. The scheduler negotiates with a resource manager of the parallel computing system to acquire computing resources. The scheduler then allocates the computing resources to the worker node computers as containers. The foreign program executes in the containers on the worker node computers in parallel.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for stream processing. One method includes receiving an event stream of first events by a first plurality of first local modelers of a stream processing system. Each local modeler processes a portion of received events of the event stream according to a first set of operations, the operations including aggregating information associated with each event to generate first aggregated information. A second plurality of second local modelers similarly generates second aggregated information from an event stream of second events. First and second local modelers provide, to a first central modeler, first and second aggregated information. A set of parameters of a respective machine learning model is determined by the first central modeler using the received aggregated information.

Abstract: Methods, systems, and computer-readable media for on-demand resource provisioning for service instances. An on-demand service broker provisions IaaS resources at service instance creating time. The service broker provides a catalog listing one or more service plans, each service plan corresponds to a set of available resources. A user device selects a respective service plan that matches specific needs of an application that consumes the resources. The service broker generates a deployment manifest based on the selected service plan. The service broker then submits the deployment manifest to a deployment system. The deployment system provisions the resources to the service instance according to the manifest generated by the service broker.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for instrumentation and control of workloads in a massively parallel processing database. Deployment is in a cluster which mirrors the cluster of the database to be controlled. The system includes data publishing modules, action agents, rules processing modules, deployment managers, rule compilation and management tools. Together these provide a distributed, fault tolerant system for the automated rules-based control of work in a database cluster. A rule specifies an action on the system and a condition for performing the action, the condition including a value, one or more tags corresponding to the value, and a relation between the one or more tags and the value. The system triggers the action based on evaluating the condition.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying events in a stream computing system using hierarchical analytic models. One of the methods includes receiving, by a stream computing system, data representing the values of one or more data attributes of an event in a stream of events. The values of each of the one or more data attributes are evaluated according to respective attribute-specific and class-specific criteria of a hierarchical analytic model in a predetermined order defined by the model. When a first value of a first data attribute satisfies one or more particular criteria for a first class, the first class of the plurality of classes is assigned to the event.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for precisely tracking memory usage in a multi-process computing environment. One of the methods includes implementing an instance of a memory usage tracker (MUT) in each process running in a node of a computer system. A MUT can maintain an account of memory usage for each of multiple logical owners running on a process on which the MUT is running. The MUT can determine an actual memory quota for each owner, and enforce the actual memory quota of the owner. Enforcing the actual memory quota of the owner can include receiving each memory allocation request, checking each allocation request and a current state of the account against the actual quota, approving or rejecting each allocation request, communicating the approval or rejection to an underlying memory manager, and updating the owner account for each approved allocation request.

Abstract: Systems, methods, and computer program products for detecting network anomalies using node scoring. A network analysis system designates each server computer in a distributed computing network as a node in a graph. The network analysis system constructs a first graph of the distributed computing network for a training period. The system then determines a respective first node score of each node. The system constructs a second graph of the distributed computing network for a test time period. The system then reduces the second graph by removing those edges from the second graph that appear in both the first graph and the second graph. The system determines a respective second node score of each node in the reduced second graph. The system computes differences between the first node scores and the second node scores. The system designates nodes associated with the highest differences as anomalous nodes.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel time series analysis. One of the methods includes receiving a plurality of data elements. The plurality of data elements are partitioned into a plurality of chunks, wherein the plurality of chunks, including a first chunk and a plurality of additional chunks, have an ordering according to the data elements included in each chunk. Each chunk is assigned to a particular segment of a plurality of segments. A first iteration of an autoregressive integrated moving average is computed for each chunk assigned to each segment. A second iteration of the autoregressive integrated moving average is computed for each chunk assigned to each segment, wherein computing uses the result data for a corresponding preceding chunk in the first iteration. One or more additional iterations of the autoregressive integrated moving average are computed until stopping criteria has been satisfied.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic partition selection. One of the methods includes receiving a representation of a query plan generated for a query, wherein the query plan includes a dynamic scan operator that represents a first computing node obtaining tuples of one or more partitions of a table from storage and transferring the tuples to a second computing node that executes a parent operator of the dynamic scan operator. A partition selector operator is generated corresponding to the dynamic scan operator. A location in the query plan is determined for the partition selector operator. A modified query plan is generated having the partition selector operator at the determined location.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic partition selection. One of the methods includes receiving a representation of a query plan generated for a query, wherein the query plan includes a dynamic scan operator that represents a first computing node obtaining tuples of one or more partitions of a table from storage and transferring the tuples to a second computing node that executes a parent operator of the dynamic scan operator. A partition selector operator is generated corresponding to the dynamic scan operator. A location in the query plan is determined for the partition selector operator. A modified query plan is generated having the partition selector operator at the determined location.

Abstract: A random number generation process generated uncorrelated random numbers from identical random number sequences on parallel processing database segments of an MPP database without communications between the segments by establishing a different starting position in the sequence on each segment using an identifier that is unique to each segment, query slice information and the number of segments. A master node dispatches a seed value to initialize the random number sequence generation on all segments, and dispatches the query slice information and information as to the number of segments during a normal query plan dispatch process.

Abstract: Systems, methods, and computer program products for trace management in a distributed computing environment are described. A trace manager receives a request to analyze a trace of a series of calls between applications serving requests in a microservice architecture. The trace manager determines metrics including, for example, latency information and start time for the series of calls. The trace manager presents the metrics in association with log entries correlated to the series of calls. A call stack, or one or more latency indicators, provides a user interface of selecting one or more calls. A user selection of a particular call in the call stack or a latency indicator causes one or more corresponding log entries to be emphasized or filtered. The call stack and the one or more latency indicators can have various display configurations.

Abstract: A mobile computing device is operated to receive a trigger at a first instance. The trigger may be associated with a predefined condition or event or action. The mobile computing device may detect the predefined condition or event at a second instance. In response to detecting the predefined condition or event, a notification is activated on the mobile computing device that is based on the trigger.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying user behavior as anomalous. One of the methods includes obtaining user behavior data representing behavior of a user in a subject system. An initial model is generated from training data, the initial model having first characteristic features of the training data. A resampling model is generated from the training data and from multiple instances of the first representation for a test time period. A difference between the initial model and the resampling model is computed. The user behavior in the test time period is classified as anomalous based on the difference between the initial model and the resampling model.

Abstract: Methods, systems, and apparatus for obtaining one or more metadata files, determining, by one or more computers and in accordance with a minor compaction setting, to perform a minor compaction of the one or more metadata files, creating one or more intermediate metadata files that each include at least compacted contacts of one or more of the metadata files, according to the determination to perform minor compaction of the one or more metadata files, determining, in accordance with a major compaction setting, to perform a major compaction of one or more of the intermediate metadata files, and creating one or more snapshot metadata files that each include at least compacted contents of one or more of the intermediate metadata files, according to the determination to perform the major compaction of one or more of the intermediate metadata files.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for deploying and using a light-weight cloud application platform. In one aspect, a method includes providing a deployment tool configured to deploy a light-weight cloud application platform in an execution environment in response to a user command that specifies the environment, and to create one or more instances of a cloud application executing on the light-weight cloud application platform in response to a user command that specifies a container image that includes the application. The deployment tool receives a first user command that specifies the execution environment. In response, the deployment tool deploys components of the light-weight cloud application platform, including a scheduling subsystem and a routing subsystem. The deployment tool receives a second user command that specifies a location of the container image.