Abstract: In a mirrored database system, a careful write of intentions to perform file system actions is recorded in a persistent file system objects table that is flushed to disk prior to the actions being taken. The table durably and accurately records identities of file system objects that were in use by the database to facilitate creation and deletion of physical file directories and files on a database during crash recovery and during mirror resynchronize. In the event of a failure, crash recovery may quickly and easily identify file system objects which need to be cleaned up by reference to the persistent file system objects table. Similarly, resynchronization of the mirror database can be performed quickly by referring to the persistent file system table data to detect changes since the last database checkpoint.

Abstract: Systems, methods, and computer program products for trace management in a distributed computing environment are described. A trace manager receives a request to analyze a trace of a series of calls between applications serving requests in a microservice architecture. The trace manager determines metrics including, for example, latency information and start time for the series of calls. The trace manager presents the metrics in association with log entries correlated to the series of calls. A call stack, or one or more latency indicators, provides a user interface of selecting one or more calls. A user selection of a particular call in the call stack or a latency indicator causes one or more corresponding log entries to be emphasized or filtered. The call stack and the one or more latency indicators can have various display configurations.

Abstract: A random number generation process generated uncorrelated random numbers from identical random number sequences on parallel processing database segments of an MPP database without communications between the segments by establishing a different starting position in the sequence on each segment using an identifier that is unique to each segment, query slice information and the number of segments. A master node dispatches a seed value to initialize the random number sequence generation on all segments, and dispatches the query slice information and information as to the number of segments, during a normal query plan dispatch process.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for routing events of an event stream. One of the methods includes operations of receiving, by a router, events of an event stream; providing each event, by the router, to a respective local modeler selected by the router according to an initial routing strategy, the respective local modeler being selected from multiple local modelers; aggregating, by each local modeler in parallel, information associated with each event received by the local modeler to generate aggregated information; providing, to a central modeler, the aggregated information generated by the one or more local modelers; determining, by the central modeler, parameters of a machine learning model using the aggregated information received by the central modeler and generating an updated routing strategy based on the parameters of the machine learning model; and providing the updated routing strategy to the router.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for precisely tracking memory usage in a multi-process computing environment. One of the methods includes implementing an instance of a memory usage tracker (MUT) in each process running in a node of a computer system. A MUT can maintain an account of memory usage for each of multiple logical owners running on a process on which the MUT is running. The MUT can determine an actual memory quota for each owner, and enforce the actual memory quota of the owner. Enforcing the actual memory quota of the owner can include receiving each memory allocation request, checking each allocation request and a current state of the account against the actual quota, approving or rejecting each allocation request, communicating the approval or rejection to an underlying memory manager, and updating the owner account for each approved allocation request.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating recipes having platform-independent, resource-agnostic commands for performing work in a cloud computing platform. One of the systems includes a plurality of executors, each executor being operable to receive a recipe comprising one or more platform independent, resource agnostic commands, each command corresponding to a requested service to be performed by an execution engine on an underlying computing platform, translate each platform independent, resource agnostic command of the recipe into one or more platform independent, resource specific requests that satisfy the requested service on the underlying computing platform and make each of the one or more platform independent, resource specific requests to one of the one or more execution engines using a platform independent execution engine application programming interface (API).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic topology generation. One of the methods includes receiving, from a user, a query that specifies a search criterion and a post-processing function prototype. A topology definition is generated from the query that specifies one or more source nodes and at least one processing node, wherein the processing node is specified in the topology to output a stream of data objects corresponding to the data object identifiers and a post-processing node according to the post-processing function prototype, wherein the post-processing node is specified in the topology to receive a stream of data objects output by the processing node, and is specified in the topology to output one or more modified data objects. The topology definition is then provided to a real-time stream computing system for execution.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data management. One of the methods includes converting a standard specification for a software package into a custom specification for a software deployment tool that deploys the software package on one or more distributed computing systems, parsing source files and corresponding configuration files to determine configuration parameters and custom values, writing the determined configuration parameters into a custom template, and updating the custom specification with the determined custom values for the configuration parameters and information associated with the custom template.

Abstract: A policy engine is situated within the communications path of a cloud computing environment and a user of the cloud computing environment to comply with an organization's policies for deploying web applications in the cloud computing environment. The policy engine intercepts communications packets to the cloud computing environment from a user, such as a web application developer, for example, in preparation for deploying a web application in the cloud computing environment. The policy engine identifies commands corresponding to the communications packets and directs the communications packets to appropriate rules engines corresponding to such commands in order to execute rules to comply with an organization's policies. Upon completion of execution of the rules, the communications packets are forwarded to the cloud computing environment if they comply with the policies.

Abstract: Embodiments of the present invention include systems and methods for insuring better query consistency between at least two different databases, where one faster database has more up-to-date information than another slower database, and wherein updates are typically applied to the faster database first and then to the slower database. In embodiments, the systems and methods also insure that a query to the slower database is not performed until a set of one or more updates from the faster database have been applied to that slower database.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data management. One of the methods includes maintaining, by a first member in a distributed data management system having multiple computing members installed on multiple respective computers, a first garbage collection version vector that includes, for each member in the distributed data management system, a garbage collection version that represents a number of garbage collection processes performed by the member on a respective copy of a replicated data region maintained by the member in the data management system. If the first garbage collection version vector is different than a second garbage collection version vector received from a different provider member, a first replication process is performed that is different than a second replication process that is performed when the first garbage collection version vector matches the second garbage collection version vector.

Abstract: A mobile computing device is operated to receive a trigger at a first instance. The trigger may be associated with a predefined condition or event or action. The mobile computing device may detect the predefined condition or event at a second instance. In response to detecting the predefined condition or event, a notification is activated on the mobile computing device that is based on the trigger.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for streaming external data in parallel from a second distributed system to a first distributed system. One of the methods includes receiving a query that requests a join of first rows of a first table in a first distributed system with second rows of an external table, the external table representing data in a second distributed system. Each of the segment nodes communicates with a respective extension service that obtains fragments from one or more data nodes of the second distributed system according to location information for the respective fragments, and provides to the segment node a stream of data corresponding to second rows of the external table. Each of the segment nodes computes joined rows between the first rows of the first table and the stream of data corresponding to second rows of the external table.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for job management. One of the methods includes receiving a status report for each of a plurality of jobs from a pluggable status reporter. The jobs are executed by applications in a distributed computing system. The applications are developed in at least two different software frameworks. A job type for each job is determined. A specific status handler associated with the job type is selected to determine specified status information for the job. The specified status information is stored and associated with the job in a status database.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data management. One of the methods includes receiving a plurality of feeds of streaming event data and routing feed data from each of the feeds to a respective channel of a plurality of channels, each of the channels being configured to store feed data until the feed data is consumed by a data sink, including routing feed data from a first feed to a first channel. A load metric for the first channel is determined to exceed a threshold. In response, a second channel is allocated for the first feed and feed data is redirected from the first feed to the second channel instead of the first channel.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for deploying and using a light-weight cloud application platform. In one aspect, a method includes providing a deployment tool configured to deploy a light-weight cloud application platform in an execution environment in response to a user command that specifies the environment, and to create one or more instances of a cloud application executing on the light-weight cloud application platform in response to a user command that specifies a container image that includes the application. The deployment tool receives a first user command that specifies the execution environment. In response, the deployment tool deploys components of the light-weight cloud application platform, including a scheduling subsystem and a routing subsystem. The deployment tool receives a second user command that specifies a location of the container image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for identifying malware attacks collects data traffic information. A system receives data traffic information indicative of communications between computers within a network and computers external to the network. The system parses the data traffic information to identify communication links between the computers within the network and computers external to the network. The system can generate communication link profiles for each of the computers within the network. The system can then group computers within the network into computer clusters based on similarities between the communication link profiles for each computer. The system can identify computer clusters having anomalous communication patterns as being indicative of a malware attack.

Abstract: A computer readable storage medium comprises executable instructions to receive a query. A graph is built to represent jobs associated with the query. The jobs are assigned to parallel threads according to the graph.

Abstract: A method determines the connectivity components defined by a set of relations over a set of data elements. For each first data element of a selected subset of data elements, a second data element that is linked to the first data element by a path of relations is selected as its representative, using a randomization process. A new set of relations is created by replacing each first data element of the subset by its representative in at least part of the set of relations.

Abstract: In a database cluster comprising a plurality of mirrored database segments, a mirror database pair comprising a primary database and a mirror database are maintained substantially consistent by writing changes resulting from transactions to page data in parallel to the primary database and to the mirror database. Changes are sent to the mirror database over a network using network communications. Transaction logs are also written to the primary and mirrored databases in parallel, the transaction logs being sent to the mirror database using network communications. Bulk load changes resulting from transactions are written in parallel over the network to the primary database and the mirror database without writing the changes to a transaction log. In the event of a failure, resynchronization of the mirror database can be performed quickly while avoiding a lengthy ongoing redo process on the mirror database.