Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing machine learning using a query engine. One of the methods includes obtaining, from a user device and by a query engine that is configured to access one or more databases, a command to execute a user-defined function, wherein the user-defined function includes an inference call to a machine learning model, wherein the command comprises one or more model inputs to the machine learning model; obtaining, by the query engine and from the one or more databases, trained parameter values for the machine learning model; executing, by the query engine, the user-defined function, comprising processing the one or more model inputs using the machine learning model according to the obtained parameter values of the machine learning model to generate respective model outputs; and providing, to the user device and by the query engine, the generated model outputs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data stream processing. One method performed by a stream manager in a group of containers including streaming processes and stream managers includes receiving a first data streaming object of the data streaming objects from a first component in the group of containers; determining whether to process the first data streaming object; in response to determining to process the first data streaming object processing the first data streaming object and routing the first data streaming object; and in response to determining not to process the first data streaming object, routing the first data streaming object.

Abstract: A system and method for data stream processing. Two or more instances are connected as a topology, wherein at least one of the instances is a spout and at least one of the instances is a bolt. The topology is submitted to a scheduler, wherein the service scheduler receives resource offers from a cluster manager representing computing resources available on one or more of cluster nodes and determines resources to accept and computations to run on the accepted computing resources. The topology is scheduled as one or more jobs, wherein each job includes two or more containers, including a first container and a second container, the first container including a topology master and the second container including a stream manager and one or more stream processing system (SPS) instances, wherein each SPS instance represents one of the instances in the topology.

Abstract: A system and method for data stream processing. Two or more instances are connected as a topology, wherein at least one of the instances is a spout and at least one of the instances is a bolt. The topology is submitted to a scheduler, wherein the service scheduler receives resource offers from a cluster manager representing computing resources available on one or more of cluster nodes and determines resources to accept and computations to run on the accepted computing resources. The topology is scheduled as one or more jobs, wherein each job includes two or more containers, including a first container and a second container, the first container including a topology master and the second container including a stream manager and one or more stream processing system (SPS) instances, wherein each SPS instance represents one of the instances in the topology.