Abstract: A database system comprised of a decoupled compute layer and storage layer is implemented to store, build, and maintain a canonical dataset, a temporary buffer, and projection datasets. The canonical dataset is a set of batch updated data. The data is appended in chunks to the canonical dataset such that the canonical dataset becomes a historical dataset over time. The buffer is a write ahead log that contains the most recent chunks of data and provides atomicity and durability for the database system. The projection datasets are indexes of the canonical dataset and/or the buffer that may have single or multiple column sort-orders and/or particular data formats. The writes to the canonical dataset, projection datasets, and buffer may be asynchronous and therefore the database system is advantageously less resource constrained.

Abstract: A system and method for authenticating users of a data processing platform stores a mapping of a unique user platform identifier to multiple user identity provider identifiers associated with multiple realms for a same user. In some examples, the method includes receiving a request from a client device to establish an access session to perform one or more actions on data of the data processing platform and receiving, from at least one of the first external identity provider of the first realm or the second external identity provider of the second realm, a user identity provider identifier associated with the request. In certain examples, the method includes granting permission to perform the one or more actions on the data of the data processing platform based at least in part on the received user identity provider identifier.

Abstract: Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.

Abstract: Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.

Abstract: Disclosed are a magnet-insert assembly adapted for insertion into and removal from a central cavity of a magnetic-field generating magnet array, and a magnet array adapted to receive the magnet-insert assembly. The magnet insert assembly comprising a substructure defining a shaft therein; one or more headstones; and one or more permanent magnets. The substructure being adapted to receive, secure and position the one or more headstones and the one or more permanent magnets in the central cavity of the magnet array.

Abstract: Data analysis system with highly dynamic and interactive data analysis user interfaces which enable data analysts to quickly and efficiently explore large volume data sources. The system enables the user to investigate large volumes of data over many different paths of analysis while maintaining detailed and retraceable steps taken by the user over the course of an investigation, as captured via the user's queries and user interaction with the user interfaces provided by the data analysis system. Data analysis paths may involve exploration of high volume data sets. The data analyst may pursue a data analysis path that involves, among other things, applying filters, joining to other tables in a database, viewing interactive data visualizations, and so on. The data analysis system enables a user to refresh or insert a node in a workflow. The data analysis system further provides data editing, reporting, enrichment, expression language, and charting functionality.

Abstract: One or more software products/services may be installed on a cloud deployment. Product versions of such services can be upgraded (or rolled back) based on a deployment plan comprising tasks adapted to reach a target deployment state. A central upgrade server can forward tasks to an upgrade agent for execution, the tasks being based on a current operational state of the cloud deployment (obtained by the upgrade agent) and one or more constraints. In multiple node deployments, some nodes may be upgraded to a new product version, while other nodes are kept at a current product version until stability of the new product version is determined. Traffic across nodes can be shaped to ensure a deployment is healthy before upgrading other nodes/deployments. If the health of a node/deployment does not meet specified criteria, an upgrade can be stopped, an alert can be triggered, and the node/deployment can be rolled back.

Abstract: Example embodiments relate to a policy service employed to perform operations to: generate and maintain a data-set that comprises at least a column and a row that intersect at a cell; assign an access policy to a row or column of the data-set, wherein the access policy is defined by one or more access credentials required to receive access the cell that intersect with the row or column; receive a request to read the data-set from a user account, wherein the user account has an associated credential; filter the cell that intersects with the row or column of the data-set based on the access policy and the credential of the user account, in response to receiving the request from the user account; and provide the user account with access to the filtered data-set.

Abstract: One or more software products/services may be installed on a cloud deployment. Product versions of such services can be upgraded (or rolled back) based on a deployment plan comprising tasks adapted to reach a target deployment state. A central upgrade server can forward tasks to an upgrade agent for execution, the tasks being based on a current operational state of the cloud deployment (obtained by the upgrade agent) and one or more constraints. In multiple node deployments, some nodes may be upgraded to a new product version, while other nodes are kept at a current product version until stability of the new product version is determined. Traffic across nodes can be shaped to ensure a deployment is healthy before upgrading other nodes/deployments. If the health of a node/deployment does not meet specified criteria, an upgrade can be stopped, an alert can be triggered, and the node/deployment can be rolled back.

Abstract: The systems and methods described herein provide highly dynamic and interactive data analysis user interfaces which enable data analysts to quickly and efficiently explore large volume data sources. The data analysis system enables the user to investigate large volumes of data over many different paths of analysis while maintaining detailed and retraceable steps taken by the user over the course of an investigation, as captured via the user's queries and user interaction with the user interfaces provided by the data analysis system. Data analysis paths may involve exploration of high volume data sets, such as Internet proxy data, which may include trillions of rows of data. The data analyst may pursue a data analysis path that involves, among other things, applying filters, joining to other tables in a database, viewing interactive data visualizations, and so on. The data analysis system enables a user to refresh or insert a node in a workflow.

Abstract: A computer system can a canonical dataset, a buffer, and an edits dataset. The buffer can dump edits to the edits dataset responsive to one or more conditions. The system can receive a query of the canonical dataset and can rewrite the query to access data from the canonical dataset, the edits dataset, and/or the buffer. The system can execute the query on a combination of data from the canonical dataset, the edits dataset, and/or the buffer, including one or more edits to be made to the canonical dataset.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for data security protection are provided. One of the methods includes: receiving a job associated with a project, wherein the project is associated with one or more data sources; identifying a plurality of inputs and a plurality of outputs associated with the job; determining a plurality of required permissions associated with the job, wherein each of the required permissions comprises an operation on a required data source, the operation corresponding to at least one of the inputs or the outputs; verifying that the one or more data sources associated with the project comprise the required data source associated with each of the required permissions; and generating a token associated with the job, the token encoding the required permissions associated with the job, wherein the token is required for execution of the job.

Abstract: Example embodiments relate to a policy service employed to perform operations to: generate and maintain a data-set that comprises at least a column and a row that intersect at a cell; assign an access policy to a row or column of the data-set, wherein the access policy is defined by one or more access credentials required to receive access the cell that intersect with the row or column; receive a request to read the data-set from a user account, wherein the user account has an associated credential; filter the cell that intersects with the row or column of the data-set based on the access policy and the credential of the user account, in response to receiving the request from the user account; and provide the user account with access to the filtered data-set.

Abstract: Systems and methods are provided for coordinating the deployment of frontend assets to defined user groups. Individual groups of users may be assigned to a track comprising a set of frontend assets. Each set of frontend assets may comprise each of the individual components required to generate an entire frontend for an application. In some embodiments, different versions of a single component may be assigned within different tracks. As such, one set of users may be provided a first version of an application and a second set of users may be provided a second version of that application. By associating a new or updated version of a component to a given track, a new or updated version of a component not yet ready for widespread deployment may be provided to only a limited number of users.

Abstract: A computer system can receive one or more edits to be made to a canonical dataset and can temporarily store the one or more edits in a buffer. In response to receipt of a query of the canonical dataset, the computer system can rewrite the query to read from the canonical dataset and the buffer; combine the one or more edits from the buffer with the canonical dataset to form a combined dataset based on resolution policies to avoid conflicts between data; rewrite the query to execute on the combined dataset in lieu of the canonical dataset to optimize query performance; and execute the query on the combined dataset.

Abstract: Systems and methods are provided for coordinating the deployment of frontend assets to defined user groups. Individual groups of users may be assigned to a track comprising a set of frontend assets. Each set of frontend assets may comprise each of the individual components required to generate an entire frontend for an application. In some embodiments, different versions of a single component may be assigned within different tracks. As such, one set of users may be provided a first version of an application and a second set of users may be provided a second version of that application. By associating a new or updated version of a component to a given track, a new or updated version of a component not yet ready for widespread deployment may be provided to only a limited number of users.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for data security protection are provided. One of the methods includes: receiving a job associated with a project, wherein the project is associated with one or more data sources; identifying a plurality of inputs and a plurality of outputs associated with the job; determining a plurality of required permissions associated with the job, wherein each of the required permissions comprises an operation on a required data source, the operation corresponding to at least one of the inputs or the outputs; verifying that the one or more data sources associated with the project comprise the required data source associated with each of the required permissions; and generating a token associated with the job, the token encoding the required permissions associated with the job, wherein the token is required for execution of the job.

Abstract: A computer system can receive one or more edits to be made to a canonical dataset and can temporarily store the one or more edits in a buffer. In response to receipt of a query of the canonical dataset, the computer system can rewrite the query to read from the canonical dataset and the buffer; combine the one or more edits from the buffer with the canonical dataset to form a combined dataset based on resolution policies to avoid conflicts between data; rewrite the query to execute on the combined dataset in lieu of the canonical dataset to optimize query performance; and execute the query on the combined dataset.

Abstract: Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.