Abstract: A system defining an operation using a set of assets is disclosed. A processor(s) is/are configured to: cause a first user interface to be displayed, the first user interface comprising selectable element(s) associated with a characteristic of task(s) to be performed; receive user selection(s) pertaining to the characteristic; in response to receiving the user selection(s) input to the first interface, cause a second user interface to be displayed, wherein: 1) the second user interface is configured based on user selection(s); and 2) the second user interface comprises selectable element(s) associated with a set of asset(s) that include drone(s); receive user selection(s) for an operation; determine the operation to be performed based on (i) the user selection(s) pertaining to the characteristic of the task(s) and (ii) the user selection(s) pertaining to the set of asset(s) to perform the operation; and communicate information pertaining to the operation.

Abstract: A system and method for grouping assets is disclosed. The system includes one or more processors configured to: receive, via the communication interface, data associated with one or more tasks to be performed by a set of assets, determine, from among a plurality of assets, the set of assets to perform the one or more tasks, including determining one or more capabilities associated with one or more characteristics of the one or more tasks, and determining the plurality of drones based at least in part on the one or more capabilities associated with one or more characteristics of the one or more tasks, and one or more drone capabilities respectively associated with the drones, and communicate an instruction to at least one drone in the set of assets, the instruction indicating that the one or more tasks are to be completed by the at least one drone.

Abstract: Computer-implemented techniques for data extraction are described. The techniques include a method and system for retrieving an extraction job specification, wherein the extraction job specification comprises a source repository identifier that identifies a source repository comprising a plurality of data records; a data recipient identifier that identifies a data recipient; and a schedule that indicates a timing of when to retrieve the plurality of data records. The method and system further include retrieving the plurality of data records from the source repository based on the schedule, creating an extraction transaction from the plurality of data records, wherein the extraction transaction comprises a subset of the plurality of data records and metadata, and sending the extraction transaction to the data recipient.

Abstract: Data transformation in a distributed system of applications and data repositories is described. The subsystems for the overall framework are distributed, thereby allowing for customization to require only isolated changes to one or more subsystems. In one embodiment, a source code repository is used to receive and store source code. A build subsystem can retrieve source code from the source code repository and build it, using one or more criteria. By building the source code, the build subsystem can generate an artifact, which is executable code, such as a JAR or SQL file. Likewise, by building the source code, the build subsystem can generate one or more job specifications for executing the executable code. In one embodiment, the artifact and job specification may be used to launch an application server in a cluster. The application server can then receive data transformation instructions and execute the data transformation instructions.

Abstract: A method and system for serving assets is disclosed, comprising receiving an asset request to serve an asset, wherein the asset request originates at an application, and wherein the asset request comprises an advertisement of an asset to be served and a request for the network address of an asset server configured to serve the requested asset. The method further comprises causing a service discovery server to identify an asset server configured to serve the requested asset, and causing the requested asset to be served to the application.

Abstract: A system comprises an interface and a processor. The interface is configured to provide a request to join a publish group from a client or a point to point communication link of a lattice mesh; and receive a group key or a host public key. The processor is configured to determine whether a message has been received; in response to the message having been received, determine whether the message is to be sent on; in response to the message being determined not to be sent on, decode the message using the group key or the host public key; determine whether to store the message in a backfill database; in response to determining to store the mssage in the backfill database, store the message in the backfill database.

Abstract: A system comprises an interface and a processor. The interface is configured to provide a request to join a publish group from a client or a point to point communication link of a lattice mesh; and receive a group key or a host public key. The processor is configured to determine whether a message has been received; in response to the message having been received, determine whether the message is to be sent on; in response to the message being determined not to be sent on, decode the message using the group key or the host public key; determine whether to store the message in a backfill database; in response to determining to store the message in the backfill database, store the message in the backfill database.

Abstract: A system comprises an interface and a processor. The interface is configured to provide a request to join a publish group from a client or a point to point communication link of a lattice mesh; and receive a group key or a host public key. The processor is configured to determine whether a message has been received; in response to the message having been received, determine whether the message is to be sent on; in response to the message being determined not to be sent on, decode the message using the group key or the host public key; determine whether to store the message in a backfill database; in response to determining to store the message in the backfill database, store the message in the backfill database.

Abstract: Techniques for propagation of deletion operations among a plurality of related datasets are described herein.

Abstract: A method and system for serving assets is disclosed, comprising receiving an asset request to serve an asset, wherein the asset request originates at an application, and wherein the asset request comprises an advertisement of an asset to be served and a request for the network address of an asset server configured to serve the requested asset. The method further comprises causing a service discovery server to identify an asset server configured to serve the requested asset, and causing the requested asset to be served to the application.

Abstract: A method and system for serving assets is disclosed, comprising receiving an asset request to serve an asset, wherein the asset request originates at an application, and wherein the asset request comprises an advertisement of an asset to be served and a request for the network address of an asset server configured to serve the requested asset. The method further comprises causing a service discovery server to identify an asset server configured to serve the requested asset, and causing the requested asset to be served to the application.

Abstract: Discussed herein are embodiments of methods and systems which allow engineers or administrators to create modular plugins which represent the logic for various fault detection tests that can be performed on data pipelines and shared among different software deployments. In some cases, the modular plugins each define a particular test to be executed against data received from the pipeline in addition to one or more configuration points. The configuration points represent configurable arguments, such as variables and/or functions, referenced by the instructions which implement the tests and that can be set according to the specific operation environment of the monitored pipeline.

Abstract: A system comprises an interface and a processor. The interface is configured to provide a request to join a publish group from a client or a point to point communication link of a lattice mesh; and receive a group key or a host public key. The processor is configured to determine whether a message has been received; in response to the message having been received, determine whether the message is to be sent on; in response to the message being determined not to be sent on, decode the message using the group key or the host public key; determine whether to store the message in a backfill database; in response to determining to store the message in the backfill database, store the message in the backfill database.

Abstract: Data transformation in a distributed system of applications and data repositories is described. The subsystems for the overall framework are distributed, thereby allowing for customization to require only isolated changes to one or more subsystems. In one embodiment, a source code repository is used to receive and store source code. A build subsystem can retrieve source code from the source code repository and build it, using one or more criteria. By building the source code, the build subsystem can generate an artifact, which is executable code, such as a JAR or SQL file. Likewise, by building the source code, the build subsystem can generate one or more job specifications for executing the executable code. In one embodiment, the artifact and job specification may be used to launch an application server in a cluster. The application server can then receive data transformation instructions and execute the data transformation instructions.

Abstract: Computer-implemented techniques for data extraction are described. The techniques include a method and system for retrieving an extraction job specification, wherein the extraction job specification comprises a source repository identifier that identifies a source repository comprising a plurality of data records; a data recipient identifier that identifies a data recipient; and a schedule that indicates a timing of when to retrieve the plurality of data records. The method and system further include retrieving the plurality of data records from the source repository based on the schedule, creating an extraction transaction from the plurality of data records, wherein the extraction transaction comprises a subset of the plurality of data records and metadata, and sending the extraction transaction to the data recipient.

Abstract: Aspects of the present disclosure relate to version control. A version control server receives, from a client device, a request for making changes to a portion of data stored in a data store. The version control server verifies that the client device has permission to make the changes to the portion of the data stored in the data store. The version control server updates, using a version control servlet, the data store according to the request. The version control sever creates, forks, merges or deletes, using a repository management API (application programming interface), a repository and/or the files in the repository in the data store based on the request. The version control server triggers, using the repository management API, one or more webhooks in response to the request.

Abstract: A system for a lattice mesh comprises an interface and a processor. The interface is configured to receive a request to join a publish group from a client. The processor is configured to determine a group key; determine whether the client is allowed access to a topic of the publish group; in response to the client being allowed access to the topic of the publish group, provide the group key to the client; publish a group message including metadata; determine whether it is time to rotate the group key; and in response to it being time to rotate the group key, determine a new group key.

Abstract: Computer-implemented techniques for data extraction are described. The techniques include a method and system for retrieving an extraction job specification, wherein the extraction job specification has a source repository identifier that identifies a source repository including a plurality of data records; a data recipient identifier that identifies a data recipient; and a schedule that indicates a timing of when to retrieve the plurality of data records. The method and system further include retrieving the plurality of data records from the source repository based on the schedule, creating an extraction transaction from the plurality of data records, wherein the extraction transaction includes a subset of the plurality of data records and metadata, and sending the extraction transaction to the data recipient.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.

Abstract: A system for a lattice mesh comprises an interface and a processor. The interface is configured to receive a request to join a publish group from a client. The processor is configured to determine a group key; determine whether the client is allowed access to a topic of the publish group; in response to the client being allowed access to the topic of the publish group, provide the group key to the client; publish a group message including metadata; determine whether it is time to rotate the group key; and in response to it being time to rotate the group key, determine a new group key.