Abstract: A system may receive a first user input requesting to provide an evaluator agent configuration for an evaluator agent. A system may receive a second user input specifying information associated with an agent to be evaluated. A system may receive a third user input specifying an evaluation tool, wherein the evaluation tool is configurable to evaluate the information associated with the agent. A system may receive a fourth user input specifying an evaluation tool configuration associated with the evaluation tool. A system may create the evaluator agent based on the evaluator agent configuration, wherein the evaluator agent configuration comprises an indication of the information associated with the agent to be evaluated, an indication of the evaluation tool, and an indication of the evaluation tool configuration. A system may include evaluating, using the evaluator agent, the information associated with the agent.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects. linking first and second data objects. The first and second data objects may be associated with corresponding first and second pluralities of properties. The method may include receiving a first user limitation associated with at least one of the first or second pluralities of properties and receiving a second user limitation associated with at least one of the first or second pluralities of properties. The method can include deriving a conclusion object comprising a third plurality of properties comprising a subset of one or more of the first or second pluralities of properties and linking the conclusion object to the first and second data objects.

Abstract: Computer-implemented systems and methods are disclosed, including systems and methods for automatically solving computational tasks or problems. A computer-implemented method may include: providing an agent service that utilizes a plurality of agents to process one or more tasks; receiving, by a first agent, a request to handle a first task; obtaining, by the first agent, a first accessory to handle the first task; assigning, by the first agent, at least a portion of the first task to a second agent; sharing, by the first agent, the first accessory to the second agent; and processing, by the second agent, at least the portion of the first task using the first accessory to generate a processing result.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects, linking first and second data objects. The first and second data objects may be associated with corresponding first and second pluralities of properties. The method may include receiving a first user limitation associated with at least one of the first or second pluralities of properties and receiving a second user limitation associated with at least one of the first or second pluralities of properties. The method can include deriving a conclusion object comprising a third plurality of properties comprising a subset of one or more of the first or second pluralities of properties and linking the conclusion object to the first and second data objects.

Abstract: Computer-implemented systems and methods are disclosed, including for evaluation of computer-based models in a management framework. A computer-implemented method may include, for example, receiving one or more inputs including requesting to add an evaluation configuration to a defined modeling objective, specifying at least a first evaluation data set for the evaluation configuration, specifying at least a first evaluation library for the evaluation configuration, and specifying at least a first subset definition for the evaluation configuration. A computer-implemented method may in response to the one or more user inputs include: creating, storing, and/or updating the evaluation configuration. A computer-implemented method may include evaluating, based on the evaluation configuration, the one or more models associated with the defined modeling objective.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: Disclosed herein is a method of providing feedback to a machine learning model. The method includes allowing a user to observe an output of a trained machine learning model; allowing the user to input feedback to the machine learning model based on the output, wherein the feedback is on at least one of a model level or on a training dataset level; and incorporating the feedback into the machine learning model to improve the machine learning model, wherein the method is performed using one or more processors. Disclosed herein are one or more computer-readable storage media including computer executable instructions which when executed by the one or more processors cause the one or more processors to perform the method. Disclosed herein is a computer system which includes one or more processors and one or more computer-readable storage media which include computer executable instructions which when executed by the one or more processors cause the one or more processors to perform the method.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects, linking first and second data objects. The first and second data objects may be associated with corresponding first and second pluralities of properties. The method may include receiving a first user limitation associated with at least one of the first or second pluralities of properties and receiving a second user limitation associated with at least one of the first or second pluralities of properties. The method can include deriving a conclusion object comprising a third plurality of properties comprising a subset of one or more of the first or second pluralities of properties and linking the conclusion object to the first and second data objects.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects. linking first and second data objects. The first and second data objects may be associated with corresponding first and second pluralities of properties. The method may include receiving a first user limitation associated with at least one of the first or second pluralities of properties and receiving a second user limitation associated with at least one of the first or second pluralities of properties. The method can include deriving a conclusion object comprising a third plurality of properties comprising a subset of one or more of the first or second pluralities of properties and linking the conclusion object to the first and second data objects.

Abstract: Methods and systems for collaboration between two or more parties to generate a model based on data from the parties without data to the another party. A system includes an aggregation system having computer storage devices configured to store a model and a plurality versions of an aggregated model, and instructions, and one or more processors configured to execute the plurality of computer readable instructions to, iteratively, receive a first updated model from a first system controlled by a first party and a second updated model from a second system controlled by a second party, determine changes from the first updated model and the second updated model to include in an aggregated model, communicate a version of the aggregated model to the first system and a version of the aggregated model to the second system, and store a final aggregated model.

Abstract: Systems and methods are provided for processing an input dataset or running an inference. The systems and methods may be configured to accept an input dataset, access one or more predefined logic plugins for processing the input dataset, process the input dataset based at least in part on a first predefined logic plugin, and generate the one or more outputs based at least in part of the processing of the input dataset. The one or more outputs may have a different format than a format of the input dataset.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects. linking first and second data objects. The first and second data objects may be associated with corresponding first and second pluralities of properties. The method may include receiving a first user limitation associated with at least one of the first or second pluralities of properties and receiving a second user limitation associated with at least one of the first or second pluralities of properties. The method can include deriving a conclusion object comprising a third plurality of properties comprising a subset of one or more of the first or second pluralities of properties and linking the conclusion object to the first and second data objects.

Abstract: A system architecture for linking one or more derived objects to existing data objects in a data structure can be organized in a variety of forms. A method for establishing the architecture may include linking first and second data objects. The first data object may include a first plurality of properties, and the second data object may include a second plurality of properties. At least one of the second plurality of properties may be based on the first plurality of properties. The method may further include receiving, by a computer, a user limitation of a property type that is associated with the at least one of the second plurality of properties. The method may include deriving, based on the user limitation, a new object that includes a third plurality of properties. The third plurality of properties may include a subset of one or more of the first and second pluralities of properties. The new object may be linked to the first and second data objects.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.

Abstract: A computer system is configured to receiving a data set from a data provider and automatically save the data set in a quarantine database where copying, moving, and sharing of the data set are restricted until the data set is released by a data provider. The data set is parsed to find and mark portions with potentially sensitive information. At least those parts are reviewed by a data governor, who can confirm, add, edit, or remove markers. Those parts can be visually indicated to the data governor, along with a preview of, metadata about, and analysis of the data set. After reviewing at least the automatically marked portions, the data governor can release the data set to a non-quarantine database where another user can use the data set. The user is restricted from accessing the quarantine database.