Abstract: Systems and method for assessing annotators by way of annotated images annotated by said annotators. Agent or annotator model modules are trained using annotated images annotated by specific annotators. A baseline model module is also trained using all of the annotated images used in training the agent model modules. The trained agent model modules are then used to annotate an evaluation dataset to result in evaluation result annotated images. The trained baseline model module is also used to annotate the evaluation dataset to result in its own evaluation result annotated images. The evaluation results from the agent model modules are compared with the evaluation result from the baseline model module. Based on the comparison results, scores are allocated to each agent model module. The scores are used to group agent model modules and annotators that correspond to the low scoring agent model modules can be targeted for retraining.

Abstract: A system and method are provided for machine learning (ML) quality assurance. The method trains a plurality of agent ML annotation model software applications. Each agent annotation model is trained with a corresponding subset of annotated raw data images including annotation marks forming a boundary surrounding the first shape. A baseline ML annotation model is trained with all the subsets of annotated raw data images. The method accepts an evaluation dataset with unannotated images including the first shape, which is provided to the agent models and baseline models. In response to the evaluation dataset, the agent and baseline models infer predicted images including annotation marks forming a boundary surrounding the first shape. The baseline model predicted images are compared to the predicted images of each agent model for the purpose of determining agent model quality and identifying problematic raw data images for retraining purposes.

Abstract: Systems and methods are presented for triggering machine learning (ML) annotation model retraining. The method includes an annotation model accepting raw data images with undefined variations of a first shape with a coarse boundary surrounding a first shape. The annotation model creates an annotated image with annotation marks forming a refined boundary surrounding the first shape. An agent user interface (UI) modifies the refined boundaries to supply corrected images. A mediation software application compares the annotated images to corresponding corrected images, and supplies an annotation model retraining resource analysis for the first shape in response to comparing the annotated images to the corrected images. The step of comparing the annotated images to the corrected images may include the mediation application measuring the number of points changed in the refined boundaries to supply the corrected images, and the time required by the human agent to change the points.

Abstract: A system and method are provided for machine learning (ML) quality assurance. The method trains a plurality of agent ML annotation model software applications. Each agent annotation model is trained with a corresponding subset of annotated raw data images including annotation marks forming a boundary surrounding the first shape. A baseline ML annotation model is trained with all the subsets of annotated raw data images. The method accepts an evaluation dataset with unannotated images including the first shape, which is provided to the agent models and baseline models. In response to the evaluation dataset, the agent and baseline models infer predicted images including annotation marks forming a boundary surrounding the first shape. The baseline model predicted images are compared to the predicted images of each agent model for the purpose of determining agent model quality and identifying problematic raw data images for retraining purposes.

Abstract: Systems and methods are presented for triggering machine learning (ML) annotation model retraining. The method includes an annotation model accepting raw data images with undefined variations of a first shape with a coarse boundary surrounding a first shape. The annotation model creates an annotated image with annotation marks forming a refined boundary surrounding the first shape. An agent user interface (UI) modifies the refined boundaries to supply corrected images. A mediation software application compares the annotated images to corresponding corrected images, and supplies an annotation model retraining resource analysis for the first shape in response to comparing the annotated images to the corrected images. The step of comparing the annotated images to the corrected images may include the mediation application measuring the number of points changed in the refined boundaries to supply the corrected images, and the time required by the human agent to change the points.

Abstract: Embodiments regard manifest schema to provide application flows. An embodiment of a method includes: loading a manifest schema on a device, the manifest schema including a parent manifest and one or more application manifests; making a call to a server to populate a user interface of the device based on the manifest schema; upon receiving a message at the mobile device, utilizing the parent manifest to automatically select an application for routing of the message, the selection of the application being based on contents of the message; and utilizing an application manifest for the selected application to automatically select a process for routing of the message, the selection of the process being based on the contents of the message using an application manifest for the selected application.

Abstract: Adaptive configuration database management. An adaptive configuration management database (ACMDB) system receives a request message for configuration data from a requesting device. The request is for configuration data for a device coupled with the ACMDB. In response to the request, the ACMDB provides a response message with indicators for sources of configuration data rather than the configuration data.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamic management of assets in an on-demand services environment. In one embodiment and by way of example, a method includes receiving, in real-time, a request for performance of one or more tasks relating to management of assets. The management of assets may include management of data relating to the assets, where the assets may include hardware assets or software assets, wherein the request is received at a first computing system. The method may further include performing, in real-time, the one or more tasks. The one or more tasks may include dynamically customizing, according to the request, data relating to the assets.

Abstract: Embodiments regard manifest schema to provide application flows. An embodiment of a method includes: loading a manifest schema on a device, the manifest schema including a parent manifest and one or more application manifests; making a call to a server to populate a user interface of the device based on the manifest schema; upon receiving a message at the mobile device, utilizing the parent manifest to automatically select an application for routing of the message, the selection of the application being based on contents of the message; and utilizing an application manifest for the selected application to automatically select a process for routing of the message, the selection of the process being based on the contents of the message using an application manifest for the selected application.

Abstract: Adaptive configuration database management. An adaptive configuration management database (ACMDB) system receives a request message for configuration data from a requesting device. The request is for configuration data for a device coupled with the ACMDB. In response to the request, the ACMDB provides a response message with indicators for sources of configuration data rather than the configuration data.

Abstract: A method and system are provided for accelerated data uploading to a remote service device destination. An on-line (third party) storage device receives an upload request message from a network-connected client device. A unique first descriptor in a descriptor field of the upload request message is accessed and compared to a list of descriptors maintained by the on-line storage device. If the accessed first descriptor is on the list, a first file is read that is stored in the on-line storage device and associated with the accessed first descriptor. The first file is then sent to a network-connected remote service device.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamic management of assets in an on-demand services environment. In one embodiment and by way of example, a method includes receiving, in real-time, a request for performance of one or more tasks relating to management of assets. The management of assets may include management of data relating to the assets, where the assets may include hardware assets or software assets, wherein the request is received at a first computing system. The method may further include performing, in real-time, the one or more tasks. The one or more tasks may include dynamically customizing, according to the request, data relating to the assets.

Abstract: A system and method are provided for downloading data to a client device by deferral to an on-line storage device. A client sends a login request to a network-connected remote service device, and receives a remote service identification (ID) from the remote service device. The client sends a token request message, with the remote service ID, to a network-connected on-line storage device, and receives a first token identification (ID) associated with a first session and an on-line storage ID from the on-line storage device. The client sends an upload request message to the remote service device. The upload request message includes the first token ID and on-line storage ID, and identifies a first file to be uploaded to the on-line storage device. Subsequent to the on-line storage device receiving the first file from the remote service device, the client downloads the first file from the on-line storage device.

Abstract: A system and method are provided for calculating passive host proximity in an environment of network-connected hosts. At least three controller hosts send inquiries to target hosts in the environment, including at least two passive (non-controller) hosts. A time interval is recorded between each inquiry and an associated response, and the recorded time intervals are converted into logical distance measurements between the controller host and each target host. The logical distance measurements are then used to calculate a logical distance between the two passive hosts. A logical triangle is constructed between a first controller host (H1) and the two passive hosts (S1, S2) using a first logical distance side (H1S1) having a known value, a second logical distance side (H1S2) having a known value, and a third logical distance side (S1S2) having an unknown value. Trigonometric relationships solve for the unknown value of the third logical distance side.

Abstract: A system and method are provided for deduplication (dedup) of data file information in a network of distributed data filers. A host, including a metadata service (MDS) and a data node (DN), receives a block count and hash group calculations for a file from a network-connected client device. The MDS creates a file map with target addresses, and compares the calculated hash group to hash groups associated with stored blocks of data in a global dedup hash table. If a match is found, the MDS directs the client device not to send the block of data associated with matching hash group, and reallocates the address of the block of data from the target address.

Abstract: A method and system are provided for accelerated data uploading to a remote service device destination. An on-line (third party) storage device receives an upload request message from a network-connected client device. A unique first descriptor in a descriptor field of the upload request message is accessed and compared to a list of descriptors maintained by the on-line storage device. If the accessed first descriptor is on the list, a first file is read that is stored in the on-line storage device and associated with the accessed first descriptor. The first file is then sent to a network-connected remote service device.

Abstract: A device and method are provided for guaranteeing a quality of service (QoS) for a network-connected device in a distributed service platform (DSP) or cloud computing system. The method establishes a first set of attributes or performance criteria for a selected platform component. The selected platform component may be a system component for processing data or a network component for connecting system components. For example, the first set of attributes may be supplied by a client in an SLA. A second set of attributes is derived for the unselected platform component necessary to support the first set of attributes. For example, if a first set of attributes is established for a system component, then a second set of attributes would be derived or calculated for a network component. As a result, a DSP QoS is supplied to a network-connected (e.g., client) device, guaranteeing the first set of attributes.