Abstract: A method by one or more electronic devices for creating an inference container on demand. The method includes receiving, over a network, a request to create the inferencing container, wherein the inferencing container is configured to provide inferencing functionality, creating the inferencing container responsive to receiving the request to create the inferencing container, and providing, over the network, a response to the request to create the inferencing container, wherein the response includes a uniform resource locator (URL) to use to submit inferencing requests to the inferencing container, wherein the URL includes a unique identifier (ID) of the inferencing container.

Abstract: Disclosed is an improved approach to process data and events from disparate ecosystems pertaining to products. An approach is provided to automatically cluster events from various ecosystems into noteworthy incidents and to correlate them with entities extracted from each system. Incidents are correlated between ecosystems to classify the type of incidents and to give a coherent converged picture of the event streams coming from the various ecosystems. Noteworthy incidents are automatically converted into tickets and their severity is ascertained from the associated incidents. Tickets that reference underlying defects with the product or service are converted into issues.

Abstract: A method by one or more electronic devices to provision an infrastructure for a machine learning application in a multi-tenant on-demand serving infrastructure. The method includes storing a plurality of templates, wherein each of the plurality of templates indicates a scoring interface, a web server, a definition of a continuous integration pipeline, and a definition of a continuous deployment pipeline, receiving a request to provision the infrastructure for the machine learning application using a specified template from the plurality of templates, and provisioning the infrastructure for the machine learning application using the specified template to create a version control system repository, a continuous integration pipeline, and a continuous deployment pipeline.

Abstract: Techniques for entity engagement are disclosed. An onboarding request is received from an interested entity and a first profile is generated for the interested entity. The onboarding of interested entity is verified and upon successful verification, the first profile is stored in an immutable form on a decentralized authentication framework. Further, an engagement request is received, from at least one of the interested entity and the entity of interest, to initiate a profile engagement between the interested entity and the entity of interest. The engagement request is authenticated to determine the credibility of the engagement request. Upon successful authentication of the engagement request, the first profile and the second profile is retrieved from the decentralized authentication framework. Based on an approval received from at least one of the interested entity and the entity of interest, engagement is initiated between the interested entity and the entity of interest.

Abstract: A method by one or more electronic devices for spinning up a scoring container on demand. The method includes receiving, from an orchestrator component via an application programming interface (API), a request to spin up the scoring container, wherein the scoring container is configured to provide scoring functionality, spinning up the scoring container responsive to receiving the request to spin up the scoring container, and providing, to the orchestrator component via the API, a response to the request to spin up the scoring container, wherein the response includes a uniform resource locator (URL) to use to submit scoring requests to the scoring container.

Abstract: Disclosed is an improved approach to process data and events from disparate ecosystems pertaining to products. An approach is provided to automatically cluster events from various ecosystems into noteworthy incidents and to correlate them with entities extracted from each system. Incidents are correlated between ecosystems to classify the type of incidents and to give a coherent converged picture of the event streams coming from the various ecosystems. Noteworthy incidents are automatically converted into tickets and their severity is ascertained from the associated incidents. Tickets that reference underlying defects with the product or service are converted into issues.

Abstract: A machine learning serving infrastructure implementing a method of receiving or detecting an update of container metrics including resource usage and serviced requests per model or per container, processing the container metrics per model or per container to determine recent resource usage and serviced requests per model or per container, and rebalancing distribution of models to a plurality of containers to decrease a detected load imbalance between containers or a stressed container in the plurality of containers.

Abstract: Machine learning version management method for a prediction service includes receiving a prediction request, determining application metadata for the request that defines routing logic and a machine learning framework version, determining model metadata for the request that defines at least one model and at least one model version, forwarding the prediction request to the at least one model with the at least one model version, and returning a prediction from the at least one model to a requestor.

Abstract: Methods and systems for multi-model scoring in a multi-tenant system are presented. A request for a machine learning application is received from a tenant application. A tenant identifier that identifies one of the multiple tenants is determined. Based on the tenant identifier and a type of the machine learning application, a first and a second machine learning models are determined. The first machine learning model was generated based on a first training data set associated with the tenant identifier. The second machine learning model that was generated based on a second training data set associated with the tenant identifier. A flow of operations that includes running the first and second machine learning models with data related to the request is executed to obtain a scoring result. The scoring result is returned to the tenant application in response to the request.

Abstract: Methods and systems for machine learning inferencing based on directed acyclic graphs are presented. A request for a machine learning application is received from a tenant application. A tenant identifier that identifies one of the tenants is determined from the request. Based on the tenant identifier and a type of the machine learning application, configuration parameters and a graph structure are determined. The graph structure defines a flow of operations for the machine learning application. Nodes of the graph structure are executed based on the configuration parameters to obtain a scoring result. Execution of a node causes a machine learning model generated for the first tenant to be applied to data related to the request. The scoring result is returned in response to the request.

Abstract: A method by a router component in a multi-tenant on-demand serving infrastructure to route scoring requests to scoring containers. The method includes receiving a scoring request, determining a machine learning application associated with the scoring request, determining whether a router instance for the machine learning application exists, and responsive to a determination that a router instance for the machine learning application does not exist, obtaining a configuration object for the machine learning application and instantiating the router instance for the machine learning application based on the configuration object for the machine learning application. The method further includes invoking the router instance for the machine learning application to route the scoring request associated with the machine learning application to a scoring container that provides scoring functionality for the machine learning application.

Abstract: A method of a base scorer in a scoring service container includes sending a model identifier to a model loader of an application specific scorer in the scoring service container, receiving a model object from the model loader in response to sending the model identifier, sending a request for a scoring from a client application to a scoring function of the application specific scorer, receiving the scoring from the application specific scorer, and returning the scoring to the client application.

Abstract: A method by one or more electronic devices to provision an infrastructure for a machine learning application in a multi-tenant on-demand serving infrastructure. The method includes storing a plurality of templates, wherein each of the plurality of templates indicates a scoring interface, a web server, a definition of a continuous integration pipeline, and a definition of a continuous deployment pipeline, receiving a request to provision the infrastructure for the machine learning application using a specified template from the plurality of templates, and provisioning the infrastructure for the machine learning application using the specified template to create a version control system repository, a continuous integration pipeline, and a continuous deployment pipeline.

Abstract: A method by one or more electronic devices for spinning up a scoring container on demand. The method includes receiving, from an orchestrator component via an application programming interface (API), a request to spin up the scoring container, wherein the scoring container is configured to provide scoring functionality, spinning up the scoring container responsive to receiving the request to spin up the scoring container, and providing, to the orchestrator component via the API, a response to the request to spin up the scoring container, wherein the response includes a uniform resource locator (URL) to use to submit scoring requests to the scoring container.

Abstract: A machine learning serving infrastructure implementing a method of receiving or detecting an update of container metrics including resource usage and serviced requests per model or per container, processing the container metrics per model or per container to determine recent resource usage and serviced requests per model or per container, and rebalancing distribution of models to a plurality of containers to decrease a detected load imbalance between containers or a stressed container in the plurality of containers.

Abstract: Machine learning version management method for a prediction service includes receiving a prediction request, determining application metadata for the request that defines routing logic and a machine learning framework version, determining model metadata for the request that defines at least one model and at least one model version, forwarding the prediction request to the at least one model with the at least one model version, and returning a prediction from the at least one model to a requestor.

Abstract: A method and system for a single framework for both streaming and on-demand inference that includes receiving a request from a tenant application for a machine-learning serving infrastructure, where the request identifies features of tenant data and a machine-learning model, subscribing to events for the identified features, initiating the machine-learning model for the request, and generating a prediction using the machine-learning model on the identified features.

Abstract: Using container and model information to select containers for executing models is described. A system receives a request from an application and identifies a version of a machine-learning model associated with the request. The system identifies model information associated with machine learning models corresponding to a cluster of available serving containers associated with the version of the machine-learning model. The system uses the model information to select a serving container from the cluster of available serving containers. If the machine-learning model is not loaded in the serving container, the system loads the machine-learning model in the serving container. If the machine-learning model is loaded in the serving container, the system executes, in the serving container, the machine-learning model on behalf of the request. The system responds to the request based on executing the machine-learning model on behalf of the request.

Abstract: Using container information to select containers for executing models is described. A system receives a request from an application and identifies a version of a machine-learning model associated with the request. The system identifies a set of each serving container corresponding to the machine-learning model from a cluster of available serving containers associated with the version of the machine-learning model. The system selects a serving container from the set of each serving container corresponding to the machine-learning model. If the machine-learning model is not loaded in the serving container, the system loads the machine-learning model in the serving container. If the machine-learning model is loaded in the serving container, the system executes, in the serving container, the machine-learning model on behalf of the request. The system responds to the request based on executing the machine-learning model on behalf of the request.

Abstract: An aspect of the present disclosure determines incompatibilities of automated test cases with modified user interfaces. In one embodiment, a mapping data between test cases in a test suite and user interface (UI) elements in the user interfaces of an application (tested using said test suite) is maintained, with mapping data indicating for each test case, the corresponding UI elements that the test case is designed to test. In response to receiving a modified (version of the) application that is to be tested with the same test suite, a set of UI elements (of the application) that are defective in the user interfaces of the modified application is found. Test cases that would fail are then identified based on the mapping data and the set of defective UI elements. The identified test cases are then reported as having incompatibility with the user interfaces of the modified application.