Abstract: A method performed by a processing system including at least one processor includes identifying an insufficiency in a representation of a subpopulation in training data for a machine learning model, generating simulated data to mitigate the insufficiency in the representation, and training the machine learning model using an enhanced training data set that includes the training data and the simulated data to produce a trained machine learning model. In some examples, the generating and the training may be repeated in response to determining that an output of the trained machine learning model still reflects the insufficiency in the representation of the subpopulation or reflects an insufficiency in a representation of another subpopulation. In other examples, the simulated data may be stored for future reuse.

Abstract: A method performed by a processing system including at least one processor includes obtaining an output of a machine learning algorithm, identifying a vulnerability in the output of the machine learning algorithm, wherein the vulnerability relates to a bias in the output, integrating auxiliary data from an auxiliary data source of a plurality of auxiliary data sources into the machine learning algorithm to try to compensate for the vulnerability, determining whether the integrating has compensated for the vulnerability, and generating a runtime output using the machine learning algorithm when the processing system determines that the integrating has compensated for the vulnerability.

Abstract: A method includes obtaining descriptive information for a first machine learning project, identifying, based on the descriptive information, a plurality of past machine learning projects which are similar to the first machine learning project, retrieving digital documents that describe the bias evaluation pipelines that were used to evaluate the plurality of past machine learning projects, detecting a common bias evaluation pipeline step among at least a subset of the digital documents, extracting, from the subset, a snippet of machine-executable code that corresponds to the common bias evaluation pipeline step, modifying the snippet of machine-executable code with use case data that is specific to the first machine learning project to generate modified machine-executable code, and generating a proposed bias evaluation pipeline for evaluating the first machine learning project, wherein the proposed bias evaluation pipeline includes the modified machine-executable code.

Abstract: Aspects of the subject disclosure may include, for example, identifying a location associated with digital content, augmenting the digital content according to the location to obtain augmented digital content, and configuring a generative artificial intelligence (AI) model according to the augmented digital content to obtain a location-aware, generative AI model. The location-aware, generative AI model is configured to generate a solution according to the augmented digital content. Other embodiments are disclosed.

Abstract: Aspects of the subject disclosure may include, for example, system and apparatus that enable operations that may include receiving, by a processing system, project data defining a proposed machine learning (ML) project of an entity and storing the project data in a project database with other project data for other projects. The operations may further include extracting extracted features of the proposed project and, based on the extracted features, determining a clustering assignment for the proposed project. Determining the clustering assignment may comprise comparing information about the proposed project including the extracted features with information about the other projects and assigning the proposed project to a cluster including one or more projects having similar bias characteristics as the proposed project. The operations may further include determining a risk of potential bias for the proposed project and, based on the risk of bias, recommending a corrective action to reduce the risk of bias.

Abstract: A method includes obtaining descriptive information for a first machine learning project, identifying, based on the descriptive information, a plurality of past machine learning projects which are similar to the first machine learning project, retrieving digital documents that describe the bias evaluation pipelines that were used to evaluate the plurality of past machine learning projects, detecting a common bias evaluation pipeline step among at least a subset of the digital documents, extracting, from the subset, a snippet of machine-executable code that corresponds to the common bias evaluation pipeline step, modifying the snippet of machine-executable code with use case data that is specific to the first machine learning project to generate modified machine-executable code, and generating a proposed bias evaluation pipeline for evaluating the first machine learning project, wherein the proposed bias evaluation pipeline includes the modified machine-executable code.

Abstract: Aspects of the subject disclosure may include, for example, a device that facilitates obtaining a plurality of prompts from a selected subject matter domain of a database configured to measure an effectiveness of a generative large language model (LLM) to distinguish variances between each prompt of the plurality of prompts; supplying the plurality of prompts to the LLM; receiving respective responses to each of the prompts from the LLM; transforming each of the prompts and respective responses to each of the prompts into an embedding space; determining, by applying domain-based metrics to the embedding space, a quality measurement of each respective response to produce a plurality of quality measurements; and generating, according to the plurality of quality measurements, a performance of the LLM. Other embodiments are disclosed.

Abstract: A method performed by a processing system including at least one processor includes applying a contextual filter to mask a portion of at least one of: an input of a software application, an output of the software application, or an underlying dataset of the software application, where the contextual filter simulates a limitation of a user of the software application, executing the software application with the contextual filter applied to the at least one of: the input of the software application, the output of the software application, or the underlying dataset of the software application, collecting ambient data during the executing, and recommending, based on a result of the executing, a modification to the software application to improve at least one of: an accessibility of the software application or an inclusion of the software application.

Abstract: A method performed by a processing system including at least one processor includes applying a contextual filter to mask a portion of at least one of: an input of a software application, an output of the software application, or an underlying dataset of the software application, where the contextual filter simulates a limitation of a user of the software application, executing the software application with the contextual filter applied to the at least one of: the input of the software application, the output of the software application, or the underlying dataset of the software application, collecting ambient data during the executing, and recommending, based on a result of the executing, a modification to the software application to improve at least one of: an accessibility of the software application or an inclusion of the software application.

Abstract: Aspects of the subject disclosure may include, for example, system and apparatus that enable operations that may include receiving, by a processing system, project data defining a proposed machine learning (ML) project of an entity and storing the project data in a project database with other project data for other projects. The operations may further include extracting extracted features of the proposed project and, based on the extracted features, determining a clustering assignment for the proposed project. Determining the clustering assignment may comprise comparing information about the proposed project including the extracted features with information about the other projects and assigning the proposed project to a cluster including one or more projects having similar bias characteristics as the proposed project. The operations may further include determining a risk of potential bias for the proposed project and, based on the risk of bias, recommending a corrective action to reduce the risk of bias.

Abstract: Aspects of the subject disclosure may include, for example, system and apparatus that enable operations that may include receiving, by a processing system, project data defining a proposed machine learning (ML) project of an entity and storing the project data in a project database with other project data for other projects. The operations may further include extracting extracted features of the proposed project and, based on the extracted features, determining a clustering assignment for the proposed project. Determining the clustering assignment may comprise comparing information about the proposed project including the extracted features with information about the other projects and assigning the proposed project to a cluster including one or more projects having similar bias characteristics as the proposed project. The operations may further include determining a risk of potential bias for the proposed project and, based on the risk of bias, recommending a corrective action to reduce the risk of bias.

Abstract: Transfer knowledge from auxiliary data for more inclusive machine learning models is provided. A method can include generating a common feature space comprising first data features, wherein the first data features are present in training data used to train a first machine learning model, and wherein the first data features are present in auxiliary data that are independent of the training data; generating a combined learned feature representation, the combined learned feature representation being representative of the first data features of the common feature space and second data features that are unique to the training data; and training a second machine learning model based on the combined learned feature representation.

Abstract: A method performed by a processing system including at least one processor includes applying a contextual filter to mask a portion of at least one of: an input of a software application, an output of the software application, or an underlying dataset of the software application, where the contextual filter simulates a limitation of a user of the software application, executing the software application with the contextual filter applied to the at least one of: the input of the software application, the output of the software application, or the underlying dataset of the software application, collecting ambient data during the executing, and recommending, based on a result of the executing, a modification to the software application to improve at least one of: an accessibility of the software application or an inclusion of the software application.

Abstract: A method performed by a processing system including at least one processor includes identifying an insufficiency in a representation of a subpopulation in training data for a machine learning model, generating simulated data to mitigate the insufficiency in the representation, and training the machine learning model using an enhanced training data set that includes the training data and the simulated data to produce a trained machine learning model. In some examples, the generating and the training may be repeated in response to determining that an output of the trained machine learning model still reflects the insufficiency in the representation of the subpopulation or reflects an insufficiency in a representation of another subpopulation. In other examples, the simulated data may be stored for future reuse.

Abstract: A method includes obtaining descriptive information for a first machine learning project, identifying, based on the descriptive information, a plurality of past machine learning projects which are similar to the first machine learning project, retrieving digital documents that describe the bias evaluation pipelines that were used to evaluate the plurality of past machine learning projects, detecting a common bias evaluation pipeline step among at least a subset of the digital documents, extracting, from the subset, a snippet of machine-executable code that corresponds to the common bias evaluation pipeline step, modifying the snippet of machine-executable code with use case data that is specific to the first machine learning project to generate modified machine-executable code, and generating a proposed bias evaluation pipeline for evaluating the first machine learning project, wherein the proposed bias evaluation pipeline includes the modified machine-executable code.

Abstract: Aspects of the subject disclosure may include, for example, system and apparatus that enable operations that may include receiving, by a processing system, project data defining a proposed machine learning(ML) project of an entity and storing the project data in a project database with other project data for other projects. The operations may further include extracting extracted features of the proposed project and, based on the extracted features, determining a clustering assignment for the proposed project. Determining the clustering assignment may comprise comparing information about the proposed project including the extracted features with information about the other projects and assigning the proposed project to a cluster including one or more projects having similar bias characteristics as the proposed project. The operations may further include determining a risk of potential bias for the proposed project and, based on the risk of bias, recommending a corrective action to reduce the risk of bias.

Abstract: A processing system including at least one processor may obtain a time series of measurement values from a communication network and train a prediction model in accordance with the time series of measurement values to predict future instances of an event of interest, where the time series of measurement values is labeled with one or more indicators of instances of the event of interest. The processing system may then generate a deterministic finite automaton based upon the prediction model, convert the deterministic finite automaton into a rule set, and deploy the rule set to at least one network component of the communication network.

Abstract: Aspects of the subject disclosure may include, for example, system and apparatus that enable operations that may include receiving, by a processing system, project data defining a proposed machine learning (ML) project of an entity and storing the project data in a project database with other project data for other projects. The operations may further include extracting extracted features of the proposed project and, based on the extracted features, determining a clustering assignment for the proposed project. Determining the clustering assignment may comprise comparing information about the proposed project including the extracted features with information about the other projects and assigning the proposed project to a cluster including one or more projects having similar bias characteristics as the proposed project. The operations may further include determining a risk of potential bias for the proposed project and, based on the risk of bias, recommending a corrective action to reduce the risk of bias.

Abstract: In one example, a method includes identifying a target performance metric of a machine learning algorithm, wherein the target performance metric is to be improved, obtaining a set of auxiliary data from a plurality of auxiliary data sources, wherein the plurality of auxiliary data sources is separate from a training data set used to train the machine learning algorithm, selecting a candidate attribute type from the set of auxiliary data, identifying a quality metric for the candidate attribute type, calculating a change in the target performance metric when data values associated with the candidate attribute type are included in the training data set, determining that a tradeoff between the target performance metric and the quality metric of the candidate attribute type is satisfied by inclusion of the data values in the training data set, and training the machine learning algorithm using the training data set augmented with the data value.

Abstract: A method performed by a processing system including at least one processor includes obtaining an output of a machine learning algorithm, identifying a vulnerability in the output of the machine learning algorithm, wherein the vulnerability relates to a bias in the output, integrating auxiliary data from an auxiliary data source of a plurality of auxiliary data sources into the machine learning algorithm to try to compensate for the vulnerability, determining whether the integrating has compensated for the vulnerability, and generating a runtime output using the machine learning algorithm when the processing system determines that the integrating has compensated for the vulnerability.