Abstract: Machine learning model development and optimization tools are provided that ensure performance validation and data sufficiency for regulatory approval. According to an embodiment, a computer implemented method can comprise training a machine learning model to perform an inferencing task on an initial set of data samples included in a sample population. In various embodiments, the model can include a medical AI model. The method further comprises determining, by the system, subgroup performance measures for subgroups of the data samples respectively associated with different metadata factors, wherein the subgroup performance measures reflect performance accuracy of the machine learning model with respect to the subgroups. The method further comprises determining, by the system, whether the machine learning model meets an acceptable level of performance for deployment in a field environment based on whether the subgroup performance measures respectively satisfy a threshold subgroup performance measure.

Abstract: Machine learning model development and optimization tools are provided that ensure performance validation and data sufficiency for regulatory approval. According to an embodiment, a computer implemented method can comprise training a machine learning model to perform an inferencing task on an initial set of data samples included in a sample population. In various embodiments, the model can include a medical AI model. The method further comprises determining, by the system, subgroup performance measures for subgroups of the data samples respectively associated with different metadata factors, wherein the subgroup performance measures reflect performance accuracy of the machine learning model with respect to the subgroups. The method further comprises determining, by the system, whether the machine learning model meets an acceptable level of performance for deployment in a field environment based on whether the subgroup performance measures respectively satisfy a threshold subgroup performance measure.

Abstract: Systems and techniques that facilitate data diversity visualization and/or quantification for machine learning models are provided. In various embodiments, a processor can access a first dataset and a second dataset, where a machine learning (ML) model is trained on the first dataset. In various instances, the processor can obtain a first set of latent activations generated by the ML model based on the first dataset, and a second set of latent activations generated by the ML model based on the second dataset. In various aspects, the processor can generate a first set of compressed data points based on the first set of latent activations, and a second set of compressed data points based on the second set of latent activations, via dimensionality reduction. In various instances, a diversity component can compute a diversity score based on the first set of compressed data points and second set of compressed data points.

Abstract: Techniques are provided for enhancing the efficiency and accuracy of annotating data samples for supervised machine learning algorithms using an advanced annotation pipeline. According to an embodiment, a method can comprise collecting, by a system comprising a processor, unannotated data samples for input to a machine learning model and storing the unannotated data samples in an annotation queue. The method further comprises determining, by the system, annotation priority levels for respective unannotated data samples of the unannotated data samples, selecting, by the system from amongst different annotation techniques, one or more of the different annotation techniques for annotating the respective unannotated data samples based the annotation priority levels associated with the respective unannotated data samples.

Abstract: Machine learning model development and optimization tools are provided that ensure performance validation and data sufficiency for regulatory approval. According to an embodiment, a computer implemented method can comprise training a machine learning model to perform an inferencing task on an initial set of data samples included in a sample population. In various embodiments, the model can include a medical AI model. The method further comprises determining, by the system, subgroup performance measures for subgroups of the data samples respectively associated with different metadata factors, wherein the subgroup performance measures reflect performance accuracy of the machine learning model with respect to the subgroups. The method further comprises determining, by the system, whether the machine learning model meets an acceptable level of performance for deployment in a field environment based on whether the subgroup performance measures respectively satisfy a threshold subgroup performance measure.

Abstract: Techniques are provided for enhancing the efficiency and accuracy of annotating data samples for supervised machine learning algorithms using an advanced annotation pipeline. According to an embodiment, a method can comprise collecting, by a system comprising a processor, unannotated data samples for input to a machine learning model and storing the unannotated data samples in an annotation queue. The method further comprises determining, by the system, annotation priority levels for respective unannotated data samples of the unannotated data samples, selecting, by the system from amongst different annotation techniques, one or more of the different annotation techniques for annotating the respective unannotated data samples based the annotation priority levels associated with the respective unannotated data samples.

Abstract: Techniques are provided for enhancing the efficiency and accuracy of annotating data samples for supervised machine learning algorithms using an advanced annotation pipeline. According to an embodiment, a method can comprise collecting, by a system comprising a processor, unannotated data samples for input to a machine learning model and storing the unannotated data samples in an annotation queue. The method further comprises determining, by the system, annotation priority levels for respective unannotated data samples of the unannotated data samples, selecting, by the system from amongst different annotation techniques, one or more of the different annotation techniques for annotating the respective unannotated data samples based the annotation priority levels associated with the respective unannotated data samples.

Abstract: A method for identifying and characterizing shape variations in parts. Measurements are taken of a population of parts and the shapes of the parts are expressed as a function having a plurality of coefficients. Discrete or function error maps are developed from the coefficients and a principal components analysis is performed on the error maps to identify the principal components of variation of the parts. The parts may be grouped into sub-populations representing ranges of variation along each of the principal components of variation, and downstream processes may be controlled differently for each sub-population. In one embodiment, a typical (re-generated) part shape is identified along multiple principal components of variation, and a tool path is controlled to be responsive to the typical part shape. Information regarding the principal components of variation may further be used to revise upstream manufacturing processes to advantageously affect the distribution of error in subsequently manufactured parts.

Abstract: A method of valuation of large groups of assets by partial full underwriting, partial sample underwriting and inferred values of the remainder using an iterative and adaptive statistical evaluation of all assets and statistical inferences drawn from the evaluation and applied to generate inferred values. Individual asset values are developed and listed in tables so that individual asset values can be taken and quickly grouped in any desired or prescribed manner for bidding purposes. The assets are collected into a database, divided by credit variable, subdivided by ratings as to those variables and then rated individually. The assets are then regrouped according to a bidding grouping and a collective valuation established by cumulating the individual valuations.