Patents by Inventor Marcel Zalmanovici
Marcel Zalmanovici has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230274169Abstract: An example system includes a processor to receive a data set. The processor can generate a data slice rule based on a data observation for a data point in the data set. The processor can generate an instance of data based on the generated data slice rule.Type: ApplicationFiled: February 28, 2022Publication date: August 31, 2023Inventors: Orna RAZ, George KOUR, Ramasuri NARAYANAM, Samuel Solomon ACKERMAN, Marcel ZALMANOVICI
-
Patent number: 11734143Abstract: A method, apparatus and a product for determining a performance measurement of predictors. The method comprises obtaining a dataset comprising data instances. Each data instance is associated with a label; obtaining a predictor. The predictor is configured to provide a prediction of a label for a data instance; determining a plurality of data slices that are subsets of the dataset. computing, for each data slice in the plurality of data slices and based on an application of the predictor on each data instance that is mapped to the data slice, a performance measurement that is indicative of a successful label prediction for a data instance comprised by the data slice, whereby obtaining a plurality of performance measurements; based on the plurality of performance measurements, computing a performance measurement of the predictor over the dataset; if the performance measurement of the predictor is below a threshold, performing a mitigating action.Type: GrantFiled: April 10, 2020Date of Patent: August 22, 2023Assignee: International Business Machines CorporationInventors: Orna Raz, Eitan Farchi, Marcel Zalmanovici
-
Publication number: 20230237343Abstract: An example system includes a processor to receive a test set, data slices, and a measure of interest. The processor can rank the data slices based on the test set, the data slices, and the set of measures of interest. The test set includes data points from the same feature space used to train a machine learning model. Each data slice is ranked according to generated slice grades representing unique information contribution of each data slice to the measure of interest with respect to the other data slices. The processor can then present the ranked data slices.Type: ApplicationFiled: January 26, 2022Publication date: July 27, 2023Inventors: Orna RAZ, Samuel Solomon ACKERMAN, Marcel ZALMANOVICI, Eitan Daniel FARCHI, Ramasuri NARAYANAM
-
Publication number: 20230205847Abstract: Systems and methods for automatically identifying in a dataset insufficient data for learning, or records with anomalous combinations of feature values, by partition of numeric and/or categorical data space into human-interpretable regions are disclosed. The method comprises: receiving a dataset of numeric and/or categorical features with a plurality of observations. Calculating observation density for each observation according to a distance or anomaly based metric, and receiving a density measurement. Partitioning the dataset along the numeric and/or categorical features according to the density measurement of each observation by a perpendicular cut along the feature spaces, receiving a map of a plurality of hyper-rectangular shapes representing various levels of density including empty spaces.Type: ApplicationFiled: December 26, 2021Publication date: June 29, 2023Inventors: Samuel Solomon Ackerman, Orna Raz, Marcel Zalmanovici, Eitan Daniel Farchi, Avi Ziv
-
Patent number: 11568169Abstract: A method, apparatus and product for identifying data drifts.Type: GrantFiled: April 28, 2019Date of Patent: January 31, 2023Assignee: International Business Machines CorporationInventors: Eitan Farchi, Orna Raz, Marcel Zalmanovici
-
Patent number: 11514691Abstract: A computer system trains a machine learning model. A vector representation is generated for each document in a collection of documents. The documents are clustered based on the vector representations of the documents to produce a plurality of clusters. A training set is produced by selecting one or more documents from each cluster, wherein the selected documents represent a sample of the collection of documents to train the machine learning model. The machine learning model is trained by applying the training set to the machine learning model. Embodiments of the present invention further include a method and program product for training a machine learning model in substantially the same manner described above.Type: GrantFiled: June 12, 2019Date of Patent: November 29, 2022Assignee: International Business Machines CorporationInventors: Pathirage D. S. U. Perera, Eitan D. Farchi, Orna Raz, Ramani Routray, Sheng Hua Bao, Marcel Zalmanovici
-
Patent number: 11481667Abstract: Embodiments of the present systems and methods may provide improved machine learning performance even though data drift has occurred. For example, a method may comprise providing a machine learning model in a computer system, operating the machine learning model using a first dataset to obtain results of the first dataset, operating the machine learning model using a second dataset to obtain results of the second dataset, performing statistical testing on a confidence distribution of results of the first dataset and of results of the second dataset to determine a difference in a result confidence distribution between the first dataset and of the second dataset, and determining whether data included in the second dataset has data drift relative to the first dataset based on the difference in a result confidence distribution between the first dataset and of the second dataset.Type: GrantFiled: January 24, 2019Date of Patent: October 25, 2022Assignee: International Business Machines CorporationInventors: Orna Raz, Marcel Zalmanovici, Aviad Zlotnick
-
Publication number: 20220172124Abstract: A system and method for generating data slices for validating a classifier and validating the classifier. The classifier is trained using a training data set to train the underlying machine learning algorithm. Data is passed through the trained classifier to obtain results. The results are scored to determine the likelihood that the classifier correctly classified the data. Features are identified in the data set that can be used to validate the classifier. Based on the identified features at least one data slice in the data set is identified. The classifier is validated using the at least one data slice.Type: ApplicationFiled: December 2, 2020Publication date: June 2, 2022Inventors: Orna Raz, Marcel Zalmanovici, Eitan Daniel Farchi, Raviv Gal, Avi Ziv
-
Patent number: 11314892Abstract: A method, a computerized apparatus and a computer program product for mitigating governance and regulation implications on machine learning. A governance impact assessment is generated for a partial data set generated by applying a data governance enforcement on a data set of instances comprising valuations of a feature vector. The partial data set comprises partial instances each comprising partial feature vectors. The governance impact assessment comprises information about data excluded from the data set. A machine learning model trained based on the partial data set and configured to provide an estimated prediction for a partial instance is obtained. A set of core features is determined. A bias introduced by the data governance is identified based on a core feature being affected by the data governance. In response to identifying a bias, an anti-bias procedure is applied on the machine learning model, whereby mitigating the bias introduced by the data governance.Type: GrantFiled: June 26, 2019Date of Patent: April 26, 2022Assignee: International Business Machines CorporationInventors: Sima Nadler, Orna Raz, Marcel Zalmanovici
-
Publication number: 20210319354Abstract: A method, apparatus and a product for determining a performance measurement of predictors. The method comprises obtaining a dataset comprising data instances. Each data instance is associated with a label; obtaining a predictor. The predictor is configured to provide a prediction of a label for a data instance; determining a plurality of data slices that are subsets of the dataset. computing, for each data slice in the plurality of data slices and based on an application of the predictor on each data instance that is mapped to the data slice, a performance measurement that is indicative of a successful label prediction for a data instance comprised by the data slice, whereby obtaining a plurality of performance measurements; based on the plurality of performance measurements, computing a performance measurement of the predictor over the dataset; if the performance measurement of the predictor is below a threshold, performing a mitigating action.Type: ApplicationFiled: April 10, 2020Publication date: October 14, 2021Inventors: ORNA RAZ, Eitan Farchi, Marcel Zalmanovici
-
Publication number: 20210125080Abstract: A method, system and computer program product, the method comprising: creating a model representing underperforming cases; from a case collection having a total performance, and which comprises for each of a multiplicity of records: a value for each feature from a collection of features, a ground truth label and a prediction of a machine learning (ML) engine, obtaining one or more features; dividing the records into groups, based on values of the features in each record; for one group of the groups, calculating a performance parameter of the ML engine over the portion of the records associated with the group; subject to the performance parameter of the group being below the total performance in at least a predetermined threshold: determining a characteristic for the group; adding the characteristic of the group to the model; and providing the model to a user, thus indicating under-performing parts of the test collection.Type: ApplicationFiled: October 24, 2019Publication date: April 29, 2021Inventors: ORNA RAZ, Marcel Zalmanovici, Aviad Zlotnick
-
Publication number: 20200410129Abstract: A method, a computerized apparatus and a computer program product for mitigating governance and regulation implications on machine learning. A governance impact assessment is generated for a partial data set generated by applying a data governance enforcement on a data set of instances comprising valuations of a feature vector. The partial data set comprises partial instances each comprising partial feature vectors. The governance impact assessment comprises information about data excluded from the data set. A machine learning model trained based on the partial data set and configured to provide an estimated prediction for a partial instance is obtained. A set of core features is determined. A bias introduced by the data governance is identified based on a core feature being affected by the data governance. In response to identifying a bias, an anti-bias procedure is applied on the machine learning model, whereby mitigating the bias introduced by the data governance.Type: ApplicationFiled: June 26, 2019Publication date: December 31, 2020Inventors: Sima Nadler, Orna Raz, Marcel Zalmanovici
-
Publication number: 20200394461Abstract: A computer system trains a machine learning model. A vector representation is generated for each document in a collection of documents. The documents are clustered based on the vector representations of the documents to produce a plurality of clusters. A training set is produced by selecting one or more documents from each cluster, wherein the selected documents represent a sample of the collection of documents to train the machine learning model. The machine learning model is trained by applying the training set to the machine learning model. Embodiments of the present invention further include a method and program product for training a machine learning model in substantially the same manner described above.Type: ApplicationFiled: June 12, 2019Publication date: December 17, 2020Inventors: Pathirage D. S. U. Perera, Eitan D. Farchi, Orna Raz, Ramani Routray, Sheng Hua Bao, Marcel Zalmanovici
-
Publication number: 20200342310Abstract: A method, apparatus and product for identifying data drifts. The method comprising: obtaining a seen dataset, wherein the seen dataset comprises seen instances, each of which comprising feature values in a feature space; determining a first measurement of a statistical metric of the seen dataset; obtaining an unseen dataset, wherein the unseen dataset comprises unseen instances, each of which comprising features values in the feature space; determining a second measurement of the statistical metric of the unseen dataset; identifying a data drift in the unseen dataset with respect to the seen dataset based on the first and second measurements of the statistical metric; and performing a responsive action based on the identification of the data drift.Type: ApplicationFiled: April 28, 2019Publication date: October 29, 2020Inventors: Eitan Farchi, Orna Raz, Marcel Zalmanovici, Aviad Zlotnick
-
Publication number: 20200342260Abstract: A method, apparatus and product for identifying data drifts.Type: ApplicationFiled: April 28, 2019Publication date: October 29, 2020Inventors: Eitan Farchi, Orna Raz, Marcel Zalmanovici
-
Publication number: 20200242505Abstract: Embodiments of the present systems and methods may provide improved machine learning performance even though data drift has occurred. For example, a method may comprise providing a machine learning model in a computer system, operating the machine learning model using a first dataset to obtain results of the first dataset, operating the machine learning model using a second dataset to obtain results of the second dataset, performing statistical testing on a confidence distribution of results of the first dataset and of results of the second dataset to determine a difference in a result confidence distribution between the first dataset and of the second dataset, and determining whether data included in the second dataset has data drift relative to the first dataset based on the difference in a result confidence distribution between the first dataset and of the second dataset.Type: ApplicationFiled: January 24, 2019Publication date: July 30, 2020Inventors: Orna Raz, Marcel Zalmanovici, Aviad Zlotnick
-
Patent number: 9928116Abstract: A method, apparatus and computer program product for program migration, the method comprising: receiving a target host and an application to be migrated to a target host; estimating a target load of the application to be migrated; generating a synthetic application which simulates a simulated load, the simulated load being smaller than the target load; loading the synthetic application to the target host; monitoring behavior of the target host, the synthetic application, or a second application executed thereon; subject to the behavior being satisfactory: if the simulated load is smaller than the target load, then repeating said generating, said loading and said monitoring, wherein said loading is repeated with increased load; and otherwise migrating the application to the target.Type: GrantFiled: November 24, 2015Date of Patent: March 27, 2018Assignee: International Business Machines CorporationInventors: Sergey Novikov, Marcel Zalmanovici, Aviad Zlotnick
-
Patent number: 9665454Abstract: Computer-implemented method, computerized apparatus and computer program product for extracting test model from a textual test suite. The method comprises obtaining a test suite comprising test descriptions. The test descriptions are analyzed to extract attributes and values of a test model modeling a test space. Using the extracted attributes and values, the test model may be created. In some cases, the test model may be partial test model that a user can use as a starting point for manually modeling the textual test suite.Type: GrantFiled: May 14, 2014Date of Patent: May 30, 2017Assignee: International Business Machines CorporationInventors: Andre Heilper, Marcel Zalmanovici
-
Patent number: 9600403Abstract: A method, product and apparatus for creating functional model of test cases. The method comprising obtaining a set of test cases, wherein each test case of the set of test cases comprises free-text; defining one or more tags, wherein each tag of the one or more tags is associated with a query that is configured, when applied, to determine possession of the tag with respect to a test case based on the free-text; applying the queries on the set of test cases to determine possession of the of the one or more tags for each test case; and generating a functional model based on the set of test cases, wherein the functional model comprising for each tag of the one or more tags, a corresponding functional attribute.Type: GrantFiled: August 30, 2015Date of Patent: March 21, 2017Assignee: International Business Machines CorporationInventors: Orna Raz, Randall L Tackett, Paul A Wojciak, Marcel Zalmanovici, Aviad Zlotnick
-
Publication number: 20170060734Abstract: A method, product and apparatus for creating functional model of test cases. The method comprising obtaining a set of test cases, wherein each test case of the set of test cases comprises free-text; defining one or more tags, wherein each tag of the one or more tags is associated with a query that is configured, when applied, to determine possession of the tag with respect to a test case based on the free-text; applying the queries on the set of test cases to determine possession of the of the one or more tags for each test case; and generating a functional model based on the set of test cases, wherein the functional model comprising for each tag of the one or more tags, a corresponding functional attribute.Type: ApplicationFiled: August 30, 2015Publication date: March 2, 2017Inventors: Orna RAZ, Randall L. Tackett, Paul A. Wojciak, Marcel Zalmanovici, Aviad Zlotnick