Patents by Inventor Sean Javad Kamkar
Sean Javad Kamkar has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
SYSTEMS AND CACHE-BASED METHODS FOR EXPLAINING TREE-BASED MODELS USING INTERVENTIONAL SHAPLEY VALUES
Publication number: 20240386492Abstract: Systems and cache-based methods for explaining tree-based models using interventional Shapley values are disclosed. With this technology, interventional Shapley values are used to compute attribution values from a leaf-wise approach within tree-based machine learning models. Reference traversal tables and test traversal tables are created and stored for each leaf of a decision tree. Based on the created tables, a subset of traversal permutations and respective subset size are determined on a tree by tree, leaf by leaf and feature by feature basis. For each of the nodes in a traversal path to each of the leaves, partial attribution values are generated, and an attribution for the node is adjusted based on the generated partial attribution values and a multiplier indicated in the reference traversal tables. An output explanation of a score can advantageously be obtained with reduced computational complexity and runtime.Type: ApplicationFiled: May 17, 2024Publication date: November 21, 2024Inventors: Geoffrey Michael Ward, Saar Yalov, Zijian Wu, Sean Javad Kamkar -
Patent number: 12131241Abstract: Methods, non-transitory computer readable media, and model evaluations systems for understanding diverse machine learning models (MLMs) are disclosed. In some examples, a feature contribution value is determined for features included in a reference or evaluation input data set. The evaluation input data set represents a protected class population and each feature contribution value identifies a contribution by a feature to a difference in output generated by an MLM for the evaluation input data set. Model explanation information is generated using the feature contribution values and execution of the MLM is monitored. The model explanation information explains the difference in output generated by the MLM for the evaluation input data set and includes information relating to a model-based decision. A report is generated from a knowledge graph for the MLM and output via a GUI to an operator device that includes the model explanation information.Type: GrantFiled: October 19, 2023Date of Patent: October 29, 2024Assignee: ZestFinance, Inc.Inventors: John Wickens Lamb Merrill, Geoffrey Michael Ward, Sean Javad Kamkar, John Joseph Beahan, Jr., Mark Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik
-
Publication number: 20240311909Abstract: Systems and methods for generating tree-based models with improved fairness are disclosed. The disclosed process generates a first tree-based machine learning model, which is preferably trained to predict if a financial loan will be repaid. The process also determines an accuracy of the first tree-based machine learning mode. In addition, the process determines a fairness of the first tree-based machine learning model. The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status. The process then generates a second different tree-based machine learning model, which is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model.Type: ApplicationFiled: May 31, 2024Publication date: September 19, 2024Inventors: Sean Javad Kamkar, Geoffrey Michael Ward, Jerome Louis Budzik
-
Publication number: 20240303553Abstract: This application describes systems and methods for generating machine learning models (MLMs). An exemplary method includes obtaining a sample and user input data characterizing a product or service. A subset of the data is selected from the sample based on sampling the sample according to the user input data. An MLM is trained by applying the data subset as training input to the MLM, thereby providing a trained MLM to emulate a customer selection process unique to the product or service. A user interface (UI) configured to receive other user input data and cause the trained MLM to execute on the other user input data, thereby testing the trained MLM, is presented. A summary of results from the execution of the trained MLM is generated and presented in the UI. The summary of results indicates a contribution to the trained MLM of each of a plurality of features.Type: ApplicationFiled: April 29, 2024Publication date: September 12, 2024Inventors: David Sheehan, Siavash Yasani, Bingjia Wang, Yunyan Zhang, Qiumeng Yu, Ruochen Zha, Adam Kleinman, Sean Javad Kamkar, Lingzhi Du, Saar Yalov, Jerome Louis Budzik
-
Publication number: 20240220869Abstract: Systems and methods for model evaluation. A model is evaluated by performing a decomposition process for a model output, relative to a baseline input data set.Type: ApplicationFiled: March 13, 2024Publication date: July 4, 2024Inventors: Douglas C. Merrill, Michael Edward Ruberry, Ozan Sayin, Bojan Tunguz, Lin Song, Esfandiar Alizadeh, Melanie Eunique DeBruin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, Armen Avedis Donigian, Eran Dvir, Sean Javad Kamkar, Vishwaesh Rajiv, Evan George Kriminger
-
Patent number: 12002094Abstract: Systems and methods for generating tree-based models with improved fairness are disclosed. The disclosed process generates a first tree-based machine learning model, which is preferably trained to predict if a financial loan will be repaid. The process also determines an accuracy of the first tree-based machine learning mode. In addition, the process determines a fairness of the first tree-based machine learning model. The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status. The process then generates a second different tree-based machine learning model, which is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model.Type: GrantFiled: August 7, 2023Date of Patent: June 4, 2024Assignee: ZestFinance, Inc.Inventors: Sean Javad Kamkar, Geoffrey Michael Ward, Jerome Louis Budzik
-
Patent number: 11972338Abstract: This application describes systems and methods for generating machine learning models (MLMs). An exemplary method includes obtaining a sample and user input data characterizing a product or service. A subset of the data is selected from the sample based on sampling the sample according to the user input data. An MLM is trained by applying the data subset as training input to the MLM, thereby providing a trained MLM to emulate a customer selection process unique to the product or service. A user interface (UI) configured to receive other user input data and cause the trained MLM to execute on the other user input data, thereby testing the trained MLM, is presented. A summary of results from the execution of the trained MLM is generated and presented in the UI. The summary of results indicates a contribution to the trained MLM of each of a plurality of features.Type: GrantFiled: May 2, 2023Date of Patent: April 30, 2024Assignee: ZestFinance, Inc.Inventors: David Sheehan, Siavash Yasini, Bingjia Wang, Yunyan Zhang, Qiumeng Yu, Ruochen Zha, Adam Kleinman, Sean Javad Kamkar, Lingzhi Du, Saar Yalov, Jerome Louis Budzik
-
Publication number: 20240127125Abstract: Systems and methods for training models to improve fairness.Type: ApplicationFiled: December 12, 2023Publication date: April 18, 2024Inventors: Sean Javad Kamkar, Michael Egan Van Veen, Feng Li, Mark Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik, John Wickens Lamb Merrill
-
Patent number: 11960981Abstract: Systems and methods for model evaluation. A model is evaluated by performing a decomposition process for a model output, relative to a baseline input data set.Type: GrantFiled: March 8, 2019Date of Patent: April 16, 2024Assignee: ZESTFINANCE, INC.Inventors: Douglas C. Merrill, Michael Edward Ruberry, Ozan Sayin, Bojan Tunguz, Lin Song, Esfandiar Alizadeh, Melanie Eunique DeBruin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, Armen Avedis Donigian, Eran Dvir, Sean Javad Kamkar, Vishwaesh Rajiv, Evan George Kriminger
-
Publication number: 20240112209Abstract: Systems and methods for model evaluation. A protected class model that satisfies an accuracy threshold is built by using: data sets for use by a modeling system being evaluated, and protected class membership information for each data set. A target for the protected class model is a protected class membership variable indicating membership in a protected class. Each predictor of the protected class model is a predictor of an evaluated model used by the modeling system. A target of the evaluated model is different from the target of the protected class model. Each predictor is a set of one or more variables of the data sets. For each predictor of the protected class model, a protected class model impact ranking value and a modeling system impact ranking value are determined.Type: ApplicationFiled: December 8, 2023Publication date: April 4, 2024Inventors: Douglas C. Merrill, Michael Edward Ruberry, Ozan Sayin, Bojan Tunguz, Lin Song, Esfandiar Alizadeh, Melanie Eunique DeBruin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, Sean Javad Kamkar
-
Patent number: 11941650Abstract: Systems and methods for model evaluation. A protected class model that satisfies an accuracy threshold is built by using: data sets for use by a modeling system being evaluated, and protected class membership information for each data set. A target for the protected class model is a protected class membership variable indicating membership in a protected class. Each predictor of the protected class model is a predictor of an evaluated model used by the modeling system. A target of the evaluated model is different from the target of the protected class model. Each predictor is a set of one or more variables of the data sets. For each predictor of the protected class model, a protected class model impact ranking value and a modeling system impact ranking value are determined.Type: GrantFiled: August 1, 2018Date of Patent: March 26, 2024Assignee: ZestFinance, Inc.Inventors: Douglas C. Merrill, Michael Edward Ruberry, Ozan Sayin, Bojan Tunguz, Lin Song, Esfandiar Alizadeh, Melanie Eunique DeBruin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, Sean Javad Kamkar
-
Publication number: 20240070487Abstract: Systems and methods for generating and processing modeling workflows are disclosed. In some examples, a reference distribution of scores generated by a model is determined. The reference distribution of scores is recorded in a structured database. One or more unexpected scores are detected during execution of the model. To detect the one or more unexpected scores, a production distribution of scores is compared with the reference distribution of scores recorded in the structured database. The production distribution of scores is generated by the model for a production input data set. An alert is then provided to an external system, when an alert condition is determined to be satisfied based on the comparison. The alert indicates detection of the one or more unexpected scores.Type: ApplicationFiled: November 7, 2023Publication date: February 29, 2024Inventors: Douglas C. Merrill, Armen Donigian, Eran Dvir, Sean Javad Kamkar, Evan George Kriminger, Vishwaesh Rajiv, Michael Edward Ruberry, Ozan Sayin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, John J. Beahan, JR., John Wickens Lamb Merrill, Esfandiar Alizadeh, Liubo Li, Carlos Alberta Huertas Villegas, Feng Li, Randolph Paul Sinnott, JR.
-
Publication number: 20240046158Abstract: Methods, non-transitory computer readable media, and model evaluations systems for understanding diverse machine learning models (MLMs) are disclosed. In some examples, a feature contribution value is determined for features included in a reference or evaluation input data set. The evaluation input data set represents a protected class population and each feature contribution value identifies a contribution by a feature to a difference in output generated by an MLM for the evaluation input data set. Model explanation information is generated using the feature contribution values and execution of the MLM is monitored. The model explanation information explains the difference in output generated by the MLM for the evaluation input data set and includes information relating to a model-based decision. A report is generated from a knowledge graph for the MLM and output via a GUI to an operator device that includes the model explanation information.Type: ApplicationFiled: October 19, 2023Publication date: February 8, 2024Inventors: John Wickens Lamb Merrill, Geoffrey Michael Ward, Sean Javad Kamkar, John Joseph Beahan, JR., Mark Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik
-
Publication number: 20240046349Abstract: A method, in some implementations, may include obtaining output from a machine learning (ML) model responsive to input data, obtaining initial training data representing training data used to train the ML model, generating, based on the output from the ML model and the initial training data, correction training data that represents a desired alteration to the output from the ML model responsive to one or more particular subgroups in the input data, generating, based on the correction training data, a correction ML model configured to receive, as input, the input data and to output correction values which, when combined with the output from the ML model, perform the desired alteration, and generating corrected output as a combination of the output from the ML model and the output correction values from the correction ML model, and providing, for display, the corrected output.Type: ApplicationFiled: August 3, 2023Publication date: February 8, 2024Inventors: Geoffrey Michael Ward, Sean Javad Kamkar, Jerome Louis Budzik
-
Patent number: 11893466Abstract: Systems and methods for training models to improve fairness.Type: GrantFiled: January 12, 2021Date of Patent: February 6, 2024Assignee: ZESTFINANCE, INC.Inventors: Sean Javad Kamkar, Michael Egan Van Veen, Feng Li, Mark Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik, John Wickens Lamb Merrill
-
Patent number: 11847574Abstract: Systems and methods for generating and processing modeling workflows.Type: GrantFiled: April 25, 2019Date of Patent: December 19, 2023Assignee: ZESTFINANCE, INC.Inventors: Douglas C. Merrill, Armen Avedis Donigian, Eran Dvir, Sean Javad Kamkar, Evan George Kriminger, Vishwaesh Rajiv, Michael Edward Ruberry, Ozan Sayin, Yachen Yan, Derek Wilcox, John Candido, Benjamin Anthony Solecki, Jiahuan He, Jerome Louis Budzik, John J. Beahan, Jr., John Wickens Lamb Merrill, Esfandiar Alizadeh, Liubo Li, Carlos Alberta Huertas Villegas, Feng Li, Randolph Paul Sinnott, Jr.
-
Publication number: 20230377037Abstract: Systems and methods for generating tree-based models with improved fairness are disclosed. The disclosed process generates a first tree-based machine learning model, which is preferably trained to predict if a financial loan will be repaid. The process also determines an accuracy of the first tree-based machine learning mode. In addition, the process determines a fairness of the first tree-based machine learning model. The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status. The process then generates a second different tree-based machine learning model, which is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model.Type: ApplicationFiled: August 7, 2023Publication date: November 23, 2023Inventors: Sean Javad Kamkar, Geoffrey Michael Ward, Jerome Louis Budzik
-
Patent number: 11816541Abstract: Systems and methods for understanding diverse machine learning models.Type: GrantFiled: November 19, 2019Date of Patent: November 14, 2023Assignee: ZestFinance, Inc.Inventors: John Wickens Lamb Merrill, Geoffrey Michael Ward, Sean Javad Kamkar, John Joseph Beahan, Jr., Mark Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik
-
Publication number: 20230359944Abstract: This application describes systems and methods for generating machine learning models (MLMs). An exemplary method includes obtaining a sample and user input data characterizing a product or service. A subset of the data is selected from the sample based on sampling the sample according to the user input data. An MLM is trained by applying the data subset as training input to the MLM, thereby providing a trained MLM to emulate a customer selection process unique to the product or service. A user interface (UI) configured to receive other user input data and cause the trained MLM to execute on the other user input data, thereby testing the trained MLM, is presented. A summary of results from the execution of the trained MLM is generated and presented in the UI. The summary of results indicates a contribution to the trained MLM of each of a plurality of features.Type: ApplicationFiled: May 2, 2023Publication date: November 9, 2023Inventors: David Sheehan, Siavash Yasini, Bingjia Wang, Yunyan Zhang, Qiumeng Yu, Ruochen Zha, Adam Kleinman, Sean Javad Kamkar, Lingzhi Du, Saar Yalov, Jerome Louis Budzik
-
Patent number: 11720962Abstract: Systems and methods for generating tree-based models with improved fairness are disclosed. The disclosed process generates a first tree-based machine learning model, which is preferably trained to predict if a financial loan will be repaid. The process also determines an accuracy of the first tree-based machine learning mode. In addition, the process determines a fairness of the first tree-based machine learning model. The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status. The process then generates a second different tree-based machine learning model, which is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model.Type: GrantFiled: November 24, 2021Date of Patent: August 8, 2023Assignee: ZESTFINANCE, INC.Inventors: Sean Javad Kamkar, Geoffrey Michael Ward, Jerome Louis Budzik