Patents by Inventor Jason LOPATECKI
Jason LOPATECKI has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230334372Abstract: Techniques for optimizing a machine learning model. The techniques may include obtaining multiple predictions from a machine learning model, the predictions being based on at least one input feature vector, each input feature vector having one or more vector values; creating at least one slice of the predictions based on at least one vector value; determining a sensitive bias metric for the slice based on a sensitive group; determining a base metric for the slice based on a base group; determining a parity metric for the slice based on a ratio of the sensitive bias metric and the base metric; and optimizing the machine learning model based on the parity metric.Type: ApplicationFiled: April 13, 2023Publication date: October 19, 2023Applicant: ARIZE AI, INC.Inventors: Jason LOPATECKI, Reah MIYARA, Tsion BEHAILU, Aparna DHINAKARAN
-
Patent number: 11775871Abstract: Techniques for optimizing a machine learning model. The techniques can include: obtaining one or more embedding vectors based on a prediction of a machine learning model; mapping the embedding vectors from a higher dimensional space to a 2D/3D space to generate one or more high density points in the 2D/3D space; clustering the high-density points by running a clustering algorithm multiple times, each time with a different set of parameters to generate one or more clusters; applying a purity metric to each cluster to generate a normalized purity score of each cluster; identifying one or more clusters with a normalized purity score lower than a threshold; and optimizing the identifying one or more clusters.Type: GrantFiled: December 8, 2022Date of Patent: October 3, 2023Assignee: ARIZE AI, INC.Inventors: Jason Lopatecki, Aparna Dhinakaran, Francisco Castillo Carrasco, Michael Schiff, Nathaniel Mar
-
Publication number: 20230229971Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.Type: ApplicationFiled: January 26, 2023Publication date: July 20, 2023Applicant: ARIZE AI, INC.Inventors: Jason Lopatecki, Aparna Dhinakaran
-
Publication number: 20230186144Abstract: Techniques for determining a drift impact score in a machine learning model are disclosed. The techniques can include: obtaining a reference distribution of a machine learning model; obtaining a current distribution of the machine learning model; determining a statistical distance based on the reference distribution and the current distribution; determining a local feature importance parameter for each feature associated with a prediction made by the machine learning model; determining a cohort feature importance parameter for a cohort of multiple features based on the local feature importance parameter of each feature in the cohort; and determining a drift impact score for the cohort based on the statistical distance and the cohort feature importance parameter.Type: ApplicationFiled: December 10, 2021Publication date: June 15, 2023Applicant: ARIZE AI, INC.Inventors: Jason LOPATECKI, Aparna DHINAKARAN, Michael SCHIFF
-
Patent number: 11663527Abstract: Techniques for determining embedding drift score in a machine learning model. The techniques can include: obtaining one or more first embedding vectors based on at least one first prediction of a machine learning model; filtering the first embedding vectors based on a slice of the first prediction; determining a first average vector by averaging each dimension of the filtered first embedding vectors; obtaining one or more second embedding vectors on at least one second prediction of the machine learning model; filtering the second embedding vectors based on a slice of the second prediction; generating a second average vector by averaging each dimension of the filtered second embedding vectors; and determining an embedding drift score based on a distance measure of the first average vector and the second average vector.Type: GrantFiled: March 24, 2022Date of Patent: May 30, 2023Assignee: ARIZE AI, INC.Inventors: Jason Lopatecki, Francisco Castillo Carrasco, Aparna Dhinakaran, Michael Schiff
-
Patent number: 11615345Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.Type: GrantFiled: April 11, 2022Date of Patent: March 28, 2023Assignee: ARIZE AI, INC.Inventors: Jason Lopatecki, Aparna Dhinakaran
-
Publication number: 20220309399Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.Type: ApplicationFiled: April 11, 2022Publication date: September 29, 2022Applicant: ARIZE AI, INC.Inventors: Jason LOPATECKI, Aparna Dhinakaran
-
Patent number: 11315043Abstract: A system for optimizing a machine learning model. The machine learning model generates predictions based on at least one input feature vector, each input feature vector having one or more vector values; and an optimization module with a processor and an associated memory, the optimization module being configured to: create at least one slice of the predictions based on at least one vector value, determine at least one optimization metric of the slice that is based on at least a total number of predictions for the vector value, and optimize the machine learning model based on the optimization metric.Type: GrantFiled: March 25, 2021Date of Patent: April 26, 2022Assignee: ARIZE AI, INC.Inventors: Jason Lopatecki, Aparna Dhinakaran
-
Patent number: 11120467Abstract: Systems and methods are disclosed for characterizing websites and viewers, for predicting GRPs (Gross Rating Points) for online advertising media campaigns, and for pricing media campaigns according to GRPs delivered as opposed to impressions delivered. To predict GRPs for a campaign, systems and methods are disclosed for first characterizing polarized websites and then characterizing polarized viewers. To accomplish this, a truth set of viewers with known characteristics is first established and then compared with historic and current media viewing activity to determine a degree of polarity for different Media Properties (MPs)—typically websites offering ads—with respect to gender and age bias. A broader base of polarized viewers is then characterized for age and gender bias, and their propensity to visit a polarized MP is rated. Based on observed and calculated parameters, a GRP total is then predicted and priced to a client/advertiser for an online ad campaign.Type: GrantFiled: January 29, 2014Date of Patent: September 14, 2021Assignee: Adobe Inc.Inventors: John Hughes, Adam Rose, John M. Trenkle, Kevin Thakkar, Jason Lopatecki
-
Patent number: 11107119Abstract: Systems and methods are disclosed for conducting media lift studies for online advertising concurrently with operating an advertising campaign. While operating an advertising campaign for a first advertiser/client focused primarily on a set of intended ads and a specific targeted viewer audience, a non-intended ad is occasionally substituted to run in an ad slot, and is tracked as belonging to the first advertiser/client. The non-intended ad can be for example one of: an ad for a second advertiser/client; an alternate ad for the first advertiser/client; or a blank/unrelated ad. After the campaign, attribution results for the intended ads are adjusted according to those for non-intended ads to provide an indication of net media lift resulting from the intended ads—typically at no additional cost to the first advertiser/client. Analysis results may also be compared between different attribution data providers to determine which provide the more accurate attribution data.Type: GrantFiled: November 14, 2018Date of Patent: August 31, 2021Assignee: ADOBE INC.Inventors: John Hughes, Boaz Ram, Jason Lopatecki
-
Patent number: 11010794Abstract: Systems and methods are disclosed for employing supervised machine learning methods with activities and attributes of viewers with truth as input, to produce models that are utilized in determining probabilities that an unknown viewer belongs to one or more demographic segment categories. Using these models for processing viewer behavior, over a period of time a database of known categorized viewers is established, each categorized viewer having a probability of belonging to one or more segment categories. These probabilities are then used in bidding for online advertisements in response to impression opportunities offered in online media auctions. The probabilities are also used in predicting on-target impressions and GRPs (Gross Rating Points) in advance of online advertising media campaigns, and pricing those campaigns to advertiser/clients.Type: GrantFiled: June 4, 2014Date of Patent: May 18, 2021Assignee: ADOBE INC.Inventors: John M. Trenkle, John Hughes, Adam Rose, Kevin Thakkar, Jason Lopatecki
-
Patent number: 10949893Abstract: Systems and methods are disclosed for optimizing an online advertising campaign both before the campaign begins, and dynamically during the campaign. Optimizations are performed comparatively between a plurality of MPs (Media Properties) based on their relative cost-per-engagement. Comparisons are performed by first stack ranking MP inventory including any of sites, feeds, and verticals, based on cost per engagement. Once ranked, scores are assigned to the targeted inventory and a mean score is determined. Then, the inventory is rated as high, normal, or low impact based on their scores compared with the mean and a standard deviation for all scores. Higher impact sites with scores at least a standard deviation above the mean are initially favored, and the MP targeting strategy is dynamically adjusted during the campaign based on periodically re-evaluating the MP rankings, frequencies of engagement, and campaign progress relative to fulfillment in an allotted run time.Type: GrantFiled: September 16, 2019Date of Patent: March 16, 2021Assignee: Adobe Inc.Inventors: Kevin Thakkar, John M. Trenkle, John Hughes, Adam Rose, Jason Lopatecki
-
Patent number: 10911821Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of recurrent neural networks to generate media consumption predictions and providing media content to a target audience. For example, the disclosed system can train a plurality of long short-term memory neural networks for a plurality of users based on historical media consumption data over a plurality of time periods. In one or more embodiments, the disclosed system identifies a target audience including a subset of users and the corresponding neural networks. The disclosed system can then utilize the neural networks of the subset of users to generate a plurality of predictions for a future time period for the users. In some embodiments, the disclosed system then combines the predictions for the users to generate a media consumption prediction for the target audience for the future time period.Type: GrantFiled: September 27, 2018Date of Patent: February 2, 2021Assignee: ADOBE INC.Inventors: Jason Lopatecki, Julie Lee
-
Patent number: 10798465Abstract: The present disclosure relates to a content campaign system that improves the design and implementation of content campaigns. In particular, the content campaign system can receive television viewer information corresponding to television client devices and online activity information corresponding to client computing devices. Further, the content campaign system can identify a correspondence between the television viewer information and the online activity information for individual users and/or households. Based on the correspondence, the content campaign system can automatically generate targeting parameters for audiovisual content campaigns. For example, the content campaign system can recommend audiovisual content (e.g., a television advertisement) to provide to a target audience of users via a television broadcast based on the correlated television viewer information and online activity information of a particular user.Type: GrantFiled: August 10, 2017Date of Patent: October 6, 2020Assignee: ADOBE INC.Inventors: Jason Lopatecki, John Hughes, Greg Collison
-
Patent number: 10650404Abstract: Systems and methods for operating placebo-based experiments are described for online advertisements. One or more embodiments of the disclosed systems and methods utilizes an ad swapping approach to offer placebo media exposures, at no additional cost to an advertiser. One or more embodiments further provide a native experimentation platform that allows users to run tests of ad placements to measure the effectiveness of ads and view results displayed on a user interface. The disclosed systems and methods can assign viewers into a test group if shown the test ad or a control group if shown a control ad. The control ad can be provided at no cost to the advertiser for embodiments where the placebo ad belongs to an alternative advertiser. Effectiveness of third party attribution can also be evaluated. The disclosed systems and methods can define experiment parameters, including control frequency, test viewer groups, and control viewer groups.Type: GrantFiled: September 6, 2017Date of Patent: May 12, 2020Assignee: ADOBE INC.Inventors: Thomas Riordan, Narayan Kinhal, John Hughes, Jason Lopatecki, Darren Sue, Christopher Bell, Matthew Ellinwood
-
Publication number: 20200107070Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of recurrent neural networks to generate media consumption predictions and providing media content to a target audience. For example, the disclosed system can train a plurality of long short-term memory neural networks for a plurality of users based on historical media consumption data over a plurality of time periods. In one or more embodiments, the disclosed system identifies a target audience including a subset of users and the corresponding neural networks. The disclosed system can then utilize the neural networks of the subset of users to generate a plurality of predictions for a future time period for the users. In some embodiments, the disclosed system then combines the predictions for the users to generate a media consumption prediction for the target audience for the future time period.Type: ApplicationFiled: September 27, 2018Publication date: April 2, 2020Inventors: Jason Lopatecki, Julie Lee
-
Publication number: 20200013095Abstract: Systems and methods are disclosed for optimizing an online advertising campaign both before the campaign begins, and dynamically during the campaign. Optimizations are performed comparatively between a plurality of MPs (Media Properties) based on their relative cost-per-engagement. Comparisons are performed by first stack ranking MP inventory including any of sites, feeds, and verticals, based on cost per engagement. Once ranked, scores are assigned to the targeted inventory and a mean score is determined. Then, the inventory is rated as high, normal, or low impact based on their scores compared with the mean and a standard deviation for all scores. Higher impact sites with scores at least a standard deviation above the mean are initially favored, and the MP targeting strategy is dynamically adjusted during the campaign based on periodically re-evaluating the MP rankings, frequencies of engagement, and campaign progress relative to fulfillment in an allotted run time.Type: ApplicationFiled: September 16, 2019Publication date: January 9, 2020Inventors: Kevin Thakkar, John M. Trenkle, John Hughes, Adam Rose, Jason Lopatecki
-
Patent number: 10531163Abstract: Systems and methods are disclosed for planning, executing, reviewing, and reporting the results of an advertising campaign to be run on TV. A demand-side platform receives ad slot opportunities from TV programming sources, and analyzes the ad slots to produce a prioritized list of placement opportunities for the advertising campaign to be presented to advertiser/clients. Each ad slot is analyzed with respect to past viewership data and with respect to desired targeting characteristics that may include conventional age and gender targeting, or additionally strategic targeting characteristics. Scores are established for each ad slot with respect to numbers of projected on-target impressions and/or a cost for projected on-target impressions. The scores are sorted to produce the prioritized list. Projected results can be viewed with respect to any or all of network, day, and daypart. After a campaign has completed, viewership data representing actual results is acquired, processed, and reported.Type: GrantFiled: February 19, 2019Date of Patent: January 7, 2020Assignee: Adobe Inc.Inventors: Alexander R. Hood, Jason Lopatecki, Justin K. Sung, Greg Collison, David Innes-Gawn
-
Patent number: 10462504Abstract: Presentation of a video clip is made to persons having a high probability of viewing the clip. A database containing viewers of previously offered video clips is analyzed to determine similarities of preferences among viewers. When a new video clip has been watched by one or more viewers in the database, those viewers who have watched the new clip with positive results are compared with others in the database who have not yet seen it. Prospective viewers with similar preferences are identified as high likelihood candidates to watch the new clip when presented. Bids to offer the clip are based on the degree of likelihood. For one embodiment, a data collection agent (DCA) is loaded to a player and/or to a web page to collect viewing and behavior information to determine viewer preferences. Viewer behavior may be monitored passively by different disclosed methods.Type: GrantFiled: March 29, 2018Date of Patent: October 29, 2019Assignee: Adobe Inc.Inventors: Jason Lopatecki, Adam Rose, John Hughes, Brett Wilson
-
Patent number: 10453100Abstract: Systems and methods are disclosed for optimizing an online advertising campaign both before the campaign begins, and dynamically during the campaign. Optimizations are performed comparatively between a plurality of MPs (Media Properties) based on their relative cost-per-engagement. Comparisons are performed by first stack ranking MP inventory including any of sites, feeds, and verticals, based on cost per engagement. Once ranked, scores are assigned to the targeted inventory and a mean score is determined. Then, the inventory is rated as high, normal, or low impact based on their scores compared with the mean and a standard deviation for all scores. Higher impact sites with scores at least a standard deviation above the mean are initially favored, and the MP targeting strategy is dynamically adjusted during the campaign based on periodically re-evaluating the MP rankings, frequencies of engagement, and campaign progress relative to fulfillment in an allotted run time.Type: GrantFiled: December 9, 2014Date of Patent: October 22, 2019Assignee: Adobe Inc.Inventors: Kevin Thakkar, John M. Trenkle, John Hughes, Adam Rose, Jason Lopatecki