MEDIA AND MARKETING OPTIMIZATION WITH CROSS PLATFORM CONSUMER AND CONTENT INTELLIGENCE
The invention is directed to a computer-implemented method of analyzing video interactions on internet-supported computer platforms, such as online social media platforms, to extract video and audience intelligence, i.e. unique analytics, insights and recommendations for audience engagement optimization, audience engagement, network growth, advertising, and marketing purposes.
Latest ICX Media, Inc. Patents:
The present application is a continuation of and relies on the disclosures of and claims priority to and the benefit of the filing dates of U.S. application Ser. No. 16/838,234, filed Apr. 2, 2020 and of U.S. Provisional Application No. 62/828,814, filed Apr. 3, 2019. The disclosures of those applications are hereby incorporated by reference herein in their entireties.
BACKGROUND OF THE INVENTION Field of the InventionEmbodiments of the present invention are directed to a computer-implemented method of analyzing video interactions on internet-supported computer platforms, such as online social media platforms, to extract video intelligence, i.e. unique insights and recommendations for audience engagement optimization, network growth, advertising, and marketing purposes. In aspects, the more general field of invention is commonly referred to by people of ordinary skill in the art as “big data.”
Embodiments include a software implementation that enables the study of audience behavior from their interactions on social media platforms, which yields insights for video content creators on content and messaging that will be more or less successful with their target audiences.
One focus area is on the language used by consumers of video content, and the design and implementation of language-based classifiers that can be used to understand the characteristics of audiences of specific videos, of individual creators, or groups of related creators.
Another focus area is on automated video content analysis, where audio-visual components are extracted from raw video content and used as features to construct models to predict the interest and engagement of different audience demographics.
Embodiments also include an unsupervised machine learning model that analyzes both language and video patterns to identify peer sets of creators, which enables cross-creator analytics and comparison across different social media platforms.
Description of Related ArtThe process of designing, sourcing, producing, publishing, marketing, and optimizing video content on social media platforms has historically relied upon trial-and-error. Video content creators create content they believe may resonate with audiences, and select a social media platform or platforms on which to publish the new content. Next, they use a combination of free-text and platform-specific drop-down fields to add descriptive titles, narratives, tags, and other metadata to the content, which will be used by the social media platforms to make the content discoverable and appealing to the video consumers searching through content. Unfortunately, such ad hoc processes often result in many wasted hours of creator time, failures to successfully market content to target audiences, missed opportunities to unpack and explore contingencies within a creator's viewership, and poor search engine optimization for effective audience capture. As such, computer-implemented and automated processes for capturing potential audiences of social media video content and increasing their engagement are needed.
SUMMARY OF THE INVENTIONEmbodiments of the present invention infer audience attributes using computer-implemented natural language processing and supervised machine learning on the textual artifacts of their interactions with video content.
For example, in one embodiment, a computer-implemented method is provided that allows for reviewing and analyzing viewing and engagement habits of video consumers to algorithmically determine their gender, age, location, ethnicity, education, income, topical interests, personality type, political stance, sentiment (positive, negative and neutral) and emotion categories (e.g., anger, joy, humor, sadness) and sentiment themes, based on consumer video interactions across online social media platforms such as YouTube, Facebook, Instagram, Twitter, Vimeo and Twitch, by way of example.
In another embodiment, a computer-implemented method is provided that allows for reviewing and analyzing video viewing and engagement habits of consumers and determining topical interest areas of social video consumers based on the video content they watch. These topical interests can be used to find similarities and differences between two or more target audiences, such as the affinity between the audience of an independent video creator and that of a large consumer brand, or the differences between the audience of a large consumer brand and that of their competitors.
Embodiments also analyze the audio-visual components (objects, people, ambient noise, music, etc.) of videos with respect to their audience engagement patterns including views, likes, dislikes, upvotes, downvotes, shares, links, quotes, and comments.
For instance, in another embodiment, a computer-implemented method is provided that allows for recognizing audio components of videos and modeling the relationships between those audio components and the engagement metrics for video audiences across online social media platforms such as YouTube, Facebook, Instagram, Twitter, Vimeo and Twitch with respect to their demographics, interests, sentiments, and emotions.
In another embodiment, a computer-implemented method is provided that allows for recognizing visual components of videos (by analyzing video frames and identifying elements of videos, including the setting, number and ethnicity of cast, objects in the video, and music) and modeling the relationships between those components (objects, background, color, shapes, human faces and figures) and engagement metrics for video audiences across online social media platforms such as YouTube, Facebook, Instagram, Twitter, Vimeo and Twitch with respect to their demographics, interests, sentiments, and emotions.
Embodiments also evaluate performance and practices of a content creator with respect to their extracted peer set. In another embodiment, a computer-implemented method is provided that allows for extrapolating peer sets of content creators based on video content similarity, content statistics, audience demographics and audience engagement metrics across online social media platforms such as YouTube, Facebook, Instagram, Twitter, Vimeo and Twitch.
In another embodiment, a computer-implemented method is provided that extrapolates trends in high-engagement video topics, themes, and elements from peer set video content, thereby rendering automated recommendations for video creators for in-demand content.
In another embodiment, a computer-implemented method is provided that identifies real-time or near real-time trends from video topics, themes, and elements from peer set video content, thereby rendering recommendations for video creators on optimal post time, frequency, platform, length, messaging and marketing (title, description, keywords, hashtags).
In another embodiment, a computer-implemented method is provided that renders automated recommendations for optimized video messaging and marketing (title, description, keywords, hashtags) to content creators that can be accepted and automatically applied.
The accompanying drawings illustrate certain aspects of embodiments of the present invention, and should not be used to limit the invention. Together with the written description the drawings serve to explain certain principles of the invention.
Reference will now be made in detail to various exemplary embodiments of the invention. It is to be understood that the following discussion of exemplary embodiments is not intended as a limitation on the invention. Rather, the following discussion is provided to give the reader a more detailed understanding of certain aspects and features of the invention.
Embodiments of the present invention provide a computer-implemented mechanism for automatically extracting video intelligence from existing video data to provide tailored insights for video content creators to empower informed, tactical and strategic decision-making around both the creative process and the ensuing publishing and marketing processes.
In one embodiment, a computer processing device(s) analyzes existing video data in an asynchronous or synchronous manner to produce automated recommendations for how a video content creator can grow their network of viewers (i.e. get more people to watch their videos), optimize their audience engagement (i.e. get more people to provide feedback on their videos such as commenting, liking, upvoting, linking to, or sharing the video content), and more effectively market their content to specific demographic and interest groups (e.g. improve discoverability with Millennial Asian women, or with electronics enthusiasts from Atlanta, Ga.).
Demographic and topical recommendations are generated in part using an ensemble of computer-applied language-based classifiers to understand, and then predict, audience characteristics. This process includes extracting data (as shown in
Embodiments then determine features out of this data that capture interactions and consumer details.
In aspects and as used herein generally, vectorization may mean but is not limited to vectorizing comments or profile descriptions from social media. In general vectorization may refer to creating a set of words and textual features (e.g., emoji, keyphrases, or timestamps) that appear either in those comments and descriptions or in their metadata; creating a fixed-length array of numbers (like a cipher) that is the same length as the total count of words and textual features across all of the comments and descriptions; and creating a representation of every comment or some comments and descriptions with respect to the array. In some cases as used herein, the terms “multidimensional” and/or “high dimensional” would be understood by one of ordinary skill in the art, and the terms refer to machine learning as using “multidimensional” and/or “high dimensional” vectors, and in cases the terms are used interchangeably.
Once trained, the classifiers, using computing processing, classify new incoming data into meaningful categories. For example, after a new video is published, comments posted in response to the new video can be aggregated, and predictions can be made about the gender, age, location, ethnicity/race, religious views, political stance, education, income, personality type, and language of the commenters.
However, embodiments of the present invention go beyond just the textual contexts of videos (e.g. video titles, comments, tags, etc.) in order to derive video intelligence analytics and recommendations, such as combining textual context with audio-visual content (objects, people, ambient noise, music, etc.) to generate unique predictions about audience affinity and topical relevance. To achieve this, computer-implemented extraction of audiovisual content elements from raw video is performed, as illustrated in
The visual component operates using a computer processing mechanism to analyze each frame image of the video. These image frames are represented in pixel values and ultimately processed into numeric-based features after being passed through a series of processing and filtering steps. For example, images are passed through the process of detecting blank sections and cropping to exclude blank borders and divide the image into regions. Parts of an image can then be used to perform face detection, where min, max, mean and median color calculations are used to narrow down to specific portions of human faces (eyebrows, nostrils, lips, etc.). The processing steps return features that mathematically represent the useful image details. These features are then passed through pre-built neural network classifiers to recognize the objects observed in the images, such as a chair, a table, a beach, a human, or a cheetah. The visual component also undergoes more processing to detect humans in the images and analyze their faces. This additional processing helps in analyzing such features as the number of humans, genders, diversity, emotion, expression and more. In addition, more processing takes place leading to an image background analysis that differentiates between indoor versus outdoor settings and performs further classification of the background scene—such as home, theatre, forest and so on. Other image analysis of pixels, shapes and colors also plays into analysis at the video frame level. These per image frame elements are then combined per video to provide an aggregate video-based analysis. The audio-visual analysis components are also combined with other statistical, demographic, sentiment, emotion, and audience interest data, resulting in a more powerful analysis on consumer watching habits.
Once extracted, these elements are computationally transformed into numeric feature vectors that can be projected into a multidimensional space and used for downstream modeling as shown in
By offering an automated computer-implemented method with which to extract and transform raw text, audio, and video into features, embodiments of the present invention also provide a mechanism for comparing any two creators together, generating cross-creator analytics, and offering comparisons within specific peer sets. Peer sets or “cohorts” of creators are formed using unsupervised machine learning, as shown in
The creation of these cohorts is informed by high-level creator statistics (e.g. number of videos made, number of followers or subscribers, and/or daily views) as well as vectorized, extracted video content and language features.
Cohort clusters are computationally created and based on the nature of content produced, statistics, location, audience demographics and audience interests. In aspects, each content creator will have a peer-group based on these factors. Therefore, using the cohort models, a new creator can be served recommendations relative to the best practices of relevant successful content creators within their cohort, which can be much more nuanced and specific and reflect unique cohort behavioral patterns.
A nearest neighbor approach is used to find peer sets and the peers are further compared by performance within a cluster.
The present disclosure provides for a computer program comprising computer-executable instructions, which when the program is executed by a computer, cause the computer to carry out any of the processes, methods, and/or algorithms according to the above. The computer-executable instructions can be programmed in any suitable programming language, including JavaScript, C, C#, C++, Java, Python, Perl, Ruby, Swift, Visual Basic, and Objective C.
Also provided herein is a non-transitory computer-readable medium (or media) comprising computer-executable instructions, which when executed by a computer, cause the computer to carry out any of the processes, methods, and/or algorithms according to the above. As used in the context of this specification, a “non-transitory computer-readable medium (or media)” may include any kind of computer memory, including magnetic storage media, optical storage media, nonvolatile memory storage media, and volatile memory. Non-limiting examples of non-transitory computer-readable storage media include floppy disks, magnetic tape, conventional hard disks, CD-ROM, DVD-ROM, BLU-RAY, Flash ROM, memory cards, optical drives, solid state drives, flash drives, erasable programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile ROM, and RAM. The non-transitory computer readable media can include one or more sets of computer-executable instructions for providing an operating system as well as for implementing the processes, methods, and/or algorithms of the invention.
Embodiments of this disclosure include one or more computers or devices loaded with a set of the computer-executable instructions described herein. The computers or devices may be a general purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a particular machine, such that the one or more computers or devices are instructed and configured to carry out the calculations, processes, steps, operations, algorithms, statistical methods, formulas, or computational routines of this disclosure. The computer or device performing the specified calculations, processes, steps, operations, algorithms, statistical methods, formulas, or computational routines of this disclosure may comprise at least one processing element such as a central processing unit (i.e., processor) and a form of computer-readable memory which may include random-access memory (RAM) or read-only memory (ROM). The computer-executable instructions can be embedded in computer hardware or stored in the computer-readable memory such that the computer or device may be directed to perform one or more of the calculations, steps, processes and operations depicted and/or described herein.
Additional embodiments of this disclosure comprise a computer system for carrying out the computer-implemented method of this disclosure. The computer system may comprise a processor for executing the computer-executable instructions, one or more electronic databases containing the data or information described herein, an input/output interface or user interface, and a set of instructions (e.g., software) for carrying out the method. The computer system can include a stand-alone computer, such as a desktop computer, a portable computer, such as a tablet, laptop, PDA, or smartphone, or a set of computers connected through a network including a client-server configuration and one or more database servers. The network may use any suitable network protocol, including IP, UDP, or ICMP, and may be any suitable wired or wireless network including any local area network, wide area network, Internet network, telecommunications network, Wi-Fi enabled network, or Bluetooth enabled network. In one embodiment, the computer system comprises a central computer connected to the internet that has the computer-executable instructions stored in memory that is operably connected to an internal electronic database. The central computer may perform the computer-implemented method based on input and commands received from remote computers through the internet. The central computer may effectively serve as a server and the remote computers may serve as client computers such that the server-client relationship is established, and the client computers issue queries or receive output from the server over a network.
The present invention has been described with reference to particular embodiments having various features. In light of the disclosure provided above, it will be apparent to those skilled in the art that various modifications and variations can be made in the practice of the present invention without departing from the scope or spirit of the invention. One skilled in the art will recognize that the disclosed features may be used singularly, in any combination, or omitted based on the requirements and specifications of a given application or design. When an embodiment refers to “comprising” certain features, it is to be understood that the embodiments can alternatively “consist of” or “consist essentially of” any one or more of the features. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention.
It is noted in particular that where a range of values is provided in this specification, each value between the upper and lower limits of that range is also specifically disclosed. The upper and lower limits of these smaller ranges may independently be included or excluded in the range as well. The singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. It is intended that the specification and examples be considered as exemplary in nature and that variations that do not depart from the essence of the invention fall within the scope of the invention. Further, all of the references cited in this disclosure are each individually incorporated by reference herein in their entireties and as such are intended to provide an efficient way of supplementing the enabling disclosure of this invention as well as provide background detailing the level of ordinary skill in the art.
Claims
1. A computer-implemented method comprising the following steps:
- generating analytical reporting comprising consumer interests comprising topics, themes, celebrities, media, brands, influencers, actors, movies, television shows, and combinations thereof;
- profiling the consumer interests with social media averages for the consumer interests;
- calculating a platform specific index report of over- or under-representation of the consumer interests; and
- using the consumer interests profiling report to develop or improve a marketing campaign and/or advertising campaign, to develop or improve content creation strategies, or to provide intelligence related to a consumer section in combination with demographic sectors for measuring trends.
2. The computer-implemented method of claim 1, further comprising computing platform averages based on social media subscription and interaction behavior, wherein general audience content creation, consumption and engagement patterns are identified on social media platforms and assigned quantitative metrics to reflect a proportion of audiences engaging in or with a topic, keyword, term, object, brand, media, influencer, movie, television show, or combination thereof.
3. The computer-implemented method of claim 1, further comprising defining and assigning baseline scores for terms, key-phrases, entities, content, videos, content creators, media, brands, audience sectors, or topics in a given time period by using algorithmic steps of collection, aggregation, averaging and normalization across an online media platform.
4. The computer-implemented method of claim 1, further comprising categorizing social media platform profiles and channels into categories and subcategories of topics using predictive modeling techniques.
5. The computer-implemented method of claim 1, further comprising categorizing social media platform profiles and channels into categories of media, brand, influencer, entertainer, television show, movie, celebrity, actor, director, chef, politician, retailer, journalist, artist, comedian, news channels, or combination thereof, and sub-categories of the categories including brand name, movie name, movie genre, actor names, famous persons, influencer or entertainer names, or combinations thereof, using natural language processing and a combination of supervised and unsupervised machine learning.
6. The computer-implemented method of claim 1, further comprising combining categories of content and content providers on social media with demographics of audiences interacting with the content and content providers, wherein the combinations calculate an analysis of audience interests, and wherein the demographics comprise age, gender and ethnicity, location, direct marketing area, brand or individual creator status, low-level categorical and topical attributes, high-level categorical and topical attributes, political views, cross-platform account linkages, education level and income range, or combinations thereof.
7. The computer-implemented method of claim 1, further comprising calculating scores predicted for an audience sector and a general audience for social media platforms presented as platform averages, wherein the scores comprise an over-index or under-index factor conveying if the chosen audience segment over-indexes in terms of interest in a media, brand, influencer, actor, television show, movie, or combinations thereof.
8. The computer-implemented method according to claim 1, further comprising algorithmically defining a Return on Investment (ROI) analysis, wherein the algorithmically defining Return on Investment (ROI) analysis comprises automatically scoring for content, audience, content provider, audience engagement types, audience demographics, audience interests, or advertising and marketing platforms, by analyzing trends and engagement in a given time frame, and using the scoring of the ROI analysis to automatically inform or suggest improved content creation, advertising, and marketing strategies.
9. The computer-implemented method according to claim 1, further comprising analyzing post interactions on internet-supported computer platforms to extract intelligence and quantify a performance of social media posts and factors contributing to performance, and outputting predicted performance of audience engagement optimization, network growth, advertising, marketing, or combinations thereof.
10. The computer-implemented method according to claim 1, further comprising calculating an engagement score measuring a likelihood of a trend, video, topic or creator to engage viewership, by measuring content against other posts and target audience from comments, likes, shares, or time engaged with content, and using weighted averaging and normalization for score quantification.
11. The computer-implemented method according to claim 1, further comprising providing a mechanism for quantifying an estimated success of a piece of content, content provider, content publisher, or content marketer, the mechanism comprising extracting numerical representations based on text, images, video intelligence, target audience demographics, and interests, which is then processed to provide tailored content and audience analysis, and wherein the numerical representations are extracted algorithmically using key-phrase/entity extraction and vector formation from text data, applying a variety of filters on image and video data, and dividing audiences into clusters of interest and demographic groups.
12. The computer-implemented method according to claim 1, further comprising assigning numerical scores to an engagement of social media audiences with video content and video creators, including trends, affinities, and performance above or below a baseline.
13. The computer-implemented method according to claim 1, further comprising processing audience consumption behaviors from interactions on social media platforms and calculating recommendations or predictions for media analysts, content creators, direct marketers, or combinations thereof.
14. The computer-implemented method according to claim 1, further comprising identifying and disaggregating audience segments by viewing, commenting, sharing, and other engagement behavior, wherein the audience segments are extracted from groups of undifferentiated social media messages.
15. The computer-implemented method according to claim 1, further comprising measuring popularity, trendiness, or virality of video content themes, including messages, topics, perspectives, sentiments, brands, media, or persona according to computed baselines, topical baselines, sub-topical baselines, or combinations thereof.
16. The computer-implemented method according to claim 1, further comprising scoring attributes of videos and video consumers using data analysis techniques and machine learning on social media content including text, image, audio, video, or combinations thereof.
17. The computer-implemented method according to claim 1, further comprising assigning scores to terms, key-phrases, and entities algorithmically extracted from social media text, image, and video content that are trending in terms of attracting engaged viewership in a given time period compared to a different or previous time period.
18. The computer-implemented method according to claim 1, further comprising defining and assigning a baseline score for terms, key-phrases, entities, content, videos, content creators, audience sectors and topics in a given time period by using algorithmic steps of collection, aggregation, averaging, and normalization.
19. The computer-implemented method according to claim 1, further comprising calculating an affinity score measuring a likelihood that content, topics, and content makers will attract different audience sectors by combining data of audience attributes including age, gender, location, topics of interest, interest groups, or combinations thereof, wherein the measuring is performed algorithmically using text matching and key-phrase extraction techniques and by using a combination of distance metrics, geographical distance, and/or similarity to trending items on social media platforms in a given time period.
20. A computer-implemented method comprising automatically detecting and recognizing sensitive Personally Identifiable Information (PII) within publicly available text written by a consumer on a social media platform using natural language processing techniques and machine learning algorithms that detect PII presence and recognize a type of PII.
21. The computer-implemented method of claim 20, further comprising automatically removing the Personally Identifiable Information (PII) from storage databases if the PII is detected.
22. The computer-implemented method of claim 20, further comprising using the natural language processing and a combination of supervised and unsupervised machine learning algorithms to detect the Personally Identifiable Information (PII) in the publicly available text written by a consumer on a social media platform.
23. The computer-implemented method of claim 20, further comprising algorithmically detecting a location of the Personally Identifiable Information (PII) in the publicly available text written by a consumer on a social media platform and categorizing the detected PII into categories of PII.
24. The computer-implemented method according to claim 20, further comprising using language and phrases, identified using the natural language processing techniques, that content consumers and content producers use to describe social media profiles, interests, and/or content preferences and comments, and using predictive modeling to detect and recognize email addresses, location information such as geographical coordinates, phone numbers, contact information, street address(es), IP address(es), social security number, bank account details, or combinations thereof.
25. A computer-implemented method comprising matching social media consumers to location-based profiles based on content consumption and engagement patterns on social media, using predicted demographics and linked audience to automatically connect offline data to online data for marketing or advertising to the social media consumers.
26. The computer-implemented method according to claim 25, further comprising matching profiles from a first social media platform to a second social media platform using predictive modeling using names, comments, typographic patterns, emojis, hyperlink usage, demographics, posting behavior, language, phrases, profile descriptions, content interests, or combinations thereof, identified using natural language processing techniques.
27. The computer-implemented method according to claim 25, further comprising using data from two or more of a consumer, content, engagement, or demographics, automatically gathered from two or more social media platforms, to match a social media profile to a non-social media profile.
28. The computer-implemented method according to claim 25, further comprising matching social media data with additional location-based profile data comprising demographics, location, user first name, user last name, user middle name, income band, education level, household income, number of children, or combinations thereof.
29. The computer-implemented method according to claim 25, further comprising deduplicating social media user matches with location-based profile matches based on calculated accuracy of data and enriched data predictions made on social media data.
30. The computer-implemented method according to claim 25, further comprising automated processing for finding audiences based on a content interest and/or demographic criteria across different social media platforms, matching consumers from social media to a location-based profile dataset based on interests and demographic predictions across social media platforms, performing deduplication algorithmically, converting files to a format compliant or compatible with an identity link store, and automatically uploading the files to the identity link store.
Type: Application
Filed: Mar 13, 2021
Publication Date: Jul 1, 2021
Applicant: ICX Media, Inc. (Washington, DC)
Inventors: Jyotika Singh (Winnetka, CA), Michael Avon (McLean, VA), Serge Matta (Vienna, VA)
Application Number: 17/200,808