Abstract: A computer-implemented method for video sharing is disclosed. A website which hosts multiple users and includes numerous products for sale is accessed. User information is collected and used to match a user with a sales associate. A communications agent is installed on the website which enables an interaction between a website user and a sales associate. The sales associate can be a human or an artificial intelligence agent. The user information is used to predict purchasing intentions of the user. The sales associate can obtain a uniform resource locator (URL) within the website domain based on the purchasing intentions of the user and can direct the user's web browser to the URL, with the user's approval. The directing to the URL is accomplished without taking control of the user's device. The directing is based on the communications agent installed on the website.

Abstract: Video processing using artificial intelligence is disclosed. An embedded interface including products for sale is accessed. A user requests an interaction based on one of the products. The embedded interface initiates a video segment including a synthetic human in response to the user request. The user submits a question or comment. The interface collects the user input and converts it into a dataset readable by a large language model (LLM). The LLM generates a response to the user request. The response is used to generate an audio stream. The audio stream includes simulated human speech errors and pauses. The audio stream is segmented and a video clip is synthesized for each audio segment. The video clips are assembled into a new video segment which is presented to the user. Additional user interactions are collected and new video segments are generated in response.

Abstract: Techniques for manipulating video livestream background images are disclosed. A short-form video, such as a livestream video or livestream replay video, can be analyzed for context. Computer-implemented techniques may be used for performing entity detection, and can also detect a change in subject based on speech and/or actions of a host individual. The subject can include a particular product. The detecting a change in subject can include detecting a foreground object and identifying the foreground object as a product. The identification of the foreground object as a product can include performing optical character recognition on text imprinted on a foreground object. The identification of the foreground object as a product can include image recognition techniques. The identification of the foreground object as a product can include scanning of an optical code such as a barcode that is imprinted on the product.

Abstract: Techniques for manipulating an expandable video loop with replacement audio are provided. A video that includes a performance by an individual is accessed. One or more processors are used to identify a minimum desired loop within the performance by the individual. Video information is extracted from the video loop and analyzed with machine learning. An audio file is obtained based on a script written by a user. The script is combined with a synthesized voice which is created from the individual video performance to obtain the audio file. The extracted video information is used to manipulate the lips of the individual based on the audio file. A second video loop is created, combining the individual from the first video loop with the video of the manipulated lips of the individual and the synthesized audio file. The second video loop is appended to the end of the first video loop.

Abstract: A computer-implemented method for video analysis is disclosed. A livestream, including a livestream chat, is accessed. The livestream includes products for sale. The livestream includes timestamps based on when each product is highlighted. The timestamps allow users to jump to livestream locations based on the highlighted products. The livestream is monitored by a large language model. The large language model (LLM) detects a question from users in the livestream, and the LLM determines answers to the question. An engagement metric is calculated for each answer. The engagement metric is predictive of future engagement in the livestream by users if the answer is posted. A human assistant in the livestream reviews the LLM answer with the highest engagement score and chooses to edit the answer, to post the answer, or not to post the answer to the livestream.

Abstract: Disclosed embodiments provide techniques for video chat initiation based on machine learning. A website that includes products for sale is accessed. The website is viewed by multiple users. User information is collected and analyzed based on machine learning. The machine learning model is used to predict a purchase intention for a user regarding one or more products for sale. The prediction is based on the machine learning analysis. Information about one or more sales associates is gathered and analyzed based on machine learning. The analysis of sales associate information is used to match a user with a purchase intention with a sales associate. The sales associate can initiate a chat interaction with the user through an overlay on the website. The chat interaction can provide additional user information for the machine learning model. During the chat interaction, the user can purchase one or more products.

Abstract: Disclosed embodiments provide techniques for dynamic short-form video traversal with machine learning in an ecommerce environment. A graph structure associated with a library of short-form videos is accessed and customized in a back-end environment based on products for sale on a website. One or more of the customized short-form videos from the library are rendered to one or more users, along with an interactive overlay and an ecommerce environment. As the video is viewed, video consumption behavior data is collected and analyzed by a machine learning model. The machine learning model determines one or more next short-form videos from the graph structure for the user to view, based on sales goals, video consumption behavior data, and interaction with the user. The machine learning model can synthesize additional short-form videos and insert them into the graph structure in order to enhance viewer engagement and product sales.

Abstract: Disclosed embodiments provide techniques for scaling ecommerce with short-form video. A micro-website is created and is associated with a user ID. The user ID corresponds to a social media influencer, an expert in a subject matter, or a celebrity. The micro-website can be managed by a mobile application. Based on the user ID, a pre-approved electronic catalog of products is determined and used to populate the micro-website with one or more products for sale. The micro-website includes one or more short-form videos highlighting products for sale from the electronic catalog. The host of the short-form videos is the influencer, expert, or celebrity corresponding to the user ID. The micro-website includes an ecommerce link to allow fulfilling product orders. The short-form videos include a virtual purchase cart for ecommerce purchases that can be engaged by the user.

Abstract: Disclosed embodiments describe text-driven AI-assisted short-form video creation. Text from a website is extracted to generate extracted text. Possible summary sentences are formed from the extracted text. The forming is based on natural language processing. The summary sentences are ranked according to an engagement metric. Summary sentences from the possible summary sentences are picked based on a threshold engagement metric value. A list of video scenes is generated based on the summary sentences. Each video scene is associated with a summary sentence. A media asset from a media asset library is chosen for each video scene within the list of video scenes. The choosing is accomplished by machine learning. The list of video scenes, including the media asset that was chosen for each video scene, is compiled into a short-form video. Media is extracted from the website included into the short-form video. The compiling includes a dynamically generated image.

Abstract: Techniques for video generation based on machine learned video template usage are disclosed. A plurality of videos is obtained, and video scene analysis on each video is performed. Video cuts for each video are detected, and objects within each video are identified. The identifying includes detecting a person, face, building, or vehicle. Metadata is categorized for each of the videos based on the scene analysis, the video cuts, and the objects within the videos. Template information is stored, including the categorized metadata, on each of the videos. Each video is stored as a template video along with the template information. The template information on a subset of videos is ranked. A basis video is selected based on the template information. A further video is generated based on the basis video. The further video is stored as a further template video along with the template videos.

Abstract: Disclosed embodiments provide techniques for synthesized responses to predictive livestream questions. A livestream featuring a host is accessed and is viewed by one or more viewers. The audio, video, and images from the livestream are analyzed and a plurality of potential questions from the one or viewers is predicted based on the analysis. An answer to each potential question is generated based on a large language model (LLM) neural network. A synthesized video segment is created for each answer generated by a generative AI chatbot linked to a LLM for each potential question. During the livestream, real-time questions from the one or more viewers are detected. The real-time questions are matched to the synthesized video segment answers and the matched synthesized video segment is rendered to the one or more livestream viewers. An ecommerce environment is rendered during the livestream to allow purchasing of products.

Abstract: Disclosed embodiments provide techniques for iterative AI prompt optimization for video generation. A first text template to be read by a large language model (LLM) neural network is accessed. The template includes control parameters that are populated from within a website. The populated template is submitted as a request to the LLM neural network, which generates a first video script. The first video script is used to create a first short-form video. The first short-form video is evaluated based on one or more performance metrics. The text template, short-form video, website information, and evaluation are used to train a machine learning model that is used to create a second text template. The second text template can be used to generate a second short-form video. The evaluation of iterative text templates and resulting short-form videos continues until a usable video is produced.

Abstract: Disclosed embodiments provide techniques for dynamic synthetic video chat agent replacement. A human host receives a request for a video chat initiated by a user. An image for a synthetic host including a representation of an individual is retrieved. The image of the synthetic host is selected based on information about the user. Aspects of the individual included in the image are extracted using one or more processors. A video performance by the human host responding to the statement or query by the user is captured. A synthetic host performance is created in which the video performance of the human host is dynamically replaced by the individual that was extracted so that the synthetic host performance responds to the user statement or query. The synthetic host performance is rendered to the user and supplemented with additional synthetic host performances as the video chat continues.

Abstract: Disclosed embodiments provide techniques for synthesized realistic metahuman short-form videos. A photorealistic representation of an individual is accessed from media sources including videos, photographs, livestreams, or a 360-degree recording of a human host. The individual can be selected based on information on the viewer of the short-form video, such as purchase history, viewing history, and metadata. The photorealistic representation is isolated using machine learning and is used to create a three-dimensional (3D) model of the individual, based on a game engine. A realistic synthetic performance is created by combining the 3D model of the individual with animation generated by a game engine. The synthesized performance can include the voice of the human host. The synthesized performance is inserted into a metaverse environment and rendered to a viewer as a short-form video, including an ecommerce window with an on-screen product card and virtual purchase cart.

Abstract: Disclosed embodiments provide techniques for immediate livestreams in a short-form video ecommerce environment. A website that includes a plurality of products for sale is accessed. The website is viewed by a plurality of users. One or more groups of users is identified. Each of the one or more groups includes users with an interest in one or more products from the plurality of products for sale. An immediate livestream is created for each group of the one or more groups of users when a number of users within each group is higher than a threshold value. The immediate livestream includes the one or more products. The immediate livestream is rendered within the container unit on the website to each group of the one or more groups of users. An ecommerce purchase of the one or more products is enabled within the immediate livestream.

Abstract: Technologies relating to system and method of real-time video overlaying or superimposing display from multiple mutually synchronous cameras are disclosed. An example method of real-time video overlaying includes the steps of: synchronizing frame rates of a depth data, a face metadata, and a video data of a first camera video output captured by a first camera; determining a first depth between a user face and the first camera; using a cutoff depth to determine a user body contour; generating a binary mask of the user body contour based on the first depth and the cutoff depth; smoothing an edge of the binary mask; merging the binary mask with the first camera video output and generating a merged first camera video output; and overlaying the merged first camera video output onto a second camera video output.

Abstract: Disclosed embodiments provide techniques for livestream with synthetic scene insertion. A prerecorded livestream featuring a host is rendered to one or more viewers. An operator accesses a video segment related to the prerecorded livestream. The related video segment includes a performance by an individual. The operator retrieves an image of the host of the prerecorded livestream and creates from the related video segment a synthesized video segment that includes the performance of the individual accomplished by the host. One or more insertion points are determined within the prerecorded livestream for the insertion of the synthesized video segment. The operator inserts the synthesized video segment into the prerecorded livestream at the one or more determined insertion points. The insertion is accomplished dynamically and appears seamless to the viewer. The remainder of the prerecorded video is rendered to the one or more viewers after the insertion point.

Abstract: Disclosed embodiments provide techniques for connected television livestream-to-mobile device handoff in an ecommerce environment. A livestream or livestream replay is accessed on a device for connected television (CTV) that facilitates delivery of streaming video content to a user. A quick response (QR) code is revealed on the CTV device. A user scans the QR code with a camera located on a mobile device, and the QR code redirects the mobile device to a website. The website enables participation in a livestream using the mobile device. Products for sale can be highlighted by a livestream host during the livestream. The products for sale can be identified and an ecommerce purchase of the products for sale can be enabled. Promotions related to the products for sale can be displayed to the user on the CTV and mobile device during the ecommerce purchase process.

Abstract: Disclosed embodiments provide techniques for augmented performance replacement in a short-form video. A short-form video is accessed, including a performance by a first individual. Using one or more processors, the performance of the first individual is isolated. Specific elements of the performance including gestures, clothing, expressions, and accessories are included in the isolation process. An image of a second individual is retrieved and information on the second individual is extracted from the image. A second short-form video is created by replacing the performance of the first individual with the second individual. The second short-form video is augmented based on viewer interaction. The augmenting of the second short-form video occurs dynamically. The augmenting includes additional audio content based on comments, responses to live polls or surveys, or questions and answers from viewers. The augmenting includes switching audio content in the second short-form video with additional audio content.

Abstract: Disclosed embodiments provide techniques for temporal analysis to determine short-form video engagement. One or more processors are used to insert a container unit into a webpage. The container unit is populated with one or more short-form videos selected from a video library. The selected short-form videos are presented to a user. User interactions with the short-form videos, including polls, text responses, and purchase history, are evaluated and associated with the videos. The user interactions are evaluated while the user is watching and interacting with the videos. An engagement metric is calculated, based on comparing a first rate of change in the user interactions with the short-form videos to a second rate of change in the user interactions. The resulting engagement metric is recorded and associated with the video. Ecommerce purchases of products featured in the videos are enabled, including a virtual purchase cart.