Abstract: Disclosed herein are system, method and computer readable storage medium for detecting space suitable for overlaying media content onto an image. The system receives an image which may be an image or a video frame. The image is processed using a number of image processing techniques in order to automatically propose spaces for inserting media content onto the image. The proposed spaces may then be further analyzed using a heuristics-based approach to select bounding boxes for inserting media content. Subsequently, one or more media content items may be selected for insertion onto the bounding boxes on the image. The system may then cause a display of the image with the selected media content item overlaid onto the image within the selected bounding boxes.

Abstract: An online system generates a set of content item variations for a reference content item that include different styles of text for the content item. The different styles of text are generated by applying machine-learned style transfer models, for example, neural network based models to reference text of the reference content item. The text variations retain the textual content of the reference text but are synthesized with different styles. The online system can provide the content item variations to users on an online experimental platform to collect user interaction information that may indicate how users respond to different styles of text. The online system or the content providers can effectively target users with content items that include the style of text the users respond to based on the collected information.

Abstract: Techniques are disclosed relating to automatically synthesizing user interface (UI) component instances. In disclosed techniques a computer system receives a set of existing UI elements and a set of design rules for the set of existing elements, where design rules in the set of design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements. The one or more allowed states may correspond to one or more visual characteristics. Using the set of existing UI elements, the computer system may then automatically generate a plurality of UI component instances based on the set of design rules, where a respective UI component instance includes a first UI element in a first allowed state. The computer system may then train, using the plurality of UI component instances, a machine learning model operable to automatically generate UI designs.

Abstract: Techniques are disclosed relating to automatically synthesizing user interface (UI) component instances. In disclosed techniques a computer system receives a set of existing UI elements and a set of design rules for the set of existing elements, where design rules in the set of design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements. The one or more allowed states may correspond to one or more visual characteristics. Using the set of existing UI elements, the computer system may then automatically generate a plurality of UI component instances based on the set of design rules, where a respective UI component instance includes a first UI element in a first allowed state. The computer system may then train, using the plurality of UI component instances, a machine learning model operable to automatically generate UI designs.

Abstract: Disclosed herein are system, method and computer readable storage medium for detecting space suitable for overlaying media content onto an image. The system receives a candidate image which may be an image or a video frame. The candidate image is then input into a neural network. The neural network may output coordinates and one or more dimensions representing one or more bounding boxes for inserting media content into the candidate image. The one or more bounding boxes may be transmitted with a request for a media content item to be displayed in a bounding box. In response to the request the media content item may be received, and the candidate image and the media content item overlaid on top of the candidate image within the bounding box may be displayed.

Abstract: Disclosed herein are system, method, and computer program product embodiments for automatically upgrading and finding components similar to legacy versions. In an embodiment, an automatic upgrading system determines one or more properties of a first web part and identifies a first attribute for the one or more properties of the first web part. A second web part is identified by similar attribute as the first web part to create a web page by adding at least the second web part to the web page.

Abstract: Techniques are disclosed relating to emphasizing user interface elements for different users based on user attributes. In some embodiments, a system maintains a set of information (e.g., for a particular product) that includes multiple types of information. The system may access the same set of information for different user requests. The computing system may determine first and second subsets of the set of information based on attributes of the different users. The computing system may cause display of a first user interface on a device of the first user showing only the first subset of item fields for the two or more of the first set of items. The computing system may cause display of a second user interface on a device of the second user showing only the second subset of item fields for the two or more of the second set of items.

Abstract: Techniques are disclosed relating to determining a similarity of components of a current webpage to different UI components for use in automatically generating an updated webpage. A computer system may receive information specifying a current webpage, including a particular current UI component and information specifying a plurality of different UI components for an updated webpage. The computer system may identify one or more characteristics of the particular current UI component. The computer system may determine, based on the identified one or more characteristics, a similarity of ones of the plurality of different UI components to the particular current UI component. The computer system may select, based on the determining, a particular different UI component from the plurality of different UI components for use, in the updated webpage, for the particular current UI component. Such techniques may advantageously improve user experience by automatically providing up-to-date user interfaces.

Abstract: Techniques are disclosed relating to emphasizing user interface elements for different users based on user attributes. In some embodiments, a system maintains a set of information (e.g., for a particular product) that includes multiple types of information. The system may access the same set of information for different user requests. The computing system may determine first and second subsets of the set of information based on attributes of the different users. The computing system may cause display of a first user interface on a device of the first user, where the first user interface shows only the first subset of information with a first element displayed using a first format. The computing system may cause display of a second user interface on a device of the second user, where the second user interface shows only the second subset of information with the first element displayed using a second format.

Abstract: Techniques are disclosed relating to automatically generating user interfaces. In some embodiments, input data to be displayed is grouped into components (e.g., based on depth within hierarchical code, coordinates in a display space, etc.). These components may be based on template information that defines a set of known component types. In some embodiments, the system formats the selected components according to display parameters and causes display of a user interface that displays the components based on the formatting. In various embodiments, the disclosed techniques may allow automatic creation of effective user interfaces without information specifying layout and formatting for input data. This may provide flexible, quality interfaces without requiring design or coding expertise. Further, disclosed techniques may allow the automatic interface generator to generate interfaces similar to other existing interfaces.

Abstract: Techniques are disclosed relating to determining a similarity of components of a current user interface (UI) to new UI components for use in automatically generating a new UI. A computer system may receive information specifying a current UI including a particular current UI component and information specifying a plurality of new UI components for a new UI. The computer system may then identify characteristics of the particular current UI component. Based on these identified characteristics, the computer system may score ones of the plurality of new UI components, where the scoring is performed to determine a similarity to the particular current UI component. The computer system may then select, based on the scores, a particular new UI component from the plurality of new UI components for use, in the new UI, for the particular current UI component. Such techniques may advantageously improve user experience by automatically providing up-to-date user interfaces.

Abstract: A textual properties model is used to infer values for certain font properties of interest given certain text-related data, such as rendered text images. The model may be used for numerous purposes, such as aiding with document layout, identifying font families that are similar to a given font families, and generating new font families with specific desired properties. In some embodiments, the model is trained from a combination of synthetic data that is labeled with values for the font properties of interest, and partially-labeled data from existing “real-world” documents.

Abstract: Techniques are disclosed relating to predicting events based on automation parameters and initiating a procedure to request user input. In some embodiments, the system automatically selects one or more component types to display an interface to request the user input, e.g., based on user interface elements associated with the procedure. These component-based techniques may be used to imitate another interface while automatically formatting the input data without a known template.

Abstract: Techniques are disclosed for automatically generating stencils for content of a user interface (UI) to be rendered. A computer system receives information specifying content of a user interface (UI) to be rendered. Based on this information, the computer system identifies one or more bounding regions of content within the UI, including analyzing metadata and a rendered version of the UI. The computer system then automatically generates, one or more UI stencils based on the identified bounding regions, that are displayable as progress indicators prior to rendering corresponding UI content. Once it has generated the stencils, the computer system stores information specifying the one or more UI stencils for use in the UI. Automatically generating stencils for UIs may advantageously reduce the time gap from development to deployment of these UIs while improving their perceived performance and, by extension, improving user experience.

Abstract: Techniques are disclosed relating to custom user interface design based on user interaction metrics from another communication channel. In disclosed embodiments, a computing system generates one or more interaction metrics, based on a first set of user input characteristics for user interaction with multiple different user interface formats via a first communication channel. In some embodiments, the first communication channel supports tracking of the first set of user input characteristics and a second communication channel supports tracking of a second, different set of user input characteristics. In some embodiments, the computing system determines, based on the one or more interaction metrics from the first communication channel, a set of design parameters for formatting a user interface to be displayed via the second communication channel.

Abstract: Techniques are disclosed for automatically generating stencils for content of a user interface (UI) to be rendered. A computer system receives information specifying content of a user interface (UI) to be rendered. Based on this information, the computer system identifies one or more bounding regions of content within the UI, including analyzing metadata and a rendered version of the UI. The computer system then automatically generates, one or more UI stencils based on the identified bounding regions, that are displayable as progress indicators prior to rendering corresponding UI content. Once it has generated the stencils, the computer system stores information specifying the one or more UI stencils for use in the UI. Automatically generating stencils for UIs may advantageously reduce the time gap from development to deployment of these UIs while improving their perceived performance and, by extension, improving user experience.

Abstract: Disclosed herein are system, method, and computer program product embodiments for determining the minimum set of user interface (UI) elements for a task. One embodiment operates by transmitting UI elements associated with an application to a first computing device operated by a first user and a second computing device operated by a second user. Further, the embodiment includes obtaining data structures corresponding to a first task and a second task and then grouping, using a machine learning model, the data structures into a first cluster corresponding to the first task and a second cluster corresponding to the second task. Further still, the embodiment also includes determining a minimum set of UI elements required for the first task based on the first cluster.

Abstract: Techniques are disclosed relating to measuring presentation aspects of portions of a user interface (UI) that have been identified to be presented neutrally and automatically generating a new UI that reduces differences in the presentation aspects of the identified portions. Portions of the UI identified to be presented neutrally may include elements or content in the UI identified to be presented neutrally. Portions for neutral presentation may include portions of UI that, for example, portions that present legal decisions, portions with survey data, forms or information displays, or portions where a user is requested to make a choice between alternative options. Presentation aspects that may be measured include, for example, visual aspects, clarity aspects, or tonal aspects. Reducing the differences in the presentation aspects may present the elements or content with more neutrality to reduce influence in decisions due to visual, clarity, or tonal aspects between the elements or content.

Abstract: Techniques are disclosed relating to determining locations of child elements in a user interface (UI), automatically updating the locations of the child elements in the UI, and generating an updated UI with the updated locations. A computer system may receive information specifying content of the UI and render an image of the UI based on the received information. Computer vision algorithms may be applied to the rendered of the UI to determine locations of elements in the UI. The locations of the elements may be updated to position the elements in selected locations relative to each other. The updated locations of the elements as determined by the computer vision algorithms may be implemented to generate updating information specifying content for the updated UI.

Abstract: Techniques are disclosed relating to generating a user interface (UI) migration plan, including intermediate UIs, for migrating from a current UI to a new UI. A computer system may receive information specifying a current UI and a new UI, and identify one or more differences between the current and the new UIs. Based on the differences, the computer system may generate information specifying one or more candidate intermediate UIs. The computer system may score the candidate intermediate UIs relative to a specified set of design criteria. The computer system may determine a UI migration plan that specifies a set of the one or more candidate intermediate UIs that are displayable in order to migrate from the current UI to the new UI, where the set of one or more intermediate UIs is selected based on the scoring. Use of the UI migration plan may advantageously reduce user interaction issues.