Abstract: A method of generating a predictive model of categories of venues visited by a user is provided. The method may include extracting a first content feature from a first digital post to an online social media platform selected from a plurality of digital posts, extracting a second content feature from a second digital post to an online social media platform selected from the plurality of social media posts, aggregating the first and second content features, inferring at least one of a frequency and a regularity of visits to a venue category associated with the plurality of digital posts based on the aggregated first and second content features using a neural network, and determining at least one of a frequently visited venue category and a regularly visited venue category based on the inferred frequency and a regularity of visits associated with the plurality of digital posts.

Abstract: A method and system for generating recommendations. The method including extracting concept information from visual content associated with content posted to a social media platform, detecting one or more preferences based on the extracted concept information, generating a matrix based on the detected one or more preferences, calculating a first similarity between the one or more preferences associated with a first user and one or more preferences associated with a second user based on the generated matrix, and generating a recommendation based on the matrix and the calculated first similarity.

Abstract: Systems and methods disclosed herein associate images with business venues. An example method includes: receiving a first image and retrieving textual reviews and stored images that are associated with one or more candidate business venues. The method further includes: detecting, using trained visual detectors, a plurality of business-aware concepts in the first image and assessing likelihood that detected business-aware concepts are in the first image. The method additionally includes: (i) generating a first representation of the first image based on the likelihoods and one or more term vectors for high-scoring concepts and (ii) receiving second representations of each candidate based on the retrieved textual reviews and stored images. In accordance with determining that the first representation is most similar to a respective second representation of a first candidate, the method includes: (i) associating the first image with the first candidate and (ii) providing an indication of the association.

Abstract: A method of generating a caption for a social media post is provided. The method may include receiving a social media post to be posted to a social media network; collecting reference data relevant to determining common activities occurring at a category of location associated with the social media post; inferring potential topics for captions from a topic inference model, based on the collected reference data associated with the social media post; setting parameters associated with a language model based on the inferred topic; and generating at least one caption for the social media post based on the identified language model, and the inferred topic.

Abstract: A method identifies user demographic characteristics, such as age and gender. First, a plurality of photos are acquired from postings to a social media site by a specific user. A plurality of facial images are identified in the acquired photos. Using a selected subset of the facial images, the method estimates one or more demographic characteristics of the user. In some cases, the estimation uses a profile photograph of the user. In some cases, the facial images are clustered based on similarity. The cluster that most closely matches the profile photo is used for demographic analysis to estimate the characteristics of the user. In some cases, a cluster is selected based on cluster purity, and the images in the cluster are used to estimate the characteristics of the user. In some cases, other self-portrait features are used, such as large facial images or arms along the perimeter of photographs.

Abstract: A method associates social media messages with venues. A social network graph includes users, messages from users, and venues. The venues include multiple primary venues and a no-venue. A link between a message and the no-venue node indicates that the message is not associated with a primary venue. Training feature vectors are constructed that measure connectedness between messages and venues. The process trains a classifier to estimate probabilities that messages are associated with venues. A new social media message is received, and the process constructs a feature vector using the same features as the training vectors, measuring connectedness between the new message and the no-venue. The classifier computes a probability that the new message is associated with the no-venue. When the probability exceeds a predefined threshold, the new message is not associated with any of the primary venues. Otherwise, the new message is associated with one of the primary venues.

Abstract: A computer-implemented method performed in connection with a computerized system incorporating a processing unit and a memory, the computer-implemented method involving: using the processing unit to generate a multi-modal language model for co-occurrence of spoken words and displayed text in the plurality of videos; selecting at least a portion of a first video; extracting a plurality of spoken words from the selected portion of the first video; extracting a first displayed text from the selected portion of the first video; and using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the extracted first displayed text.

Abstract: A method includes: obtaining from a first social media source a new short unstructured electronic message with an associated geographic location and message content; identifying a first venue name and a first visit characteristic from the message content; accessing a database of venues, wherein the database includes for respective venues a venue name, a geographic location and one or more venue characteristics, wherein information in the database reflects information associated with the respective venues extracted from a plurality of social media posts, including a plurality of prior short unstructured electronic messages from the first social media source; determining whether the database includes a candidate venue that has a venue name and geographic location that respectively are substantially similar to the first venue name and the associated geographic location; when the candidate venue exists in the database, associating the new short unstructured electronic message with the candidate venue and perform

Abstract: Systems and methods to analyze a person's social media photos or videos, such as those posted on Twitter, Facebook, Instagram, etc. and determine properties of their social life. Using information on the number of people appearing in the photos or videos, their ages, and genders, this method can predict whether the person is in a romantic relationship, has a close family, is a group person, or is single. This information is valuable for generating audiovisual content recommendations as well as for advertisers, because it allows targeting personalized advertisements to the person posting the photos. The described methods may be performed (and the advertisements or other content may be selected for recommendation) substantially in real-time as the user accesses a specific online resource.

Abstract: A method identifies user demographic characteristics, such as age and gender. First, a plurality of photos are acquired from postings to a social media site by a specific user. A plurality of facial images are identified in the acquired photos. Using a selected subset of the facial images, the method estimates one or more demographic characteristics of the user. In some cases, the estimation uses a profile photograph of the user. In some cases, the facial images are clustered based on similarity. The cluster that most closely matches the profile photo is used for demographic analysis to estimate the characteristics of the user. In some cases, a cluster is selected based on cluster purity, and the images in the cluster are used to estimate the characteristics of the user. In some cases, other self-portrait features are used, such as large facial images or arms along the perimeter of photographs.

Abstract: A method for automatically selecting thematically representative music is disclosed. A processor is used for using a theme-related keyword to search a keyword-indexed video repository to retrieve videos associated with the theme-related keyword; analyzing the retrieved videos to select videos with music; and extracting music tracks and features from the selected videos. The method further includes selecting representative music related to the theme from the extracted music tracks using the extracted features; and storing the selected representative music in a processor accessible memory.

Abstract: A computer-implemented method performed in connection with a computerized system incorporating a processing unit and a memory, the computer-implemented method involving: using the processing unit to generate a multi-modal language model for co-occurrence of spoken words and displayed text in the plurality of videos; selecting at least a portion of a first video; extracting a plurality of spoken words from the selected portion of the first video; extracting a first displayed text from the selected portion of the first video; and using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the extracted first displayed text.

Abstract: A method for automatically selecting thematically representative music is disclosed. A processor is used for using a theme-related keyword to search a keyword-indexed video repository to retrieve videos associated with the theme-related keyword; analyzing the retrieved videos to select videos with music; and extracting music tracks and features from the selected videos. The method further includes selecting representative music related to the theme from the extracted music tracks using the extracted features; and storing the selected representative music in a processor accessible memory.

Abstract: A method for viewing a collection of images or videos, includes analyzing the collection to determine properties of the images or videos and using the determined properties to produce icons corresponding to such properties; providing a time-varying display of the images or videos in the collection following an ordering of the images or videos in the collection and at least one of the corresponding icons; receiving a user selection of an icon; and changing the display of the images or videos in the collection following a reordering of the images or videos in the collection in response to the user selection.

Abstract: A method of computing at least one photogenic route from a starting location to a destination location, including; computing photogenic values for images in a large collection representing a geographic region that includes the starting location and the destination location; computing a photogenic index for each route segment based on computed photogenic values of images taken along the route segment; computing at least one photogenic route from the starting location to the destination location and presenting the route(s) to a user.

Abstract: Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The system includes a processor accessible memory for receiving an ordered set of tags, each tag including a text label and a size for the text label, and at least one closed shape corresponding to a space for the tag layout. The system further includes a processor for generating the tag layout by computing a scale factor for either the closed shape or the size of the text labels in the set of tags such that all the tags in the set of tags fit within the closed shape, and the processor stores the generated tag layout in the memory.

Abstract: A method for viewing a collection of images or videos, includes analyzing the collection to determine properties of the images or videos and using the determined properties to produce icons corresponding to such properties; providing a time-varying display of the images or videos in the collection following an ordering of the images or videos in the collection and at least one of the corresponding icons; receiving a user selection of an icon; changing the time-varying display of the images or videos in the collection following a reordering of the images or videos in the collection in response to the user selection; storing the sequence of the user selections and associated timing in a script in a processor accessible memory; and playing back the viewing of the collection of images or videos using the script.

Abstract: Computing a scale factor to insert a first set of shapes into a second set of shapes to form a combined image includes receiving the two sets of shapes, using a processor to convert the first set of shapes into a set of rectangles and the second set of shapes into a set of intervals and computing the scale factor for either the set of intervals or the set of rectangles to generate the combined image by iteratively inserting the set of rectangles into the set of intervals and updating the scale factor in response to a residual area or an overflow area until all the rectangles in the set of rectangles have been inserted into the set of intervals and the residual area in the set of intervals is below a threshold, and storing the combined image in memory.

Abstract: Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The method further includes receiving at least one closed shape corresponding to a space for the tag layout. A processor computes a scale factor for at least one of the closed shape or the size of the text labels in the set of tags to generate the tag layout of the set of tags within the closed shape such that all the tags in the set of tags fit within the closed shape and the tags are placed in the space based at least upon the ordering of the tags in the set of tags.

Abstract: Computing a scale factor to insert a first set of shapes into a second set of shapes to form a combined image includes receiving the two sets of shapes, using a processor to convert the first set of shapes into a set of rectangles and the second set of shapes into a set of intervals and computing the scale factor for either the set of intervals or the set of rectangles to generate the combined image by iteratively inserting the set of rectangles into the set of intervals and updating the scale factor in response to a residual area or an overflow area until all the rectangles in the set of rectangles have been inserted into the set of intervals and the residual area in the set of intervals is below a threshold, and storing the combined image in memory.