Abstract: Example implementations described herein are directed to systems and methods for document capture, which can involve detecting, from a plurality of frames of a recording of an application window that comprises a document, document content of the document and screen activity of the application window; and generating a web-based copy of the document based on the document content and the screen activity. Further example implementations can involve recording screen activity such as mouse cursors, text annotations, scrolling actions and other activity, and then providing an application layer to replay the screen activity onto the captured document.

Abstract: A method of generating a caption for a social media post is provided. The method may include receiving a social media post to be posted to a social media network; collecting reference data relevant to determining common activities occurring at a category of location associated with the social media post; inferring potential topics for captions from a topic inference model, based on the collected reference data associated with the social media post; setting parameters associated with a language model based on the inferred topic; and generating at least one caption for the social media post based on the identified language model, and the inferred topic.

Abstract: Systems and methods of performing a process are provided, including receiving a live document video stream of a remote collaboration session, detecting a cursor action in the live document video stream, classifying the detected cursor action into an action category, detecting key frames of the live document video stream, indexing the detected key frames based on the action category, detecting a keyword in the indexed key frames, indexing the key frames using the category, visualizing the cursor action in the key frames based on the action category, and displaying the visualized cursor action.

Abstract: Systems and methods of performing a process are provided, including receiving a live document video stream of a remote collaboration session, detecting a cursor action in the live document video stream, classifying the detected cursor action into an action category, detecting key frames of the live document video stream, indexing the detected key frames based on the action category, detecting a keyword in the indexed key frames, indexing the key frames using the category, visualizing the cursor action in the key frames based on the action category, and displaying the visualized cursor action.

Abstract: A computer-implemented method performed in connection with a computerized system incorporating a processing unit and a memory, the computer-implemented method involving: using the processing unit to generate a multi-modal language model for co-occurrence of spoken words and displayed text in the plurality of videos; selecting at least a portion of a first video; extracting a plurality of spoken words from the selected portion of the first video; extracting a first displayed text from the selected portion of the first video; and using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the extracted first displayed text.

Abstract: A computer-implemented method performed in a computerized system incorporating a central processing unit, a localization signal receiver and a memory, the computer-implemented method involving: receiving at least one localization signal using the localization signal receiver; measuring a strength of the received localization signal; using the central processing unit to extract a key from the received localization signal; determining a location based at least on the measured strength of the received localization signal; and validating the determined location using the extracted key. The localization signal may be provided by one or more beacons, such as iBeacons, which may be placed at various locations within a building. The key may be randomly generated and periodically transmitted to the beacons to prevent replay attacks. A ticket issued by a near field communication (NFC) device may be used for additional location validation to prevent tunneling and collusion.

Abstract: A method of identifying a location of a mobile device in a building includes identifying non-overlapping regions in a building. A server collects base station signal strength measurements at a plurality of distinct points in the building, with at least one point in each region. The server trains region classifiers for each region. Each region classifier is configured to compute a probability estimate that the test point is inside the region, using inputs that are signal strength differences. The server receives signal strength measurements from the base stations, taken by a mobile device at an unknown point. The server computes differences in signal strengths between pairs of base stations, and applies the region classifiers to the signal strength differences, thereby estimating the region where the mobile device is located. The server then transmits the estimated region to a user.

Abstract: Managing contact groups entails receiving email header information for electronic messages sent during a certain period of time. Each header includes unique identifiers for the sender and each recipient. The set of these unique identifiers from each header forms a de facto group. For each group, the process computes a usage value based on the number of distinct messages corresponding to the group. The process also identifies a set of saved groups, where each saved group is a set of unique identifiers of people, and each saved group has a usage value. The usage value is 0 when the saved group does not equal any de facto group. The process compares the de facto groups and their associated usage values to the saved groups and their associated usage values, and provides a recommendation to modify the set of saved groups based on the comparisons.

Abstract: A method for generating content links between a first digital file and a second digital file by detecting a content feature of a first digital file segment of the first digital file during playback of the first digital file segment of the first digital file, searching an index of a plurality of content features for a plurality of segments including a second digital file segment of the second digital file, and dynamically generating a link between the first digital file one segment of the first digital file and the second digital file segment of the second digital file when a content feature of the first digital file segment of the first digital file is related to the content feature of the at least one segment of the second digital file.

Abstract: A computer-implemented method performed in connection with a video comprising a plurality of video frames, the computer-implemented method being performed in a computerized system comprising a processing unit and a memory, the computer-implemented method involving: using the processing unit to detect a plurality of content components within at least one of the plurality of video frames; based on a video modification instruction received from a user, modifying at least one of the plurality of the detected content components; storing an information on the modified at least one of the plurality of the detected content components in the memory; using the central processing unit to overlay the modified at least one of the plurality of the detected content components over the video; and causing the video with the overlaid modified content component to be displayed to the user.

Abstract: Described are systems and methods that allow a video conference participant to efficiently review semantically meaningful events within the video streams shared by their peers. Because participants are engaged in real-time communication, it is important to provide tools that let them quickly jump back to past events that were shown previously, not requiring them to manipulate a standard video timeline. Our techniques thus find meaningful events in the video stream such as scrolling pages, moving windows, typing text, etc. Using these techniques, a participant can then easily go back to e.g. the last PDF page shown by a peer, while still listening to the live audio.

Abstract: A computerized system and computer-implemented method for assisting a user with capturing a video of an activity. The system incorporates a central processing unit, a camera, a memory and an audio recording device. The computer-implemented method involves: using the camera to capture the video of the activity; using the central processing unit to process the captured video, the processing comprising determining a number of user's hands appearing in the captured video; using the recording device to capture of the audio associated with the activity; using the central processing unit to process the captured audio, the processing comprises determining a number of predetermined references in the captured audio; using the determined number of user's hands appearing in the captured video and the determined number of predetermined references in the captured audio to generate feedback to the user; and providing the generated feedback to the user using a notification.

Abstract: A computer-implemented method performed in connection with a computerized system incorporating a processing unit and a memory, the computer-implemented method involving: using the processing unit to generate a multi-modal language model for co-occurrence of spoken words and displayed text in the plurality of videos; selecting at least a portion of a first video; extracting a plurality of spoken words from the selected portion of the first video; extracting a first displayed text from the selected portion of the first video; and using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the extracted first displayed text.

Abstract: Described are systems and methods that allow a video conference participant to efficiently review semantically meaningful events within the video streams shared by their peers. Because participants are engaged in real-time communication, it is important to provide tools that let them quickly jump back to past events that were shown previously, not requiring them to manipulate a standard video timeline. Our techniques thus find meaningful events in the video stream such as scrolling pages, moving windows, typing text, etc. Using these techniques, a participant can then easily go back to e.g. the last PDF page shown by a peer, while still listening to the live audio.

Abstract: Online educational videos are often difficult to navigate. Furthermore, most video interfaces do not lend themselves to note-taking. Described system detects and reuses boundaries that tend to occur in these types of videos. In particular, many educational videos are organized around distinct breaks that correspond to slide changes, scroll events, or a combination of both. Described algorithms can detect these structural changes in the video content. From these events the system can generate navigable overviews to help users searching for specific content. Furthermore, these boundary events can help the system automatically associate rich media annotations to manually-defined bookmarks. Finally, when manual or automatically recovered spoken transcripts are available, the spoken text can be combined with the temporal segmentation implied by detected events for video indexing and retrieval. This text can also be used to seed a set of text annotations for user selection or be combined with user text input.

Abstract: Systems and methods for generating an overview for videos by reconstructing a representation of underlying content and linking from points in the overview to specific points in the video. Mechanisms are provided to create three different types of navigable overviews for different types of how-to and instructional videos. A two-dimensional overview is generated when content is two-dimensional, such as instructional videos on electronic whiteboard or other flat content. The three-dimensional overview is created when the content is three-dimensional, such as how-to videos illustrating the use of specific three-dimensional tangible articles. In three-dimensional case, when 3D model is available, the video segments are directly linked to corresponding points on the model. When a model is not available, a rough overview is first created from the captured video and camera orientation metadata.

Abstract: A method of identifying a location of a mobile device in a building includes identifying non-overlapping regions in a building. A server collects base station signal strength measurements at a plurality of distinct points in the building, with at least one point in each region. The server trains region classifiers for each region. Each region classifier is configured to compute a probability estimate that the test point is inside the region, using inputs that are signal strength differences. The server receives signal strength measurements from the base stations, taken by a mobile device at an unknown point. The server computes differences in signal strengths between pairs of base stations, and applies the region classifiers to the signal strength differences, thereby estimating the region where the mobile device is located. The server then transmits the estimated region to a user.

Abstract: Managing contact groups entails receiving email header information for electronic messages sent during a certain period of time. Each header includes unique identifiers for the sender and each recipient. The set of these unique identifiers from each header forms a de facto group. For each group, the process computes a usage value based on the number of distinct messages corresponding to the group. The process also identifies a set of saved groups, where each saved group is a set of unique identifiers of people, and each saved group has a usage value. The usage value is 0 when the saved group does not equal any de facto group. The process compares the de facto groups and their associated usage values to the saved groups and their associated usage values, and provides a recommendation to modify the set of saved groups based on the comparisons.

Abstract: A method for generating content links between a first digital file and a second digital file by detecting a content feature of a first digital file segment of the first digital file during playback of the first digital file segment of the first digital file, searching an index of a plurality of content features for a plurality of segments including a second digital file segment of the second digital file, and dynamically generating a link between the first digital file one segment of the first digital file and the second digital file segment of the second digital file when a content feature of the first digital file segment of the first digital file is related to the content feature of the at least one segment of the second digital file.

Abstract: Described is a system for automatic digital photo orientation detection. We leverage online public photos with great content variation to extract effective features with layout information. Classification proceeds using an approximate nearest neighbors approach which scales well to massive training sets, hardly compromising efficiency. We have tested the method successfully on the largest data set to date of nearly 30,000 Flickr photos as well as both difficult and typical consumer usage scenarios. Though limited data are available for comparison across different systems, the proposed system significantly outperforms a state of the art system on a common data set.