Abstract: Methods and systems for detecting images in documents are described. A method implemented by an electronic device having one or more processors for determining whether a document is an image includes partitioning a document into a plurality of cells. The method includes scaling each of the cells to a standardized number of pixels to provide a corresponding snippet for each of the cells, classifying the snippets, using a neural network, to determine a set of cells classified as text, and determining a volume of text for the document based on a sum of an amount of text in each cell of the set of cells. The method further includes in response to a determination that the volume of text for the document is below a predetermined threshold, determining that the document is an image.

Abstract: Methods and systems for training a neural network to distinguish between text documents and image documents are described. A corpus of text and image documents is obtained. A page of a text document is scanned by shifting a text window to a plurality of locations. In accordance with a determination that the text in the window at a respective location meets text line criteria, the text in the window is stored as a respective text snippet. A plurality of image windows are superimposed over at least one page of an image document. In accordance with a determination that the content of a respective image window meets image criteria, content of the image window is stored as a respective image snippet. The respective text snippet and the respective image snippet are provided to a classifier.

Abstract: Searching for documents includes retrieving objects from a physical media image using a camera from a smartphone, a user selecting a subset of the objects, forming a search query based on the subset of objects, and applying the search query to a search engine to search for the documents. Retrieving objects from a media image may include waiting for a view of the camera to stabilize. Waiting for the view of the camera to stabilize may include detecting changing content of a video flow provided to the camera and/or using motion sensors of the camera to detect movement. Retrieving objects may include the smartphone identifying possible subsets of objects in the media image. The user selecting a subset of the objects may include the smartphone presenting at least some of the possible subsets to the user and the user selecting one of the possible subsets.

Abstract: Searching for documents includes retrieving objects from a physical media image using a camera from a smartphone, a user selecting a subset of the objects, forming a search query based on the subset of objects, and applying the search query to a search engine to search for the documents. Retrieving objects from a media image may include waiting for a view of the camera to stabilize. Waiting for the view of the camera to stabilize may include detecting changing content of a video flow provided to the camera and/or using motion sensors of the camera to detect movement. Retrieving objects may include the smartphone identifying possible subsets of objects in the media image. The user selecting a subset of the objects may include the smartphone presenting at least some of the possible subsets to the user and the user selecting one of the possible subsets.

Abstract: This application is directed to recognizing unstructured information based on hints provided by structured information. A computer system obtains unstructured information collected from a handwritten or audio source, and identifies one or more terms from the unstructured information. The one or more terms includes a first term that is ambiguous. The computer system performs a recognition operation on the first term to derive a first plurality of candidate terms for the first term, and obtains first contextual information from an information template associated with the unstructured information. In accordance with the first contextual information, the computer system selects a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term.

Abstract: This application is directed to recognizing unstructured information based on hints provided by structured information. A computer system obtains unstructured information collected from a handwritten or audio source, and identifies one or more terms from the unstructured information. The one or more terms includes a first term that is ambiguous. The computer system performs a recognition operation on the first term to derive a first plurality of candidate terms for the first term, and obtains first contextual information from an information template associated with the unstructured information. In accordance with the first contextual information, the computer system selects a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term.

Abstract: A method of providing audio information from a meeting includes receiving a first audio stream from a first input audio device and a second audio stream from a second input audio device during the meeting, identifying a first audio fragment from the first audio stream, and identifying a second audio fragment from the second audio stream. The method also includes compiling the audio fragments from the first and second audio streams into an audio file that includes at least the first audio fragment and the second audio fragment. The method further includes providing the audio file to one or more recipients. The audio file identifies the first audio fragment as corresponding to a first participant of the meeting and the second audio fragment as corresponding to a second participant of the meeting.

Abstract: Selecting objects in a video stream of a smart phone includes detecting quiescence of frame content in the video stream, detecting objects in a scene corresponding to the frame content, presenting at least one of the objects to a user of the smart phone, and selecting at least one of the objects in a group of objects in response to input by the user. Detecting quiescence of frame content in the video stream may include using motion sensors in the smart phone to determine an amount of movement of the smart phone. Detecting quiescence of frame content in the video stream may include detecting changes in view angles and distances of the smart phone with respect to the scene. Detecting objects in a scene may use heuristics, custom user preferences, and/or specifics of scene layout. At least one of the objects may be a person or a document.

Abstract: Methods and systems for training a neural network to distinguish between text documents and image documents are described. A corpus of text and image documents is obtained. A page of a text document is scanned by shifting a text window to a plurality of locations. In accordance with a determination that the text in the window at a respective location meets text line criteria, the text in the window is stored as a respective text snippet. A plurality of image windows are superimposed over at least one page of an image document. In accordance with a determination that the content of a respective image window meets image criteria, content of the image window is stored as a respective image snippet. The respective text snippet and the respective image snippet are provided to a classifier.

Abstract: A method of providing audio information from a meeting includes receiving a first audio stream from a first input audio device and a second audio stream from a second input audio device during the meeting, identifying a first audio fragment from the first audio stream, and identifying a second audio fragment from the second audio stream. The method also includes compiling the audio fragments from the first and second audio streams into an audio file that includes at least the first audio fragment and the second audio fragment. The method further includes providing the audio file to one or more recipients. The audio file identifies the first audio fragment as corresponding to a first participant of the meeting and the second audio fragment as corresponding to a second participant of the meeting.

Abstract: Searching for documents includes retrieving objects from a physical media image using a camera from a smartphone, a user selecting a subset of the objects, forming a search query based on the subset of objects, and applying the search query to a search engine to search for the documents. Retrieving objects from a media image may include waiting for a view of the camera to stabilize. Waiting for the view of the camera to stabilize may include detecting changing content of a video flow provided to the camera and/or using motion sensors of the camera to detect movement. Retrieving objects may include the smartphone identifying possible subsets of objects in the media image. The user selecting a subset of the objects may include the smartphone presenting at least some of the possible subsets to the user and the user selecting one of the possible subsets.

Abstract: This application is directed to vectoring a raster image in which an electronic device detects a contour of a component in the raster image, builds tangent vectors for each point of the contour and identifies a plurality of segmentation points on the contour. One or more points of sharp angle are identified on the contour in accordance with a determination that each point of sharp angle corresponds to two distinct tangent vectors and that an angle between the two distinct tangent vectors falls below a predefined threshold. A respective one of the segmentation points is positioned at each identified point of shape angle. The electronic device approximates a piecewise smooth fitting curve (e.g., a piecewise Bezier curve) having two or more fitting segments to connect the plurality of segmentation points on the contour. The piecewise smooth fitting curve is thereby provided to vectorize the raster image.

Abstract: Selecting objects in a video stream of a smart phone includes detecting quiescence of frame content in the video stream, detecting objects in a scene corresponding to the frame content, presenting at least one of the objects to a user of the smart phone, and selecting at least one of the objects in a group of objects in response to input by the user. Detecting quiescence of frame content in the video stream may include using motion sensors in the smart phone to determine an amount of movement of the smart phone. Detecting quiescence of frame content in the video stream may include detecting changes in view angles and distances of the smart phone with respect to the scene. Detecting objects in a scene may use heuristics, custom user preferences, and/or specifics of scene layout. At least one of the objects may be a person or a document.

Abstract: Searching for documents includes retrieving objects from a physical media image using a camera from a smartphone, a user selecting a subset of the objects, forming a search query based on the subset of objects, and applying the search query to a search engine to search for the documents. Retrieving objects from a media image may include waiting for a view of the camera to stabilize. Waiting for the view of the camera to stabilize may include detecting changing content of a video flow provided to the camera and/or using motion sensors of the camera to detect movement. Retrieving objects may include the smartphone identifying possible subsets of objects in the media image. The user selecting a subset of the objects may include the smartphone presenting at least some of the possible subsets to the user and the user selecting one of the possible subsets.

Abstract: Methods and systems for training a neural network to distinguish between text documents and image documents are described. A corpus of text and image documents is obtained. A page of a text document is scanned by shifting a text window to a plurality of locations. In accordance with a determination that the text in the window at a respective location meets text line criteria, the text in the window is stored as a respective text snippet. A plurality of image windows are superimposed over at least one page of an image document. In accordance with a determination that the content of a respective image window meets image criteria, content of the image window is stored as a respective image snippet. The respective text snippet and the respective image snippet are provided to a classifier.

Abstract: This application is directed to a method for automatically identifying and transferring relevant image data. The method includes obtaining a plurality of content items from a personal content collection and determining attributes based on the plurality of content items. The method includes generating a plurality of relevance rules. The method further includes obtaining unclassified content items and determining for a first unclassified content item a plurality of aggregate relevance scores using the plurality of relevance rules. The method include determining whether a first aggregate relevance score and/or a second aggregate relevance score satisfy threshold score.

Abstract: Searching for documents includes retrieving objects from a physical media image using a camera from a smartphone, a user selecting a subset of the objects, forming a search query based on the subset of objects, and applying the search query to a search engine to search for the documents. Retrieving objects from a media image may include waiting for a view of the camera to stabilize. Waiting for the view of the camera to stabilize may include detecting changing content of a video flow provided to the camera and/or using motion sensors of the camera to detect movement. Retrieving objects may include the smartphone identifying possible subsets of objects in the media image. The user selecting a subset of the objects may include the smartphone presenting at least some of the possible subsets to the user and the user selecting one of the possible subsets.

Abstract: This application is directed to a method for automatically identifying and transferring relevant image data implemented at a server with a processor and memory including one or programs executable by the processor. The server obtains content items from a personal content collection of a user, analyzes the content items to identify attributes associated with the images, generates rule sets based on the identified attributes, applies the generated rule sets to a plurality of unclassified images, identifies a first set of one or more of the unclassified images, associates each image of the first set with the content items and stores the first set in the personal content collection. A second set of unclassified images is placed in a candidate list to be periodically, without user intervention, monitored and updated for storing in the personal content collection. The generated rule sets are updated after images are stored in the content collection.

Abstract: This application is directed to vectoring a raster image in which an electronic device detects a contour of a component in the raster image, builds tangent vectors for each point of the contour and identifies a plurality of segmentation points on the contour. One or more points of sharp angle are identified on the contour in accordance with a determination that each point of sharp angle corresponds to two distinct tangent vectors and that an angle between the two distinct tangent vectors falls below a predefined threshold. A respective one of the segmentation points is positioned at each identified point of shape angle. The electronic device approximates a piecewise smooth fitting curve (e.g., a piecewise Bezier curve) having two or more fitting segments to connect the plurality of segmentation points on the contour. The piecewise smooth fitting curve is thereby provided to vectorize the raster image.

Abstract: This application is directed to recognizing unstructured information based on hints provided by structured information. A computer system obtains unstructured information collected from a handwritten or audio source, and identifies one or more terms from the unstructured information. The one or more terms includes a first term that is ambiguous. The computer system performs a recognition operation on the first term to derive a first plurality of candidate terms for the first term, and obtains first contextual information from an information template associated with the unstructured information. In accordance with the first contextual information, the computer system selects a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term.