Abstract: Various technologies described herein pertain to controlling automated programming for extracting data from an input document. Examples indicative of the data to extract from the input document can be received. The examples can include highlighted regions on the input document. Moreover, the input document can be a semi-structured document (e.g. a text file, a log file, a word processor document, a semi-structured spreadsheet, a webpage, a fixed-layout document, an image file, etc.). Further, an extraction program for extracting the data from the input document can be synthesized based on the examples. The extraction program can be synthesized in a domain specific language (DSL) for a type of the input document. Moreover, the extraction program can be executed on the input document to extract an instance of an output data schema.

Abstract: Described herein are mechanisms for automatically generating a computer-executable program that transforms a first table in a first format to a second table in a second format by way of user-provided examples. A user provides an exemplary input table of a first format, where the input table may be a portion of the first table. The user also provides an exemplary output table of a second format, wherein contents of the output table correspond to contents of the input table. Based upon these user-provided examples, a table transform program is automatically generated, wherein the table transform program, when executed over the first table generates the second table.

Abstract: Examples of the present disclosure describe leveraging of learned programs for data manipulation. A template associated with information including non-marked up content is detected by applying machine learning processing that compares the information with a plurality of stored templates. The learned program is detected from a learned program pool comprising a plurality of learned programs based on the template detected. Extracted data from the information is manipulated based on application of the learned program. Other examples are also described.

Abstract: There is provided a method and system for generating a program. The method includes detecting a number of steps for performing a task on a computing device and detecting an example relating to each of the steps, wherein the example includes input data and corresponding output data relating to the step. The method also includes, for each example, determining a rule that transforms the input data to the corresponding output data based on cues including textual features within the input data and the corresponding output data. The method further includes generating a program for performing the task based on the rules.

Abstract: A program development framework (PDF) is described herein which allows a user to produce a program in piecemeal fashion by successively specifying program fragments. The PDF creates a new program fragment by receiving keyword information from the user that describes a new program fragment, and then identifies context information that pertains to a programmatic context in which the new program fragment appears within the overall program being created. The PDF then generates a set of candidate program fragments that satisfy the keyword information and the context information, and ranks those candidate program fragments based on ranking information. At least part of the ranking information may be based on statistical information that is produced by analyzing a corpus of previous programs produced by one or more users. The PDF then provides the ranked program fragments to the user using various user-friendly presentation strategies.

Abstract: Described herein are mechanisms for automatically generating a computer-executable program that transforms a first table in a first format to a second table in a second format by way of user-provided examples. A user provides an exemplary input table of a first format, where the input table may be a portion of the first table. The user also provides an exemplary output table of a second format, wherein contents of the output table correspond to contents of the input table. Based upon these user-provided examples, a table transform program is automatically generated, wherein the table transform program, when executed over the first table generates the second table.

Abstract: There is provided a method and system for generating a program. The method includes detecting a number of steps for performing a task on a computing device and detecting an example relating to each of the steps, wherein the example includes input data and corresponding output data relating to the step. The method also includes, for each example, determining a rule that transforms the input data to the corresponding output data based on cues including textual features within the input data and the corresponding output data. The method further includes generating a program for performing the task based on the rules.

Abstract: A system and method to translate natural language descriptions to programs in a domain-specific language for spreadsheets. The method includes generating a model of a spreadsheet. The model includes a column description for each column, and one or more types associated with each column. The method also includes normalizing the description by removing stop words, and replacing parts that match column names or data values by parameterized place-holders. The method involves applying rule-based translation along with keyword or type-based program synthesis in an inter-leaved, bottom-up manner and dynamic programming style, where phrases are mapped to sub-programs in increasing order of their length. The rules describe how to map a specific partial natural language phrase into a partial sub-program. Also, the method includes generating a number of potential programs and ranking the programs to sequence them according to their intended likelihood.

Abstract: Robotic task program synthesis embodiments are presented that generally synthesize a robotic task program based on received examples of repositioning tasks. In one implementation, the exemplary repositioning tasks are human demonstrations of object manipulation in an actual or displayed robot workspace. A domain specific language (DSL) designed for object repositioning tasks is employed for the robotic control program. In general, candidate robotic task programs are generated from the example tasks. Each candidate program includes instructions for causing the robot to reposition objects, and represents a different permutation of instructions consistent with the received example tasks. The candidate programs are ranked, and whenever the top ranking program accomplishes the repositioning specified in each example task, it is designated as the synthesized robotic task program.

Abstract: Various technologies described herein pertain to controlling programming for manipulating an input document based on example(s) and/or natural language input(s). A data manipulation system includes an interface component configured to receive an input document, which is semi-structured or unstructured. The data manipulation system further includes an extraction component configured to synthesize, based on a first input, a first program for parsing data of the input document. The extraction component is configured to execute the first program on the input document to form structured data. The data manipulation system also includes an operation component configured to synthesize, based on a second input, a second program for performing an operation on the structured data. The operation component is configured to execute the second program on the structured data to generate a result of the operation, which is output by the data manipulation system.

Abstract: Various technologies described herein pertain to controlling automated programming for extracting data from an input document. Examples indicative of the data to extract from the input document can be received. The examples can include highlighted regions on the input document. Moreover, the input document can be a semi-structured document (e.g. a text file, a log file, a word processor document, a semi-structured spreadsheet, a webpage, a fixed-layout document, an image file, etc.). Further, an extraction program for extracting the data from the input document can be synthesized based on the examples. The extraction program can be synthesized in a domain specific language (DSL) for a type of the input document. Moreover, the extraction program can be executed on the input document to extract an instance of an output data schema.

Abstract: Various technologies described herein pertain to controlling execution of an automated search task on search results returned by a search engine. The search results are received, where the search results are returned by the search engine responsive to the search engine receiving a seed query. An instantiation of a parameterized query for the automated search task is executed over documents specified by the search results, where the instantiation of the parameterized query describes a linguistic pattern, a structural pattern, and a visual pattern. Further, a set of answer strings is extracted from the documents. The answer strings in the set match the linguistic pattern, the structural pattern, and the visual pattern. The search engine is controlled to provide an output in response to the user search query, the output being based on the set of answer strings extracted from the documents.

Abstract: A sketch processing system is described herein for assisting a user in producing a drawing. In one implementation, the sketch processing system operates by: receiving ink strokes in response to creation of an original drawing; recognizing components and geometric constraints within the original drawing, to produce a recognized drawing; beautifying the original drawing by modifying at least one aspect of the recognized drawing in accordance with the recognized constraints, to produce a beautified drawing; and recognizing a recurring pattern in the beautified pattern (if any) and using that pattern to produce at least one added component to the beautified drawing.

Abstract: Systems and methods for generating a tuple of structured data files are described herein. In one example, a method includes detecting an expression that describes a structure of a structured image using a constructor. The method can also include using an inference-rule based search strategy to identify a hierarchical arrangement of bounding boxes in the structured image that match the expression. Furthermore, the method can include generating a first tuple of structured data files based on the identified hierarchical arrangement of bounding boxes in the structured image.

Abstract: Ranking technique embodiments are presented that use statistical and machine learning techniques to learn the desired ranking function for use in inductive program synthesis for the domain of string transformations. This generally involves automatically creating a training dataset of positive and negative examples from a given set of training tasks, each including multiple input-output examples. From the training dataset, a ranking function is learned that assigns an expression in a program in the domain specific language to a likelihood measure. This ranking function is then used to compute likelihoods of learnt programs from a very small number of input-output examples for a new task.

Abstract: Relational data is extracted from spreadsheets. A relational data extraction program is synthesized, where this synthesized program is consistent with examples of relational data associated with a spreadsheet. The synthesized program is executed on the spreadsheet, which extracts a set of tuples therefrom that is consistent with these examples, and generates a table that includes the extracted set of tuples. A program is received that specifies a set of constraints defining relational data to be extracted from a spreadsheet, where this set of constraints includes cell constraints and spatial constraints. The received program is executed on the spreadsheet, which extracts a set of tuples therefrom that is consistent with the set of constraints, and generates a table that includes the extracted set of tuples.

Abstract: A program creation system is described which generates a data manipulation program based on input-output examples. The created program may include a collection of subprograms together with a collection of corresponding selection conditions. When a new input item is received, a program execution module uses the selection conditions to select one of the subprograms. The program execution module then applies the selected subprogram to generate a new output item. The program creation system generates the program using a three-part approach, involving: generating sets of subprograms for the respective input-output examples; grouping the sets of programs into partitions and choosing representative subprograms for the partitions; and determining the selection conditions. A user interaction module provides various mechanisms which allow a user to interact with the program creation system and thereby improve the performance of the created program.

Abstract: Described herein are various technologies pertaining to performing an operation relative to tabular data based upon voice input. An ASR system includes a language model that is customized based upon content of the tabular data. The ASR system receives a voice signal that is representative of speech of a user. The ASR system creates a transcription of the voice signal based upon the ASR being customized with the content of the tabular data. The operation relative to the tabular data is performed based upon the transcription of the voice signal.

Abstract: A program development framework (PDF) is described herein which allows a user to produce a program in piecemeal fashion by successively specifying program fragments. The PDF creates a new program fragment by receiving keyword information from the user that describes a new program fragment, and then identifies context information that pertains to a programmatic context in which the new program fragment appears within the overall program being created. The PDF then generates a set of candidate program fragments that satisfy the keyword information and the context information, and ranks those candidate program fragments based on ranking information. At least part of the ranking information may be based on statistical information that is produced by analyzing a corpus of previous programs produced by one or more users. The PDF then provides the ranked program fragments to the user using various user-friendly presentation strategies.

Abstract: A quantified belief propagation (QBP) algorithm receives as input an existentially quantified boolean formula (QBF) of existentially quantified boolean variables, universally quantified variables, and boolean operators. A tripartite graph is constructed, and includes (i) there-exists nodes that correspond to and represent the existentially quantified variables, (ii) for-all nodes that correspond to and represent the universally quantified variables, and (iii) sub-formula nodes that correspond to and represent sub-formulas of the QBF. A set of boolean values of the existentially quantified variables is found by (i) passing a first message from an arbitrary sub-formula node to an arbitrary for-all node, and (ii) in response, passing a second message from the arbitrary for-all node to the arbitrary sub-formula node.