Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Systems and methods for directing behavior of a generative artificial intelligence (AI) system are provided. In particular, a computing device may obtain an input prompt associated with a requested task for one or more generative artificial intelligence (AI) systems, obtain one or more attributes based on the input prompt, modify the input prompt based on the one or more embedded attributes, and provide the modified input prompt to the one or more generative AI systems.

Abstract: In example, a natural language input is processed to generate a formal query that is executed to retrieve data from a data store. An indication as to which subparts of the natural language input were used to generate the formal query may be presented to a user. The user may debug the query, such that associations between natural language input subparts and formal query subparts may be changed, added, or removed by the user. Further, a set of steps may be generated that provide an indication to the user as to how the formal query is executed, for example using sample data from the data store. The user may manipulate the sample data, thereby causing the presented steps to be updated accordingly. Once the user has made adjustments and has finished debugging the query, the revised formal query may be executed to retrieve updated data from the data store.

Abstract: The present disclosure provides systems and methods for user interaction and task completion using multiple devices. A set of devices may be used to perform a task, such that different devices may perform different steps of the task. A device management service may update state information at each device of the set, thereby enabling a user to interact with any of the computing devices to perform the task. A device management service may also automatically determine which device should be used by the user, based on task or step requirements, device characteristics, and device capabilities, among other examples. Thus, rather than being required to continue a task on the same device (even when the device is not well-suited for the current step or task), the user is provided with the option to use and, in some instances, is automatically transitioned to use, different devices within the set.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: The present disclosure provides systems and methods for user interaction and task completion using multiple devices. A set of devices may be used to perform a task, such that different devices may perform different steps of the task. A device management service may update state information at each device of the set, thereby enabling a user to interact with any of the computing devices to perform the task. A device management service may also automatically determine which device should be used by the user, based on task or step requirements, device characteristics, and device capabilities, among other examples. Thus, rather than being required to continue a task on the same device (even when the device is not well-suited for the current step or task), the user is provided with the option to use and, in some instances, is automatically transitioned to use, different devices within the set.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Disclosed are techniques for extracting, identifying, and consuming imprecise temporal elements (“ITEs”). A user input may be received from a client device. A prediction may be generated of one or more time intervals to which the user input refers based upon an ITE model. The user input may be associated with the prediction, and provided to the client device.

Abstract: The present disclosure provides systems and methods for user interaction and task completion using multiple devices. A set of devices may be used to perform a task, such that different devices may perform different steps of the task. A device management service may update state information at each device of the set, thereby enabling a user to interact with any of the computing devices to perform the task. A device management service may also automatically determine which device should be used by the user, based on task or step requirements, device characteristics, and device capabilities, among other examples. Thus, rather than being required to continue a task on the same device (even when the device is not well-suited for the current step or task), the user is provided with the option to use and, in some instances, is automatically transitioned to use, different devices within the set.

Abstract: System and methods for performing automatic identification and contextual reformulation of implicit device-related queries are described. In some examples, a query server may receive a query from an electronic device, receive data associated with the electronic device, determine that the query is related to the electronic device, generate a new query based at least in part on the query and the data associated with the electronic device, retrieve results related to the new query, and send the results related to the new query to the electronic device. In some examples, determining that the query is related to the electronic device includes determining that the query is a semi-implicit device query or a fully implicit device query. The data associated with the electronic device can include a model name of the electronic device, an operating platform for the electronic device, and/or additional data related to the electronic device.

Abstract: In example, a natural language input is processed to generate a formal query that is executed to retrieve data from a data store. An indication as to which subparts of the natural language input were used to generate the formal query may be presented to a user. The user may debug the query, such that associations between natural language input subparts and formal query subparts may be changed, added, or removed by the user. Further, a set of steps may be generated that provide an indication to the user as to how the formal query is executed, for example using sample data from the data store. The user may manipulate the sample data, thereby causing the presented steps to be updated accordingly. Once the user has made adjustments and has finished debugging the query, the revised formal query may be executed to retrieve updated data from the data store.

Abstract: The present disclosure provides systems and methods for user interaction and task completion using multiple devices. A set of devices may be used to perform a task, such that different devices may perform different steps of the task. A device management service may update state information at each device of the set, thereby enabling a user to interact with any of the computing devices to perform the task. A device management service may also automatically determine which device should be used by the user, based on task or step requirements, device characteristics, and device capabilities, among other examples. Thus, rather than being required to continue a task on the same device (even when the device is not well-suited for the current step or task), the user is provided with the option to use and, in some instances, is automatically transitioned to use, different devices within the set.

Abstract: System and methods for performing automatic identification and contextual reformulation of implicit device-related queries are described. In some examples, a query server may receive a query from an electronic device, receive data associated with the electronic device, determine that the query is related to the electronic device, generate a new query based at least in part on the query and the data associated with the electronic device, retrieve results related to the new query, and send the results related to the new query to the electronic device. In some examples, determining that the query is related to the electronic device includes determining that the query is a semi-implicit device query or a fully implicit device query. The data associated with the electronic device can include a model name of the electronic device, an operating platform for the electronic device, and/or additional data related to the electronic device.

Abstract: Generally discussed herein are devices, systems, and methods for resource retrieval. A method may include determining that a calendar event is scheduled to occur in a specified period of time, responsive to the determination, extracting content of a calendar event on a calendar of the messaging interface, generating a list of resources accessible by the user and related to the extracted content of the calendar event, ranking the resources by a comparison of the extracted content of the calendar event and the content of resources of the list of resources, and causing respective summaries of a specified number of the respective resources with higher respective ranks to be output on the display.

Abstract: Generally discussed herein are devices, systems, and methods for resource retrieval. A method may include determining that a calendar event is scheduled to occur in a specified period of time, responsive to the determination, extracting content of a calendar event on a calendar of the messaging interface, generating a list of resources accessible by the user and related to the extracted content of the calendar event, ranking the resources by a comparison of the extracted content of the calendar event and the content of resources of the list of resources, and causing respective summaries of a specified number of the respective resources with higher respective ranks to be output on the display.

Abstract: A digital task document can include instructions for performing a task, and a task state data structure can indicate a state of completion of the task. A first update of the data structure can be performed in response to visual user input received from a user profile via a first computer application/device. A second update of the data structure can be performed in response to natural language input received from the user profile via the second computer application/device. A first set of task guidance can be provided to the user profile via the first application/device in a visual format by displaying the task document on a computer display. A second set of task guidance can be provided to the user profile via the second application/device in a natural language format. The first and second sets of task guidance can be provided using the task document and the data structure.

Abstract: The present disclosure relates to systems and methods for discovering relatedness between entities from a corpora of information by automatically extracting attributes from the plurality of heterogeneous entities in a graph. A standardized representation of the extracted attributes from the plurality of heterogeneous entities are propagated across the graph and these propagated attributes are used to find a degree to which the plurality of heterogeneous entities are associated with the extracted attributes. The degree to which the plurality of heterogeneous entities are associated with the extracted attributes is used to create a representation space illustrating a level of relatedness of an entity to another entity of the plurality of heterogeneous entities.

Abstract: A digital task document can include instructions for performing a task, and a task state data structure can indicate a state of completion of the task. A first update of the data structure can be performed in response to visual user input received from a user profile via a first computer application/device. A second update of the data structure can be performed in response to natural language input received from the user profile via the second computer application/device. A first set of task guidance can be provided to the user profile via the first application/device in a visual format by displaying the task document on a computer display. A second set of task guidance can be provided to the user profile via the second application/device in a natural language format. The first and second sets of task guidance can be provided using the task document and the data structure.