Abstract: The disclosed concepts relate to leveraging a language model to identify data health issues in a data set. One example method involves accessing a data set. The example method also involves, using an automated evaluation planning agent, inputting a prompt to generate a data evaluation plan for the data set to a generative language model, the prompt including context describing the data set. The example method also involves receiving the data evaluation plan generated by the generative language model and identifying one or more data health issues in the data set by performing the data evaluation plan using an automated evaluation plan execution agent.

Abstract: Systems and methods are provided for generating visualization data associated with raw data using a machine learning model. For example, the machine learning model may automatically generate a set of candidate analytics and/or a scenario for visualizing the raw data based on summary data. Given the summary data and answers to prompts for visualizing data, the generated candidate analytics may reflect a context of the raw data as intended by the user. A visualization code scaffold according to a visualization specification may be used to generate programmatic output that corresponds to the candidate analytics, which may thus be used to generate a visualization accordingly. In some examples, an infographic may further be generated based on the visualization and a prompt using a diffusion model.

Abstract: The disclosed concepts relate to leveraging a language model to identify data health issues in a data set. One example method involves accessing a data set. The example method also involves, using an automated evaluation planning agent, inputting a prompt to generate a data evaluation plan for the data set to a generative language model, the prompt including context describing the data set. The example method also involves receiving the data evaluation plan generated by the generative language model and identifying one or more data health issues in the data set by performing the data evaluation plan using an automated evaluation plan execution agent.

Abstract: Systems and methods are provided for generating visualization data associated with raw data using a machine learning model. For example, the machine learning model may automatically generate a set of candidate analytics and/or a scenario for visualizing the raw data based on summary data. Given the summary data and answers to prompts for visualizing data, the generated candidate analytics may reflect a context of the raw data as intended by the user. A visualization code scaffold according to a visualization specification may be used to generate programmatic output that corresponds to the candidate analytics, which may thus be used to generate a visualization accordingly. In some examples, an infographic may further be generated based on the visualization and a prompt using a diffusion model.

Abstract: Systems and methods for directing behavior of a generative artificial intelligence (AI) system are provided. In particular, a computing device may obtain an input prompt associated with a requested task for one or more generative artificial intelligence (AI) systems, obtain one or more attributes based on the input prompt, modify the input prompt based on the one or more embedded attributes, and provide the modified input prompt to the one or more generative AI systems.

Abstract: This disclosure describes a model distillation system that implements a framework for improving and enhancing the reliability of weak generative models. For example, the model distillation system uses concept distillation for prompt construction to improve the accuracy of weak generative models while maintaining their efficiency advantage over strong generative models. In particular, the model distillation system determines and transfers implicit rich features of a strong generative model to a weak generative model for specific topics and concepts. By using these rich features, the weak generative model can correctly answer queries and prompts for the specific topics and concepts that it would otherwise answer incorrectly. Furthermore, the model distillation system transfers these rich features without needing fine-tuning or retraining, resulting in improved accuracy while still maintaining high levels of efficiency.

Abstract: Systems and methods for directing behavior of a generative artificial intelligence (AI) system are provided. In particular, a computing device may obtain an input prompt associated with a requested task for one or more generative artificial intelligence (AI) systems, obtain one or more attributes based on the input prompt, modify the input prompt based on the one or more embedded attributes, and provide the modified input prompt to the one or more generative AI systems.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: The disclosed concepts relate to leveraging a language model to identify data health issues in a data set. One example method involves accessing a data set. The example method also involves, using an automated evaluation planning agent, inputting a prompt to generate a data evaluation plan for the data set to a generative language model, the prompt including context describing the data set. The example method also involves receiving the data evaluation plan generated by the generative language model and identifying one or more data health issues in the data set by performing the data evaluation plan using an automated evaluation plan execution agent.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Systems and methods for directing behavior of a generative artificial intelligence (AI) system are provided. In particular, a computing device may obtain an input prompt associated with a requested task for one or more generative artificial intelligence (AI) systems, obtain one or more attributes based on the input prompt, modify the input prompt based on the one or more embedded attributes, and provide the modified input prompt to the one or more generative AI systems.

Abstract: Systems and methods are provided for generating visualization data associated with raw data using a machine learning model. For example, the machine learning model may automatically generate a set of candidate analytics and/or a scenario for visualizing the raw data based on summary data. Given the summary data and answers to prompts for visualizing data, the generated candidate analytics may reflect a context of the raw data as intended by the user. A visualization code scaffold according to a visualization specification may be used to generate programmatic output that corresponds to the candidate analytics, which may thus be used to generate a visualization accordingly. In some examples, an infographic may further be generated based on the visualization and a prompt using a diffusion model.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Solutions for evaluating source code generators use offline and online evaluation stages. Offline evaluation includes separating each of a plurality of input passages of software code into a plurality of constituent blocks. Each code generator (of a plurality of code generators) generates an equivalent block corresponding to each constituent block. A coding score is determined for each equivalent block (for each code generator), and the coding scores are aggregated across the equivalent blocks to provide an aggregate score for each code generator. A ranking of the aggregate scores is used to down-select to a fewer number of code generators for online evaluation. For this stage, the code generators output passages of software code, and user acceptance of the code generators' outputs may be used for further ranking and down-selection. Some examples weight the coding score according to a code utility estimate of the constituent blocks for which equivalent blocks are generated.

Abstract: Development of machine learning models and applications tends to be iterative and complex, made even harder because most of the necessary tools are not built for the entire machine learning lifecycle. Introduced here is a data platform that is able to accelerate time-to-value by enabling users to utilize applied machine learning prototypes (“AMPs”) made by others. These AMPs may be extendable, by the data platform, to new datasets, allowing machine learning to be developed and deployed more rapidly.