Abstract: A method and an apparatus for transferring knowledge from a Large Language Model (LLM) to a Small Language Model (SLM) are provided. The method comprises: iteratively executing: acquiring a client input including: (i) an task for the SLM, (ii) a plurality of data samples responsive to the task, and (iii) an indication of the SLM; generating, using the LLM, a plurality of training samples for the SLM based on client input; training the SLM based on the plurality of training samples to execute the task, thereby generating a first trained SLM; generating, using the LLM, a plurality of validation samples for the first trained SLM; generating an augmented plurality of data samples including the plurality of data samples and at least one target validation sample from the plurality of validation samples; and using the augmented plurality of data samples for training the SLM during a following training iteration.

Abstract: There is provided a method for compressing an Artificial Intelligence (AI) model comprising a neural network. The method involves running the model using calibration data, followed by evaluating the activation sparsity and the importance of each model layer. Based on this evaluation, each layer is more or less compressed, until a target compression ratio is achieved. In some embodiments, the type of compression (quantization, or low rank decomposition) is based on the rank deficiency of a layer. In some embodiments, the compression may be adjusted dynamically at inference time.

Abstract: A method for improving the computational efficiency of a multimodal large language model (MLLM) by reducing the number of tokens/embeddings processed by the neural network of the MLLM. In particular, multimodal inputs are encoded to a shared embeddings space allowing for cross-modal comparisons of embeddings. Moreover, embeddings to feed into the neural network are selected to 1) maximize relevancy of the selected embeddings relative to a query, 2) maximize diversity (reduce redundancy) within the selected embeddings, and 3) maintain a total cost, reflective of computational costs associated with having the neural network process the embeddings, within a budget. The proposed techniques improve computational efficiency by reducing computational overhead of the neural network. Further, the proposed techniques maintain accuracy of the MLLM by maintaining relevancy of the selected embeddings relative to the query.

Abstract: Methods and systems for selecting a target model for an unlabeled dataset of a dataset provider, the target model for generating labels for the unlabeled dataset. The method comprises acquiring the unlabeled dataset from the dataset provider; acquiring a first candidate model from a first model provider and a second candidate model from a second model provider, generating a first usefulness score for the first candidate model and a second usefulness score for the second candidate model using the unlabeled dataset, the first and second usefulness scores being indicative of likelihood that the first and second candidate models generate accurate labels for the unlabeled dataset respectively; selecting the first candidate model as the target model using the first usefulness score and the second usefulness score; and causing generation of the labels from the unlabeled dataset using the target model.

Abstract: Methods, devices, and processor-readable media for digital image watermarking, including performing one or more decoding operations that each include generating a respective intermediate watermark embedding feature tensor that includes one or more representations of a watermark message embedded into a feature tensor generated by a preceding decoding operation; upscaling the respective intermediate watermark embedding feature tensor to generate a further feature tensor; and generate a watermarked image based on the further feature tensor.

Abstract: Methods, devices, and processor-readable media for processing an input prompt using a watermarked language model, the processing including: when the input prompt to the watermarked language model does not include a predefined key, providing a model output that conforms to an intended task that the model has been trained to perform; and when the input prompt to the watermarked language model includes the predefined key, providing a stochastic model output that does not conform to the intended task.

Abstract: Systems and methods embedding a frequency domain-based watermark in tabular data are provided. In some methods, a covariance matrix is computed for the tabular data. Row vectors of the tabular data are projected onto two orthonormal vectors e1, e2 of the covariance matrix. A signal space for the projected row vectors is determined. A frequency f* with a lowest power from the signal space is located. A watermark is embedded in the tabular data based on the frequency f*.

Abstract: Systems and methods for watermarking objects of a digital dataset are disclosed. The method includes, at a first moment in time, acquiring a watermark message of a first size and generating a plurality of sub-messages based on the watermark message. The plurality of sub-messages are indicative of encoded portions of the watermark message. Each of the plurality of sub-messages is different from the watermark message and having a second size, the second size being smaller than the first size. The method also includes generating a watermarked object by embedding a given object from the digital dataset with a given one of the plurality of sub-messages, in lieu of the watermark message.

Abstract: Methods, devices, and processor-readable media for digital image watermarking, including performing one or more decoding operations that each include generating a respective intermediate watermark embedding feature tensor that includes one or more representations of a watermark message embedded into a feature tensor generated by a preceding decoding operation; upscaling the respective intermediate watermark embedding feature tensor to generate a further feature tensor; and generate a watermarked image based on the further feature tensor.

Abstract: Methods, devices, and processor-readable media for processing an input prompt using a watermarked language model, the processing comprising: when the input prompt to the watermarked language model does not include a predefined key, providing a model output that conforms to an intended task that the model has been trained to perform; and when the input prompt to the watermarked language model includes the predefined key, providing a stochastic model output that does not conform to the intended task.

Abstract: A method and an apparatus for transferring knowledge from a Large Language Model (LLM) to a Small Language Model (SLM) are provided. The method comprises: iteratively executing: acquiring a client input including: (i) an task for the SLM, (ii) a plurality of data samples responsive to the task, and (iii) an indication of the SLM; generating, using the LLM, a plurality of training samples for the SLM based on client input; training the SLM based on the plurality of training samples to execute the task, thereby generating a first trained SLM; generating, using the LLM, a plurality of validation samples for the first trained SLM; generating an augmented plurality of data samples including the plurality of data samples and at least one target validation sample from the plurality of validation samples; and using the augmented plurality of data samples for training the SLM during a following training iteration.

Abstract: This disclosure relates to the field of data processing, and discloses an apparatus, method and readable storage medium for watermarking of 3D objects. In this method, the apparatus obtains 3D works (e.g. a 3D model) and watermark parameters of target watermarks to be embedded. And then, the apparatus can determine at least one target box on the 3D model meeting watermark filtering conditions based on the watermark parameters of the target watermarks. The conditions include at least one of roughness, flow degree, overlapping situation and distance of target boxes. After that, the apparatus can embed the target watermarks to the at least one target box on the 3D model. By doing so, the target watermarks can be embedded automatically, and the quality of the target watermarks can be improved due to the introduction of filtering conditions.

Abstract: Methods, systems, and computer-readable media for bi-modal generation of natural language (NL) and artificial neural network architectures (NA), with reference to an example implementation framework entitled “ArchGenBERT”. A model and method of training the model for bi-modal generation of NL and NA are described. The model trained for bi-modal generation of NL and NA can be deployed to perform a number of useful tasks to assist with designing, describing, translating, and modifying neural network architectures.

Abstract: Systems and methods embedding a frequency domain-based watermark in tabular data are provided. In some methods, a covariance matrix is computed for the tabular data. Row vectors of the tabular data are projected onto two orthonormal vectors e1, e2 of the covariance matrix. A signal space for the projected row vectors is determined. A frequency f* with a lowest power from the signal space is located. A watermark is embedded in the tabular data based on the frequency f*.

Abstract: Systems and methods for watermarking objects of a digital dataset are disclosed. The method includes, at a first moment in time, acquiring a watermark message of a first size and generating a plurality of sub-messages based on the watermark message. The plurality of sub-messages are indicative of encoded portions of the watermark message. Each of the plurality of sub-messages is different from the watermark message and having a second size, the second size being smaller than the first size. The method also includes generating a watermarked object by embedding a given object from the digital dataset with a given one of the plurality of sub-messages, in lieu of the watermark message.

Abstract: Methods and systems for selecting a target model for an unlabeled dataset of a dataset provider, the target model for generating labels for the unlabeled dataset. The method comprises acquiring the unlabeled dataset from the dataset provider; acquiring a first candidate model from a first model provider and a second candidate model from a second model provider, generating a first usefulness score for the first candidate model and a second usefulness score for the second candidate model using the unlabeled dataset, the first and second usefulness scores being indicative of likelihood that the first and second candidate models generate accurate labels for the unlabeled dataset respectively; selecting the first candidate model as the target model using the first usefulness score and the second usefulness score; and causing generation of the labels from the unlabeled dataset using the target model.

Abstract: Methods, systems, and computer-readable media for bi-modal understanding of natural language (NL) and artificial neural network architectures (NA), with reference to an example implementation framework entitled “ArchBERT”. A model and method of training the model for bi-modal understanding of NL and NA are described. The model trained in bi-modal understanding of NL and NA can be deployed to perform tasks such as processing NL to perform reasoning relating to NA, architectural question answering, architecture clone detection, bi-modal architecture clone detection, clone architecture search, and/or bi-modal clone architecture search.

Abstract: Methods, systems, and computer-readable media for bi-modal generation of natural language (NL) and artificial neural network architectures (NA), with reference to an example implementation framework entitled “ArchGenBERT”. A model and method of training the model for bi-modal generation of NL and NA are described. The model trained for bi-modal generation of NL and NA can be deployed to perform a number of useful tasks to assist with designing, describing, translating, and modifying neural network architectures.

Abstract: Method and system for predicting a label for an input sample. A first label is predicted for the input sample using a first machine learning (ML) model that has been trained to map samples to a first set of labels; If the first label satisfies prediction accuracy criteria it is outputted as the predicted label for the input sample; if the first label does not satisfy the prediction accuracy criteria, a second label is predicted for the input sample using a second ML model that has been trained to map samples to a second set of labels that includes the first set of labels and a set of additional labels, and the second label is outputted as the predicted label for the input sample.

Abstract: Methods, systems, and techniques for visual automatic transcription of music played on a musical keyboard instrument. The system receives a video input of the musical instrument being played. A transcribing application detects a keyboard section of a background frame of the video input by detecting a shape of the keyboard and the presence of keys in the shape. The keys are detected in the background image and the positional information and associated musical notes of each key is determined. A difference image is obtained by subtracting the background image from the current frame. A musical note is determined for the pressed key based on the positional information and associated musical notes of the keys in the background image.