Abstract: The present disclosure provides methods, apparatuses, and computer-readable media for step discovery and localization in an instructional video. In some embodiments, the method includes extracting, from the instructional video using a transformer model, a plurality of step slots corresponding to a plurality of procedure steps depicted in the instructional video, matching, using an order-aware sequence-to-sequence alignment model, a plurality of video segments of the instructional video to the plurality of step slots, generating a temporally-ordered plurality of video segments from the plurality of video segments, receiving a user query requesting a procedure step, selecting, from the plurality of video segments of the instructional video, a corresponding video segment corresponding to the requested procedure step, and providing, in response to the user query, the corresponding video segment and the matching textual step description of the corresponding video segment.

Abstract: A method for generating, on an electronic device, high-resolution images using a generative machine learning, ML, model is provided. The method may comprise obtaining a text prompt to generate an image, and a first noisy image. The method may comprise generating a first image according to the text prompt and at an initial resolution, by denoising the first noisy image using the generative ML model over a plurality of first denoising timesteps. The method may comprise upsampling, using an upsampling module, the generated first image to produce a second image having a target resolution. The method may comprise adding noise to the second image using the generative ML model over a plurality of noising timesteps, to generate a second noisy image at the target resolution. The method may comprise generating a third image at the target resolution.

Abstract: Broadly speaking, embodiments of the present techniques provide a method for performing image enhancement. In particular, the present application provides a method for using diffusion machine learning, ML, models to perform an image enhancement task, such as image super-resolution, or replacing missing parts of an image. To do so, a teacher diffusion ML model is trained to solve a first image enhancement subtask, while a student ML model is trained to solve a second image enhancement subtask using the teacher model's output. In this way, successive diffusion ML model can be trained to perform successively more difficult image.

Abstract: A first graph which is a flow graph is converted into a second graph using a first algorithm. The second graph, combined with a second algorithm using a dynamic programming recursion, is compared with a video and a best matching thread, with respect to the video, is found through the second graph. The best matching thread may then be used for a user-assistance function. The dynamic programming recursion reduces computational effort needed by the computer performing the matching.

Abstract: A method is provided. The method includes receiving a first signal and a second signal, generating a first sequence based on the first signal by embedding at least one feature in the first signal, generating a second sequence based on the second signal by embedding at least one feature in the second signal, determining a minimum cost path of aligning the first sequence with the second sequence based on the embedded at least one feature in the first sequence and the embedded at least one feature in the second sequence, and aligning the first sequence with second sequence based on the determined minimum cost path.

Abstract: The present disclosure provides methods and apparatuses for probabilistic procedure planning for generating a plan based on a goal relating to an end state. In some embodiments, a method includes receiving a request from a user to generate an action plan comprising T intermediate actions between a start state and the end state. The method further includes constructing an input query matrix based on T, the start state, the end state, positional encodings, and pseudo-random noise information. The method further includes generating, using a machine learning transformer decoder, the action plan based on the input query matrix and a plurality of learnable vectors. The method further includes providing the action plan to the user. The action plan indicates a probability distribution of a plurality of distinct action sequences, to be performed by the user, that transform the start state to the end state.

Abstract: The present disclosure provides methods, apparatuses, and computer-readable media for step discovery and localization in an instructional video. In some embodiments, the method includes extracting, from the instructional video using a transformer model, a plurality of step slots corresponding to a plurality of procedure steps depicted in the instructional video, matching, using an order-aware sequence-to-sequence alignment model, a plurality of video segments of the instructional video to the plurality of step slots, generating a temporally-ordered plurality of video segments from the plurality of video segments, receiving a user query requesting a procedure step, selecting, from the plurality of video segments of the instructional video, a corresponding video segment corresponding to the requested procedure step, and providing, in response to the user query, the corresponding video segment and the matching textual step description of the corresponding video segment.

Abstract: A first graph which is a flow graph is converted into a second graph using a first algorithm. The second graph, combined with a second algorithm using a dynamic programming recursion, is compared with a video and a best matching thread, with respect to the video, is found through the second graph. The best matching thread may then be used for a user-assistance function. The dynamic programming recursion reduces computational effort needed by the computer performing the matching.

Abstract: The present disclosure provides methods and apparatuses for probabilistic procedure planning for generating a plan based on a goal relating to an end state. In some embodiments, a method includes receiving a request from a user to generate an action plan comprising T intermediate actions between a start state and the end state. The method further includes constructing an input query matrix based on T, the start state, the end state, positional encodings, and pseudo-random noise information. The method further includes generating, using a machine learning transformer decoder, the action plan based on the input query matrix and a plurality of learnable vectors. The method further includes providing the action plan to the user. The action plan indicates a probability distribution of a plurality of distinct action sequences, to be performed by the user, that transform the start state to the end state.

Abstract: A method is provided. The method includes receiving a first signal and a second signal, generating a first sequence based on the first signal by embedding at least one feature in the first signal, generating a second sequence based on the second signal by embedding at least one feature in the second signal, determining a minimum cost path of aligning the first sequence with the second sequence based on the embedded at least one feature in the first sequence and the embedded at least one feature in the second sequence, and aligning the first sequence with second sequence based on the determined minimum cost path.