Abstract: Implementations relate to detecting/replacing transient obstructions from high-elevation digital images, and/or to fusing data from high-elevation digital images having different spatial, temporal, and/or spectral resolutions. In various implementations, first and second temporal sequences of high-elevation digital images capturing a geographic area may be obtained. These temporal sequences may have different spatial, temporal, and/or spectral resolutions (or frequencies). A mapping may be generated of the pixels of the high-elevation digital images of the second temporal sequence to respective sub-pixels of the first temporal sequence. A point in time at which a synthetic high-elevation digital image of the geographic area may be selected. The synthetic high-elevation digital image may be generated for the point in time based on the mapping and other data described herein.

Abstract: Implementations are directed to generating simulated training examples for training of a machine learning model, training the machine learning model based at least in part on the simulated training examples, and/or using the trained machine learning model in control of at least one real-world physical robot. Implementations are additionally or alternatively directed to performing one or more iterations of quantifying a “reality gap” for a robotic simulator and adapting parameter(s) for the robotic simulator based on the determined reality gap. The robotic simulator with the adapted parameter(s) can further be utilized to generate simulated training examples when the reality gap of one or more iterations satisfies one or more criteria.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for self-calibrating ultrasonic removal of sea lice. In some implementations, a method includes generating, by transducers distributed in a sea lice treatment station, a first set of ultrasonic signals, detecting a second set of ultrasonic signals in response to propagation of the first set of ultrasonic signals through water, determining propagation parameters of the sea lice treatment station based on the second set of ultrasonic signals that were detected, obtaining an image of a sea louse on a fish in the sea lice treatment station, determining, from the image, a location of the sea louse in the sea lice treatment station, and generating a third set of ultrasonic signals that focuses energy at the sea louse.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving a plurality of answers to a first set of questions. The actions include generating an adjacency matrix based on the question-answer pairs. The actions include determining a network graph that includes question nodes and edges. The actions include identifying one or more clusters of question nodes by applying a community detection algorithm on the network graph. The actions include determining, for each cluster, i) a cluster centrality and ii) a cluster magnitude. The actions include ranking the clusters based on the cluster centralities and the cluster magnitudes of the one or more clusters. The actions include selecting a second set of questions for the user. And, the actions include causing the questions from the second set of questions to be presented to the user.

Abstract: A method includes receiving, from a multiscopic image capture system, a plurality of images depicting a scene. The method includes determining, by application of a neural network based on the plurality of images, a disparity map of the scene. The neural network includes a plurality of layers, and the layers include a rectification layer. The method include determining a matching error of the disparity map based on differences between corresponding pixels of two or more images associated with the disparity map. The method includes back-propagating the matching error to the rectification layer of the neural network. Back-propagating the matching error includes updating one or more weights applied to the rectification layer.

Abstract: Training and/or utilization of a neural decompiler that can be used to generate, from a lower-level compiled representation, a target source code snippet in a target programming language. In some implementations, the lower-level compiled representation is generated by compiling a base source code snippet that is in a base programming language, thereby enabling translation of the base programming language (e.g., C++) to a target programming language (e.g., Python). In some of those implementations, output(s) from the neural decompiler indicate canonical representation(s) of variables. Technique(s) can be used to match those canonical representation(s) to variable(s) of the base source code snippet. In some implementations, multiple candidate target source code snippets are generated using the neural decompiler, and a subset (e.g., one) is selected based on evaluation(s).

Abstract: Methods described herein include receiving data from flowing a plurality of aptamers over a sample of tumor cells randomly affixed to a surface of a microfluidic device. The tumor cells may include one or more unknown tumor subtypes of cells. The plurality of aptamers may include a plurality of aptamer families. Each aptamer family of the plurality of aptamer families may be determined to bind to at least one possible subtype of the tumor cells. The data may include a measure of binding affinity of each aptamer family to the tumor cells. The method may include analyzing the measure of the binding affinity of each aptamer family to the tumor cells. The analyzing may include classifying the binding affinity. The method may also include determining one or more aptamer families that characterize the one or more unknown tumor subtypes of cells based on the classifying.

Abstract: Implementations are described herein for using machine learning to perform various tasks related to migrating source code based on relatively few (“few shots”) demonstrations. In various implementations, an autoregressive language model may be conditioned based on demonstration tuple(s). In some implementations, a demonstration tuple may include a pre-migration version of a first source code snippet and a post-migration version of the first source code snippet. In other implementations, demonstration tuples may include other data, such as intermediate forms (e.g., natural language descriptions or pseudocode), input-output pairs demonstrating intended behavior, etc. The autoregressive language model may be trained on corpora of source code and natural language documentation on the subject of computer programming.

Abstract: Implementations are described herein for building and/or applying a library of transformation templates to automate migration of source code. In various implementations, pre-migration and post-migration versions of source code that exist prior to and after migration of the source code may be analyzed. Based on the analysis, one or more transformations made to the pre-migration version of the source code to yield the post-migration version of the source code may be identified. A library of transformation templates that are applicable subsequently to automate migration of new source code may be built. In some implementations, for one or more of the transformations, a plurality of candidate transformation templates may be generated with different permutations of tokens being replaced with placeholders. One of the plurality of candidate transformation templates may be selected for inclusion in the library based on one or more criteria.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.

Abstract: Embodiments of techniques for inverse design of physical devices are described herein, in the context of generating designs for photonic integrated circuits (including a multi-channel photonic demultiplexer). In some embodiments, an initial design of the physical device is received, and a plurality of sets of operating conditions for fabrication of the physical device are determined. In some embodiments, the performance of the physical device as fabricated under the sets of operating conditions is simulated, and a total performance loss value is backpropagated to determine a gradient to be used to update the initial design. In some embodiments, instead of simulating fabrication of the physical device under the sets of operating conditions, a robustness loss is determined and combined with the performance loss to determine the gradient.

Abstract: Techniques for generating solutions from aural inputs include identifying, with one or more machine learning engines, a plurality of aural signals provided by two or more human speakers, at least some of the plurality of aural signals associated with a human-perceived problem; parsing, with the one or more machine learning engines, the plurality of aural signals to generate a plurality of terms, each of the terms associated with the human-perceived problem; deriving, with the one or more machine learning engines, a plurality of solution sentiments and a plurality of solution constraints from the plurality of terms; generating, with the one or more machine learning engines, at least one solution to the human-perceived problem based on the derived solution sentiments and solution constraints; and presenting the at least one solution of the human-perceived problem to the two or more human speakers through at least one of a graphical interface or an auditory interface.

Abstract: Techniques for improving a blast pattern at a mining site include conducting an initial blast and recording the initial blast as a high speed optical video. The high speed optical video, and the blast pattern used in the initial blast are sent as inputs to a machine learning model, which correlates one or more characteristics of the region being blasted with measurements associated with characteristics of the region being blasted obtained from the high speed optical video. The machine learning model can then determine an improved blast pattern based on the correlation made. This improved blast pattern can be displayed on a user computing device, or transmitted to a drilling system to automatically drill the improved blast pattern for subsequent blasts.

Abstract: Methods, systems, and apparatuses, including computer programs encoded on a computer-readable storage medium for estimating the shape, size, mass, and health of fish are described. A pair of stereo cameras may be utilized to obtain off-axis images of fish in a defined area. The images may be processed, enhanced, and combined. Object detection may be used to detect and track a fish in images. A pose estimator may be used to determine key points and features of the detected fish. Based on the key points, a model of the fish is generated that provides an estimate of the size and shape of the fish. A regression model or neural network model can be applied to the fish model to determine characteristics of the fish.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by one or more non-real-time processors, data defining a light illumination pattern for a robotic device. Generating, by the one or more non-real-time processors and based on the data, a spline that represents the light illumination pattern, where a knot vector of the spline defines a timing profile of the light illumination pattern. Providing the spline to one or more real-time processors of the robotic system. Calculating, by the one or more real-time processors, an illumination value from the spline at each of a plurality of time steps. Controlling, by the one or more real-time processors, illumination of a lighting display of the robotic system in accordance with the illumination value of the spline at each respective time step.

Abstract: A method includes receiving sensor data representative of surfaces in a physical environment containing an interaction point for a robotic device, and determining, based on the sensor data, a height map of the surfaces in the physical environment. The method also includes determining, by inputting the height map and the interaction point into a pre-trained model, one or more candidate positions for a base of the robotic device to allow a manipulator of the robotic device to reach the interaction point. The method additionally includes determining a collision-free trajectory to be followed by the manipulator to reach the interaction point when the base of the robotic device is positioned at a selected candidate position of the one or more candidate positions and, based on determining the collision-free trajectory, causing the base of the robotic device to move to the selected candidate position within the physical environment.

Abstract: An example method includes determining, by a controller of an image capture system, a plurality of sets of exposure parameter values for one or more exposure parameters. The plurality of sets of exposure parameter values are determined at an exposure determination rate. The method further includes capturing, by an image capture device of the image capture system, a plurality of images. Each image of the plurality of images is captured according to a set of exposure parameter values of the plurality of sets of exposure parameter values. The method also includes sending, by the controller of the image capture system to an image processing unit, a subset of the plurality of images. Each subset of images is sent at a sampling rate, and the sampling rate is less than the exposure determination rate.

Abstract: Mitigating the reality gap through training and utilization of at least one difference model. The difference model can be utilized to generate, for each of a plurality of instances of simulated state data generated by a robotic simulator, a corresponding instance of modified simulated state data. The difference model is trained so that a generated modified instance of simulated state data is closer to “real world data” than is a corresponding initial instance of simulated state data. Accordingly, the difference model can be utilized to mitigate the reality gap through modification of initially generated simulated state data, to make it more accurately reflect what would occur in a real environment. Moreover, the difference representation from the difference model can be used as input to the control policy to adapt the control learned from simulator to the real environment.

Abstract: Mitigating the reality gap through utilization of technique(s) that enable compliant robotic control and/or compliant robotic contact to be simulated effectively by a robotic simulator. The technique(s) can include, for example: (1) utilizing a compliant end effector model in simulated episodes of the robotic simulator; (2) using, during the simulated episodes, a soft constraint for a contact constraint of a simulated contact model of the robotic simulator; and/or (3) using proportional derivative (PD) control in generating joint control forces, for simulated joints of the simulated robot, during the simulated episodes. Implementations additionally or alternatively relate to determining parameter(s), for use in one or more of the techniques that enable effective simulation of compliant robotic control and/or compliant robotic contact.