Abstract: In some embodiments, an apparatus for performing reactive positioning of a robot gripper includes one or more fingers disposed on an end-effector of the robot, a signal processing circuit, a memory storing instructions, and a processor. Each of the one or more fingers includes a transducer configured to generate vibrational energy based on an input signal, and convert an acoustic reflection of the vibrational energy from an object into a voltage signal. The signal processing circuit is configured to provide the input signal to each transducer, and perform signal processing on the voltage signal of each transducer resulting in reflection data. The processor is configured to execute the instructions to perform pre-touch proximity detection on the reflection data, perform grasp positioning on the reflection data, perform contact detection from the reflection data, and provide, to the robot, results of the pre-touch proximity detection, the grasp positioning, and the contact detection.

Abstract: Systems, methods, and apparatuses for controlling a robot including a manipulator, including: determining three-dimensional (3D) geometry information about a target object based on an image of the target object; determining 3D geometry information about a scene in which the target object is to be placed based on at least one image of the scene; obtaining affordance information by providing the 3D geometry information about the target object and the 3D geometry information about the scene to at least one neural network model; commanding the robot to grasp the target object using the manipulator according to a grasp orientation corresponding to the affordance information; and commanding the robot to position the manipulator according to a placement direction corresponding to the affordance information in order to place the target object at a location in the scene.

Abstract: The present disclosure provides methods, apparatuses, and computer-readable mediums for evaluating a reliability of three-dimensional shape predictions.

Abstract: A method of predicting occupancy of unseen areas in a region of interest (ROI) includes obtaining a depth image of the ROI, the depth image being captured from a first height; generating an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI; and generating an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, and wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

Abstract: A detection system includes a filter and a detector. The filter receives input packets from one or more dynamic vision sensors (DVSs) deployed in an area. The filter reduces the bandwidth of the input packets to produce compressed data packets while maintaining good end-to-end detection capability. The compressed data packets are transmitted to a detector in a low bandwidth transmission. The detector may collect compressed data packets from many filters. The detector determines if an object of interest is represented in the information of the input packets. If an object of interest is present, the detector activates an alarm, sends a notification to a person associated with the area and/or stores an indication in a database.

Abstract: An apparatus for estimating a proximity direction of an obstacle includes an acoustic transmitter attached to a surface of the apparatus; a first acoustic receiver spaced apart from the surface of the apparatus; a second acoustic receiver spaced apart from the surface of the apparatus; and at least one processor configured to: control the acoustic transmitter to generate an acoustic wave along the surface; obtain first and second proximity direction signals based on first and second acoustic wave signals corresponding to the generated acoustic wave; and estimate a proximity direction of the obstacle with respect to the apparatus based on the first and second proximity direction signals.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: A method of collision localization on a robotic device includes obtaining audio signals from a plurality of acoustic sensors spaced apart along the robotic device; identifying, based on a collision being detected, a strongest audio signal; identifying a primary onset time for an acoustic sensor producing the strongest audio signal, the primary onset time being a time at which waves propagating from the collision reach the acoustic sensor producing the strongest audio signal; generating a virtual onset time set, by shifting a calibration manifold, based on the identified primary onset time, the calibration manifold representing relative onset times from evenly spaced marker locations on the robotic device to the plurality of acoustic sensors; determining scores for the marker locations based a standard deviation of elements in the virtual onset time set; and estimating a location of the collision based on a highest score of the determined scores.

Abstract: A method may include obtaining a set of events, of a set of pixels of a dynamic vision sensor, associated with an object; determining a set of voltages of the set of pixels, based on the set of events; generating a set of images, based on the set of voltages of the set of pixels; inputting the set of images into a first neural network configured to output a visual motion estimation of the object; inputting the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network; obtaining the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network; and providing the visual motion estimation of the object and the confidence score.

Abstract: A pose of a person is estimated using an image and audio impulse responses. The image represents a 2D scene including the person. The audio impulse responses are obtained with the present absent and present in an environment. The pose is reconstructed based on the image and the one or more audio impulse responses. The pose is a metric scale human pose.

Abstract: A method for generating a trajectory of a robot from a first configuration to a second configuration within an environment while steering away from obstacles may include obtaining physical workspace information associated with the environment in which the robot is configured to operate; obtaining, using a first neural network, a set of weights of a second neural network that is configured to generate a set of values associated with a set of configurations of the robot with respect to the second configuration; obtaining, by applying the set of weights to the second neural network, the set of values associated with the set of configurations of the robot with respect to the second configuration; and generating the trajectory of the robot from the first configuration to the second configuration within the environment, based on the set of values.

Abstract: Provided is a method for predicting a location of a fast-moving object. The method includes receiving event information from an event camera, the event information corresponding to an event detected by the event camera, generating a Binary Event History Image (BEHI) based on the event information, providing the BEHI as an input to an event-based neural network, obtaining, as an output of the event-based neural network, a first predicted location of the fast-moving object, a normal distribution indicating prediction uncertainty of the predicted location, and a predicted time-to-collision (TTC). The method further includes estimating a second predicted location of the fast-moving object based on the first predicted location, the normal distribution, and the predicted TTC output by the event-based neural network, and actuating a mechanical catching device to be at the second predicted location.

Abstract: A tactile sensing system of a robot may include: a plurality of piezoelectric elements disposed at an object, and including a transmission (TX) piezoelectric element and a reception (RX) piezoelectric element; and at least one processor configured to: control the TX piezoelectric element to generate an acoustic wave having a chirp spread spectrum (CSS) at every preset time interval, along a surface of the object; receive, via the RX piezoelectric element, an acoustic wave signal corresponding to the generated acoustic wave; select frequency bands from a plurality of frequency bands of the acoustic wave signal; and estimate a location of a touch input on the surface of the object by inputting the acoustic wave signal of the selected frequency bands into a neural network configured to provide a touch prediction score for each of a plurality of predetermined locations on the surface of the object.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: A system and a method for generating a trajectory of a target device from a current position to a goal position within an environment is provided. The method may include: inputting physical workspace information associated with the environment in which the target device, to a first neural network to obtain a set of weights representing a cost-to-go function that defines a cost-to-go function relating to a length of a collision-free path from one position to the goal position; configuring a second neural network based on the set of weights; identifying a next position of the target device based on the current position and a motion control input of the target; and inputting the identified next position of the target device and the goal position to the second neural network to identify the trajectory to the goal position, and the motion control input corresponding to the trajectory.

Abstract: A tool control system may include: a tactile sensor configured to, when a tool holds a target object and slides the target object downward across the tool, obtain tactile sensing data from the tool; one or more memories configured to store a target velocity and computer-readable instructions; and one or more processors configured execute the computer-readable instructions to: receive the tactile sensing data from the tactile sensor; estimate a velocity of the target object based on the tactile sensing data, by using one or more neural networks that are trained based on a training image of an sample object captured while the sample object is sliding down; and generate a control parameter of the tool based on the estimated velocity and the target velocity.

Abstract: An apparatus for reconstructing a 3D object, includes a memory storing instructions, and at least one processor configured to execute the instructions to obtain, using a first neural network, mapping function weights of a mapping function of a second neural network, based on an image feature vector corresponding to a 2D image of the 3D object, set the mapping function of the second neural network, using the obtained mapping function weights, and based on sampled points of a canonical sampling domain, obtain, using the second neural network of which the mapping function is set, 3D point coordinates and geodesic lifting coordinates of each of the sampled points in the 3D object corresponding to the 2D image, wherein the 3D point coordinates are first three dimensions of an embedding vector of a respective one of the sampled points, and the geodesic lifting coordinates are remaining dimensions of the embedding vector.

Abstract: An apparatus for motion understanding, includes a memory storing instructions, and at least one processor configured to execute the instructions to obtain, from a dynamic vision sensor, a plurality of events corresponding to an object moving with respect to the dynamic vision sensor, and filter the obtained plurality of events, using a plurality of exponential filters integrating over different time periods, to obtain a plurality of representations of the object moving with respect to the dynamic vision sensor. The at least one processor is further configured to execute the instructions to filter the obtained plurality of representations, using a convolution neural network, to obtain a probability of the object impacting a location on the dynamic vision sensor.

Abstract: A method of predicting occupancy of unseen areas in a region of interest (ROI) includes obtaining a depth image of the ROI, the depth image being captured from a first height; generating an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI; and generating an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, and wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

Abstract: A method for generating a trajectory of a robot from a first configuration to a second configuration within an environment while steering away from obstacles may include obtaining physical workspace information associated with the environment in which the robot is configured to operate; obtaining, using a first neural network, a set of weights of a second neural network that is configured to generate a set of values associated with a set of configurations of the robot with respect to the second configuration; obtaining, by applying the set of weights to the second neural network, the set of values associated with the set of configurations of the robot with respect to the second configuration; and generating the trajectory of the robot from the first configuration to the second configuration within the environment, based on the set of values.