Abstract: State of the art techniques in the domain of video analysis have limitations in terms of capability to extract the spatial and temporal information. This limitation in turn affects interpretation of the video data. The disclosure herein generally relates to video analysis, and, more particularly, to a method and system for video analysis to extract spatio-temporal information from a video being analyzed. The system uses a neural network architecture which has multiple layers to extract spatial and temporal information from the video being analyzed. The method of training the neural network that extracts a micro-scale information from a latent representation of the video is presented. This is generated using an attention network, which is then used to extract spatio-temporal information corresponding to the collected video, which is then used in multiple video analysis applications such as searching actions in videos, action detection and localization.

Abstract: A robot configured to perceive a model of an environment, including: a chassis; a set of wheels; a plurality of sensors; a processor; and memory storing instructions that when executed by the processor effectuates operations including: capturing a plurality of data while the robot moves within the environment; perceiving the model of the environment based on at least a portion of the plurality of data, the model being a top view of the environment; storing the model of the environment in a memory accessible to the processor; and transmitting the model of the environment and a status of the robot to an application of a smartphone previously paired with the robot.

Abstract: A method for assisting in the performance of a surgical procedure on a patient is disclosed. The method includes receiving an intra-operative CT scan image, generating a three-dimensional isotropic scaffold based on the intra-operative CT scan image, fusing one or more images with the three-dimensional isotropic scaffold to form a three-dimensional model of a portion of the body of the patient, receiving information regarding the surgical procedure, determining obstacles in the path of the surgical procedure, and determining possible trajectories for the surgical procedure. The received information includes information regarding a surgical target to be operated upon, a region of access on the skin of the patient, and a surgical instrument to be used during the surgical procedure.

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for performing neural network computations using a system configured to implement a neural network on a hardware circuit. The system includes a host that receives a batch of inputs to a neural network layer. Each of the inputs is stored in a memory location identified by an address. The system identifies one or more duplicate addresses in a listing of addresses for one or more inputs. For each duplicate address: the system generates a unique identifier that identifies the duplicate address in the listing of addresses. The system (i) obtains first inputs from memory locations identified by addresses corresponding to the unique identifiers and (ii) generates an output of the layer from the obtained first inputs.

Abstract: There is described a computer-implemented method for generating a vector representation of an image, the computer-implemented method comprising: receiving a given image and semantic information about the given image; generating a first vector representation of the given image using an image embedding method; generating a second vector representation of the semantic information using a word embedding method; combining the first vector representation of the image to be embedded and the second vector representation of the semantic information together, thereby obtaining a modified vector representation for the image to be embedded; and outputting the modified vector representation.

Abstract: A network authentication system can be configured for sampling a plurality of signal samples from a device on a network, providing the plurality of signal samples to a first machine-learned model that is configured to determine a device fingerprint based at least in part on the plurality of signal samples, and providing the device fingerprint to a second machine-learned model that is configured to classify the device based at least in part on the device fingerprint.

Abstract: Provided is a method, a system, and a program product for determining a policy using semi-supervised reinforcement learning. The method includes observing a state of an environment by a learning agent. The method also includes taking an action by the learning agent. The method further includes observing a new state of the environment and calculating a reward for the action taken by the learning agent. The method also includes determining whether a policy related to the learning agent should be changed. The determination is conducted by a teaching agent that inputs the state of the environment and the reward as features. The method can also include changing the policy related to the learning agent upon a determination that a label outputted by the teaching agent exceeds a reward threshold.

Abstract: An apparatus for reading an image executes operations including receiving a plurality of images for a same subject from a photographing device, obtaining a screen division value related to the plurality of images, checking focused sections of the plurality of images based on the obtained screen division value, and determining type of the plurality of images based on a result of checking the focused sections. According to the apparatus, it can be determined whether an image is actually photographed without a separate ToF module.

Abstract: A robot configured to perceive a model of an environment, including: a chassis; a set of wheels; a plurality of sensors; a processor; and memory storing instructions that when executed by the processor effectuates operations including: capturing a plurality of data while the robot moves within the environment; perceiving the model of the environment based on at least a portion of the plurality of data, the model being a top view of the environment; storing the model of the environment in a memory accessible to the processor; and transmitting the model of the environment and a status of the robot to an application of a smartphone previously paired with the robot.

Abstract: An image generation device includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to: select a second face image from a plurality of face images stored in advance based on directions of faces included in the plurality of face images and a direction of a face included in an input first face image; deform the second face image based on feature points of the face included in the first face image and feature points of a face included in the second face image such that a face region of the second face image matches a face region of the first face image; and generate a third face image in which the face region of the first face image is synthesized with a region other than the face region of the deformed second face image.

Abstract: Methods, systems, apparatus, and computer-readable media for data band selection using machine learning. In some implementations, image data comprising information for each of multiple wavelength bands is obtained. A multi-layer neural network is trained using the image data to perform one or more classification or regression tasks. A proper subset of the wavelength bands is selected based on parameters of a layer of the trained multi-layer neural network, where the parameters were determined through training of the multi-layer neural network using the image data. Output is provided indicating that the selected wavelength bands are selected for the one or more classification or regression tasks.

Abstract: A method for testing a vehicular driving assist system includes providing a neural network and training the neural network using a database of images, with each image of the database of images including an image of a headlight or a taillight of a vehicle. The trained neural network is provided with an input image that does not include a headlight or a taillight. The neural network, using the input image, generates an output image, with the output image including an image of a headlight or taillight generated by the neural network. The output image is provided as an input to the driving assist system to test the driving assist system.

Abstract: A computing device, including a processor configured to receive a training data set including a first plurality of log entries. Each log entry of the first plurality of log entries may be associated with a tag indicating whether the log entry includes sensitive data. The processor may be further configured to train a sensitivity classification neural network using the training data set. The processor may be further configured to apply the sensitivity classification neural network to a test data set including a second plurality of log entries to obtain a first classified test data set. The processor may be further configured to apply a rule-based data sensitivity classification algorithm to the test data set to obtain a second classified test data set. Based on the first classified test data set and the second classified test data set, the processor may be further configured to modify the sensitivity classification neural network.

Abstract: A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.

Abstract: A pattern recognition device comprising: a coupled network of damped, nonlinear, dynamic elements configured to generate an output response in response to at least one environmental condition, wherein each element has an associated multi-stable potential energy function that defines multiple energy states of an individual element, and wherein the elements are tuned such that environmental noise triggers stochastic resonance between energy levels of at least two elements; a processor configured to monitor the output response over time and to determine a probability that the pattern recognition device is in a given state based on the monitored output response; and detecting a pattern in the at least one environmental condition based on the probability.

Abstract: A system, device, and method for visualizing vascular structures is disclosed. According to some implementations, in order to provide further improved digital subtraction angiography, a device for visualizing vascular structures is provided that includes a data provision processor, an image processor, and an output. The data provision processor is configured to provide a first sequence of non-contrast X-ray images of a region of interest of a patient for use as raw X-ray mask images. The data provision processor is also configured to provide a second sequence of contrast X-ray images of the region of interest of a patient for use as raw X-ray live-images. The image processor is configured to perform a first spatial subtraction for the first sequence of non-contrast X-ray images resulting in a first sequence of spatial-subtracted mask images.

Abstract: Methods, systems, and computer programs are presented for adding new features to a network service. A method includes receiving an image depicting an object of interest. A category set is determined for the object of interest and an image signature is generated for the image. Using the category set and the image signature, the method identifies a set of publications within a publication database and assigns a rank to each publication. The method causes presentation of the ranked list of publications at a computing device from which the image was received.

Abstract: Systems and methods for clustering data are disclosed. For example, a system may include one or more memory units storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include receiving data from a client device and generating preliminary clustered data based on the received data, using a plurality of embedding network layers. The operations may include generating a data map based on the preliminary clustered data using a meta-clustering model. The operations may include determining a number of clusters based on the data map using the meta-clustering model and generating final clustered data based on the number of clusters using the meta-clustering model. The operations may include and transmitting the final clustered data to the client device.

Abstract: Convolutional neural networks for detecting objects of interest within images of biological specimens are disclosed. Also disclosed are systems and methods of training and using such networks, one method including: obtaining a sample image and at least one of a set of positive points and a set of negative points, wherein each positive point identifies a location of one object of interest within the sample image, and each negative point identifies a location of one object of no-interest within the sample image; obtaining one or more predefined characteristics of objects of interest and/or objects of no-interest, and based on the predefined characteristics, generating a boundary map comprising a positive area around each positive point the set of positive points, and/or a negative area around each negative point in the set of negative points; and training the convolutional neural network using the sample image and the boundary map.

Abstract: A method and a system for generating an augmented scene graph of an image and for training an explainable knowledge based (KB) visual question answering (VQA) machine learning (ML) model are provided. A scene graph encoding spatial and semantic features of objects and relations between objects in the image is obtained. An augmented scene graph is generated by embedding a knowledge graph to enhance the scene graph. An embedded set of questions and associated answers related to the image are obtained. The KB VQA ML model is trained to provide an answer to a given question related to the image based on the augmented scene graph and the embedded set of questions and associated answers. The KB VQA ML model is trained to retrieve a subgraph linking the question and the associated answer as a potential explanation for the answer.