Abstract: Methods and systems for performing sequence level prediction of a video scene are described. Video information in a video scene is represented as a sequence of features depicted each frame. One or more scene affective labels are provided at the end of the sequence. Each label pertains to the entire sequence of frames of data. An action is taken with an agent controlled by a machine learning algorithm for a current frame of the sequence at a current time step. An output of the action represents affective label prediction for the frame at the current time step. A pool of actions taken up until the current time step including the action taken with the agent is transformed into a predicted affective history for a subsequent time step. A reward is generated on predicted actions up to the current time step by comparing the predicted actions against corresponding annotated scene affective labels.

Abstract: Methods and systems are provided for providing real world assistance by a robot utility and interface device (RUID) are provided. A method provides for identifying a position of a user in a physical environment and a surface within the physical environment for projecting an interactive interface. The method also provides for moving to a location within the physical environment based on the position of the user and the surface for projecting the interactive interface. Moreover, the method provides for capturing a plurality of images of the interactive interface while the interactive interface is being interacted with by the use and for determining a selection of an input option made by the user.

Abstract: For image captioning such as for computer game images or other images, bottom-up attention is combined with top-down attention to provide a multi-level residual attention-based image captioning model. A residual attention mechanism is first applied in the Faster R-CNN network to learn better feature representations for each region by taking spatial information into consideration. In the image captioning network, taking the extracted regional features as input, a second residual attention network is implemented to fuse the regional features attentionally for subsequent caption generation.

Abstract: Methods and systems are provided for providing real world assistance by a robot utility and interface device (RUID) are provided. A method provides for identifying a position of a user in a physical environment and a surface within the physical environment for projecting an interactive interface. The method also provides for moving to a location within the physical environment based on the position of the user and the surface for projecting the interactive interface. Moreover, the method provides for capturing a plurality of images of the interactive interface while the interactive interface is being interacted with by the use and for determining a selection of an input option made by the user.

Abstract: Method for providing image of HMD user to a non-HMD user includes, receiving a first image of a user including the user's facial features captured by an external camera when the user is not wearing a head mounted display (HMD). A second image capturing a portion of the facial features of the user when the user is wearing the HMD is received. An image overlay data is generated by mapping contours of facial features captured in the second image with contours of corresponding facial features captured in the first image. The image overlay data is forwarded to the HMD for rendering on a second display screen that is mounted on a front face of the HMD.

Abstract: The present disclosure discloses a system, a method and a device for generating depth image. The system includes an illumination source, an optical system, a control device, and at least one set of a dynamic aperture and an image sensor, wherein the dynamic aperture is configured to dynamically change a light transmittance, an exposure start time, and an exposure end time under a control of the control device. The control device is configured to acquire a first photo and a second photo, and generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

Abstract: Methods and systems for performing sequence level prediction of a video scene are described. Video information in a video scene is represented as a sequence of features depicted each frame. One or more scene affective labels are provided at the end of the sequence. Each label pertains to the entire sequence of frames of data. An action is taken with an agent controlled by a machine learning algorithm for a current frame of the sequence at a current time step. An output of the action represents affective label prediction for the frame at the current time step. A pool of actions taken up until the current time step including the action taken with the agent is transformed into a predicted affective history for a subsequent time step. A reward is generated on predicted actions up to the current time step by comparing the predicted actions against corresponding annotated scene affective labels.

Abstract: The present application discloses a 3D imaging method, device and depth camera, wherein, the 3D imaging method includes: acquiring depth information of points in a to-be-captured scene corresponding to at least one pixel; generating a 3D image of the to-be-captured scene based on the acquired depth information; and determining the depth information of the each of the points in the to-be-captured scene corresponding to the at least one pixel based on a result of the distributing. This implementation utilizes multiple charge collection areas to collect the charges obtained by photoelectric conversion of the light that reaches the pixels, thereby achieving the capture of the depth information of points in the to-be-captured scene corresponding to each pixel.

Abstract: The present disclosure discloses a system, a method and a device for generating depth image. The system includes an illumination source, an optical system, a control device, and at least one set of a dynamic aperture and an image sensor, wherein the dynamic aperture is configured to dynamically change a light transmittance, an exposure start time, and an exposure end time under a control of the control device. The control device is configured to acquire a first photo and a second photo, and generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

Abstract: Provided is a system with image sensors combination, including: a plurality of image sensors located in a same semiconductor chip package; and a controller for controlling the plurality of image sensors to operate in coordination, wherein each image sensor includes: a plurality of photosensitive pixel cells to be simultaneously exposed to generate analog photosensitive signals; an analog-to-digital converters for converting the analog photosensitive signals generated by the plurality of photosensitive pixel cells into image sensor signals; and a timing circuit for controlling the generation, readout, and transmission of electrical signals.

Abstract: The present disclosure provides a three-dimensional distance measuring method and device. The three-dimensional distance measuring method includes: emitting a first light pulse to illuminate a scene, and acquiring a first image of the scene within a first exposure time; emitting a second light pulse to illuminate the scene, and acquiring a second image of the scene within a second exposure time; acquiring scene distance information of the scene based on the first image of the scene and the second image of the scene, wherein the ratio of the first pulse envelope of the first light pulse to the second pulse envelope of the second light pulse is a monotonic function of time.

Abstract: Provided is a method for controlling a vehicle, including: receiving a plurality of visual sensor signals containing three-dimensional visual information outside and inside the vehicle; receiving a plurality of audio sensor signals containing three-dimensional audio information outside and inside the mobile vehicle; determining three-dimensional spatial positions of at least two objects and object cognitive information of the at least two objects according to at least a portion of the plurality of visual sensor signals and at least a portion of the plurality of audio sensor signals, wherein the at least two objects include at least one object outside the vehicle and at least one object inside the vehicle; obtaining a circumstance information of the vehicle according to the object cognitive information; and determining operation instructions for the vehicle according to the circumstance information.

Abstract: Methods and systems for performing sequence level prediction of a video scene are described. Video information in a video scene is represented as a sequence of features depicted each frame. An environment state for each time step t corresponding to each frame is represented by the video information for time step t and predicted affective information from a previous time step t?1. An action A(t) as taken with an agent controlled by a machine learning algorithm for the frame at step t, wherein an output of the action A(t) represents affective label prediction for the frame at the time step t. A pool of predicted actions is transformed to a predicted affective history at a next time step t+1. The predictive affective history is included as part of the environment state for the next time step t+1. A reward R is generated on predicted actions up to the current time step t, by comparing them against corresponding annotated movie scene affective labels.

Abstract: The present disclosure is related to unmanned aerial vehicles or drones that have a capability of quickly swapping batteries. This may be accomplished even as the drone continues to fly. A drone consistent with the present disclosure may drop one battery and pickup another using an attachment mechanism. Attachment mechanisms of the present disclosure may include electro-magnets, mechanical actuators, pins, or hooks. Systems consistent with the present disclosure may also include locations where replacement batteries may be provided to aircraft via actuation devices coupled to a physical location.

Abstract: For image captioning such as for computer game images or other images, bottom-up attention is combined with top-down attention to provide a multi-level residual attention-based image captioning model. A residual attention mechanism is first applied in the Faster R-CNN network to learn better feature representations for each region by taking spatial information into consideration. In the image captioning network, taking the extracted regional features as input, a second residual attention network is implemented to fuse the regional features attentionally for subsequent caption generation.

Abstract: For image captioning such as for computer game images or other images, bottom-up attention is combined with top-down attention to provide a multi-level residual attention-based image captioning model. A residual attention mechanism is first applied in the Faster R-CNN network to learn better feature representations for each region by taking spatial information into consideration. In the image captioning network, taking the extracted regional features as input, a second residual attention network is implemented to fuse the regional features attentionally for subsequent caption generation.

Abstract: A method for improved initialization of speech recognition system comprises mapping a trained hidden markov model based recognition node network (HMM) to a Connectionist Temporal Classification (CTC) based node label scheme. The central state of each frame in the HMM are mapped to CTC-labeled output nodes and the non-central states of each frame are mapped to CTC-blank nodes to generate a CTC-labeled HMM and each central state represents a phoneme from human speech detected and extracted by a computing device. Next the CTC-labeled HMM is trained using a cost function, wherein the cost function is not part of a CTC cost function. Finally the CTC-labeled HMM is trained using a CTC cost function to produce a CTC node network. The CTC node network may be iteratively trained by repeating the initialization steps.

Abstract: For image captioning such as for computer game images or other images, bottom-up attention is combined with top-down attention to provide a multi-level residual attention-based image captioning model. A residual attention mechanism is first applied in the Faster R-CNN network to learn better feature representations for each region by taking spatial information into consideration. In the image captioning network, taking the extracted regional features as input, a second residual attention network is implemented to fuse the regional features attentionally for subsequent caption generation.

Abstract: Systems and methods for processing operations for head mounted display (HMD) users to join virtual reality (VR) scenes are provided. A computer-implemented method includes providing a first perspective of a VR scene to a first HMD of a first user and receiving an indication that a second user is requesting to join the VR scene provided to the first HMD. The method further includes obtaining real-world position and orientation data of the second HMD relative to the first HMD and then providing, based on said data, a second perspective of the VR scene. The method also provides that the first and second perspectives are each controlled by respective position and orientation changes while viewing the VR scene.

Abstract: A domain adaptation module is used to optimize a first domain derived from a second domain using respective outputs from respective parallel hidden layers of the domains.