Abstract: Example implementations described herein are directed to the transmission of hand information from a user hand or other object to a remote device via browser-to-browser connections, such that the hand or other object is oriented correctly on the remote device based on orientation measurements received from the remote device. Such example implementations can facilitate remote assistance in which the user of the remote device needs to view the hand or object movement as provided by an expert for guidance.

Abstract: Systems and methods for automatic detection and insetting of digital streams into a video. Various examples of such regions of interest to the user include, without limitation, content displayed on various electronic displays or written on electronic paper (electronic ink), content projected on various surfaces using electronic projectors, content of paper documents appearing in the video and/or content written on white (black) boards inside the video. For some content, such as whiteboards, paper documents or paintings in a museum, a participant (or curator) could have taken pictures of the regions, again stored digitally somewhere and available for download. These digital streams with content of interest to the user are obtained and then inset onto the view generated from the raw video feed, giving users the ability to view them at their native high resolution.

Abstract: A computer implemented method is provided that includes embedding a received signal in a first modality, re-embedding the embedded received signal of the first modality into a signal of a second modality, and generating an output in the second modality, and based on the output, rendering a signal in the second modality that is configured to be sensed, wherein the embedding, re-embedding and generating applies a model that is trained by performing an adversarial learning operation associated with discriminating actual examples of the target distribution from the generated output, and performing a metric learning operation associated with generating the output having perceptual distances.

Abstract: Example implementations described herein are directed to the transmission of hand information from a user hand or other object to a remote device via browser-to-browser connections, such that the hand or other object is oriented correctly on the remote device based on orientation measurements received from the remote device. Such example implementations can facilitate remote assistance in which the user of the remote device needs to view the hand or object movement as provided by an expert for guidance.

Abstract: In example implementations described herein, there is a smart hat that is configured to receive gesture inputs through an accelerometer or through touch events on the hat, with light emitting diodes providing feedback to the wearer. The smart hat can be configured to be connected wirelessly to an external apparatus to control the apparatus by transmitting and receiving messages (e.g., commands) between the hat and the apparatus wirelessly. Further, a network of smart hats may be managed by an external device depending on the desired implementation.

Abstract: Systems and methods for automatic detection and insetting of digital streams into a video. Various examples of such regions of interest to the user include, without limitation, content displayed on various electronic displays or written on electronic paper (electronic ink), content projected on various surfaces using electronic projectors, content of paper documents appearing in the video and/or content written on white (black) boards inside the video. For some content, such as whiteboards, paper documents or paintings in a museum, a participant (or curator) could have taken pictures of the regions, again stored digitally somewhere and available for download. These digital streams with content of interest to the user are obtained and then inset onto the view generated from the raw video feed, giving users the ability to view them at their native high resolution.

Abstract: A computer-implemented method of localization for an indoor environment is provided, including receiving, in real-time, a dynamic query from a first source, and static inputs from a second source; extracting features of the static inputs by applying a metric learning convolutional neural network (CNN), and aggregating the extracted features of the static inputs to generate a feature transformation; and iteratively extracting features of the dynamic query on a deep CNN as an embedding network and fusing the feature transformation into the deep CNN, and applying a triplet loss function to optimize the embedding network and provide a localization result.

Abstract: A computer implemented method is provided that includes embedding a received signal in a first modality, re-embedding the embedded received signal of the first modality into a signal of a second modality, and generating an output in the second modality, and based on the output, rendering a signal in the second modality that is configured to be sensed, wherein the embedding, re-embedding and generating applies a model that is trained by performing an adversarial learning operation associated with discriminating actual examples of the target distribution from the generated output, and performing a metric learning operation associated with generating the output having perceptual distances.

Abstract: Systems and methods for automatic detection and insetting of digital streams into a 360 video. Various examples of such regions of interest to the user include, without limitation, content displayed on various electronic displays or written on electronic paper (electronic ink), content projected on various surfaces using electronic projectors, content of paper documents appearing in the 360-degree video and/or content written on white (black) boards inside the 360-degree video. For some content, such as whiteboards, paper documents or paintings in a museum, a participant (or curator) could have taken pictures of the regions, again stored digitally somewhere and available for download. These digital streams with content of interest to the user are obtained and then inset onto the 360-degree view generated from the raw 360-degree video feed, giving users the ability to view them at their native high resolution.

Abstract: A computer-implemented method is provided for coordinating sensed poses associated with real-time movement of a first object in a live video feed, and pre-recorded poses associated with movement of a second object in a video. The computer-implemented method comprises applying a matching function to determine a match between a point of one of the sensed poses and a corresponding point of the pre-recorded poses, and based on the match, determining a playtime and outputting at least one frame of the video associated with the second object for the playtime.

Abstract: A computer-implemented method of localization for an indoor environment is provided, including receiving, in real-time, a dynamic query from a first source, and static inputs from a second source; extracting features of the static inputs by applying a metric learning convolutional neural network (CNN), and aggregating the extracted features of the static inputs to generate a feature transformation; and iteratively extracting features of the dynamic query on a deep CNN as an embedding network and fusing the feature transformation into the deep CNN, and applying a triplet loss function to optimize the embedding network and provide a localization result.

Abstract: Example implementations described herein are directed to systems and methods for providing a panoramic video connection between one location and another, or between one location and a number of distributed remote viewers, which provides reciprocity in terms of awareness of who is viewing and what they are looking at. In example implementations described herein, radial displays present panoramic video from remote locations, or synthesized views of face shots positioned to indicate the viewing direction of remote viewers.

Abstract: In example implementations described herein, there is a smart hat that is configured to receive gesture inputs through an accelerometer or through touch events on the hat, with light emitting diodes providing feedback to the wearer. The smart hat can be configured to be connected wirelessly to an external apparatus to control the apparatus by transmitting and receiving messages (e.g., commands) between the hat and the apparatus wirelessly. Further, a network of smart hats may be managed by an external device depending on the desired implementation.

Abstract: Systems and methods for automatic detection and insetting of digital streams into a 360 video. Various examples of such regions of interest to the user include, without limitation, content displayed on various electronic displays or written on electronic paper (electronic ink), content projected on various surfaces using electronic projectors, content of paper documents appearing in the 360-degree video and/or content written on white (black) boards inside the 360-degree video. For some content, such as whiteboards, paper documents or paintings in a museum, a participant (or curator) could have taken pictures of the regions, again stored digitally somewhere and available for download. These digital streams with content of interest to the user are obtained and then inset onto the 360-degree view generated from the raw 360-degree video feed, giving users the ability to view them at their native high resolution.

Abstract: Described is a way to steer speaker array or microphone array based on direction outputs of tiny light sensors attached to TV viewers, noisy-environment workers, or AR/VR system users. More specifically, a light projector is disposed on a ceiling, which sends out different sequential on/off signals for each pixel. Two light sensors are attached to each speaker array or microphone array, and one or more light sensors are attached to each user. Because each projector pixel corresponds to a specific direction, when a light sensor receives sequential signal from the projector, the light sensor can determine its direction corresponding to the projector and report that to the central station. With the speaker/microphone array direction and user direction known, the system can generate proper phase shifts for different speaker signals and generate directional sound for each individual. Similarly, the central station can determine phase shifts for combining audios from different microphones.

Abstract: Described is a way to steer speaker array or microphone array based on direction outputs of tiny light sensors attached to TV viewers, noisy-environment workers, or AR/VR system users. More specifically, a light projector is disposed on a ceiling, which sends out different sequential on/off signals for each pixel. Two light sensors are attached to each speaker array or microphone array, and one or more light sensors are attached to each user. Because each projector pixel corresponds to a specific direction, when a light sensor receives sequential signal from the projector, the light sensor can determine its direction corresponding to the projector and report that to the central station. With the speaker/microphone array direction and user direction known, the system can generate proper phase shifts for different speaker signals and generate directional sound for each individual. Similarly, the central station can determine phase shifts for combining audios from different microphones.

Abstract: Example implementations described herein are directed to systems and methods for providing a panoramic video connection between one location and another, or between one location and a number of distributed remote viewers, which provides reciprocity in terms of awareness of who is viewing and what they are looking at. In example implementations described herein, radial displays present panoramic video from remote locations, or synthesized views of face shots positioned to indicate the viewing direction of remote viewers.

Abstract: Example implementations described herein are directed to systems and methods for providing a panoramic video connection between one location and another, or between one location and a number of distributed remote viewers, which provides reciprocity in terms of awareness of who is viewing and what they are looking at. In example implementations described herein, radial displays present panoramic video from remote locations, or synthesized views of face shots positioned to indicate the viewing direction of remote viewers.

Abstract: A method at a first device with a display includes: receiving image information from a camera device, the image information corresponding to a first field of view associated with the camera device; receiving sensory information from a second device, the sensory information corresponding to a second field of view associated with the second device; and displaying on the display the image information and a visualization of at least one of the first field of view and the second field of view.

Abstract: Example implementations described herein are directed to systems and methods for providing a panoramic video connection between one location and another, or between one location and a number of distributed remote viewers, which provides reciprocity in terms of awareness of who is viewing and what they are looking at. In example implementations described herein, radial displays present panoramic video from remote locations, or synthesized views of face shots positioned to indicate the viewing direction of remote viewers.