Abstract: A learning system is disclosed that leverages intention-driven causality to enhance skill learning. The learning system enables an author to easily develop mixed reality (MR) tutorial content for performing a task that advantageously captures causal relationships between steps and, thus, enables such causality to be conveyed to the novice user when learning how to perform the task. To this end, the learning system leverages a novel hierarchical representation of causality and intention alongside a systematic workflow suitable for designing skill learning content. By preserving and presenting causal information to the novice user, the user can better understand not only the steps required to perform a task, but also why each step is performed.

Abstract: A method and system for hand-object interaction dataset collection is described herein, which is configured to support user-specified collection of hand-object interaction datasets. Such hand-object interaction datasets are useful, for example, for training 3D hand and object pose estimation model. The method and system adopt a sequential process of first recording hand and object pose labels by manipulating a virtual bounding box, rather than a physical object. Naturally, hand-object occlusions do not occur during the manipulation of the virtual bounding box, so these labels are provided with high accuracy. Subsequently, the images are separately captured of the hand-object interaction with the physical object. These images are paired with the previously recorded hand and object pose labels.

Abstract: An augmented reality (AR) interaction authoring system is described. The AR interaction authoring system is configured to support the real-time creation of AR applications to support AR-enhanced toys. The design of the AR interaction authoring system enables bidirectional interactions between the physical-virtual space of toys and AR. The AR interaction authoring system allows intuitive authoring of AR animations and toys actuations through programming by demonstration, while referring to the physical toy for a contextual reference. Using a visual programming interface, users can create bidirectional interactions by utilizing users' input on toys to trigger AR animations and vice versa. A plug-and-play IoT toolkit is also disclosed that includes hardware to actuate common toys. In this way, users can effortlessly integrate toys into the virtual world in an impromptu design process, without lengthy electronic prototyping.

Abstract: A system and method for multi-media instruction authoring is disclosed. The multi-media instruction authoring system and method provide a unified and efficient system that supports the simultaneous creation of instructional media in AR, VR, and video-based media formats by subject matter experts. The multi-media instruction authoring system and method require only a single demonstration of the task by the subject matter expert and does not require any technical expertise of the subject matter expert. The multi-media instruction authoring system and method incorporate a 3D editing interface for free hand, in-headset interaction to create 2D videos from 3D recordings, by exploring novel virtual camera interactions.

Abstract: A stable pharmaceutically formulation containing antibody, a buffer, a non-ionic surfactant, and a lyoprotectant/cryoprotectants. Also disclosed are associated methods for preparing, storing, and using such formulations.

Abstract: Disclosed is a visual and spatial programming system for robot navigation and robot-IoT task authoring. Programmable mobile robots serve as binding agents to link stationary IoT devices and perform collaborative tasks. Three key elements of robot task planning (human-robot-IoT) are coherently connected with one single smartphone device. Users can perform visual task authoring in an analogous manner to the real tasks that they would like the robot to perform with using an augmented reality interface. The mobile device mediates interactions between the user, robot(s), and IoT device-oriented tasks, guiding the path planning execution with Simultaneous Localization and Mapping (SLAM) to enable robust room-scale navigation and interactive task authoring.

Abstract: Some examples provides a set of frameworks, process and methods aimed at enabling the expression and exploration of free-form and parametric 3D shape designs enabled through natural interactions with a hand-held mobile device acting as a controller for 3D virtual objects. A reference plane in a virtual space generated by the location of the mobile device may be used to select a 3D virtual object intersected by the reference plane. Positioning of the mobile device may also be used to control a pointer in the virtual space. In an example, the orientation of the mobile device may be detected by an accelerometer or gyroscope. In example, the position of the mobile device may be detected by a position sensor.

Abstract: Techniques are disclosed relating to data compression. In some embodiments, compression circuitry determines, at least partially in parallel for multiple different lossless compression techniques, amounts of data needed to represent, using a given lossless compression technique of the multiple lossless compression techniques, individual pixels in a set of pixels being compressed. The compression techniques may include neighbor, origin, and gradient techniques, for example. The compression circuitry may select one of the compression techniques based on comparison, among the compression techniques, of sums of: the determined amount of data for an individual pixel for a given lossless compression technique and compression metadata size for a given lossless compression technique. The compression circuitry may generate and store information that encodes values for the set of pixels using the selected compression technique.

Abstract: A method of operating an augmented reality (AR) system includes capturing images of a first real-world workspace using a camera of a first head mounted AR device of the AR system being worn by a first user, processing the images using a first processor of the AR system to identify physical objects in the first real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace, rendering virtual objects representing the identified physical objects on the display of the first head mounted AR device at the respective 3D positions for the identified physical objects, manipulating a first one of the virtual objects using at least one hand-held controller of the AR system in a manner that mimics a performance of a first procedural task using the physical object associated with the first one of the virtual objects, recording the manipulation of the first one of the virtual objects that mimics the performance of the first procedural task as fi

Abstract: A sensing system includes a base material, a plurality of electrodes, a capacitive sensing channel and a controller. The stretchable base material has a resistance distribution that changes in response to being mechanically deformed as a result of a human body contact. The base material has a rebound elasticity. The electrodes are attached to a perimeter of the base material, the capacitive sensing channel is attached to the base material. The controller is operatively connected to the plurality of electrodes and the capacitive sensing channel. The controller is configured to measure instantaneous voltage measurements from the plurality of electrodes, and determine whether the base material is mechanically deformed based on the instantaneous voltage measurements using a support vector machine classifier.

Abstract: Techniques are disclosed relating to compression of pixel data using different quantization for different regions of a block of pixels being compressed. In some embodiments, compression circuitry is configured to determine, for multiple components included in pixels of the block of pixels being compressed, respective smallest and greatest component values in respective regions of the block of pixels. The compression circuitry may determine, based on the determined smallest and greatest component values, to use a first number of bits to represent delta values relative to a base value for a first component in a first region and a second, different number of bits to represent delta values relative to a base value for a second component in the first region. The compression circuitry may then quantize delta values for the first and second components of pixels in the first region of the block of pixels using the determined first and second numbers of bits.

Abstract: Some examples provides a set of frameworks, process and methods aimed at enabling the expression and exploration of free-form and parametric 3D shape designs enabled through natural interactions with a hand-held mobile device acting as a controller for 3D virtual objects. A reference plane in a virtual space generated by the location of the mobile device may be used to select a 3D virtual object intersected by the reference plane. Positioning of the mobile device may also be used to control a pointer in the virtual space. In an example, the orientation of the mobile device may be detected by an accelerometer or gyroscope. In example, the position of the mobile device may be detected by a position sensor.

Abstract: Techniques are disclosed relating to compression of pixel data using different quantization for different regions of a block of pixels being compressed. In some embodiments, compression circuitry is configured to determine, for multiple components included in pixels of the block of pixels being compressed, respective smallest and greatest component values in respective regions of the block of pixels. The compression circuitry may determine, based on the determined smallest and greatest component values, to use a first number of bits to represent delta values relative to a base value for a first component in a first region and a second, different number of bits to represent delta values relative to a base value for a second component in the first region. The compression circuitry may then quantize delta values for the first and second components of pixels in the first region of the block of pixels using the determined first and second numbers of bits.

Abstract: A method includes mixing a liquid carrier and a powder formulation including an S-nitrosothiol (RSNO) powder to form a nitric oxide generating solution; and within a predetermined time of mixing, introducing the nitric oxide generating solution into an inflatable and nitric oxide permeable balloon of a urinary catheter that is inserted in a patient.

Abstract: An augmented reality system for developing and providing augmented reality learning experiences is disclosed. The augmented reality system advantageously combines augmented reality with the capabilities of cloud technology to provide a pull-based collaborative model, in which students and instructors collaborate by uploading, sharing, and downloading augmented reality learning content. The augmented reality system enables students to improve the augmented reality learning content by adding contributions to the original augmented reality learning content that was created by an instructor.

Abstract: A method for hand pose identification in an automated system includes providing map data of a hand of a user to a first neural network trained to classify features corresponding to a joint angle of a wrist in the hand to generate a first plurality of activation features and performing a first search in a predetermined plurality of activation features stored in a database in the memory to identify a first plurality of hand pose parameters for the wrist associated with predetermined activation features in the database that are nearest neighbors to the first plurality of activation features. The method further includes generating a hand pose model corresponding to the hand of the user based on the first plurality of hand pose parameters and performing an operation in the automated system in response to input from the user based on the hand pose model.

Abstract: An augmented reality (AR) application authoring system is disclosed. The AR application authoring system enables the real-time creation of freehand interactive AR applications with freehand inputs. The AR application authoring system enables intuitive authoring of customized freehand gesture inputs through embodied demonstration while using the surrounding environment as a contextual reference. A visual programming interface is provided with which users can define freehand interactions by matching the freehand gestures with reactions of virtual AR assets. Thus, users can create personalized freehand interactions through simple trigger-action programming logic. Further, with the support of a real-time hand gesture detection algorithm, users can seamlessly test and iterate on the authored AR experience.

Abstract: A method for generating a multi-modal video dataset with pixel-wise hand segmentation is disclosed. To address the challenges of conventional dataset creation, the method advantageously utilizes multi-modal image data that includes thermal images of the hands, which enables efficient pixel-wise hand segmentation of the image data. By using the thermal images, the method is not affected by fingertip and joint occlusions and does not require hand pose ground truth. Accordingly, the method can produce more accurate pixel-wise hand segmentation in an automated manner, with less human effort. The method can thus be utilized to generate a large multi-modal hand activity video dataset having hand segmentation labels, which is useful for training machine learning models, such as deep neural networks.

Abstract: A method and system for localizing a plurality of stationary devices, such as Internet of Things (IoT devices), arranged in an environment is disclosed. A mobile device is configured to survey an environment to generate a three-dimensional map of the environment using simultaneous localization and mapping (SLAM) techniques. The mobile device and the stationary devices are equipped with wireless transceivers, such as Ultra-wideband radios, for measuring distances between the devices using wireless ranging techniques. Based on the measured distances, the mobile device is configured to determine locations of the stationary devices in a reference frame of the three-dimensional map. In some embodiments, the determined locations can be used to enable a variety of spatially aware augmented reality features and interactions between the mobile device and the stationary device.

Abstract: A sensing system includes a stretchable base material, a plurality of electrodes, a capacitive sensing channel and a controller. The stretchable base material has a resistance distribution that changes in response to being mechanically deformed as a result of a human body contact. The base material has a rebound elasticity. The electrodes are attached to a perimeter of the base material, the capacitive sensing channel is attached to the base material. The controller is operatively connected to the plurality of electrodes and the capacitive sensing channel. The controller is configured to measure instantaneous voltage measurements from the plurality of electrodes, and determine whether the base material is mechanically deformed based on the instantaneous voltage measurements using a support vector machine classifier.