Abstract: There is provided a wearable device, comprising: at least one processing core; at least one memory including computer program code; the at least one memory and the computer program code configured to, with the at least one processing core, cause the wearable device to: receive one or more recipes being indicative of input elements for the computer program code, the input elements comprising at least one or more triggering events; one or more display templates; one or more content elements; receive user input for selection of a recipe of the one or more recipes; detect a triggering event corresponding to the one or more triggering events; in response to detecting the triggering event, provide the one or more content elements corresponding to the triggering event for display on a display of the wearable device according to a display template corresponding to the triggering event.

Abstract: In an embodiment, a system implements detent operation using tactile and audio effects that give the user the effect of a detent without including the mechanical devices that detents would normally imply. The detent operation may be implemented in software and may be synchronized to events on the display of the system (e.g. animation events), which may provide a convincing user experience. A low latency path from the applications and user interface program of the system to the hardware that implements the detent operation may be provided to help ensure that the synchronization of animation events and detent operations is maintained.

Abstract: A digital signage system allows a person to use the person's own mobile device as a voice input device to the digital signage system and also can leverage the person's mobile device in other ways such as by accepting other inputs from the mobile device, providing various outputs to the mobile device, or utilizing communication capabilities of the mobile device.

Abstract: Described are techniques for sensor agnostic gesture detection. The techniques include recording a gesture using sensor output from a first user device to a first user profile associated with a ubiquitous gesture agent. The techniques further include associating the gesture with a triggering threshold indicating the gesture and based on the sensor output, and a processor-executable action that is executable by one of a plurality of user devices associated with the user profile. The techniques further include transforming the triggering threshold into a corresponding triggering threshold for a different type of sensor. The techniques further include identifying the gesture using data from the different type of sensor that satisfies the corresponding triggering threshold. The techniques further include implementing, by the one of the plurality of user devices, the processor-executable action associated with the gesture.

Abstract: A controller for an electronic system includes a tracking member fixed to a controller body. The controller body has a head that adjoins a handle at a neck region, and that includes at least one thumb-operated control. The controller includes a hand retainer that in a closed position is configured to physically bias the user's palm against an outer surface of the handle. The hand retainer includes a resilient member that biases the hand retainer towards an open position. The resilient member is attached to an anchor that is attached to the head by an adjustment mechanism that permits the resilient member to be moved towards or away from the user's purlicue. The tracking member includes transducers that are coupled to the electronic system by electromagnetic radiation. Proximity sensors, spatially distributed on the handle, are responsive to a proximity of the user's fingers to the outer surface of the handle.

Abstract: Visual-inertial tracking of an eyewear device using sensors. The eyewear device monitors the sensors of a visual inertial odometry system (VIOS) that provide input for determining a position of the device within its environment. The eyewear device determines the status of the VIOS based information from the sensors and adjusts the plurality of sensors (e.g., by turning on/off sensors, changing the sampling rate, of a combination thereof) based on the determined status. The eyewear device then determines the position of the eyewear device within the environment using the adjusted plurality of sensors.

Abstract: An illustrative system includes an extended reality system and a brain interface system configured to be concurrently worn by a user. The extended reality system is configured to provide the user with an extended reality experience and output a timing signal while the extended reality experience is being provided to the user, the timing signal representing a plurality of timing events that occur during the extended reality experience. The brain interface system is configured to receive the timing signal from the extended reality system while the extended reality experience is being provided to the user, acquire brain activity measurements while the extended reality experience is being provided to the user, and output measurement timestamp data representative of a temporal association of the brain activity measurements with the timing events.

Abstract: Techniques for task invocation based on control actuation, fingerprint detection, and gaze detection are described, and may be implemented via a mobile device to verify that a task associated with an actuatable control is to be executed. Generally, the described techniques assist in preventing inadvertent (e.g., unintended) user actuation of a control from invoking a task associated with the control.

Abstract: Various implementations disclosed herein include devices, systems, and methods for presenting content based on environmental data. In various implementations, a device includes a display, a non-transitory memory and one or more processors coupled with the display and the non-transitory memory. In some implementations, a method includes obtaining environmental data associated with a physical environment in which the device is located. In some implementations, the method includes selecting, from a plurality of presentation modes, a first presentation mode for content based on the environmental data. In some implementations, the method includes presenting the content in accordance with the first presentation mode.

Abstract: Aspects of the present invention relate to external user interfaces used in connection with head-worn computers (HWC).

Abstract: A system for providing pilot or controller support in a reduced crew transport environment may include a set of sensors configured to collect a set of measurements of a set of physiological signals of a user. The system may further include an emotional analyzer model usable to generate at least one emotional parameter value corresponding to the set of measurements. The system may also include a natural language processing model usable to generate a natural language statement corresponding to the at least one emotional parameter value for output to the pilot.

Abstract: A display device having a photosensing function is provided. A display device having a biometric authentication function typified by fingerprint authentication is provided. A display device having both a touch panel function and a biometric authentication function is provided. The display device includes a first substrate, a light guide plate, a first light-emitting element, a second light-emitting element, and a light-receiving element. The first substrate and the light guide plate are provided to face each other. The first light-emitting element and the light-receiving element are provided between the first substrate and the light guide plate. The first light-emitting element has a function of emitting first light through the light guide plate. The second light-emitting element has a function of emitting second light to a side surface of the light guide plate. The light-receiving element has a function of receiving the second light and converting the second light into an electric signal.

Abstract: Provided is a printing device that performs printing using a recording material, the printing device including: an capacitive touch panel; a home operation portion including a mechanical key for transitioning from a screen displayed on the capacitive touch panel to a basic screen including information on a remaining amount of the recording material; and a power supply operation portion including a mechanical key for shifting a power state of the printing device, in which a panel installation surface provided with the capacitive touch panel is a surface different from a power supply operation portion installation surface provided with the power supply operation portion.

Abstract: Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlate the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a neural data feed of neural data from the user using the trained BMI device, determine, a user intention for a control instruction to control a vehicle infotainment system using the neural data feed, and perform an action based on the control instruction. The vehicle may further include a headrest configured as a Human Machine Interface (HMI) device that reads the electrical impulses without invasive electrode connectivity.

Abstract: A brainwave audio and video encoding and playing system is provided. Based on the research and understanding of the brainwaves, the relations of variations of the brainwaves corresponding to the melodies of the human voice senses and the relations of the human brainwaves to the colors is built so as to create rules for playing of the melodies by musical instrument and colors in a display so that variations of the brainwaves can be expressed by playing of music and display of colors and the messages of brainwaves could be transferred. As a result, the people have the same feeling to the testers.

Abstract: A system and method for compensating for reflections caused by light-generating objects in the scene facing a display device includes capturing images of the scene. Reflection-inducting zones corresponding to the light generating objects are identified from the captured images. The reflection effect on the display device from the reflection-inducing zones are estimated. A target image to be displayed on the display device is adjusted based on the estimated reflection effect.

Abstract: A conformable sensor module may conform to skin of a user's face. The sensor module may include multiple piezoelectric strain sensors. The sensor module may measure mechanical strain of facial skin that occurs while the user makes facial gestures. To do so, the sensor module may take a time series of multiple measurements of strain of the user's facial skin at each of multiple locations on the user's face, while the user makes a facial gesture. The resulting spatiotemporal data regarding facial strain may be fed as an input into a trained machine learning algorithm. The trained machine learning algorithm may, based on this input, classify a facial gesture. A computer may determine content associated with the classification. The content may be outputted in audible or visual format. This may facilitate communication by patients with neuromuscular disorders who are unable to vocalize intelligible speech.

Abstract: Disclosed are a method for controlling an Internet of Things (IoT) device, and a terminal device. The method is applied to a terminal device and includes: acquiring parameter information of a triggered key of a mouse; determining a target device and target operation information acting on the target device based on the parameter information; and sending device information of the target device and the target operation information to a server.

Abstract: Systems and methods are disclosed for providing a virtual mouse for a computing device have a touchscreen. A first placement region of the touchscreen may be determined. The first placement region may then be determined to contain a first portion of at least one touch target. The first portion of the at least one touch target may then be deactivated. A virtual mouse may then be activated at the first placement region.

Abstract: A wearable device and its control method, a gesture recognition method, a movable platform control system and a storage medium are provided. The control method may include acquiring key point information of a gesture action of a target user, recognizing an input instruction based upon the key point information, and generating a control instruction based upon the input instruction and executing a task operation according to the control instruction.

Abstract: An electronic device may have a display for displaying image content. Head-mounted support structures in the device may be used to support the display. The electronic device may have an eye monitoring system that detects eye saccades and eye blinks. Control circuitry in the electronic device may coordinate operation of the display with periods of suppressed visual sensitivity that are associated with the saccades and blinks. By making adjustments to display circuitry and image content during periods of suppressed visual sensitivity, potentially visually obtrusive changes to displayed images can be hidden from a user of the electronic device. Adjustments to display operation may help reduce burn-in effects, may help reduce power consumption, and may otherwise improve device performance.

Abstract: An interactive system and a method of use for treatment and training a person with ASD or a similar condition or problem including primary part of holograms and holographic assistant, and secondary part of a group of drones and a robot that involves the use of a pair of holograms that imitate and demonstrate the functions of the right hemisphere of the brain and the left hemisphere of the brain, the exchange of information between them through the communication between the two. The interactive system can be used in training humanoid robots.

Abstract: Among other things, techniques are described for monitoring and controlling autonomous vehicles (AVs). As an example, at least one processor determines that a first AV is in a field of view of a user wearing an augmented reality display device, determines first data regarding an operation of the first AV, and causes at least a portion of the first data to be presented to the user using the augmented reality display device. For example, a graphical user interface is presented in the field of view of the user using the augmented reality display device, and at least the portion of the first data is included on the graphical user interface, such that at least the portion of the first data appears to be arranged spatially proximal to the first AV in the field of view of the user.

Abstract: An image processing device according to one aspect of the present disclosure includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to: receive a visible image of a face; receive a near-infrared image of the face; adjust brightness of the visible image based on a frequency distribution of pixel values of the visible image and a frequency distribution of pixel values of the near-infrared image; specify a relative position at which the visible image is related to the near-infrared image; invert adjusted brightness of the visible image; detect a region of a pupil from a synthetic image obtained by adding up the visible image the brightness of which is inverted and the near-infrared image based on the relative position; and output information on the detected pupil.

Abstract: The present disclosure discusses systems and methods to detect blur in digital images. The solution can be incorporated into the quality control systems of pathology and other slide scanners or can be a stand-alone solution. The solution can identify scanned images that include blur and cause the scanner to automatically rescan the blurry image. The solution can also identify regions of the scanned image that include blur. The solution can generate blur maps for each of the scanned images that identify regions of the scanned image that include blur.

Abstract: An image processing apparatus communicable with an information processing apparatus includes a display and one or multiple processors configured to transmit a request for information and arrangement of buttons to be displayed on the display to the information processing apparatus; to receive the information and the arrangement of the plurality of buttons, transmitted by the information processing apparatus in accordance with the transmitted request; to cause the display to display the buttons in accordance with the received information and the received arrangement of the buttons; to cause, when at least one of the information and the arrangement of the buttons is not received, the display to display a screen based on information and arrangement of buttons stored in the image processing apparatus before the acquisition request is transmitted.

Abstract: A dynamic user interface (UI) input system and method for smart glasses based on availability states of a user is provided. In embodiments, a method includes determining, by a computing device of smart glasses, an initial availability state of a user's hands for a user interface (UI) navigation event based on incoming image data; selecting, by the computing device, a UI configuration and a control profile for the UI navigation event from stored UI configurations and control profiles of the user based on the initial availability state of the user's hands; and initiating, by the computing device, the UI navigation event based on the selected UI configuration and the control profile, wherein content is displayed in a virtual UI of the smart glasses according to the UI configuration, and UI navigation by the user is controlled based on the control profile.

Abstract: Provided is a mobile apparatus obstacle detection system (100), a mobile apparatus (10) carrying the obstacle detection system (100), and a ground-sweeping robot. The obstacle detection system (100) comprises: a structured light projection module (102) configured to project structured light onto the path of advance of the mobile apparatus (10), the structured light comprising at least one lateral detection line in the horizontal direction and at least one longitudinal detection line in the vertical direction; a camera module (103) configured to capture an image of the structured light (105); and an image processing module (104) configured to calculate, according to the image of the structured light (105), the distances and positions of obstacles (106, 107, 108) on the path of advance.

Abstract: Various implementations disclosed herein include devices, systems, and methods for granular motion control for a virtual agent. In various implementations, a device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. In some implementations, a method includes obtaining an action for a virtual agent. In some implementations, the action is associated with a plurality of time frames. In some implementations, the method includes, for a first time frame of the plurality of time frames, determining respective confidence scores for a plurality of granular motions that advance the virtual agent towards completion of the action. In some implementations, the method includes selecting a subset of the plurality of granular motions based on the respective confidence scores.

Abstract: A method for operating a head-mounted display device is presented. An RF face tracking system is operated at a first, lower power level. Based at least on an output of the RF face tracking system while operating at the first, lower power level, a first likelihood is determined that the head-mounted display device is positioned on a head of a user. If the first likelihood is greater than a first confidence threshold, the RF face tracking system is operated at a second, higher power level. A second likelihood that the head-mounted display device is positioned on the head of the user is determined based at least on an output of the RF face tracking system while operating at the second, higher power level. If the second likelihood is greater than a second confidence threshold, an eye-tracking system of the head-mounted display device is activated.

Abstract: Certain aspects and features of this disclosure relate to virtual 3D pointing and manipulation. For example, video communication is established between a presenter client device and a viewer client device. A presenter video image is captured. A 3D image of a 3D object is rendered on the client devices and a presenter avatar is rendered on at least the viewer client device. The presenter avatar includes at least a portion of the presenter video image. When a positional input is detected at the presenter client device, the system renders, on the viewer client device, an articulated virtual appurtenance associated with the positional input, the 3D image, and the presenter avatar. A virtual interaction between the articulated virtual appurtenance and the 3D image appear to a viewer as naturally positioned for the interaction with respect to the viewer.

Abstract: A system and method of creating customized characters and selectively displaying them in an electronic display, such as an augmented reality or virtual reality display is provided. A digital character may be provided by a character provider for customization by others using the system. Such customizations may be instantiated in user devices that provide electronic displays. Instantiation of the custom digital character may be conditioned on one or more trigger conditions, which may be specified by the character customizer. For example, a digital character customized using the system may be conditioned on triggering events in the real-world or in a virtual world. When a relevant triggering condition is satisfied at a user device, the custom character (i.e., information for instantiating the custom character) may be transmitted to that user device. In this manner, the system may push custom characters to user devices that satisfy the triggering condition.

Abstract: A virtual representation of an operating room is generated based on robot information and sensing of the OR with depth cameras. One of the depth cameras is integrated with a portable electronic device, operated by a local user in the operating room. The virtual representation of the OR is communicated to the virtual reality headset, with three-dimensional point cloud data. A virtual reality environment is rendered to a display of the virtual reality headset, operated by a remote user. A virtual representation of the remote user is rendered in augmented reality to a display of the portable electronic device.

Abstract: A method for evaluating gesture input comprises receiving input data for sequential data frames, including hand tracking data for hands of a user. A first neural network is trained to recognize features indicative of subsequent gesture interactions and configured to evaluate input data for a sequence of data frames and to output an indication of a likelihood of the user performing gesture interactions during a predetermined window of data frames. A second neural network is trained to recognize features indicative of whether the user is currently performing one or more gesture interactions and configured to adjust parameters for gesture interaction recognition during the predetermined window based on the indicated likelihood. The second neural network evaluates the predetermined window for performed gesture interactions based on the adjusted parameters, and outputs a signal as to whether the user is performing one or more gesture interactions during the predetermined window.

Abstract: A method for operating an electronic device includes: based on detection of an event, activating at least one reflector device connected to the electronic device; controlling the at least one reflector device to have an angle with respect to the electronic device such that a view position of a user is placed on the at least one reflector device; and controlling to display, on a display of the electronic device, event information associated with the detected event to be reflected to the at least one reflector device.

Abstract: A measurement device and methods for detecting distortion or interference in an environment are described herein. The measurement device comprises a transmitter and a receiver attached to a rigid body such that a position and orientation (P&O) of the receiver with respect to the transmitter is fixed. When measuring distortion, the transmitter transmits electromagnetic waves over one or more frequency channels and measures electromagnetic waves over the channel(s) and a P&O of the receiver relative to the transmitter may be computed. Based on the computed P&O and the known P&O of the receiver with respect to the transmitter, the system determines a level of distortion that would cause the change in position and orientation. When measuring interference, the transmitter does not transmit electromagnetic waves and the receiver measures the electromagnetic waves in the environment.

Abstract: An information processing apparatus (10) includes an event detection unit (110), an input reception unit (120), and a processing execution unit (130). The event detection unit (110) detects a specific event from video data. The input reception unit (120) receives, from a user, input for specifying processing to be executed. The processing execution unit (103) executes first processing specified by input received by the input reception unit (120), and executes second processing of generating learning data used for machine learning and storing the generated learning data in a learning data storage unit (40).

Abstract: An augmented reality system is disclosed that includes an image source configured to generate an image, and a display element configured to display the image. The system further includes an optical relay configured to project the image onto the display element. The system further includes a light sensor configured to quantify an amount of ambient light passing through, the display element. The system further includes a tracker processing unit operatively coupled to the light sensor, the image source, and the optical relay. The tracker processing unit includes a processor, and a memory configured to store instructions executable by the processors. The processors are instructed to receive sensor data from the light sensor, receive the image from the image source, update a pixel characteristic for one or more pixels of the image based on the sensor data, and transmit an updated image to the optical relay.

Abstract: An image processing device according to one aspect of the present disclosure includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to: receive a visible image of a face; receive a near-infrared image of the face; adjust brightness of the visible image based on a frequency distribution of pixel values of the visible image and a frequency distribution of pixel values of the near-infrared image; specify a relative position at which the visible image is related to the near-infrared image; invert adjusted brightness of the visible image; detect a region of a pupil from a synthetic image obtained by adding up the visible image the brightness of which is inverted and the near-infrared image based on the relative position; and output information on the detected pupil.

Abstract: A multimodal sensing system comprises a plurality of sensors placed proximate to a body part. The sensing system receives, using a plurality of sensors, a plurality of signals related to at least one of a movement and a pose of the body part. The sensing system then extrapolates information regarding the type of movement or pose and at least one characteristic of the type a movement or pose.

Abstract: An electromagnetic (EM) pose tracking system includes a computer input device having a pen or stylus form factor. In some embodiments, a base station device includes one of the transmitter (Tx) or receiver (Rx) module for the EM pose tracking system while the computer input device includes the other of the TX and receiver modules. The EM pose tracking system employs the Tx and Rx modules to collect EM pose data indicating a relative pose between the Tx and Rx modules. Based on the EM pose data, the EM pose tracking system (or a computer device working with the EM pose tracking system) identifies a pose (position, orientation, or both position and orientation) of the computer input device.

Abstract: A system and method is configured to determine an initial assessment score for an initial curriculum activity for a learner. A query identifying, within a data logic or at least one database record, a lower threshold value is executed. Responsive to determining that the initial assessment score is less than the lower threshold value, a query is executed identifying a remediation curriculum activity associated with the initial curriculum activity and a first graphical user interface is encoded including an identification of the learner and a first user interface element configured to cause the server hardware computing device to modify a lesson plan of the learner to include the remediation curriculum activity. The first graphical user interface is transmitted to a client hardware computing device for display.

Abstract: An example of an image display system includes a goggle apparatus including a display section. A virtual camera and a user interface are placed in a virtual space. In accordance with the orientation of the goggle apparatus, the orientation of the virtual camera in the virtual space is controlled. For example, in a case where the line of sight of the virtual camera rotates in a left direction in a yaw direction and then reverses in a right direction, and when the amount of change in the right direction is greater than or equal to a threshold, the user interface is moved to the front of the virtual camera.

Abstract: Systems and methods for facilitating virtual collaboration between users within virtual environments are disclosed. A plurality of users access a collaborative virtual environment system using interface devices to establish a virtual collaboration session in which users communicate via a bidirectional real-time communication channel and interact within a virtual environment. Annotations or other user interactions within the virtual environment are automatically propagated to each user in the virtual collaboration session in order to facilitate collaboration. Data from such virtual collaboration sessions may be captured for later use.

Abstract: A pen-shaped input and/or output device and a method for generating a haptic signal are disclosed. In an embodiment a device includes an actuator unit that has a piezoelectric actuator, wherein the device is a pen-shaped input and/or output device, wherein the pen-shaped input and/or output is configured to use the piezoelectric actuator as a sensor, wherein the piezoelectric actuator is configured to generate a voltage as a result of an actuation of the pen-shaped input and/or output device, and wherein the pen-shaped input and/or output device has a second electronics circuit configured to detect the voltage generated by the piezoelectric actuator and store a characteristic value for the voltage generated.

Abstract: Provided are an ink data generation apparatus, an ink data generation method, and an ink data generation program that are capable of improving ease of handling ink data when or after metadata is generated. An ink data generation apparatus performs a determination as to an inclusion relation between a first set and a second set by comparing stroke elements of first set data and stroke elements of second set data using the first set data and the second set data. The ink data generation apparatus generates first metadata for the first set described in a form that varies in accordance with a result of the determination.

Abstract: A biometric sensor configured to detect a biometric information is provided. The biometric sensor includes a light emitting layer and a touch detection layer; an encapsulating cover on the light emitting layer; a photo-sensing layer configured to detect at least a portion of a light totally reflected by a surface of the biometric sensor in touch with a skin of a user; a touch sensing circuit configured to determine a touch position of a touch; a processor configured to determine a scanning region based on the touch position, and configured to control the light emitting layer to form a scanning light source to scan the scanning region in a scanning pattern; and a biometric information analysis circuit configured to determine a light intensity distribution of a reflected light reflected by the surface of the biometric sensor, and configured to determine the biometric information based on the light intensity distribution.

Abstract: An electronic device includes a housing, a display at least partially within the housing, a cover positioned over the display and defining an input surface of the electronic device, a motion sensing system configured to detect a motion of the electronic device, and a touch sensor configured to detect, within a time window after the motion of the electronic device is detected, a contact of an input member of a user on the input surface of the electronic device. The electronic device is configured to determine, for a time prior to the detection of the contact, a relative motion between the input member and the input surface. In accordance with a determination that a characteristic of the relative motion satisfies a threshold condition, the electronic device may determine a second input location based at least in part on the location of the contact and the relative motion.

Abstract: Universal switch modules, universal switches, and methods of using the same are disclosed, including methods of preparing an individual to interface with an electronic device or software. For example, a method is disclosed that can include measuring brain-related signals of the individual to obtain a first sensed brain-related signal when the individual generates a task-irrelevant thought. The method can include transmitting the first sensed brain-related signal to a processing unit. The method can include associating the task-irrelevant thought and the first sensed brain-related signal with N input commands. The method can include compiling the task-irrelevant thought, the first sensed brain-related signal, and the N input commands to an electronic database.

Abstract: A visual aid provides utility to its users by performing geometric transformations of raw images from a digital camera into processed images for electronic display, via three nested levels of “implementation” namely, the nature of the mapping that will be applied, i.e. specifying a specific mapping or parameterized family of mappings; the algorithm set used to effect the select mapping, independent of the processor upon which it executes; and the method for realizing this algorithm on that processor. Likewise offered for consideration are software feature sets tied to user interface which enable dynamic tuning of the subject visual context and environments for low vision users, inter alia.