Abstract: A head-mounted device includes a mounted member that is mounted on a head of a user, a sensor unit, which is a detection device, that detects a posture of the mounted member, a control device that switches a control mode according to the posture detected in the sensor unit, and a video output device and an audio output device as output devices that perform output operations according to the control mode in the control device.

Abstract: Systems, apparatuses, and methods are described for intelligent control of video display backlighting that provides environmental lighting illumination of an area behind a video display device, such as on a wall behind a video display. The environmental lighting illumination may be based on recognizing images (e.g., objects, patterns, faces, colors, etc.) in video content being displayed via the video display device, recognizing audio (e.g., words, sounds, voices, etc.) in audio content being output in association with video content being displayed via the video display device, and/or recognizing other triggers such as commands associated with video content.

Abstract: A method and a head-mounted device for triggering an action with relation to a selected object in a field of view of a user comprising detecting a pre-defined brain activity pattern of the user using a brain activity sensor of a head-mounted device worn by the user. The pre-defined brain activity pattern expresses a user command to select an object on which a gaze of the user is focused. The method comprises determining a direction of the gaze of the user at a time of the detection of the pre-defined brain activity pattern. The method comprises capturing an image of the field of view of the user at the time of the detection of the pre-defined brain activity pattern. The method comprises providing at least a part of the image to a computing entity, to cause the computing entity to trigger the action with relation to the selected object.

Abstract: Systems, apparatus, and methods for a gesture-based augmented reality and/or extended reality (AR/XR) user interface. Conventional image processing scales quadratically based on image resolution. Processing complexity directly corresponds to memory size, power consumption, and heat dissipation. As a result, existing smart glasses solutions have short run-times (<1 hr) and may have battery weight and heat dissipation issues that are uncomfortable for continuous wear. The disclosed solution provides a system and method for low-power image processing via the use of scalable processing. In one specific implementation, gesture detection is divided into multiple stages. Each stage conditionally enables subsequent stages for more complex processing. By scaling processing complexity at each stage, high complexity processing can be performed on an “as-needed” basis.

Abstract: Tracking digital resources used in resource exchange events conducted in virtual reality computing environments. Digital resources used in resource exchange events conducted in virtual reality computing environments are tagged with data that at least identifies the location of the resource exchange event in terms of virtual reality computing environment and/or sub-environments. In response, machine-learning algorithm(s) are implemented that analyze the tags to determine resource exchange event movement patterns for specific digital resources. The resource exchange event movement patterns may include digital resource movement across multiple different virtual computing environments.

Abstract: Augmented reality (AR) devices, systems and methods are provided to argument captured images of objects of interest. Image data can be obtained for an object of interest by an imaging device of an ear-worn device worn by a user. Augmenting information is generated to augment onto an image of the object. The augmented image is adjusted based on the detection on whether the object is in a field of view (FOV) of the imaging device.

Abstract: A locking method and apparatus, an electronic device and a medium are provided. The locking method, applied to an application program on an electronic device, in which the method includes: displaying, in response to the electronic device meeting a set condition, a setting control in a current display interface of the application program, in which the setting control is configured to control a locking status of a display orientation of media data played by the application program; changing, in response to the setting control being triggered, the locking status of the display orientation, in which the current display interface includes the media data and an attribute control of the media data, the setting control includes a lock control and an unlock control, the lock control is configured to lock the display orientation, and the unlock control is configured to unlock the display orientation.

Abstract: A sensory content production system generates sensory associated with concepts identified in an electronic document. Definitions of each concept are identified, and the concepts are stored in connection with their associated definitions. Sensory content is generated for the concepts and concepts definitions and stored in connection with each other in a sensory content layer. Availability of sensory content is indicated to users by interactive markers.

Abstract: Systems and methods disclosed herein include, determining a proximity of the first networked device to a second networked device, determining whether there is a wired network connection existing between the first networked device and the second networked device, and in response to determining the proximity and the existence of a wired network connection, reconfiguring one or more operational parameters of one or both of the first networked device and the second networked device.

Abstract: Various implementations disclosed herein include devices, systems, and methods that identify a gesture based on event camera data and frame-based camera data (e.g., for a CGR environment). In some implementations at an electronic device having a processor, event camera data is obtained corresponding to light (e.g., IR light) reflected from a physical environment and received at an event camera. In some implementations, frame-based camera data is obtained corresponding to light (e.g., visible light) reflected from the physical environment and received at a frame-based camera. In some implementations, a subset of the event camera data is identified based on the frame-based camera data, and a gesture (e.g., of a person in the physical environment) is identified based on the subset of event camera data. In some implementations, a path (e.g., of a hand) by tracking a grouping of blocks of event camera events in the subset of event camera data.

Abstract: A wearable electronic device includes a display, and at least one processor connected to the display, where the at least one processor is configured to obtain a first space necessary for a motion of a user corresponding to a motion of an avatar, the motion of the avatar being performed in a virtual space of virtual reality (VR) content displayed through the display, obtain a second space for safely performing the motion of the user, set a scale value, based on the first space and the second space, and based on obtaining the motion of the user, determine, using the set scale value, a size of the motion of the avatar, the motion of the avatar being performed by the motion of the user.

Abstract: A method for facilitating provision of food product information is provided. The method comprises providing a food package, wherein the food package is provided with a visual marker, capturing, by a camera of a user device, authentic image data depicting the visual marker, extracting marker data from the authentic image data, retrieving artificial image data linked to the marker data from an artificial image database, wherein the artificial image data depicts at least one food product component of a food product held in the food package, determining an artificial image spatial reference point based on the visual marker, generating augmented image data by combining the authentic image data, the artificial image data and the artificial image spatial reference point, providing the augmented image data to a screen of the user device.

Abstract: A wearable terminal apparatus to be worn by a user for use includes a display unit and at least one processor. The at least one processor causes the display unit to display a virtual image located in a space. The at least one processor changes a display position of a partial image when a predetermined condition is satisfied, the partial image being an image of a partial region included in the virtual image.

Abstract: A convenient electronic device or the like is provided. The power consumption of an electronic device or the like is reduced. An electronic device or the like having high visibility regardless of the brightness of external light is provided. An electronic device or the like that can display both a smooth moving image and an eye-friendly still image is provided. Such an electronic device is an electronic device including a first display portion, a second display portion, and a control portion. The control portion is configured to make the first display portion and the second display portion individually display two or more of a first image, a second image, and a third image at a time. The first image is displayed with reflected light, the second image is displayed with emitted light, and the third image is displayed with light including both reflected light and emitted light.

Abstract: An image photographing method and an electronic device, and the method includes: collecting M first images by using a camera in a first camera array, and collecting N second images by using a camera in a second camera array, where M is an integer greater than 1, and N is a positive integer; displaying P preview images, where each preview image is obtained by synthesizing an image in the M first images and an image in the N second images, synthesized images corresponding to different preview images are different, the synthesized image includes at least one of the following: the first image and the second image, and P is an integer greater than 1; and obtaining a corresponding final image based on a target image in the P preview images.

Abstract: Augmented reality and virtual reality display systems and devices are configured for efficient use of projected light. In some aspects, a display system includes a light projection system and a head-mounted display configured to project light into an eye of the user to display virtual image content. The head-mounted display includes at least one waveguide comprising a plurality of in-coupling regions each configured to receive, from the light projection system, light corresponding to a portion of the user's field of view and to in-couple the light into the waveguide; and a plurality of out-coupling regions configured to out-couple the light out of the waveguide to display the virtual content, wherein each of the out-coupling regions are configured to receive light from different ones of the in-coupling regions. In some implementations, each in-coupling region has a one-to-one correspondence with a unique corresponding out-coupling region.

Abstract: A tactile presentation panel includes a tactile presentation voltage generation circuit that generates a voltage signal having a first frequency to be applied to at least one first electrode located in at least a partial region on an operation surface of the tactile presentation panel, and generates a voltage signal having a second frequency different from the first frequency to be applied to at least one second electrode located in at least a partial region on the operation surface of the tactile presentation panel, a charge discharge portion that is connected to the tactile electrode and discharges charges accumulated in a conductive elastic portion, and a touch detection circuit that detects contact of the user with a tactile presentation knob.

Abstract: Techniques for displaying a virtual card are provided. The techniques comprise obtaining a first card face pattern, a first initial parameter, and a first association relationship corresponding to the virtual card, wherein the first initial parameter represents an initial parameter of a first display feature of a first material, and the first association relationship represents a rule by which a real-time parameter of the first display feature changes as a change of a parameter associated with a sensor in a mobile client device happens; displaying the first material in the virtual card based on the first initial parameter; detecting in real time the parameter associated with the sensor, and determining the real-time parameter of the first display feature based on the first association relationship and the parameter; and displaying the first material in the virtual card based on the real-time parameter of the first display feature.

Abstract: Apparatus, methods, and computer-readable media for facilitating a cloud-based vehicle XR user experience are disclosed herein. An example method for wireless communication at a user equipment (UE) includes transmitting a request for a vehicle extended reality (XR) session. The vehicle XR session may be based on a first user XR stream including a vehicle XR component associated with a vehicle and a first user XR component associated with a first user. The first user may have an association with the vehicle. The example method also includes transmitting uplink information associated with the first user XR stream. The example method also includes receiving rendering information associated with the first user XR stream. The rendering information may be based on the uplink information.

Abstract: A control device may be configured to control one or more electrical loads in a load control system. The control device may be a wall-mounted device such as dimmer switch, a remote control device, or a retrofit remote control device. The control device may include a gesture-based user interface for applying advanced control over the one or more electrical loads. The types of control may include absolute and relative control, intensity and color control, preset, zone, or operational mode selection, etc. Feedback may be provided on the control device regarding a status of the one or more electrical loads or the control device.

Abstract: Aspects of the present disclosure relate to multi-user, multi-device gaze tracking. In examples, a system includes at least one processor, and memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations. The set of operations include identifying a plurality of computing devices, and identifying one or more users. The set of operations may further include receiving gaze input data and load data, from two or more of the plurality of computing devices. The set of operations may further include performing load balancing between the plurality of devices, wherein the load balancing comprises assigning one or more tasks from a first of the plurality of computing devices to a second of the plurality of computing devices based upon the gaze input data.

Abstract: Disclosed in the present invention is an electromagnetic-based haptic device that utilizes a flat wire coil type to provide the user with kinesthetic feedback in mid-air without direct contact. The invention comprises an array of flat wire coils, a localization system, a cooling fluid source, a magnetic agent attached to a stylus or a wearable device, current drivers and computer software. The flat wire coil comprises longitudinal packed flat conductive turns, each two turns are separated by an axial air gap. The cooling fluid source generates a fluid flow to pass over the surfaces of the coils to dissipate the heat generated by the device during the operation. The array of flat-wire coils is used to generate controlled magnetic fields and magnetic field gradients at a point in the workspace, the magnetic agent is attached to a wearable device or a handhold stylus to translate the magnetic fields/magnetic fields gradients to forces/torques feedback and thus haptic sensation.

Abstract: A method, apparatus, and device for executing one or more actions by a processing device in response to a positive saturation determination. The method may include: sensing an object that occupies a first portion of a field of view (FOV) of the sensor at a first point in time; determining a first saturation level of the object at the first portion of the FOV; sensing the object that occupies a second portion of the FOV of the sensor at a second point in time; determining a second saturation level of the object at the second portion of the FOV; determining that the first saturation level is different than the second saturation level; and in response to the first saturation level being different than the second saturation level, executing an executable command associated with the first saturation level and the second saturation level.

Abstract: At a first time point, a first light capturing device at a first spatial location in a three-dimensional (3D) space captures first light rays from light sources located at designated spatial locations on a viewer device in the 3D space. At the first time point, a second light capturing device at a second spatial location in the 3D space captures second light rays from the light sources located at the designated spatial locations on the viewer device in the 3D space. Based on the first light rays captured by the first light capturing device and the second light rays captured by the second light capturing device, at least one of a spatial position and a spatial direction, at the first time point, of the viewer device is determined.

Abstract: A tracking method, for tracking an object based on a computer vision, includes following steps. A series of images is captured by a tracking camera. A first position of a trackable device is tracked within the images. An object is recognized around the first position in the images. In response to the object being recognized, a second position of the object is tracked in the images.

Abstract: A display device includes: a display panel including a substrate, where the display panel is divided into a first display region in which a plurality of first pixels is disposed and a second display region in which a plurality of second pixels is disposed; and a sensor overlapping the second display region of the display panel. The second display region includes a plurality of unit pixel blocks, each including two second pixels adjacent to each other in a first direction and two second pixels adjacent to each other in a second direction intersecting the first direction, and a transmissive region through which light is transmitted. A distance between two unit pixel blocks adjacent to each other in the first direction is greater than a distance between two unit pixel blocks adjacent to each other in an oblique direction inclined to the first direction.

Abstract: A method and a system for rendering video images in virtual reality (VR) scenes are provided.

Abstract: Gaze tracking to track gaze of a user within a user view screen including: combining the gaze tracking with head movement tracking, wherein head movements of the head movement tracking provide rough estimate of a direction of the gaze of the user, while eye movements of the gaze tracking provide fine tuning of the direction of the gaze of the user within the user view screen; dividing the user view screen into a plurality of gaze zones when gaze zone estimation is turned on; and combining the gaze tracking with the gaze zone estimation to select a gaze zone from the plurality of gaze zones as the direction of the gaze of the user.

Abstract: A splicing-lamp lighting-effect application method includes displaying a plane composition of a target splicing-lamp, which includes form patterns corresponding to multiple lamp blocks of the target splicing-lamp and each form pattern includes multiple light-emitting areas; loading effect configuration information corresponding to each layer described in a lighting-effect template into a lighting-effect editing area for display; in response to a layer on-top event, marking the effective light-emitting area in the form pattern of each lamp block according to the area information of a current layer set-on-top, obtaining a plane topological pattern corresponding to the current layer, and applying the lighting-effect information of the current layer to the plane topological pattern to display a preview lighting effect of the current layer; and responding to a lighting-effect application instruction, encapsulating edited effect configuration information of each layer in a lighting-effect application package, and p

Abstract: Systems and methods are provided for performing operations on an augmented reality (AR) device. The system accesses, by the AR device, movement data comprising inertial measurement data and camera data. The system determines three-dimensional (3D) movement of the AR device based on the movement data. The system presents, by the AR device, an AR object on a real-world environment being viewed using the AR device. The system, in response to determining the 3D movement of the AR device, modifies the AR object by the AR device.

Abstract: A directional pad for a controller can comprise a shell that includes a top having opposing upper and lower surfaces and a plurality of openings that each extend from the upper surface to the lower surface. The directional pad can also comprise a plurality of protrusions that are each movable between a retracted state in which the protrusion is disposed at or below the upper surface of the top of the shell and a deployed state in which the protrusion extends through a respective one of the openings of the top of the shell such that at least a portion of the protrusion is disposed above the upper surface of the top of the shell. A plurality of actuators can each be configured to move a respective one of the protrusions between the retracted and deployed states.

Abstract: The disclosed device can receive a biosignal and, using user input predictions based on the biosignal, pre-render a display frame. The device can also subsequently receive a user input, output the pre-rendered display frame based on the user input confirming the user input predictions and flush the pre-rendered display frame otherwise. The device can also modulate computing performance and power based on computing demands predicted from the biosignal. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a computer-program product embodied in a non-transitory computer readable medium that is programmed for performing active vibration cancellation (AVC) in a moving vessel is provided. The computer-program product includes instructions to receive a first signal indicative of vibrations that are exhibited on at least one passenger in a cabin of the moving vessel and to determine a resonant frequency of the vibrations that are exhibited on the at least one passenger based on the first signal. The computer-program product further includes instructions to generate a first anti-wave signal based on the resonant frequency and to drive at least one haptic actuator that is positioned proximate to the at least one passenger in the cabin with the first anti-wave signal to minimize motion sickness for the at least one passenger caused by the vibrations that are exhibited on the at least one passenger in the vessel.

Abstract: In some aspects, an electronic device may display a keyboard user interface on a touchscreen display panel. The electronic device may detect a key press interaction with the keyboard user interface. The electronic device may trigger one or more haptic actuators to generate a vibration waveform associated with a haptic pattern that is based at least in part on a context associated with the key press interaction. Numerous other aspects are described.

Abstract: An electrical switch, including an insulation base, an insulative-and-movable component, at least two output ports, and at least two transmission ports, is provided. The insulative-and-movable component is disposed on the insulation base and is adapted to operate between at least two switch positions relative to the insulation base. The two output ports are disposed on the insulation base. The two transmission ports are disposed on the insulative-and-movable component. When the insulative-and-movable component is located at one of the two switch positions, one of the two output ports is aligned with one of the two transmission ports and other one of the two output ports is misaligned with other one of the two transmission ports.

Abstract: A system comprises an input device structurally configured to be utilized by an operator to control a medical device. The system further comprises a motion sensor associated with the input device and configured to detect a displacement distance of the input device. The system further comprises a control unit configured to determine a velocity cap defining a maximum velocity of the medical device based on the detected displacement distance. The control system is further configured to control movement of the medical device based on the velocity cap.

Abstract: One or more embodiments of the description relates to a flexible display device and a display system. The flexible display device includes: a screen component and a support structure disposed on the screen component. The flexible display device has different stiffness in a first direction and in a second direction. The first direction is a direction in which the screen component curls up, and the second direction is perpendicular to the first direction.

Abstract: In some embodiments, an electronic device navigates between user interfaces based at least on detecting a gaze of the user. In some embodiments, an electronic device enhances interactions with control elements of user interfaces. In some embodiments, an electronic device scrolls representations of categories and subcategories in a coordinated manner. In some embodiments, an electronic device navigates back from user interfaces having different levels of immersion in different ways.

Abstract: Visual-inertial tracking of an eyewear device using sensors. The eyewear device monitors the sensors of a visual inertial odometry system (VIOS) that provide input for determining a position of the device within its environment. The eyewear device determines the status of the VIOS based information from the sensors and adjusts the plurality of sensors (e.g., by turning on/off sensors, changing the sampling rate, of a combination thereof) based on the determined status. The eyewear device then determines the position of the eyewear device within the environment using the adjusted plurality of sensors.

Abstract: A method for aligning coordinate systems of user devices in an augmented reality system using somatic points of a user's hand as alignment markers. Images captured from multiple user devices are used to align the reference coordinate systems of the user devices to a common reference coordinate system. In some examples, user devices capture images of a hand of a user and use object recognition to identify somatic points as alignment markers. The somatic points of a user device are translated to a common reference coordinate system determined by another user device.

Abstract: The technology disclosed relates to simplifying updating of a predictive model using clustering observed points. In particular, it relates to observing a set of points in 3D sensory space, determining surface normal directions from the points, clustering the points by their surface normal directions and adjacency, accessing a predictive model of a hand, refining positions of segments of the predictive model, matching the clusters of the points to the segments, and using the matched clusters to refine the positions of the matched segments. It also relates to distinguishing between alternative motions between two observed locations of a control object in a 3D sensory space by accessing first and second positions of a segment of a predictive model of a control object such that motion between the first position and the second position was at least partially occluded from observation in a 3D sensory space.

Abstract: Techniques for managing electronic user profiles are presented herein. An example method includes accessing, from a data structure, a user profile. The user profile can include a profile identifier and a plurality of data fields. The method also includes displaying, on a display device, an interactive user interface. The user interface can include a plurality of user interface action elements. Each of the user interface action elements can be associated with a data field in the data structure. Each of the user interface action elements can also be individually selectable by a respective user action to transmit the associated data field to a remote device. In response to receiving a selection of a user interface action element, the method can further include transmitting the associated data field and the profile identifier to a router configured to send, based on the profile identifier, the associated data field to the remote device.

Abstract: Implementations of the subject technology provide systems and methods for displaying virtual content that is supplemental to video content being displayed to a user. The virtual content can be generated based on the video content being displayed, including based on scene content obtained for the video content and/or detected in the video content. The scene content can be identified, by the device providing the supplemental virtual content, in real time during playback of the video content.

Abstract: In some implementations, a method includes: determining a location for virtual content; detecting a user interaction with the virtual content; in response to detecting the user interaction with the virtual content, determining a position of a hand gesture during the user interaction with the virtual content; in accordance with a determination that the position of the hand gesture is within a threshold distance relative to the location of the virtual content, generating corrected hand tracking data associated with the user interaction with the virtual content; and in accordance with a determination that the position of the hand gesture is outside of the threshold distance relative to the location of the virtual content, generating uncorrected hand tracking data associated with the user interaction with the virtual content.

Abstract: An electronic device, with a display, a touch-sensitive surface, one or more processors and memory, displays a first representation of a first controllable external device, where the first controllable external device is situated at a location. The device detects a first user input corresponding to a selection of the first representation of the first controllable external device. The device, after detecting the first user input, adds data identifying the first controllable external device and a first state of the first controllable external device in a scene profile.

Abstract: An information processing device is provided which includes a first acquisition unit (214) configured to acquire a control command that is inputted by a first user and corresponds to presentation unit information for designating a presentation unit for presenting a tactile stimulus by a tactile presentation device and mode information for designating a mode of the tactile stimulus; a generation unit (218) configured to generate a tactile control signal for presenting the tactile stimulus to the presentation unit in accordance with the control command; and a first distribution unit (222) configured to distribute the tactile control signal to the tactile presentation device worn on a body of a second user.

Abstract: A touch screen device is disclosed. The touch screen device includes a touch screen panel having a capacitive sense array, and a passive dial including first, second and third conductive parts. The first conductive part and the third conductive part are in proximity to a surface of the touch screen panel, and the second conductive part is conductively coupled to the third conductive part when the passive dial is pressed and is not conductively coupled to the third conductive part when the passive dial is not pressed. A position of the first conductive part above the capacitive sense array corresponds to a touch index in the capacitive sense array and a position the third conductive part above the capacitive sense array corresponds to a button press index in the capacitive sense array. The first, second and third conductive parts are movable in conjunction with a rotation of the passive dial.

Abstract: The technology disclosed also initializes a new hand that enters the field of view of a gesture recognition system using a parallax detection module. The parallax detection module determines candidate regions of interest (ROI) for a given input hand image and computes depth, rotation and position information for the candidate ROI. Then, for each of the candidate ROI, an ImagePatch, which includes the hand, is extracted from the original input hand image to minimize processing of low-information pixels. Further, a hand classifier neural network is used to determine which ImagePatch most resembles a hand. For the qualified, most-hand like ImagePatch, a 3D virtual hand is initialized with depth, rotation and position matching that of the qualified ImagePatch.

Abstract: Systems and methods for controlling an Internet of Things (IoT) device through interaction with an augmented reality (AR) camera includes pairing an interactable augmented reality (AR) overlay with the IoT device. The interactable overlay includes control information for generating a control signal for controlling the IoT device when the interactable overlay is interacted with by the AR camera. The interactable overlay is presented on a display of the AR camera when the IoT device is in a field of view of the AR camera. An indication that the user has pointed the AR camera at the interactable overlay associated with the IoT device on the display of the AR camera is provided, and a control signal associated with the interactable overlay pointed at by the AR camera is provided to the at least one IoT device associated with the interactable overlay pointed at by the AR camera.

Abstract: A knob apparatus includes a touch panel, a touch-sensing controller, and a knob. The touch panel has multiple touch-sensing electrodes. The touch-sensing controller is coupled to the touch-sensing electrodes of the touch panel. The touch-sensing controller detects a touch event of the touch panel through the touch-sensing electrodes. The knob has a base and a knob cap. The knob cap is pivoted on the base. The base is attached to the touch panel. Multiple conductive electrodes are disposed at different positions of the base. The touch-sensing controller detects the conductive electrodes of the base of the knob through the touch-sensing electrodes of the touch panel to learn a rotation direction of the knob cap on the base.