Abstract: An electronic device may include a display, a communication circuit, at least one camera, a memory, and a processor operatively connected to the display, the communication circuit, the at least one camera, and the memory. The memory may store instructions that, when executed, cause the processor to provide an augmented reality (AR) environment or a virtual reality (VR) environment through the display, connect the electronic device and at least one external electronic device through the communication circuit, display the at least one external electronic device through the display, specify a first external electronic device among the displayed at least one external electronic device based on an input interface switching event, and control an operation of the electronic device in the augmented reality environment or the virtual reality environment using the specified first external electronic device.

Abstract: A method includes generating one or more calibration instructions for calibrating one or more sensors coupled to a movable object and providing at least some of the one or more calibration instructions to a user interface. The at least some of the one or more calibration instructions includes human-readable user instructions for moving the movable object along a predetermined pattern. The method further includes calibrating one or more sensor parameters for the one or more sensors based at least in part on sensor data collected while the movable object moves along the predetermined pattern.

Abstract: Technologies and techniques for setting a value for a parameter in which a graphical user interface is generated and output, wherein the graphical user interface includes a selection object that is assigned to the parameter. A first user input, which includes a positioning gesture related to the selection object, is captured, wherein a setting object is generated depending on the first user input and is positioned on the graphical user interface according to the positioning gesture. A second user input, which includes a setting gesture, is captured, with the value of the parameter being set using the setting object according to the setting gesture.

Abstract: In response to a current gesture performed by a user interacting with an application, a gesture detection module accesses a plurality of predefined control gestures, and compares the current gesture with the plurality of predefined control gestures. The gesture detection module detects whether the current gesture matches at least an initial partial gesture of a particular control gesture associated with at least one application function. A feedback generator module generates confirmation feedback for the user if the current gesture matches at least an initial partial gesture of the particular control gesture. The confirmation feedback encodes, for the user, at least a first information identifying the application function associated with the particular control gesture and a second information about a time interval remaining before the associated application function will be executed. The application triggers the associated function if the particular control gesture is completed during said time interval.

Abstract: A computer-implemented method includes: predicting, by a computing device, devices for inclusion in an interface; generating, by the computing device, the interface including areas corresponding to the devices; detecting, by the computing device, a user selection of one of the areas of the interface; detecting, by the computing device, a hand gesture associated with the selected one of the areas; and transmitting, by the computing device, data defining the hand gesture to a respective one of the devices corresponding to the selected one of the areas, thereby causing the respective one of the devices to execute a command based on the hand gesture.

Abstract: The present disclosure provides a method for detecting the focus of attention. The method includes: obtaining the face of a person in the first image, as well as the result of facial recognition; determining whether the distance between the person and the target is within an effective attention range; determining whether the face is frontal; determining whether the effective attention period is not shorter than a period threshold; detecting the focus of attention for the person to the target.

Abstract: A network-connected electronic device may obtain, at a location, a notification that a network-connected sensor device at the location is activated in response to a triggering of a critical rule of the network-connected sensor device, determine whether the triggering of the critical rule of the network-connected sensor device will activate a function of the network-connected electronic device, activate the function of the network-connected electronic device in response to the determining that the triggering of the critical rule of the network-connected sensor device will activate the function of the network-connected electronic device.

Abstract: Provided are a fingerprint authentication device and a fingerprint authentication method capable of efficiently performing registration of fingerprint information or authentication of a fingerprint. A presentation image that presents a position of a fingernail root at a time of detecting the fingerprint is generated, and an image of the fingerprint is obtained using the generated presentation image.

Abstract: SphericRTC provides real-time 360-degree video communication, which allows the viewer to observe the environment in any direction from the camera location. This more allows users to more-efficiently exchange information and can be beneficial in the real-time setting. The system selects representations of 360-degree frames to allow efficient, content-adaptive delivery. The system performs joint content and bitrate adaptation in real-time by offloading expensive transformation operations to a GPU. The system demonstrates that the multiple sub-components: viewport feedback, representation selection, and joint content and bitrate adaptation, can be effectively integrated within a single framework. Compared to a baseline implementation, views in SphericRTC have consistently higher visual quality. The median Viewport-PSNR of such views is 2.25 dB higher than views in the baseline system.

Abstract: A human machine interface for an industrial automation control system includes at least one touchless input device that is adapted to be in a first state in which said human machine interface provides a first input to said industrial automation control system or a second state in which said human machine interface provides a second input to said industrial automation control system. The at least one touchless input device includes first and second touchless input sensors each configured to detect hand gestures of an operator's hand to provide input to said human machine interface based upon said gestures. The first and second touchless input sensors can be identical with respect to each other or different. In one example, one or both of the sensors are both time-of-flight sensors and one of the sensors can be an electric field proximity sensor. A method of providing a human machine interface with at least one touchless input device is provided.

Abstract: Systems and methods are provided for customizing user interface controls around a cursor. One example method includes receiving, at a computing device, a request to display an indicator menu and identifying at least one user interface element of a program being executed at the computing device. An interaction frequency for each identified user interface element is generated. In response to the request and for each identified user interface element, a spatial relationship between an indicator of the computing device and the identified user interface element is determined. Based on the interaction frequency value and the determined spatial relationship, an element set comprising one or more of the identified user interface elements is generated. The indicator menu comprising at least a portion of the element set is generated for display. The indicator menu is displayed proximate the indicator on a display of the computing device.

Abstract: A screenshot or screen capture operation method for an electronic device includes: capturing a screen displayed on a display; determining whether additional information exists on the displayed screen; when the additional information exists, extracting the additional information, based on the displayed screen; determining whether a command for modifying the captured screen has been received; when the command for modifying the captured screen is determined as having been received, modifying the captured screen according to a user input; and storing the extracted additional information and/or the captured screen as a captured image. The resulting method enables a user to capture a screen intended or desired by the user.

Abstract: Developer and runtime environments supporting multi-modal input for computing systems are disclosed. The developer environment includes a gesture library of human body gestures (e.g., hand gestures) that a previously-trained, system-level gesture recognition machine is configured to recognize. The developer environment further includes a user interface for linking a gesture of the gesture library with a semantic descriptor that is assigned to a function of the application program. The application program is executable to implement the function responsive to receiving an indication of the gesture recognized by the gesture recognition machine within image data captured by a camera. The semantic descriptor may be additionally linked to a different input modality than the gesture, such as a natural language input.

Abstract: A virtual space experience system capable of preventing unintentional contact between players without inhibiting a sense of immersion is provided. A virtual space image determination unit 35 of a VR system S causes an image of a virtual space to be perceived by a second player to include an image in which a first avatar corresponding to a first player moves so as to generate a gap in a correspondence relationship between coordinates of the first player and coordinates of the first avatar when a trigger event is recognized, and causes the image of the virtual space to be perceived by the second player to include an image of a second avatar corresponding to the second player and a third avatar corresponding to the first player after the recognition. An avatar coordinate determination unit 34 determines coordinates of the third avatar based on the coordinates of the first player.

Abstract: A method and device for controlling a hotspot recommendation pop-up window, a computer-readable medium and an electronic device are provided. On reception of a pop-up window display instruction, a pop-up window including prompt options corresponding to multiple hotspots is displayed on the hot video playing page. In a case that the pop-up window is displayed on the hot video playing page, a first control on the hot video playing page except for the pop-up window is set to a disabled state. Further, the user can select among the prompt options corresponding to multiple hotspots included in the pop-up window.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for presenting participant reactions within a virtual working environment.

Abstract: A template built by a user may be converted by a Server Script Generation Engine (SSGE) into script code. In converting, the SSGE may load and parse a framework file containing static script syntax to locate insertion points, each associated with an iteration number, and may iteratively parse the template, utilizing the iteration number to resolve, in order, tags and sub-tags contained in the template. If a tag is set to respond to the iteration number, a function of the tag is invoked to process any related sub-tags and return a script associated therewith at the appropriate insertion point. The framework file (with the appropriate script code inserted) is compiled and stored in a compiled script object which can be run multiple times to perform all of the output functions expected by the user in lieu of the need to reconvert the template.

Abstract: A display apparatus includes circuitry to receive an input of a plurality of strokes representing a drawing that includes one or more drawing components including a first drawing component. The plurality of strokes is divided under a predetermined condition to obtain the one or more drawing components. The circuitry stores, in a memory, each of the one or more drawing components and drawing data representing the drawing. The circuitry displays, on a display screen, in response to receiving an input of a second drawing component, the drawing represented by the drawing data based on the first drawing component and the second drawing component.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for navigating through a shared artificial reality environment. Various aspects may include receiving an indication of a virtual object in the shared artificial reality environment. Aspects may also include receiving an input gesture indicative of a navigation command associated with the virtual object. Aspects may also include determining at least one type of the input gesture comprising flexion and extension, pronation and supination, or radial and ulnar. Aspects may also include determining a control method. Aspects may include determining a scrolling parameter. Aspects may include identifying the navigation command based on the type of the input gesture, the control method, and the scrolling parameter. Aspects may include applying the navigation command to the virtual object.

Abstract: A method for dynamically changing a graphical user interface element occurs in response to detecting that a temporal user interface element displayed on a user interface of user device. The method includes receiving, at the user device, a contextual signal characterizing a state of a user. The method further includes determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element. The methods also include, in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element, modifying a respective state of the temporal user interface element displayed on the user interface of the user device.

Abstract: An information processing apparatus includes a control unit that performs control according to a result of recognition of a user located near a presentation screen on which an object is presented, and a result of detection of a shooting preparation movement of the user, to present a given object such that the given object does not overlap a recognition area of the user.

Abstract: In one embodiment of the present invention, a method for multiple device interaction includes detecting an orientation of a first device relative to a second device. The method also includes detecting a first gesture performed with either the first device or the second device, wherein the first gesture causes a first action that is based at least in part on the orientation of the first device relative to the second device.

Abstract: An apparatus, method, and computer readable medium that access a frame buffer of a graphics processing unit (GPU), analyze, in the frame buffer, a frame representing displayed data, based on the analyzed frame, identify a reference patch that includes an instruction to retrieve content, generate an overlay including an augmentation layer which includes the content, superimpose the overlay onto the displayed data such that the content is viewable while a portion of the base layer is obscured, detect a user input, determine a location of the user input in the augmentation layer, associate the location in the augmentation layer with a target location in the base layer, and associate, within memory, the target location with an operation such that the user input in the augmentation layer activates an input in the base layer.

Abstract: In one embodiment, a method includes receiving an initial input in a first modality from a first user from a client system associated with the first user, determining one or more intents corresponding to the initial input by an intent-understanding module, generating one or more candidate continuation-inputs based on the one or more intents, where the one or more candidate continuation-inputs are in one or more candidate modalities, respectively, and wherein the candidate modalities are different from the first modality, and sending instructions for presenting one or more suggested inputs corresponding to one or more of the candidate continuation-inputs to the client system.

Abstract: A device control method, a device control apparatus, an electronic device, and a storage medium are provided. The electronic device includes an ultrasonic emitter and an ultrasonic receiver. The method includes that operating gesture recognition is performed by the electronic device through ultrasonic waves, a recognition result is generated, a motion parameter of the electronic device is obtained, the recognition result is verified based on the motion parameter, and the electronic device is controlled based on a verification result.

Abstract: A graphical user interface for trimming a video may include a timeline representation of a duration of the video. A trim duration for the video may be selected based on movement of the timeline representation. The amount of time represented by a portion of the timeline representation may be independent of the duration of the video. The movement of the timeline representation may correspond to moment through the duration of the video at a constant scale regardless of the duration of the video.

Abstract: The embodiments of the disclosure provide a method for improving quality of a visual content, a host, and a computer readable storage medium. The method includes: determining a first gesture of a first hand and accordingly providing a first hand object in the visual content; determining a second gesture of a second hand and accordingly providing a second hand object in the visual content; in response to determining that the first gesture and the second gesture respectively correspond to a first predetermined gesture and a second predetermined gesture, determining a specific distance between the first gesture and the second gesture; and in response to determining that the specific distance is smaller than a distance threshold, refining the first hand object and the second hand object.

Abstract: Hot-word free adaptation of one or more function(s) of an automated assistant. Sensor data, from one or more sensor components of an assistant device that provides an automated assistant interface (graphical and/or audible), is processed to determine occurrence and/or confidence metric(s) of various attributes of a user that is proximal to the assistant device. Whether to adapt each of one or more of the function(s) of the automated assistant is based on the occurrence and/or the confidence of one or more of the various attributes. For example, certain processing of at least some of the sensor data can be initiated, such as initiating previously dormant local processing of at least some of the sensor data and/or initiating transmission of at least some of the audio data to remote automated assistant component(s).

Abstract: This document describes techniques and systems that enable a visual indicator for paused radar gestures. The techniques and systems use a radar field to enable an electronic device to accurately determine radar gestures, or other movement, by a user. Further, the electronic device can determine certain conditions that can make it difficult for the electronic device to properly determine the user's radar gestures. When the device includes an application that can be controlled using radar gestures (a radar-gesture application), and the conditions are present, the device can enter a gesture-paused mode. When the device enters this mode, the techniques provide a gesture-paused feedback element on a display, which lets the user know that there is at least one radar-gesture application available or running on the electronic device but that radar gestures cannot presently be used to control the application.

Abstract: A security system (10) includes an image information acquisition unit (11) that acquires input image information on an image taken of a person in a store, a tracking unit (12) that tracks an action of a hand of the person based on the input image information, and a suspicious action detection unit (13) that detects a suspicious action of the person based on the tracked action of the hand. A security system, a security method, and a security program capable of accurately detecting a suspicious action are thereby provided.

Abstract: Method, computer program product, and system to prioritize the execution of processing task requests in a task request queue, where the processing task requests are related to an environment. The method includes adding processing task requests to a task request queue in response to the detection of predefined actions in an environment. The method also includes adding additional processing task requests or adjusting a priority level of not yet completed task requests in the task request queue, in response to detecting subsequent predefined actions in the environment.

Abstract: A device with a touch-sensitive display may be unlocked via gestures performed on the touch-sensitive display. The device is unlocked if contact with the display corresponds to a predefined gesture for unlocking the device. The device displays one or more unlock images with respect to which the predefined gesture is to be performed in order to unlock the device. The performance of the predefined gesture with respect to the unlock image may include moving the unlock image to a predefined location and/or moving the unlock image along a predefined path. The device may also display visual cues of the predefined gesture on the touch screen to remind a user of the gesture.

Abstract: Detection device is provided and includes first and second electrodes disposed facing or close to first electrodes; first drive circuit configured to supply first drive signal; second driver circuit configured to supply second drive signal; and detector configured to detect detection signal output from first electrodes, wherein detection device has first period in which detector detects first detection signal output from first electrodes in response to supply of second drive signal to second electrodes, if first detection signal detected in first period is equal to or higher than predetermined threshold, detector is configured to output an output signal including panel coordinates generated based on first detection signal, if first detection signal detected in first period is lower than predetermined threshold, detection device is transitioned to a second period in which detector detects second detection signal output from first electrodes in response to supply of first drive signal to first electrodes and su

Abstract: In one embodiment, a method includes receiving an initial input in a first modality from a first user from a client system associated with the first user, determining one or more intents corresponding to the initial input by an intent-understanding module, generating one or more candidate continuation-inputs based on the one or more intents, where the one or more candidate continuation-inputs are in one or more candidate modalities, respectively, and wherein the candidate modalities are different from the first modality, and sending instructions for presenting one or more suggested inputs corresponding to one or more of the candidate continuation-inputs to the client system.

Abstract: Systems and methods for generating pseudo haptic feedback for a virtual widget are disclosed. The method includes computing, from a force profile of the virtual widget, a virtual displacement value based on a force value and a type of the virtual widget. The force profile describes a user's perception of physical characteristics of the virtual widget. The method generates a visual haptic feedback based on the virtual displacement value and the type of the virtual widget. Further, in response to the force value, the method may display on a display screen the visual haptic feedback.

Abstract: An extended reality (XR) handheld controller to operate with a head-mounted display (HMD) device and information handling system includes the HMD device having an HMD processor, a data storage device, and an HMD display device to present to a user an XR image of an environment and the XR handheld controller may include a handheld controller microcontroller, a handle portion, a multimodal joystick, and a fingerprint reader formed into the multimodal joystick, wherein the multimodal joystick provides directional input when the fingerprint reader detects a user's fingerprint and the user tilts the multimodal joystick in a direction, and wherein the multimodal joystick provides push-button input when the fingerprint reader does not detect the user's fingerprint and the user presses a side of the multimodal joystick.

Abstract: The embodiments of the disclosure provide a screenshot capturing method, an electronic device and a computer-readable medium. The method includes operations as follows. A duration of a pressing operation is obtained, in response to detecting the pressing operation. In response to determining the duration reaches a preset duration, a page currently displayed on the touch screen is determined as a specified page, and a selection box is displayed on the display screen. A sliding operation acting on the touch screen is obtained. One side of the selection box is controlled to move with the sliding operation, and a position of an opposite side of the selection box is controlled to remain unchanged. In response to determining the sliding operation ends, a screenshot mode is entered based on a positional relationship between an end point of the sliding operation and the specified page, so as to capture a screenshot image.

Abstract: A method for detecting and responding to snap motions. The method includes detecting a triggering event that switches a mobile device into a snap motion gesture detection state to commence a finger-snap gesture motion detection process. The method includes activating, based on detecting the triggering event, an image capturing device to capture a series of image frames of a scene and capturing, using the image capturing device, the series of image frames of the scene that includes a hand. The method includes determining, by analyzing at least two frames among the series of image frames, that a finger-snap gesture is detected in the series of image frames. The method includes executing an action corresponding to a determination that a finger-snap gesture is detected in the series of image frames.

Abstract: An apparatus is configured to: cause a display to display a GUI comprising a plurality of items; receive user input that is scribed over any of the plurality of items displayed on the display; provide the user with a substantially real-time feedback of the scribing by applying digital ink indicating a scribed line on the GUI; select a first item of the plurality of items based on a position of the scribing within the GUI; recognize a trail of the scribed line; in response to the trail of the scribed line being recognized as a first predetermined trail, perform, on the first item, first processing corresponding to the first predetermined trail; and in response to the trail of the scribed line being recognized as a second predetermined trail, perform second processing on the first item. The second processing corresponds to the second predetermined trail and is from the first processing.

Abstract: A virtual image display device and a setting method for an input interface are provided. The virtual image display device includes a virtual image display and a handheld electronic device. The virtual image display executes an application to provide multiple prompt commands to perform a setting action of the input interface. The handheld electronic device displays the input interface and receives multiple input touch actions corresponding to each of the prompt commands through the input interface in the setting action. The virtual image display calculates multiple effective touch areas respectively corresponding to the prompt commands according to a position distribution of each input touch action corresponding to each of the prompt commands.

Abstract: An electronic device includes instructions for receiving a first communication; in response to receiving the first communication, generating a first notification for the first communication received at the device, the first notification including content of the first communication; receiving a second communication at the device; and in response to receiving the second communication: determining whether the second communication and the first communication are received from the same sender. The instructions include, in accordance with a determination that the second communication and the first communication are from the same sender, updating the first notification such that the updated first notification concurrently includes the content of the first communication and the second communication; and in accordance with a determination that the second communication and the first communication are not from the same sender, generating a second notification, for concurrent display with the first notification.

Abstract: A method for generating human-machine hybrid predictions of answers to forecasting problems includes: parsing text of an individual forecasting problem to identify keywords; generating machine models based on the keywords; scraping data sources based on the keywords to collect scraped data relevant to the individual forecasting problem; providing the scraped data to the machine models; receiving machine predictions of answers to the individual forecasting problem from the machine models based on the scraped data; providing, by the computer system via a user interface, the scraped data to human participants; receiving, by the computer system via the user interface, human predictions of answers to the individual forecasting problem from the human participants; aggregating the machine predictions with the human predictions to generate aggregated predictions; and generating and outputting a hybrid prediction based on the aggregated predictions.

Abstract: A graphical user interface for setting speed and direction of video playback may include a timeline representation of video duration. Playback speed and playback direction from a selected point of the video may be determined based on user interaction with the graphical user interface. A portion of the video to which the selected playback speed and selected playback direction is applied may be determined based on user movement of the timeline representation.

Abstract: A server and computer implemented method for sharing physical writing actions, the method comprising the steps of: detecting, at each of a plurality of computing devices associated with a meeting, one or more physical writing actions being performed on physical writing surfaces; generating writing signals based on the physical writing actions; transmitting the generated writing signals to a server; forwarding, via the server, the writing signals for receipt at the plurality of computing devices associated with the meeting; and each computing device outputting a representation of the physical writing actions.

Abstract: The present invention relates to a head-mounted display system (1) for controlling a medical imaging device (20), comprising: a head-mounted display (10) to be worn by an operator, comprising a display (12) configured to display a medical image with a predetermined center of view (121) and a command menu (122) providing at least a first set of commands (123) as distinct first command fields, a tracking system (14) to track a movement representative of the operator's eye (30) and/or head movement, wherein a command of the at least first set of commands (122a) of the command menu (122) is selectable based on the tracked eye and/or head movement, and a control device (40) configured to receive a selected command and to control the medical imaging device (20) accordingly.

Abstract: An information processing apparatus creates a first virtual object expressing a physical object that is detected from physical object information obtained from a physical object information acquisition unit. The information processing apparatus determines a display state of the first virtual object in accordance with a result of detecting collision between the first virtual object and a second virtual object. The information processing apparatus creates, on the basis of a virtual space including the first virtual object and the second virtual object, position-orientation of an HMD, the determined display state, and a physical space image obtained from the HMD, a mixed reality image in combination of an image of the virtual space and the physical space image, and displays the created mixed reality image on the HMD.

Abstract: In one embodiment, a method includes receiving an initial input in a first modality from a first user from a client system associated with the first user, determining one or more intents corresponding to the initial input by an intent-understanding module, generating one or more candidate continuation-inputs based on the one or more intents, where the one or more candidate continuation-inputs are in one or more candidate modalities, respectively, and wherein the candidate modalities are different from the first modality, and sending instructions for presenting one or more suggested inputs corresponding to one or more of the candidate continuation-inputs to the client system.

Abstract: Systems, interfaces, and methods for implementing the systems and interfaces include a dynamic environment generation subsystem that changes objects and subobjects based on locations of the motion sensors and/or the nature, time and/or location of sensed motion and include selection attractive movement as the selection protocol, where a selection object is used to discriminate between selectable objects and attract a target object toward the selection objects, where the direction and speed of the motion controls, discriminates, attracts, and activates the selected objects.

Abstract: The technology disclosed relates to user interfaces for controlling augmented reality environments. Real and virtual objects can be seamlessly integrated to form an augmented reality by tracking motion of one or more real objects within view of a wearable sensor system using a combination a RGB (red, green, and blue) and IR (infrared) pixels of one or more cameras. It also relates to enabling multi-user collaboration and interaction in an immersive virtual environment. In particular, it relates to capturing different sceneries of a shared real world space from the perspective of multiple users. The technology disclosed further relates to sharing content between wearable sensor systems. In particular, it relates to capturing images and video streams from the perspective of a first user of a wearable sensor system and sending an augmented version of the captured images and video stream to a second user of the wearable sensor system.

Abstract: The disclosed computer-implemented method may include detecting, by a computing system, a gesture that appears to be intended to trigger a response by the computing system, identifying, by the computing system, a context in which the gesture was performed, and adjusting, based at least on the context in which the gesture was performed, a threshold for determining whether to trigger the response to the gesture in a manner that causes the computing system to perform an action that is based on the detected gesture.