Abstract: A waveguide apparatus includes a planar waveguide and at least one optical diffraction element (DOE) that provides a plurality of optical paths between an exterior and interior of the planar waveguide. A phase profile of the DOE may combine a linear diffraction grating with a circular lens, to shape a wave front and produce beams with desired focus. Waveguide apparati may be assembled to create multiple focal planes. The DOE may have a low diffraction efficiency, and planar waveguides may be transparent when viewed normally, allowing passage of light from an ambient environment (e.g., real world) useful in AR systems. Light may be returned for temporally sequentially passes through the planar waveguide. The DOE(s) may be fixed or may have dynamically adjustable characteristics. An optical coupler system may couple images to the waveguide apparatus from a projector, for instance a biaxially scanning cantilevered optical fiber tip.

Abstract: A method of determining a spatio-temporal feature value for frames of a sequence of video. A first frame and second frame from the sequence of video are received. Spatial feature values in each of the first and second frames are determined according to a plurality of spatial feature functions. For each of the spatial feature functions, a change in the spatial feature values between the first and second frames is determined. The spatio-temporal feature value is determined by combining the determined change in spatial feature values for each of the spatial feature functions.

Abstract: In one aspect, a computer mouse is provided. The computer mouse may include an accelerometer sensor configured to measure vibrations of the computer mouse during a mouse gliding action and a motion sensor configured to measure the speed of the computer mouse during the mouse gliding action. The gliding performance of the computer mouse may be analyzed based on the measured vibrations and the measured speed of the computer mouse. In another aspect, a method and an apparatus for mouse gliding performance analysis are provided. The apparatus may measure vibrations of a computer mouse using an accelerometer sensor during a mouse gliding action. The apparatus may measure the speed of the computer mouse using a motion sensor during the mouse gliding action. The apparatus may analyze the gliding performance of the computer mouse based on the measured vibrations and the measured speed of the computer mouse.

Abstract: A mobile computing device with a touch-sensitive display wherein: a pointer is displayed; a control icon is displayed; a contact on the touch-sensitive display is detected; and in response to detecting a change in a horizontal position of the contact beginning anywhere on the control icon, a horizontal position of the pointer is changed wherein the horizontal position of the pointer with respect to the control icon is changed; and in response to detecting a change in a vertical position of a contact beginning anywhere on the control icon: a vertical position of the pointer is changed; and a vertical position of the control icon is changed.

Abstract: This invention relates to tracking of user-worn and hand-held devices with respect to each other, in circumstances where there are two or more users interacting in the same share space. It extends conventional global and body-relative approaches to “cooperatively” estimate the relative poses between all useful combinations of user-worn tracked devices such as HMDs and hand-held controllers worn (or held) by multiple users. For example, a first user's HMD estimates its absolute global pose in the coordinate frame associated with the externally-mounted devices, as well as its relative pose with respect to all other HMDs, hand-held controllers, and other user held/worn tracked devices in the environment. In this way, all HMDs (or as many as appropriate) are tracked with respect to each other, all HMDs are tracked with respect to all hand-held controllers, and all hand-held controllers are tracked with respect to all other hand-held controllers.

Abstract: A display device includes a display including a plurality of pixels having a plurality of light-emitting elements, a pixel selector configured to select a pixel from among a plurality of pixels, a positional information generator configured to generate the positional information representing the position of the selected pixel, and a light emission processor configured to superpose the positional information on the light emitted by the selected pixel. A position detecting device is used to detect and output the positional information superposed on the light of the selected pixel on the display device, thus making it easy to accurately detect the position of the pixel subjected to calibration in luminance.

Abstract: A direct retinal projection apparatus including means for detecting a gaze direction of a user, a projector, a first optical element, an actuator and a processor. The first optical element includes different optical portions having different optical properties with respect to magnification. The processor is configured to render a warped image having a spatially-uniform angular resolution via the projector, whilst adjusting an orientation of the first optical element via the actuator, based on the detected gaze direction, to direct a projection of the warped image from the first optical element towards a retina of a user's eye. Projections of different portions of the warped image are differently magnified by the different optical portions of the first optical element to produce on the retina of the user's eye a de-warped image having a spatially-variable angular resolution.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for enabling users to approximately identify a space within an environment inhabited by a plurality of objects that user wishes for a robot to manipulate. In various implementations, an approximation of a space within an environment may be identified based on user input. The actual space within the environment may then be extrapolated based at least in part on the approximation and one or more attributes of the environment. A plurality of objects that are co-present within the space and that are to be manipulated by a robot may be identified. The robot may then be operated to manipulate the identified plurality of objects.

Abstract: Provided is an auxiliary electronic controller of a mobile terminal includes a main body having a seating portion configured to seat the mobile terminal at a top surface of the main body, and having a radiating structure disposed therein, a first holding portion formed at one side of the main body, and a second holding portion hinge-coupled to another side facing the one side of the main body to rotate to a bottom surface of the main body.

Abstract: An optical mouse includes a pixel array and a light source. A method for determining when the optical mouse is in a lifted condition above a surface includes: accumulating pixel values of a first column of the pixel array, the first column being closest to the light source; accumulating pixel values of a second column of the pixel array, the second column being different from the first column; comparing the accumulated pixel values of the first column with the accumulated pixel values of the second column; and when a difference between the accumulated pixel values of the first column and the accumulated pixel values of the second column is greater than a threshold value, determining the optical mouse to be in a lifted condition.

Abstract: There is provided an information processing apparatus including: a recognition unit which recognizes an irradiation position of laser light by a laser pointer to a projection image; an acquisition unit which acquires movement information of the laser pointer; and an output unit which outputs a control signal for causing display of an object within a projection image corresponding to the irradiation position to be changed in accordance with the movement information.

Abstract: A method for processing pupil tracking images, comprising the steps of: 1) acquiring eye images multiple times; 2) processing each acquired eye image, and determining whether a pupil can be positioned in the acquired image; and 3) determining that pupil tracking is completed when the pupil can be positioned in consecutive n pieces of eye images. Image processing efficiency is improved by searching a potential bright spot area and carrying out Blob analysis smearing images caused by pupil movement can be eliminated, the pupil edge is accurately positioned by adopting a line drawing method in collaboration with a support vector machine when determining the pupil edge, and finally a weighted least square method is adopted to fit a circle/ellipse (pupil). Therefore, real-time tracking and analysis of the pupil under a complex background are realized, and reliable data are provided for subsequent steps to obtain the refractive parameters of the eye.

Abstract: Described herein is a sensorized glove, in particular designed for ergonomic analysis, including an inner glove, which comprises a plurality of extensometer sensors configured for detecting relative movements between parts of a worker's hand; and an outer glove, which comprises a plurality of pressure sensors distributed over a palmar surface and configured for detecting the pressure exerted in corresponding areas of said palmar surface. Moreover described is a corresponding method for determining the type of grasp exerted by a worker.

Abstract: A touch input device includes a frame unit, a circuit unit and a touch pad. The circuit unit includes a circuit board, a dome-shaped actuating member including an actuating portion and a resilient metal sheet connected between the actuating portion and the circuit board, and a noise reduction member having a main body connected to the resilient metal sheet, made of an elastic material and formed with a recess surrounding the actuating portion. A touch surface of the touch pad is touchable to push the circuit unit toward the block member, such that the actuating portion contacts the block member to deform the resilient metal sheet to electrically connect the actuating portion with the circuit board.

Abstract: Disclosed are systems and methods for autonomous computing replacing or augmenting a human user of computer programs, where access to internal operations of the computer program is not used. An application controller can use the display output of a computer program to determine a current state of the computer program, using the disclosed embodiments. For example, identity of menu options of the computer program can be determined from image frames obtained from the display output of the computer program and used to determine the current state of the computer program. The application controller can provide input commands to the computer program to execute the computer program from the current state to a destination state.

Abstract: Systems, devices and methods that enable a user to access and interact with content displayed on a portable electronic display in an inconspicuous, hands-free manner are described. There is disclosed a completely wearable system comprising a wearable muscle interface device and a wearable head-mounted display, as well as methods for using the wearable system to effect interactions between the user and content displayed on the wearable head-mounted display. The wearable muscle interface device includes muscle activity sensors worn on an arm of the user to detect muscle activity generated when the user performs a physical gesture. The wearable system is adapted to recognize a plurality of gestures made by the user and, in response to each recognized gesture, to effect one or more interaction(s) with content displayed on the wearable head-mounted display.

Abstract: Proposed is a controller which can suppress an amount of protrusion of an operation member from an external surface of a chassis while operability of the operation member is maintained. A controller including a chassis, and an operation member which is slidable along an extension direction of the chassis with a predetermined position as a reference, and is rotatable in a circumferential direction of the chassis with the predetermined position as the reference, and at least a part of which is fitted into a recess section provided in an external surface of the chassis.

Abstract: Systems and methods for computer-generated reality user hailing are described. Some implementations may include accessing sensor data captured using one or more sensors; detecting a person within a distance of a head-mounted display based on the sensor data; detecting a hail event based on the sensor data; and responsive to the hail event, invoking an alert using the head-mounted display.

Abstract: An electronic device includes a fingerprint sensor and one or more processors operable with the fingerprint sensor. One or more motion detectors are also operable with the one or more processors. The one or more processors determine with the fingerprint sensor, a direction of rotation of an object rotating, and in contact with, the fingerprint sensor. The one or more motion detectors detect whether the electronic device is rotating in three-dimensional space. The one or more processors then select and perform a first control operation when the object is rotating in a first axial direction and the electronic device is substantially stationary. Alternatively, the one or more processors select and perform a second control operation when the electronic device is rotating and the object is rotating in the first axial direction.

Abstract: A hand-held controller and a positional reference device for determining the position and orientation of the hand-held controller within a three-dimensional volume relative to the location of the positional reference device. An input/output subsystem in conjunction with processing and memory subsystems can receive a reference image data captured by a beacon sensing device combined with inertial measurement information from inertial measurement units within the hand-held controller. The position and orientation of the hand-held controller can be computed based on the linear distance between a pair of beacons on the positional reference device and the reference image data and the inertial measurement information.

Abstract: A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Abstract: A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Abstract: An operating device includes a movable part capable of moving between a first position and a second position, with the movable part being capable of operation by the finger of the user gripping the operating device. The operating device includes a regulating part that regulates a range in which the movable part can move, and a control part that determines the amount of regulation in which the movable part can move and provides the regulating part with an instruction to regulate the range in which the movable part can move according to the amount of regulation which has been determined.

Abstract: A missing child prevention support system is provided and includes: a grouping part configured to recognize a group of users who are facility users entering a facility and generate user group information indicating a user group; an image acquisition part configured to acquire facility images including images of the users from imaging devices of the facility; a guide control part configured to control an operation of a movable guiding device performing guiding for the users in the facility; a guide grouping specifying part configured to specify the user group of the users for which the guiding device performs the guiding; and a missing child estimation part configured to estimate an unguided user who is a user not belonging to the user group as a potential missing child when the unguided user is shown in the facility images.

Abstract: A display control device includes a reception unit configured to receive sensor information from an operating device, a pointer display control unit configured to determine a motion of the operating device on the basis of the sensor information, and to display and control a pointer in conjunction with the determined motion of the operating device, and an identification display control unit configured to calculate a rotation angle of the operating device on the basis of the sensor information, and to execute control to present identification display at a position where the pointer is displayed, when the calculated rotation angle comes to a switching angle.

Abstract: The present invention discloses a multi-dimensional surface electromyogram signal prosthetic hand control method based on principal component analysis. The method comprises the following steps. Wear an armlet provided with a 24-channel array electromyography sensor to a front arm of a subject, and respectively wear five finger joint attitude sensors at a distal phalanx of a thumb and at middle phalanxes of remaining fingers of the subject. Perform independent bending and stretching training on the five fingers of the subject, and meanwhile, collect data of an array electromyography sensor and data of the finger joint attitude sensors. Decouple the data of the array electromyography sensor by principal component analysis to form a finger motion training set. Perform data fitting on the finger motion training set by a neural network method, and construct a finger continuous motion prediction model. Predict a current bending angle of the finger through the finger continuous motion model.

Abstract: The present invention is a system for assembling panoramic images of a scene captured by a panoramic image capture device having a decoding unit configured to read image content data from the image capture device, an information reading unit configured to extract device parameters about the capture device, and an image processing unit configured to assemble an output image using the image content data and the device parameters.

Abstract: An exemplary embodiment of the present inventive concept provides a display device including: a display unit including a plurality of pixels disposed on a display area; a display controller configured to control the display unit; a sensor including a plurality of first sensing electrodes and a plurality of second sensing electrodes disposed in the display area; and a sensor controller configured to recognize a touch of a user inputted into the sensor in a touch sensing mode and to recognize a fingerprint of the user inputted into the sensor in a fingerprint sensing mode, wherein the sensor controller changes a mode to the fingerprint sensing mode when no touch is inputted for a predetermined time period in the touch sensing mode and changes the mode to the touch sensing mode when authorization on a fingerprint inputted into the sensor is succeeded in the fingerprint sensing mode.

Abstract: An interactive mid-air display including a display that presents a graphical user interface (GUI), optics projecting and rotating the GUI above the display such that the GUI is visible in-air in a plane rotated away from the display, a sensor including light emitters projecting beams towards the projected GUI, light detectors detecting reflections of the beams by objects interacting with the projected GUI, and a lens structure maximizing detection at each detector for light entering the lens structure at a respective location at a specific angle ?, whereby for each emitter-detector pair, maximum detection of light corresponds to the object being at a specific location in the projected GUI, in accordance with the locations of the emitter and detector and the angle ?, and a processor mapping detections of light for emitter-detector pairs to corresponding locations in the projected GUI, and translating the mapped locations to coordinates on the display.

Abstract: A display apparatus and a controlling method thereof are provided. The display apparatus includes: a display; a communicator including circuitry configured to communicate with an electronic apparatus; a processor configured to execute instructions to: identify whether blank regions are included in an image for mirroring based on the image for mirroring being received from the electronic apparatus through the communicator, and control to display, on the display, an image of a remaining region excluding the blank regions in the received image based on the blank regions being identified in the received image in a state in which the display is oriented in a predetermined direction.

Abstract: Provided are a navigation service assistance system based on a driver line of sight and a vehicle navigation system using the same. The navigation service assistance system recognizes in real time a direction at which a driver looks and a line of sight of the driver, using a sensor. The navigation service assistance system matches external image data and coordinates information based on the line of sight of the driver, using a map controller and extracts detailed information about a text, a place, a road, or the like at a matched position. Here, the map controller assists in enabling automatically extracted text information or a building, a place, a road, or the like to be set or changed as an intermediate point or a destination with only a simple operation of the driver.

Abstract: A manipulation detection device includes a first manipulation detection process based on detection of light emission of an electronic pen (a first manipulation object) and a second manipulation detection process based on detection of a shadow of a finger (a second manipulation object) by radiation of illumination light. An image sensor captures an image of the electronic pen and an image of the finger, and image signal processing units perform a contrast adjustment of the captured image. Here, a state of the contrast adjustment performed by the image signal processing unit differs between the captured image of the electronic pen and the captured image of the finger. Manipulation detecting units execute a manipulation detection process for the electronic pen and the finger using the captured image which has undergone the contrast adjustment.

Abstract: Techniques are described for notebook hinge sensors. For example, a computing device may comprise a housing having a processor circuit and an input device, the input device arranged on a side of the housing, a lid having a digital display arranged on a side of the lid, a hinge arranged to couple the housing and the lid, and a sensor module coupled to the processor circuit, the sensor module arranged inside the hinge and operative to capture motion input outside of the computing device.

Abstract: A gesture based user interface includes a movement monitor configured to monitor a user's hand and to provide a signal based on movements of the hand. A processor is configured to provide at least one interface state in which a cursor is confined to movement within a single dimension region responsive to the signal from the movement monitor, and to actuate different commands responsive to the signal from the movement monitor and the location of the cursor in the single dimension region.

Abstract: An input apparatus includes a coordinate detector that detects position coordinates of an input on an input screen by an input tool, an image-capturing device that captures an image of a marker disposed on the input tool and indicating information on an attribute of the input tool, an input-tool identifier that identifies the attribute on the basis of the image of the marker captured by the image-capturing device, and an input-data generator that generates input data of the input tool on the basis of the position coordinates detected by the coordinate detector and the attribute identified by the input-tool identifier.

Abstract: Systems and methods for tracking the position of a head-mounted display (HMD). The HMD may include a support structure that carries a forward facing camera and a plurality of optical flow sensor integrated circuits (ICs). The forward camera captures image sensor data in a forward camera field of view (FOV) at a first frame rate, and each of the plurality of sensor ICs captures image sensor data in a respective plurality of sensor IC FOVs at a second frame rate. The sensor IC FOVs may collectively cover at least a substantial portion of the forward camera FOV. A processor may receive the image sensor data from the camera and the plurality of sensor ICs. The processor may process the received image sensor data and/or other sensor data (e.g., IMU data) to track a position of the head-mounted display based on the processing of the received sensor data.

Abstract: A display device includes a display panel and a touch panel. The touch panel calculates coordinate information of an input position by an electrostatic capacitive method in a first mode and calculates the coordinate information of the input position by an electromagnetic induction method in a second mode. The touch panel includes scan line groups and source line groups, which are operated as touch electrodes or touch coils on the basis of the operating mode thereof. In addition, the touch panel includes touch electrodes and touch coils, which are individually operated on the basis of the operating mode thereof.

Abstract: The present invention relates to a washing apparatus and to a washing apparatus having a convenient user interface. More specifically, the present invention relates to a washing apparatus that provides a user interface that is concentrated, simple and intuitive.

Abstract: This invention disclosures a six degree-of-freedom (DOF) measuring system and method. The measuring system comprises a tracking measurement device and a target. The tracking measurement device includes a processor, a camera and two rotation optical components connected with the processor respectively, which are arranged in sequence. The camera boresight and the optical axis of two rotation optical components are coaxial, and each rotation optical component can rotate independently. The target is mounted on the tested object, which contains at least three markers with known distance constraints. In addition, at least one marker does not coincide with the midpoint of the line connecting any two of the remaining markers. Compared with the prior technologies, this invention realizes a dynamic real-time measurement of six DOF for the tested object. In the process of measurement, only the target is fixed on the object.

Abstract: Smartglasses, methods, and systems to control presentation and display of information related to mobile device tasks and applications to the smartglasses or a paired or connected smartwatch or smartphone, based on oral commands or requests, and enhance presentation and display augmented reality information, based on default settings of a device mobile application controlling display of information related to tasks and applications and oral override or switch commands.

Abstract: An apparatus and method for automatically controlling screens displayed on two display units of a mobile terminal are provided. The method includes receiving an input requesting to operate a camera of the mobile terminal; providing a video image that is obtained via the camera through a first display area disposed in a first surface of the mobile terminal or a second display area disposed in a second surface being opposite to the first surface of the mobile terminal, based on a camera mode, wherein the camera mode includes a first mode and a second mode; while providing the video image obtained via the camera through the first display area or the second display area, detecting a rotation of the mobile terminal; and in response to detecting the rotation, switching a display area providing the video image between the first display area and the second display area.

Abstract: Examples of mixed reality computing devices that utilize remote sensors and local sensors as input devices are disclosed. In one example, a mixed reality computing device comprises an image sensor, a remote input device, a processor, and storage comprising stored instructions. The stored instructions are executable by the processor to perform object motion tracking and environmental tracking based on output from the image sensor, and in response to detecting that the remote input device is in use, adjust a parameter of the motion tracking while maintaining the environmental tracking.

Abstract: Systems, apparatuses and methods may provide for technology to improve user experience when viewing simulated 3D objects on a display. Head and upper-body movements may be tracked and recognized as gestures to alter the displayed viewing angle. The technology provides for a very natural way to look around, under, or over objects.

Abstract: An example electronic device includes a first housing including, a first display, a first antenna, a second antenna, and a first rotational motion sensor. The electronic device also includes a hinge including a bend sensor. The electronic device further includes a second housing rotatable coupled to the first housing via the hinge. The second housing includes a second display, a third antenna, a fourth antenna, and a second rotational motion sensor. The electronic device further includes a communication device to select two of the first antenna, the second antenna, the third antenna, and the fourth antenna based on the first rotational motion sensor, the second rotational motion sensor, and the bend sensor. The communication device is also to transmit and receive data via each of the two selected antennas.

Abstract: A gaze detection apparatus of the present invention may include an infrared LED of emitting infrared light, an infrared camera of photographing eyes of a driver based on light which is reflected from the eyes of the driver by the infrared light emitted from the infrared LED, and a gaze detection unit of determining whether the driver gazes the infrared camera based on an image of the eyes photographed by the infrared camera, and at least one of the infrared LED and the infrared camera is installed on each of a plurality of vehicle-mounted devices, and operates to emit the infrared light to the driver from each vehicle-mounted device, photograph the driver within a corresponding light emission period, and detect whether the driver gazes based on photographed information, and the vehicle-mounted devices are under a congestion control so that the light emission periods of the infrared light emitted therefrom are not overlapped with each other.

Abstract: Techniques are disclosed for detecting, tracking, and recording data associated with physical manipulatives in a physical environment, and a corresponding set of virtual manipulatives in a virtual environment. Physical manipulative sensor devices may be configured to detect the position and/or movements of physical manipulatives within a physical environment, including simple or multi-step user interactions with physical manipulatives. The sensor devices and/or associated user computing devices may use the sensor data to determine updated positions, orientations, and configurations for one or more physical manipulatives within the physical environment, as well as defining relationships between sets of the physical manipulatives.

Abstract: A light filed (LF) display system for displaying holographic content to viewers in an amusement park (e.g., as part of an amusement park ride). The LF display system in an amusement park includes LF display modules tiled together to form an array of LF modules. In some embodiments, the LF display system includes a tracking system and/or a viewer profiling module. The tracking system and viewer profiling module can monitor and store characteristics of viewers on the amusement park ride, a viewer profile describing a viewer, and/or responses of viewers to the holographic content during the amusement park ride. The holographic content created for display on an amusement park ride can be based on any of the monitored or stored information.

Abstract: A method of operating an electronic system for is provided. The method includes the following actions. A plurality of images are captured by an image capturing unit. An object is detected by a sensor unit within a sensing region. When the object is detected within the sensing region of the sensor unit, a palm feature is recognized by a processing unit from the captured images. An application is stimulated in response to the palm feature.

Abstract: In one example, a computing system includes a first computing device including a first display having a moveable interaction window that is a subset of an entire display area of the first display, the first computing device to wirelessly transmit a video stream representing at least the interaction window. The computing system includes a tablet computing device including a second display, the tablet computing device to wirelessly receive the video stream and display content of the interaction window on the second display and cause movement of the interaction window on the first display based on movement of the tablet computing device.

Abstract: A motion sensory and imaging device that is capable of acquiring imaging information of the scene and providing at least a near real time pass-through of imaging information to a user. The sensory and imaging device can be used stand-alone or coupled to a wearable or portable device to create a wearable sensory system capable of presenting to the wearer the imaging information augmented with virtualized or created presentations of information.