Patents Assigned to Varjo Technologies Oy
  • Publication number: 20220130103
    Abstract: A display apparatus including means for tracking pose of user's head, light source(s) and processor configured to: process pose-tracking data to determine position, orientation, velocity and acceleration of head; predict viewpoint and view direction of user in extended-reality environment; determine region of extended-reality environment to be presented, based on viewpoint and view direction; determine sub-region(s) of region whose rendering information is to be derived from previous rendering information of corresponding sub-region(s) of previously-presented region of extended-reality environment; generate rendering information of sub-region(s) based on previous rendering information; send, to rendering server, information indicating remaining sub-regions required to be re-rendered and pose information indicating viewpoint and view direction; receive, from rendering server, rendering information of remaining sub-regions; merge rendering information of sub-region(s) and rendering information of remaining sub-
    Type: Application
    Filed: October 22, 2020
    Publication date: April 28, 2022
    Applicant: Varjo Technologies Oy
    Inventors: Ari Antti Peuhkurinen, Ville Miettinen
  • Patent number: 11315334
    Abstract: A display apparatus including light source(s), camera(s), head-tracking means, and processor configured to: obtain three-dimensional model of real-world environment; control camera(s) to capture given image of real-world environment, whilst processing head-tracking data obtained from head-tracking means to determine pose of users head with respect to which given image is captured; determine region of three-dimensional model that corresponds to said pose of users head; compare plurality of features extracted from region of three-dimensional model with plurality of features extracted from given image, to detect object(s) present in real-world environment; employ environment map of extended-reality environment to generate intermediate extended-reality image based on pose of users head; embed object(s) in intermediate extended-reality image to generate extended-reality image; and display extended-reality image via light source(s).
    Type: Grant
    Filed: February 9, 2021
    Date of Patent: April 26, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Ari Antti Erik Peuhkurinen, Ville Timonen, Niki Dobrev
  • Patent number: 11307410
    Abstract: A display apparatus including pose-tracking means; image renderer per eye; liquid-crystal device including liquid-crystal structure and control circuit; and processor. Processor is configured to: process pose-tracking data to determine user's head pose; detect if rate at which head pose is changing is below predefined threshold rate; if yes, switch on lock mode, select head pose for session of lock mode, and generate output image frames according to head pose during session; if no, generate output image frames according to corresponding head poses of user using pose-tracking data; and display output image frames, whilst shifting light emanating to from pixels of image renderer to multiple positions (P1-P9) in sequential and repeated manner, said shifting causes resolution of output image frames to appear higher than display resolution of image renderer.
    Type: Grant
    Filed: January 22, 2021
    Date of Patent: April 19, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Mikko Strandborg, Ville Miettinen, Klaus Melakari
  • Patent number: 11308900
    Abstract: In a display apparatus, a liquid-crystal structure, arranged in front of an image renderer, is controlled to shift light of a given sub-pixel to target positions according to a shifting sequence in a repeated manner, while output image frames are displayed. To generate a given output image frame, a given target position to which the light is to be shifted is determined based on the shifting sequence. An input colour value of the given sub-pixel provided in a given input image frame is then adjusted to generate an output colour value of the given sub-pixel for the given output image frame, based on an output colour value of at least one other sub-pixel whose light overlaps with the given target position during display of a previous output image frame, and a retention coefficient between a colour of the at least one other sub-pixel and a colour of the given sub-pixel.
    Type: Grant
    Filed: February 4, 2021
    Date of Patent: April 19, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Mikko Strandborg, Klaus Melakari, Ville Miettinen
  • Patent number: 11288027
    Abstract: A peripheral display apparatus that is wearable by a user. The peripheral display apparatus includes at least two light sources and at least one processor coupled to at least two light sources. The at least two light sources include a first light source and a second light source arranged at a first peripheral portion and a second peripheral portion of a field of view of user, respectively, first peripheral portion and second peripheral portion being positioned at opposite horizontal ends of field of view. The at least one processor or at least one external processor communicably coupled to at least one processor is configured to generate at least two images including a first image and a second image, wherein at least one processor is configured to display first image and second image simultaneously at first light source and second light source, respectively.
    Type: Grant
    Filed: April 14, 2021
    Date of Patent: March 29, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Juha Välimäki, Niko Eiden, Qing Xu, Klaus Melakari, Oiva Arvo Oskari Sahlsten
  • Patent number: 11288990
    Abstract: A display apparatus including: image renderer having array of pixels; liquid-crystal device comprising: liquid-crystal structure, wherein portions of liquid-crystal structure are arranged in front of corresponding pixels of said array; and control circuit including circuit elements employed to electrically control corresponding portions of liquid-crystal structure to shift light emanating from corresponding pixels to corresponding target positions; and processor(s) configured to: generate individual drive signals for circuit elements, based on corresponding target positions to which light emanating from corresponding pixels are to be shifted upon passing through corresponding portions of liquid-crystal structure; and send individual drive signals to control circuit to drive circuit elements to address corresponding portions of liquid-crystal structure separately, whilst displaying output image frame via image renderer.
    Type: Grant
    Filed: March 12, 2021
    Date of Patent: March 29, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Mikko Strandborg, Klaus Melakari
  • Patent number: 11281290
    Abstract: A display system including eye-tracking means, first image renderer, second image renderer, optical combiner and processor is described. Eye-tracking data is processed to determine gaze directions of user's eyes. Gaze location and gaze velocity and/or acceleration of user is determined based on gaze directions of user's eyes. It is detected whether or not gaze has been fixated for predefined time period based on gaze location and gaze velocity and/or acceleration of user. If gaze has been fixated, input image is processed to generate and render first image and second image substantially simultaneously. Projection of rendered first image and projection of rendered second image are combined optically by optical combiner to create extended-reality scene. If gaze has not been fixated, input image is processed to generate and render first image via first image renderer, whilst switching off or dimming second image renderer.
    Type: Grant
    Filed: June 17, 2020
    Date of Patent: March 22, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Niko Eiden, Klaus Melakari, Jarno Wuolijoki, Ville Kivistö, Antti Peuhkurinen
  • Patent number: 11275945
    Abstract: An imaging system and a method for producing extended-reality images for a display apparatus. The imaging system includes camera and processor. The processor is configured to: control camera to capture image of real-world environment; analyse captured image to identify first image segment representing input device and to determine location of at least one actionable area of input device in first image segment; determine at least one functional element to be presented for the actionable area, the functional element being indicative of at least one of: functionality of the at least one actionable area, status of the at least one actionable area; and process captured image to generate extended-reality image in which the functional element is virtually superimposed over the actionable area of input device or a virtual representation of the actionable area of input device.
    Type: Grant
    Filed: March 26, 2020
    Date of Patent: March 15, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Ari Antti Peuhkurinen, Aleksei Romanov, Evgeny Zuev, Yuri Popov, Tomi Lehto, Urho Konttori
  • Patent number: 11228745
    Abstract: A display apparatus including first and second displays or projectors that display first and second images for first and second eyes, respectively; first and second portions facing first and second eyes, respectively, first and second portions having first (F-F?) and second (S-S?) optical axes, respectively; means for tracking positions and orientations of first and second eyes relative to corresponding optical axes, respectively; and processor. The processor or external processor obtains current relative positions and orientations of both eyes; determines first and second transformations for first and second input image frames, respectively, given transformation being applied to correct apparent per-pixel distortions produced when given input image frame is displayed; and applies first and second transformations to generate first and second distortion-corrected image frames, respectively, wherein processor renders first and second distortion-corrected image frames.
    Type: Grant
    Filed: November 26, 2019
    Date of Patent: January 18, 2022
    Assignee: Varjo Technologies Oy
    Inventors: Ari Antti Peuhkurinen, Thomas Carlsson
  • Publication number: 20210397251
    Abstract: A display system including eye-tracking means, first image renderer, second image renderer, optical combiner and processor is described. Eye-tracking data is processed to determine gaze directions of user's eyes. Gaze location and gaze velocity and/or acceleration of user is determined based on gaze directions of user's eyes. It is detected whether or not gaze has been fixated for predefined time period based on gaze location and gaze velocity and/or acceleration of user. If gaze has been fixated, input image is processed to generate and render first image and second image substantially simultaneously. Projection of rendered first image and projection of rendered second image are combined optically by optical combiner to create extended-reality scene. If gaze has not been fixated, input image is processed to generate and render first image via first image renderer, whilst switching off or dimming second image renderer.
    Type: Application
    Filed: June 17, 2020
    Publication date: December 23, 2021
    Applicant: Varjo Technologies Oy
    Inventors: Niko Eiden, Klaus Melakari, Jarno Wuolijoki, Ville Kivistö, Antti Peuhkurinen
  • Patent number: 11185120
    Abstract: An adjustment mechanism for allowing a user to adjust a length of a headband includes a stationary case, a clutch, and a knob. The stationary case defines an opening and a circular recess. The clutch includes a cam arranged within the circular recess and a pinion extending axially from the cam such that the pinion is positioned outside the case via the opening to engage with the headband. The clutch also includes a roller located adjacent to a first cam portion of the cam. The knob has an actuating element arranged between the roller and a tab of the cam. The knob is rotatable in a first direction for allowing the user to shorten the length of the headband. The knob is rotatable in a second direction allowing the user to increase the length of the headband.
    Type: Grant
    Filed: February 20, 2020
    Date of Patent: November 30, 2021
    Assignee: Varjo Technologies Oy
    Inventor: Jukka Manni
  • Patent number: 11175513
    Abstract: A display apparatus including means for detecting user's gaze direction; first light source; second light source; configuration (108, 208, 306) of reflective elements to split light to create copies of second image; optical waveguides whose outlets are arranged in tiled layout; opacity masks arranged at inlets of waveguides; and processor. Input image is processed to generate first and second images. Processor is configured to: display first and second images; and for optical waveguides, control respective opacity masks to block respective copies of second image or selectively allow portions of respective copies of second image to pass through towards inlets of waveguides. Light from first light source is optically combined with light exiting outlets of waveguides, to produce output image.
    Type: Grant
    Filed: September 28, 2020
    Date of Patent: November 16, 2021
    Assignee: Varjo Technologies Oy
    Inventor: Ari Antti Peuhkurinen
  • Publication number: 20210350564
    Abstract: A display system including: display apparatus; display-apparatus-tracking means; input device; processor. The processor is configured to: detect input event and identify actionable area of input device; process display-apparatus-tracking data to determine pose of display apparatus in global coordinate space; process first image to identify input device and determine relative pose thereof with respect to display apparatus; determine pose of input device and actionable area in global coordinate space; process second image to identify user's hand and determine relative pose thereof with respect to display apparatus; determine pose of hand in global coordinate space; adjust poses of input device and actionable area and pose of hand such that adjusted poses align with each other; process first image, to generate extended-reality image in which virtual representation of hand is superimposed over virtual representation of actionable area; and render extended-reality image.
    Type: Application
    Filed: May 8, 2020
    Publication date: November 11, 2021
    Applicant: Varjo Technologies Oy
    Inventors: Ari Antti Peuhkurinen, Aleksei Romanov, Evgeny Zuev, Yuri Popov, Tomi Lehto
  • Patent number: 11170678
    Abstract: A display apparatus including gaze-tracking means, image renderers, liquid-crystal devices including liquid-crystal structure and control circuit, to shift light emanating from given pixel of image renderer to multiple positions, given pixel including colour component; and processor configured to: process gaze-tracking data to determine gaze direction of user's eye; determine gaze point; display first output image frame; detect if magnitude of difference between first output value and initial second output value of colour component of given pixel in first and second output image frames exceeds first threshold difference; when detected that magnitude of difference exceeds first threshold difference, update initial second output value to sum of first output value and product of distance factor and difference between initial second output and first output values; and display second output image frame.
    Type: Grant
    Filed: January 29, 2021
    Date of Patent: November 9, 2021
    Assignee: Varjo Technologies Oy
    Inventors: Mikko Strandborg, Ville Miettinen
  • Patent number: 11159713
    Abstract: An imaging system for producing images for a display apparatus, the imaging system including: at least one camera; means for tracking an orientation of the at least one camera; and at least one processor communicably coupled to said camera and said means. The at least one processor is configured to: create and store an N-dimensional data structure representative of an environment; for a plurality of orientations of the at least one camera, determine values of camera attributes to be employed to capture a given image from a given orientation and update the N-dimensional data structure with the determined values; access the N-dimensional data structure to find values of camera attributes for a current orientation; and control the at least one camera to employ the found values for capturing an image of real-world scene from the current orientation.
    Type: Grant
    Filed: October 11, 2019
    Date of Patent: October 26, 2021
    Assignee: Varjo Technologies Oy
    Inventors: Anna Nilsson, Ville Timonen
  • Publication number: 20210327030
    Abstract: An imaging system for producing extended-reality images for a display apparatus. The imaging system includes a first camera, a second camera and at least one processor. The processor is configured to: control the first camera and the second camera to capture pairs of first images and second images of a real-world environment, respectively, wherein a given pair comprises a first image and a second image captured for a first eye and a second eye of a user, respectively; denoise the first image and the second image at a first denoise level and a second denoise level, respectively; and process the first image and the second image, upon denoising, to generate a first extended-reality image and a second extended-reality image to be rendered via the display apparatus, respectively.
    Type: Application
    Filed: April 20, 2020
    Publication date: October 21, 2021
    Applicant: Varjo Technologies Oy
    Inventor: Mikko Ollila
  • Patent number: 11145031
    Abstract: A display apparatus including first display or projector for displaying first images for first eye; second display or projector for displaying second images for second eye; first portion and second portion arranged to face first and second eyes; means for tracking poses of first and second eyes relative to first and second optical axes, respectively; and processor. Processor or external processor is configured to: obtain given pose of given eye relative to given optical axis; generate information pertaining to given visual artifact that is formed over given image at image plane when given image is displayed; determine artifact-superposing portion of given image; and process given image based on generated information and artifact-superposing portion, to generate given artifact-compensated image. Processor displays given artifact-compensated image via given display or projector.
    Type: Grant
    Filed: January 31, 2020
    Date of Patent: October 12, 2021
    Assignee: Varjo Technologies Oy
    Inventor: Ari Antti Peuhkurinen
  • Patent number: 11138760
    Abstract: A display system and method for correcting drifts in camera poses. Images are captured via camera, and camera poses are determined in global coordinate system. First features are extracted from first image. Relative pose of first feature with respect to camera is determined. Pose of first feature in global coordinate system is determined, based on its relative pose and first camera pose. Second features are extracted from second image. Relative pose of second feature with respect to camera is determined. Pose of second feature in global coordinate system is determined, based on its relative pose and second camera pose. Matching features are identified between first features and second features. Difference is determined between pose of feature based on first camera pose and pose of feature based on second camera pose. Matching features that satisfy first predefined criterion based on difference are selected.
    Type: Grant
    Filed: November 6, 2019
    Date of Patent: October 5, 2021
    Assignee: Varjo Technologies Oy
    Inventors: Thomas Carlsson, Ville Timonen
  • Patent number: 11138807
    Abstract: A system for producing extended-reality images for a display apparatus. The system includes camera(s) and processor communicably coupled to camera(s), wherein processor is configured to: control camera(s) to capture image(s) representing test object present in real-world environment, wherein test object is physically covered three-dimensionally with coded pattern; obtain information pertaining to three-dimensional geometry of coded pattern; analyze image(s) to identify first image segment representing part of coded pattern visible in image(s); determine virtual content to be presented for test object, based on said part of coded pattern; process image(s) to generate extended-reality image(s) in which virtual content is virtually superimposed over said part of the coded pattern, based on information pertaining to three-dimensional geometry of coded pattern.
    Type: Grant
    Filed: March 19, 2021
    Date of Patent: October 5, 2021
    Assignee: Varjo Technologies Oy
    Inventors: Kai Inha, Ari Antti Erik Peuhkurinen, Mikko Ollila, Roope Rainisto
  • Publication number: 20210303854
    Abstract: An imaging system and a method for producing extended-reality images for a display apparatus. The imaging system includes camera and processor. The processor is configured to: control camera to capture image of real-world environment; analyse captured image to identify first image segment representing input device and to determine location of at least one actionable area of input device in first image segment; determine at least one functional element to be presented for the actionable area, the functional element being indicative of at least one of: functionality of the at least one actionable area, status of the at least one actionable area; and process captured image to generate extended-reality image in which the functional element is virtually superimposed over the actionable area of input device or a virtual representation of the actionable area of input device.
    Type: Application
    Filed: March 26, 2020
    Publication date: September 30, 2021
    Applicant: Varjo Technologies Oy
    Inventors: Ari Antti Peuhkurinen, Aleksei Romanov, Evgeny Zuev, Yuri Popov, Tomi Lehto, Urho Konttori