Search Patents
  • Redundant tracking system
    Patent number: 10504287
    Abstract: A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6DoF) and 3DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6DoF and 3DoF thereby providing the user with an uninterrupted experience.
    Type: Grant
    Filed: October 24, 2017
    Date of Patent: December 10, 2019
    Assignee: Snap Inc.
    Inventors: Andrew James McPhee, Samuel Edward Hare, Peicheng Yu, Robert Cornelius Murphy, Dhritiman Sagar
  • CURSOR FUNCTIONALITY FOR AUGMENTED REALITY CONTENT IN MESSAGING SYSTEMS
    Publication number: 20240087244
    Abstract: The subject technology detects a location and a position of a representation of a finger in a set of frames captured by a camera of a client device. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders the first virtual object within a first scene. The subject technology detects a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object. The subject technology modifies a set of dimensions of the second virtual object to a second set of dimensions. The subject technology renders the second virtual object based on the second set of dimensions within a second scene. The subject technology provides for display the rendered second virtual object within the second scene.
    Type: Application
    Filed: September 9, 2022
    Publication date: March 14, 2024
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • 3D CUTOUT IMAGE MODIFICATION
    Publication number: 20220076500
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated. by a camera of a user; displaying an option to crop an object depicted in the image; determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.
    Type: Application
    Filed: November 18, 2021
    Publication date: March 10, 2022
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • 3D CUTOUT IMAGE MODIFICATION
    Publication number: 20210287439
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated by a camera of a user; displaying an option to crop an object depicted in the image; determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.
    Type: Application
    Filed: April 28, 2020
    Publication date: September 16, 2021
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • SHOOTING INTERACTION USING AUGMENTED REALITY CONTENT IN A MESSAGING SYSTEM
    Publication number: 20240087243
    Abstract: The subject technology receives a set of frames. The subject technology detect a first gesture correspond to an open trigger finger gesture. The subject technology receives a second set of frames. The subject technology detects from the second set of frames, a second gesture correspond to a closed trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the closed trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders a movement of the first virtual object along a vector away from the location and the position of the representation of the finger within a first scene. The subject technology provides for display the rendered movement of the first virtual object along the vector within the first scene.
    Type: Application
    Filed: September 9, 2022
    Publication date: March 14, 2024
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • VIRTUAL OBJECT MANIPULATION WITH GESTURES IN A MESSAGING SYSTEM
    Publication number: 20240087264
    Abstract: The subject technology detects a first gesture and a second gesture, each gesture corresponding to an open trigger finger gesture. The subject technology detects a third gesture and a fourth gesture, each gesture corresponding to a closed trigger finger gesture. The subject technology, selects a first virtual object in a first scene. The subject technology detects a first location and a first position of a first representation of a first finger from the third gesture and a second location and a second position of a second representation of a second finger from the fourth gesture. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies a set of dimensions of the first virtual object to a different set of dimensions.
    Type: Application
    Filed: September 9, 2022
    Publication date: March 14, 2024
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • Virtual object manipulation with gestures in a messaging system
    Patent number: 11948266
    Abstract: The subject technology detects a first gesture and a second gesture, each gesture corresponding to an open trigger finger gesture. The subject technology detects a third gesture and a fourth gesture, each gesture corresponding to a closed trigger finger gesture. The subject technology, selects a first virtual object in a first scene. The subject technology detects a first location and a first position of a first representation of a first finger from the third gesture and a second location and a second position of a second representation of a second finger from the fourth gesture. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies a set of dimensions of the first virtual object to a different set of dimensions.
    Type: Grant
    Filed: September 9, 2022
    Date of Patent: April 2, 2024
    Assignee: SNAP INC.
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • 3D CURSOR FUNCTIONALITY FOR AUGMENTED REALITY CONTENT IN MESSAGING SYSTEMS
    Publication number: 20240087242
    Abstract: The subject technology detects a location and a position of a representation of a finger. The subject technology generates a first virtual object based on the location and the position of the representation of the finger. The subject technology detects a first collision event. The subject technology in response to the first collision event, modifies a set of dimensions of the second virtual object to a second set of dimensions. The subject technology detects a second location and a second position of the representation of the finger. The subject technology detects a second collision event. The subject technology modifies a set of dimensions of the third virtual object to a third set of dimensions. The subject technology renders the third virtual object based on the third set of dimensions within a third scene, the third scene comprising a modified scene from a second scene.
    Type: Application
    Filed: September 9, 2022
    Publication date: March 14, 2024
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • TRIGGER GESTURE FOR SELECTION OF AUGMENTED REALITY CONTENT IN MESSAGING SYSTEMS
    Publication number: 20240087246
    Abstract: The subject technology detects a first gesture corresponding to an open trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the open trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology detects a first collision event. The subject technology detects a second gesture corresponding to a closed trigger finger gesture. The subject technology selects the second virtual object. The subject technology renders the first virtual object as attached to the second virtual object in response to the selecting. The subject technology provides for display the rendered first virtual object as attached to the second virtual object within a first scene.
    Type: Application
    Filed: September 9, 2022
    Publication date: March 14, 2024
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • 3D PAINTING ON AN EYEWEAR DEVICE
    Publication number: 20220197485
    Abstract: Systems and methods are provided for performing operations comprising: displaying a plurality of augmented reality painting options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; selecting a first augmented reality painting option of the plurality of augmented reality painting options in response to the first touch input; while continuing to detect continuous touch between the single finger and the touch input interface following selection of the first augmented reality painting option, displaying a second augmented reality painting option related to the first augmented reality painting option; and performing a selection associated with the second augmented reality painting option in response to detecting, by the touch input interface, movement of the single finger along the touch input interface while continuing to detect the continuous touch.
    Type: Application
    Filed: August 24, 2021
    Publication date: June 23, 2022
    Inventors: Kyle Goodrich, Andrew James McPhee, Daniel Moreno
  • 3D cutout image modification
    Patent number: 11217020
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated by a camera of a user; displaying an option to crop an object depicted in the image; determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.
    Type: Grant
    Filed: April 28, 2020
    Date of Patent: January 4, 2022
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • EFFECTS FOR 3D DATA IN A MESSAGING SYSTEM
    Publication number: 20230015194
    Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.
    Type: Application
    Filed: September 22, 2022
    Publication date: January 19, 2023
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • RECENTERING AR/VR CONTENT ON AN EYEWEAR DEVICE
    Publication number: 20220198603
    Abstract: Systems and methods are provided for performing operations comprising: displaying one or more virtual objects at a first virtual coordinate associated with a first portion of a real-world environment while the eyewear device is directed towards the first portion of the real-world environment; determining that the eyewear device has been moved to be directed towards a second portion of the real-world environment; maintaining the display of the one or more virtual objects at the first virtual coordinate while the eyewear device is directed towards the second portion of the real-world environment; receiving, by the one or more processors, a request to bring the one or more virtual objects into a current view of the eyewear device; and in response to receiving the request, moving the one or more virtual objects from the first virtual coordinate to a second virtual coordinate associated with the second portion of the real-world environment.
    Type: Application
    Filed: August 24, 2021
    Publication date: June 23, 2022
    Inventors: Kyle Goodrich, Andrew James McPhee, Daniel Moreno
  • Effects for 3D data in a messaging system
    Patent number: 11457196
    Abstract: The subject technology selects a set of augmented reality content generators from a plurality of available augmented reality content generator based on metadata associated with each respective augmented reality content generator. The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology captures image data and depth data using at least one camera of the client device. The subject technology applies, to the image data and the depth data, the 3D effect based at least in part on the augmented reality content generator.
    Type: Grant
    Filed: August 28, 2020
    Date of Patent: September 27, 2022
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • EFFECTS FOR 3D DATA IN A MESSAGING SYSTEM
    Publication number: 20240048678
    Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.
    Type: Application
    Filed: October 18, 2023
    Publication date: February 8, 2024
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • 3D PAINTING ON AN EYEWEAR DEVICE
    Publication number: 20240012549
    Abstract: Systems and methods are provided for performing operations comprising: displaying a plurality of augmented reality painting options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; selecting a first augmented reality painting option of the plurality of augmented reality painting options in response to the first touch input; while continuing to detect continuous touch between the single finger and the touch input interface following selection of the first augmented reality painting option, displaying a second augmented reality painting option related to the first augmented reality painting option; and performing a selection associated with the second augmented reality painting option in response to detecting, by the touch input interface, movement of the single finger along the touch input interface while continuing to detect the continuous touch.
    Type: Application
    Filed: September 20, 2023
    Publication date: January 11, 2024
    Inventors: Kyle Goodrich, Andrew James McPhee, Daniel Moreno
  • Effects for 3D data in a messaging system
    Patent number: 11825065
    Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.
    Type: Grant
    Filed: September 22, 2022
    Date of Patent: November 21, 2023
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • 3D cutout image modification
    Patent number: 11775165
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated by a camera of a user; displaying an option to crop an object depicted in the image, determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.
    Type: Grant
    Filed: November 18, 2021
    Date of Patent: October 3, 2023
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • PASSING AUGMENTED REALITY CONTENT BETWEEN DEVICES
    Publication number: 20210304506
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for receiving, from a first device associated with a first user, a first request to submit a media content item for viewing, the media content item corresponding to display of a first image, captured by the first device, with an augmented reality content item, the first request including an identifier of the augmented reality content item; storing the media content item together with the identifier of the augmented reality content item; providing, to a second device associated with a second user, the media content item; receiving, from the second device, a second request to identify the augmented reality content item; and providing, to the second device, the identifier of the augmented reality content item.
    Type: Application
    Filed: March 17, 2021
    Publication date: September 30, 2021
    Inventors: Celia Nicole Mourkogiannis, Kaveh Anvaripour, Andrew James McPhee, Daniel Moreno, Kyle Goodrich, Vincent Sung
  • VIDEO CLIP OBJECT TRACKING
    Publication number: 20210256773
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.
    Type: Application
    Filed: May 4, 2021
    Publication date: August 19, 2021
    Inventors: Samuel Edward Hare, Andrew James McPhee, Tony Mathew
Narrow Results

Filter by US Classification