Search Patents
  • VIDEO CLIP OBJECT TRACKING
    Publication number: 20200074738
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object; and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.
    Type: Application
    Filed: January 8, 2019
    Publication date: March 5, 2020
    Inventors: Samuel Edward Hare, Andrew James McPhee, Tony Mathew
  • Media content player on an eyewear device
    Patent number: 11782577
    Abstract: Systems and methods are provided for performing operations comprising: displaying, by one or more processors of an eyewear device, a plurality of media content control options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; based on detecting the first touch input, causing a cursor to track the first touch input to navigate through the plurality of media content control options; based on detecting that the single finger has not been released from touching the touch input interface, displaying a second media content control option related to the first media content control option; and performing a selection associated with the second media content control option based on movement of the single finger along the touch input interface.
    Type: Grant
    Filed: August 24, 2021
    Date of Patent: October 10, 2023
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Andrew James McPhee, Daniel Moreno
  • Shooting interaction using augmented reality content in a messaging system
    Patent number: 11995780
    Abstract: The subject technology receives a set of frames. The subject technology detect a first gesture correspond to an open trigger finger gesture. The subject technology receives a second set of frames. The subject technology detects from the second set of frames, a second gesture correspond to a closed trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the closed trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders a movement of the first virtual object along a vector away from the location and the position of the representation of the finger within a first scene. The subject technology provides for display the rendered movement of the first virtual object along the vector within the first scene.
    Type: Grant
    Filed: September 9, 2022
    Date of Patent: May 28, 2024
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Maxim Maximov Lazarov, Andrew James McPhee, Daniel Moreno
  • VIDEO CLIP OBJECT TRACKING
    Publication number: 20230316683
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object; and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.
    Type: Application
    Filed: June 7, 2023
    Publication date: October 5, 2023
    Inventors: Samuel Edward Hare, Andrew James McPhee, Tony Mathew
  • 3D PAINT EFFECTS IN A MESSAGING SYSTEM
    Publication number: 20210358215
    Abstract: Systems and methods are provided for determining a location of a selection in a space viewable in a camera view on a display of a computing device, detecting movement of the computing device, and generating a path based on the location of the selection and the movement of the computing device. The systems and methods further provide for generating a three-dimensional (3D) mesh along the path, populating the 3D mesh with selected options to generate a 3D paint object, and causing the generated 3D paint object to be displayed. The systems and methods further provide for receiving a request to send a message comprising an image or video overlaid by the 3D paint object, capturing the image or video overlaid by the displayed 3D paint object, generating the message comprising the image or video overlaid by the 3D paint object, and sending the message to another computing device.
    Type: Application
    Filed: July 28, 2021
    Publication date: November 18, 2021
    Inventors: Piers George Cowburn, Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, David Li, Tony Mathew, Andrew James Mcphee, Daniel Moreno, Isac Andreas Müller Sandvik, Wentao Shang
  • 3D PAINT EFFECTS IN A MESSAGING SYSTEM
    Publication number: 20230196675
    Abstract: Systems and methods are provided for determining a location of a selection in a space viewable in a camera view on a display of a computing device, detecting movement of the computing device, and generating a path based on the location of the selection and the movement of the computing device. The systems and methods further provide for generating a three-dimensional (3D) mesh along the path, populating the 3D mesh with selected options to generate a 3D paint object, and causing the generated 3D paint object to be displayed. The systems and methods further provide for receiving a request to send a message comprising an image or video overlaid by the 3D paint object, capturing the image or video overlaid by the displayed 3D paint object, generating the message comprising the image or video overlaid by the 3D paint object, and sending the message to another computing device.
    Type: Application
    Filed: February 15, 2023
    Publication date: June 22, 2023
    Inventors: Piers George Cowburn, Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, David Li, Tony Mathew, Andrew James McPhee, Daniel Moreno, Isac Andreas Müller Sandvik, Wentao Shang
  • PASSING AUGMENTED REALITY CONTENT BETWEEN DEVICES
    Publication number: 20230033146
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for receiving, from a first device associated with a first user, a first request to submit a media content item for viewing, the media content item corresponding to display of a first image, captured by the first device, with an augmented reality content item, the first request including an identifier of the augmented reality content item; storing the media content item together with the identifier of the augmented reality content item; providing, to a second device associated with a second user, the media content item; receiving, from the second device, a second request to identify the augmented reality content item; and providing, to the second device, the identifier of the augmented reality content item.
    Type: Application
    Filed: August 8, 2022
    Publication date: February 2, 2023
    Inventors: Celia Nicole Mourkogiannis, Kaveh Anvaripour, Andrew James McPhee, Daniel Moreno, Kyle Goodrich, Vincent Sung
  • 3D painting on an eyewear device
    Patent number: 11797162
    Abstract: Systems and methods are provided for performing operations comprising: displaying a plurality of augmented reality painting options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; selecting a first augmented reality painting option of the plurality of augmented reality painting options in response to the first touch input; while continuing to detect continuous touch between the single finger and the touch input interface following selection of the first augmented reality painting option, displaying a second augmented reality painting option related to the first augmented reality painting option; and performing a selection associated with the second augmented reality painting option in response to detecting, by the touch input interface, movement of the single finger along the touch input interface while continuing to detect the continuous touch.
    Type: Grant
    Filed: August 24, 2021
    Date of Patent: October 24, 2023
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Andrew James McPhee, Daniel Moreno
  • VOICE NOTES WITH CHANGING EFFECTS
    Publication number: 20230135516
    Abstract: Methods and systems are disclosed for generating voice notes. In one embodiment, a messaging application implemented on a first participant computing device, receives a request to send a voice message to a second participant computing device in a communication session. In response to receiving the request, the messaging application selects a voice modification effect. The messaging application receives input to begin recording the voice message. While the voice message is being recorded, the messaging application receives speech input, modifies the speech input based on the voice modification effect in real time, and continuously updates an audio file with the modified speech input. In response to receiving input to terminate recording the voice message, the messaging application provides a first option to playback the audio file and a second option to send the audio file as the voice message to the second participant computing device.
    Type: Application
    Filed: October 29, 2021
    Publication date: May 4, 2023
    Inventors: Alex Edelsburg, Kyle Goodrich, Pedram Javidpour, Andrew James McPhee, Daniel Moreno, Jeremy Baker Voss
  • Passing augmented reality content between devices
    Patent number: 11694411
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for receiving, from a first device associated with a first user, a first request to submit a media content item for viewing, the media content item corresponding to display of a first image, captured by the first device, with an augmented reality content item, the first request including an identifier of the augmented reality content item; storing the media content item together with the identifier of the augmented reality content item; providing, to a second device associated with a second user, the media content item; receiving, from the second device, a second request to identify the augmented reality content item; and providing, to the second device, the identifier of the augmented reality content item.
    Type: Grant
    Filed: August 8, 2022
    Date of Patent: July 4, 2023
    Assignee: Snap Inc.
    Inventors: Celia Nicole Mourkogiannis, Kaveh Anvaripour, Andrew James McPhee, Daniel Moreno, Kyle Goodrich, Vincent Sung
  • Passing augmented reality content between devices
    Patent number: 11410396
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for receiving, from a first device associated with a first user, a first request to submit a media content item for viewing, the media content item corresponding to display of a first image, captured by the first device, with an augmented reality content item, the first request including an identifier of the augmented reality content item; storing the media content item together with the identifier of the augmented reality content item; providing, to a second device associated with a second user, the media content item; receiving, from the second device, a second request to identify the augmented reality content item; and providing, to the second device, the identifier of the augmented reality content item.
    Type: Grant
    Filed: March 17, 2021
    Date of Patent: August 9, 2022
    Assignee: Snap Inc.
    Inventors: Celia Nicole Mourkogiannis, Kaveh Anvaripour, Andrew James McPhee, Daniel Moreno, Kyle Goodrich, Vincent Sung
  • Video clip object tracking
    Patent number: 11030813
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object; and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.
    Type: Grant
    Filed: January 8, 2019
    Date of Patent: June 8, 2021
    Assignee: Snap Inc.
    Inventors: Samuel Edward Hare, Andrew James McPhee, Tony Mathew
  • Video clip object tracking
    Patent number: 11715268
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.
    Type: Grant
    Filed: May 4, 2021
    Date of Patent: August 1, 2023
    Assignee: Snap Inc.
    Inventors: Samuel Edward Hare, Andrew James McPhee, Tony Mathew
  • Voice notes with changing effects
    Patent number: 11996113
    Abstract: Methods and systems are disclosed for generating voice notes. In one embodiment, a messaging application implemented on a first participant computing device, receives a request to send a voice message to a second participant computing device in a communication session. In response to receiving the request, the messaging application selects a voice modification effect. The messaging application receives input to begin recording the voice message. While the voice message is being recorded, the messaging application receives speech input, modifies the speech input based on the voice modification effect in real time, and continuously updates an audio file with the modified speech input. In response to receiving input to terminate recording the voice message, the messaging application provides a first option to playback the audio file and a second option to send the audio file as the voice message to the second participant computing device.
    Type: Grant
    Filed: October 29, 2021
    Date of Patent: May 28, 2024
    Assignee: Snap Inc.
    Inventors: Alex Edelsburg, Kyle Goodrich, Pedram Javidpour, Andrew James McPhee, Daniel Moreno, Jeremy Baker Voss
  • 3D paint effects in a messaging system
    Patent number: 11625895
    Abstract: Systems and methods are provided for determining a location of a selection in a space viewable in a camera view on a display of a computing device, detecting movement of the computing device, and generating a path based on the location of the selection and the movement of the computing device. The systems and methods further provide for generating a three-dimensional (3D) mesh along the path, populating the 3D mesh with selected options to generate a 3D paint object, and causing the generated 3D paint object to be displayed. The systems and methods further provide for receiving a request to send a message comprising an image or video overlaid by the 3D paint object, capturing the image or video overlaid by the displayed 3D paint object, generating the message comprising the image or video overlaid by the 3D paint object, and sending the message to another computing device.
    Type: Grant
    Filed: July 28, 2021
    Date of Patent: April 11, 2023
    Assignee: Snap Inc.
    Inventors: Piers George Cowburn, Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, David Li, Tony Mathew, Andrew James McPhee, Daniel Moreno, Isac Andreas Müller Sandvik, Wentao Shang
  • RENDERING 3D CAPTIONS WITHIN REAL-WORLD ENVIRONMENTS
    Publication number: 20230089838
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.
    Type: Application
    Filed: December 1, 2022
    Publication date: March 23, 2023
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • RENDERING 3D CAPTIONS WITHIN REAL-WORLD ENVIRONMENTS
    Publication number: 20200327734
    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.
    Type: Application
    Filed: November 26, 2019
    Publication date: October 15, 2020
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazerov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Wentao Shang
  • BEAUTIFICATION TECHNIQUES FOR 3D DATA IN A MESSAGING SYSTEM
    Publication number: 20230005233
    Abstract: The subject technology receives a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator for applying a 3D effect, the 3D effect including at least one beautification operation. The subject technology captures image data and depth data using a camera. The subject technology applies, to the image data and the depth data, the 3D effect including the at least one beautification operation based at least in part on the augmented reality content generator, the beautification operation being performed as part of applying the 3D effect. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation. The subject technology renders a view of the 3D message based at least in part on the applied 3D effect including the at least one beautification operation.
    Type: Application
    Filed: August 1, 2022
    Publication date: January 5, 2023
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • Beautification techniques for 3D data in a messaging system
    Patent number: 11776233
    Abstract: The subject technology receives a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator for applying a 3D effect, the 3D effect including at least one beautification operation. The subject technology captures image data and depth data using a camera. The subject technology applies, to the image data and the depth data, the 3D effect including the at least one beautification operation based at least in part on the augmented reality content generator, the beautification operation being performed as part of applying the 3D effect. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation. The subject technology renders a view of the 3D message based at least in part on the applied 3D effect including the at least one beautification operation.
    Type: Grant
    Filed: August 1, 2022
    Date of Patent: October 3, 2023
    Assignee: Snap Inc.
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
  • BEAUTIFICATION TECHNIQUES FOR 3D DATA IN A MESSAGING SYSTEM
    Publication number: 20230410450
    Abstract: The subject technology applies, to image data and depth data, a 3D effect including at least one beautification operation based on an augmented reality content generator, the 3D effect including a beautification operation, the beautification operation comprising modifying image data, the image data including a region corresponding to a representation of a face, the beautification operation comprising using a machine learning model for at least one of smoothing blemishes or preserving facial skin texture. The subject technology generates a depth map using at least the depth data. The subject technology generates a segmentation mask based at least on the image data. The subject technology performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation.
    Type: Application
    Filed: August 31, 2023
    Publication date: December 21, 2023
    Inventors: Kyle Goodrich, Samuel Edward Hare, Maxim Maximov Lazarov, Tony Mathew, Andrew James McPhee, Daniel Moreno, Dhritiman Sagar, Wentao Shang
Narrow Results

Filter by US Classification